Nicky Breen from the consumer advocacy group Choice said the delays were especially frustrating for people waiting on Christmas deliveries.
"The biggest bugbear is for a consumer to walk out of their house and find a missed delivery notification in their mailbox, even though they'd been home all day," she said.
Many people then face lengthy queues at the post office to collect their parcels.
Earlier this year, a Choice survey of 1,025 Australians found that more than one in four had experienced problems with parcel deliveries in the past 12 months.
The most common complaint was about receiving "missed delivery" notifications when the customers were actually at home.
The survey included all parcel delivery services and was not confined to Australia Post — but it is the largest provider.
Ms Breen said if people believed drivers were not making genuine delivery attempts, they should notify Australia Post.
"It's much easier now to trace delivery to drivers, so if you're having an issue ... make a complaint," she said.
'Shortcuts taken' during Christmas rush
The union representing postal workers said the problem was being caused by subcontracted drivers, who were paid as little as $1.00 per delivery.
Communication, Electrical and Plumbing Union organiser Peter Chaloner said drivers were under enormous pressure in December.
"When the heat's on at Christmas time, there's only so many hours in the day," he said.
"You're not getting paid for extra hours, you're just getting paid for extra articles, so shortcuts are taken."
He said Australia Post needed to consider the welfare of subcontracted drivers when negotiating tenders for package delivery services.