Growers, unable to access backpacker labour due to the international travel restrictions, have had to abandon some trees and let fruit fall to the ground.
In other cases, growers have been unable to get premium prices because they could not get the fruit harvested at the optimum time.
One major grower near Shepparton, who did not wish to be identified, estimated his own losses at close to $1 million this season.
Growers have been unable to access international backpackers, and the Victorian Government has been unable to facilitate the arrival of Pacific Island labourers in time for the harvest peak.
The state government's scheme to attract local labour has also been unsuccessful.
The amount of abandoned fruit in GV orchards has also raised concerns there will be a reservoir for fruit fly.
A month ago, the NSW Farmers Association estimated crop losses at $50 million.
At Shepparton East, the Mladenov family grows fruit on a small orchard for their popular shop-front business, Boris's Fruit Shed.
While the excellent growing conditions this season has produced good fruit for their business, tonnes of fruit lies abandoned on the ground because it couldn't be harvested in time.
Jeff Mladenov said they could not find labour through the usual sources, including the harvest labour office, and had to rely on a small band of loyal workers to bring in the harvest.
Several crops matured at the same time during the peak, resulting in peaches, nashis and nectarines being left on the ground.
“At the peak of the season we are getting up to 20 tonnes a day, but the most we could bring in this year was about 15 tonnes,” Mr Mladenov said.
The family was disappointed more unemployed local people hadn't pitched in to help with the harvest.
Family matriarch Emily Mladenov is 79, and still picks fruit each day for the retail outlet.
Worker Hadiseh was living in north-eastern Victoria when she picked up the Mladenovs’ contact number by word of mouth and ended up staying for the season.
She is picking about four to six bins a day and saving to open up her own car detailing business.
Fruit Growers Victoria crop services manager Michael Crisera said as well as the fruit that might be left on the ground, other fruit could not be picked at optimum maturity, resulting in penalties for growers.
He understood that the first flights of Pacific Island workers were due to arrive in Tasmania for quarantine. They now face a two-week wait before they can go onto Victorian farms.
Only about 40,000 working holiday makers remain in Australia, compared to around 200,000 pre-COVID levels — and they are leaving Australia to return home at the rate of about 1000 each week.
Meanwhile, the Labor Party and labour unions have been critical of some agricultural enterprises which have been underpaying workers.
A new report has been released from Unions NSW based on a survey of job advertisements for fruit pickers and interviews with workers.
Federal Opposition Leader Anthony Albanese called for the Morrison Government to tackle wage theft as he launched the report on Friday.
“We need to do much much better and we need to address this now,” Mr Albanese said.
“The report's authors reviewed more than 1000 ads published, mainly on social media, between December 2019 and September 2020.
“They also interviewed over 100 horticultural workers .Of the workers they interviewed, none met the minimum legal wage.”
The vast majority — 88 per cent — of job ads promised "piece rates" rather than an hourly wage, meaning workers were paid per volume of work.