Crews were called to the Bright Moon Buddhist Temple in Springvale South about 8pm on Sunday as large flames engulfed the building.
It took about 80 firefighters more than two hours to control the blaze.
About 30 residents in the surrounding area had to be evacuated, while others were told to close doors and windows and turn off heating and cooling systems.
No one was injured in the blaze, Fire Rescue Victoria confirmed.
Firefighters were continuing to put out spot fires on Monday morning, Assistant Chief Fire Officer Paul Foster said.
"This was an extensive fire, very rapidly developing and quite spectacular to look at," he told ABC News Breakfast on Monday morning. "It was seen for many kilometres around."
It was still too early to determine the cause of the blaze, Mr Foster said.
Fire Rescue Victoria will work with the local Buddhist community as they investigate the fire.
"There's a very culturally sensitive operation we've conducted here," Mr Foster said.
"Not only is it a place of worship, it's a place of gathering for the local Buddhist community."
It's understood some members of the Springvale Buddhist community had their loved ones' ashes kept inside the temple.
8°C