After Travis Head's century and Scott Boland's defence initially repelled the Proteas, Australia lost their last wicket with 28.5 overs to play on the final day on Thursday.
It means the series will head to the third and final Test in Cape Town next Tuesday with the series level at 1-1.
Aided by rain that washed out the morning session, Australia did well to hold on for as long as they did on Thursday with Boland soaking up 166 balls as nightwatchman.
But after Head (117) and Steve Smith fell before tea, Boland was bowled shortly after the break to open the door for the Proteas at St George's Park.
And when Kagiso Rabada (4-77) bowled a hard-swinging Mitchell Starc and Josh Hazlewood edged Marco Jansen to slip, the game was over.
Ultimately, Australia have been left to lick their wounds in a match they should never have been in a position to lose.
Pat Cummins' men had South Africa 6-110 early on the third morning, after Boland's five-wicket haul ripped through the Proteas on a wet opening two days.
But they allowed Temba Bavuma to lead a South African recovery to 9-425 declared.
And when Australia recorded their lowest first-innings score in almost 10 years on Thursday and were bowled out for 99, they were fighting to save the match.
This Test will go down as one of the Proteas' great wins given the position they were in, with Bavuma and Marco Jansen the heroes.
For Australia, though, there will be recriminations.
Marnus Labuschagne, Nic Maddinson and Beau Webster are all under pressure to keep their spots, particularly with Cameron Green and Josh Inglis fit again.
Boland and Head briefly looked likely to save their blushes on Thursday, with their 59-run fifth-wicket stand surviving through the first 80 minutes.
But when Head played a Wiaan Mulder delivery that seamed back in at him onto his own stumps, South Africa could sniff victory again.
Smith then inexplicably attempted to pull Kagiso Rabada (3-73) seven overs later against the new ball, skying him to Matthew Breetzke at mid-wicket.
That shot was one of several questionable ones for Australia over the past two days, with Labuschagne's into gusty winds on the fourth evening with his side 1-100 equally poor.
Starc was also dismissed swinging hard at a Rabada delivery late as threatening clouds began rolling in, before Hazlewood guided Jansen straight to slip and left Alex Carey (37no) with no more support.
The pair appeared frustrated by their shots, but it is the batters that must accept the blame for Australia's crushing defeat.
23°C