The 35-year-old Ukraine captain took his tally to 48 in 108 international appearances with two superb headers within seven minutes of each other - his opposite number Zlatan Ibrahimovic's neat 52nd minute finish had give the Swedes the lead.
The victory sees Ukraine - who are co-hosting the tournament with Poland - top the table after the other group match between France and England ended 1-1.
"We deserved it; we played great," said Shevchenko after the game, clearly moved by the occasion. "I want to thank the fans and the supporters who believed in us. It's a great achievement."
There had been little indication of the fairytale on the evidence of the first-half as Shevchenko looked way off the pace and when he was found free on the right of the box the 35-year-old, who has been struggling with injury for the past year, scuffed his shot.
The Ukrainians carved out the best chances with another veteran Andrei Voronin forcing Andreas Isaksson to punch to safety from long range while one of the younger generation, Andrei Yarmolenko, was also denied by the Swedish keeper at closer range.
However, the Swedes should have broken the deadlock in the 40th minute, but their captain Ibrahimovic saw his free header just go the wrong side of the far post. The AC Milan star then struck in the 52nd minute, this time making no mistake.
The Ukrainians failed to clear their lines and Kim Kallstrom managed to lose his marker, collect Sebastian Larsson's cross and pass inside to Ibrahimovic, who coolly slotted it into the net and embarked on a typical flamboyant celebration, arm raised straight in the air.
The goal had been coming after the Swedes had come out pumped-up from the half-time break with Markus Rosenberg going close. However, their lead last justed three minutes as Yarmolenko delivered a superb cross into the area and Shevchenko - like in the days of old - got ahead of his marker, veteran Olof Mellberg, to head powerfully into the net.
The crowd, the scorer and his team went berserk and they were to even outdo that noise seven minutes later as, from an Oleg Gusev corner, Shevchenko nipped in ahead of Ibrahimovic, this time to head in at the near post. The charismatic striker charged down the touchline to embrace his fellow former European player of the year, coach Oleg Blokhin.
Sweden kept pressing for the equaliser with Kallstrom going the closest with a fierce effort from outside the area which the not always convincing Andrei Pyatov fumbled away and was fortunate that Ibrahimovic was unable to put the rebound past him.
The Swedes should have levelled the game in the dying minutes as Ibrahimovic's divine flick set up Johan Elmander, but showing the rustiness that has seen him only return to action after a month out, he flashed his shot across the post.