Arsenal were 2-3 down with 11 minutes to go, but super-subs Aaron Ramsey and Olivier Giroud propped up with a goal each to give the Gunners a great win.
Arsenal woes continued after half-time when Vardy headed his second after a brilliant Riyad Mahrez corner.
Substitute Aaron Ramsey rifled home to level before Giroud - introduced alongside the Wales global - headed in an 85th-minute victor to settle a breathless encounter in Arsenal's favour.
In his post-match reaction, Arsenal boss Wenger was understandably delighted with how his new signing was settling into proceedings. "I know how important he is not only as a player but also in the group".
Giroud scores the dramatic victor to save Arsenal, a simply sensation moment and Leicester has their hearts broken again against Arsenal with a late goal scored against them.
"We kept going and the spirit of the team was absolutely outstanding", Wenger said.
Arsenal's new striker Alexandre Lacazette is the proud joint record-holder for the fastest goal scored on a Premier League opening day.
"We're gonna win the league!" sang the Leicester fans and they nearly had a third goal to cheer when Okazaki's header from Christian Fuchs's cross bounced just wide of Cech's left-hand post. The Foxes also looked to unleash the pace of Jamie Vardy at every opportunity, teeing him up to run into the open spaces to the side of the wide centre-backs. He'll play 30 games, he'll give you 15 goals.
Arsene Wenger also added after the game that he was delighted that the 30-year-old had chose to stay at the Emirates, after he had offered him the opportunity to leave. One was a corner, one was a ball we lost in the build-up and the third goal was a corner again. We want him to be a nuisance.
Petr Cech has seen better days after conceding three goals, and he said Leicester did well to pounce on Arsenal's errors.
'It's impossible. You know that you can not keep them happy when they don't play, ' Wenger told Sky Sports.
"It would be unwise of him not to consult experts as it relates to his persisting knee injuries in order to help him because his place with Watford is seriously threatened and I don't see him staying past this season at this rate".