There were only six minutes of normal time left when David Vaughan's powerful shot drew Sunderland level in front of their own fans at the Stadium of Light.
And just when it seemed the hosts would have to settle for a draw, Rovers conceded a free-kick on the edge of their box and Sweden international Larsson curled a shot around the defensive wall that flew in at former England goalkeeper Paul Robinson's near post.
Victory meant Sunderland, who recently sacked Steve Bruce as their manager, climbed out of the relegation zone while Rovers remained in the bottom three.
"The confidence was a little bit low, as you'd expect with where we are in the league, but hopefully we can get on a good run now," said Vaughan, after what was only the north-east side's third league success this season.
Larsson added: "Martin told us he believes in us. This will do the world of good for us. We needed this kind of win and we deserved it."
Rovers midfielder Simon Vukcevic opened the scoring in the 17th minute when he headed in after Sunderland goalkeeper Keiren Westwood could only parry Christopher Samba's shot.
The Black Cats should have equalised six minutes later but Kieran Richardson failed to get enough power into a shot from six yards out and his effort was well saved by Robinson.
And four minutes before half-time Sunderland nearly fell 2-0 behind when Rovers' Scott Dann out the ball in the net only for his 'goal' to be disallowed for a foul by Samba on the keeper.
Blackburn seemed content to defend their lead for most of the second half, rather than press for a second goal, and it was a policy that looked as if it would result in a precious away win for Rovers manager Steve Kean until Sunderland's late revival.
Former Leicester and Celtic boss O'Neill had been out of management since ending his four-year spell in charge of Aston Villa last year by resigning from the Birmingham club.
Source: AFP
Source: AFP