Out of all seven, my absolute favorite is "Harry Potter and the Prisoner of Azkaban". In a previous thread, I mention that it is also my favorite movie.
Although I love "Deathly Hallows", I have to say that I love "Prisoner of Azkaban" more.
It might just be because I considered Lupin a great character, or because the Dementors are so enthralling and fascinating to me.
For the first time, I think, you see the true Magical community, you are thrust into a very different situation than before. It starts off with Harry jumping out into the unknown, all of a sudden he's not at the Dursley's anymore - he's run away - and while his little escapade was short lived, it was thrilling. He's alone, in the magical world for a couple of weeks. He is enjoying himself in Diagon Alley, he's actually doing his homework with diligence. This is the most freedom he's had up until this point, so we can see more of the wizarding world.
This is also where the story starts to mature, and in many ways the turning point of the story. Yes, Goblet of Fire is the middle of the story and thusly where the story's course changes to that fateful and epic conclusion we see in DH, but it all started with Wormtail's escape at Hogwarts.
We have this magnificent backstory about Harry's parents, and we get this glimpse into what his father's family was like. We are introduced to Sirius at last (he is mentioned in the first chapter of PoS, and the name stuck in my head--I wanted to know who he was!)