Here's my theory: Petunia's parents were magical, and since she didn't inherit the same magical qualities, she is now resentful of anything magical. You know how Filch is always after the students because he is jealous that they have magic? I think it's the same thing. Petunia probably had a very hard time dealing with being so different growing up that now she will do anything she can to stay away from it. But that's just what i think.