Originally Posted by Connie

It sounds like a fine theory until you remember that Petunia became resentful of Lily when they were just children. She called Lily hurtful names because of her powers and was horribly jealous of her. Being rejected by Dumbledore made her resent Lily and didn't even want to go with the family when they took Lily to King's Cross.

Also if the Horcrux caused the Dursleys to be bitter and hateful to Harry, then everyone else close to him would be just as mean and hateful.

My own theory that I haven't seen anyone else say anything about is that the accidental Horcrux Lord Voldemort made Harry would have caused suffering instead of hate. Harry's scar was the connection between Harry and Voldemort. I think it was where the Horcrux entered Harry instead of the piece of curse that rebounded back on Voldemort. That seems like a more likely scenario for his headaches and glimpsing into what Voldemort was seeing or feeling; the Horcrux caused pain when they saw into the mind of the other because Voldemort was evil and Harry was pure. The Dursleys weren't affected by the Horcrux; they were just mean prejudiced people against anyone not like them.