I don't think people always get influenced for the worse that way. Some of them can stay true to themselves. and if Harry hated them so much, he wouldn't want to be like them. maybe he should have been mean to them, but not other people.