It has been said many times that J K Rowling pulled Alan Rickman aside in the first movie and told him something, something no one but JKR knew.
I think she told him he had grown up with and loved Lily Evans Potter. I think that is all that would have been needed. Every time he looked at Harry he saw James, whom he hated and Lily whom he loved.
Anybody thinking the same thing?