It isn't just the physical presence of the father that matters- it's his engagement and involvement. An emotionally remote or rejecting or actively punitive father leads to girls' feeling pretty apprehensive around men.