The 24-year-old posted to Instagram in April in honour of Braff's 45th birthday, showing him and their foster dog.
Within minutes, she had to "turn off the comments" on the post due to the incessant "hateful comments" she received.
People had very strong opinions about her relationship, mostly because of the 21-year age gap between the couple.
Appearing on the Sue Perkins: An hour or so with... podcast this week, Pugh spoke about the reaction.
"I've always found it funny, how I can be good enough for people to watch my work and support my work and pay for tickets, and I'm old enough to be an adult and pay taxes, but I'm not old enough to know who I should and should not have sex with?" Pugh told host Perkins.
"Yet again, once again, it's making a young woman feel like sh*t for no reason."
Pugh further defended her relationship in an interview with Elle UK in May.