Frank addresses the way he treated Sam and the way fans talk about it