Why doesn't Obama's personal affiliations with known US haters bother more people?

He has ties to William Ayres (a terrorist that helped found the Weatherman) Rev Wright and his own wife. I just don't get it. Please don't say I'm picking on him because I beleave this is a valid point for somebody who wants to be commander in chief