Republican students have a right to invite whomever they want to say whatever they want (short of a targeted call for violence) unmolested and without censorship. So if University of California, Berkeley, where Yiannopoulous' appearance triggered riots by armed leftist hoodlums, stick to their vow to have him back (along with right-wing conspiracy theorist Alex Jones) then so be it. Berkley is a public university and is required not just by the First Amendment but its own mission to be a broad purveyor of ideas to create a "safe space" for Yiannopoulos. And CPAC is a private gathering that can put whomever it wants on its roster — and also remove those same people.
But if Republicans have a right to invite Yiannaopoulos, others have the right to judge them for the company they keep. And Yiannopoulos is very, very bad company.