Twitter freaked out last night because of this (not endorsing everything at that link, but it explains it well enough).
Anyway, I do think the whole thing is pretty stupid with a minor caveat. There is a legitimate concern with this kind of humor, that your audience isn't laughing at your satire, they're laughing at the racism you have used in your satire. Dave Chappelle had to deal with this.
I saw the bit when it aired, and who it's aimed at is obvious (Dan Snyder and the racist name Redskins). Still invoking racism to be satirical about racism is always going to be a problematic. I think Colbert understands this well, and it's part of his brilliance (which is why I think this particular issue is stupid), but less smart comedians don't always get it right. Because they don't get it.