Hi @Chris Guzek - thanks for starting this important conversation.
I have experienced this situation before (more than once) and I handled it by letting go of the CSM each time.
The reason is, your A players see this person and know just as you do that they shouldn't be there.
Yes, it will cause you immediate pain with the backfill needed and being short-staffed but it's the only way to improve things moving forward.
You have to choose between short-term or long-term pain.