Is it only X that breaks? Have you tried Wayland?
I think the problem may be upstream from X, near the driver or its use maybe. I've used X for 30 years, had some problems in a distant past, but nothing like that. X has been smooth sailing for me for the last 15 years. "Wayland" is perhaps not a cure-all. Besides, it's not compatible with Fvwm, which I really, really like.
Sébastien.
