“As a byproduct of our study, we uncover a bug in the Java implementation that can cause the sorting method to fail during the execution.”
http://drops.dagstuhl.de/opus/volltexte/2018/9467/ This might impact Spark since we took the Java based TimSort implementation. I have seen in the wild TimSort failing in the past. Maybe this is the cause.
