Github user juliuszsompolski commented on the issue: https://github.com/apache/spark/pull/19181 LGTM. It'd be nicer if there was an invariant that a nested spill just never can happen, i.e. there are no places that can spill on the code path that already spills. But this fix does seem to render nested spill harmless in this particular case. @hvanhovell In my original use case I hit this bug after hours of running on a big cluster, so reproducing it on that case is quite unpractical for me. I'll see if I'll observe it again.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org