Github user squito commented on the issue: https://github.com/apache/spark/pull/23058 can we also make the same change to `TaskResultGetter`? We could avoid the same inefficiency for large task results that also get fetched to disk. (That would actually get us closer to removing the 2gb block limit on task results, but to fully remove it, we'd need to fix the task serialization in the executor, which is a little trickier.)
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org