HyukjinKwon edited a comment on issue #26611: [SPARK-29939][CORE] Add spark.shuffle.mapStatus.compression.codec conf URL: https://github.com/apache/spark/pull/26611#issuecomment-558385771 Using difficult compression codec usually has its general purpose I think such as the efficiency given the pattern of the data. I dont believe zstd is always better in all cases. However, yeah I agree that it is important to describe the reason in general, and it better had to be clarified.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
