Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/2247#issuecomment-54362759
It's not an interface in the sense that program output doesn't change. It's
only a config setting for an optimization.
Anyway, I agree with Reynold -- I'd like to see an example where
spark.shuffle.spill.compress and spark.shuffle.compress need to be at different
values, and see performance numbers for that. It seems to me that you're
spilling the same kind of objects in both, so there will be the same tradeoff
between I/O and compute time.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
