Github user rxin commented on the pull request:
https://github.com/apache/spark/pull/1091#issuecomment-46405754
Thanks for working on this, @ScrapCodes. I talked with Matei and while we
both agree compression would be better set in per-RDD basis, adding another
boolean flag to StorageLevel is not ideal.
Matei suggested deferring this and we will come up with a proper design
later.
```
We should come up with a proper design for this. I think one viable design
is to make StorageLevel get constructed via a builder pattern. More generally
in the future Iâd like to have something called StorageStrategy that can also
convert the data into a format (e.g. columnar or something). Kind of like
including a serializer in the storage level.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
