Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15327
Somehow github didn't email me at all. I think we can follow something like
what Spark SQL does, i.e. two settings: one for the size of each partition, and
another for the cost of opening a file.
PS: Would it make more sense to just add binary file support to Spark SQL,
and then call it a day?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
