Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21143
This is a very good point! Unfortunately Spark SQL doesn't support change
the physical plan at a per-split fashion, and I'd say this feature is
non-trivial to implement, and needs a design doc.
After we have this feature, it might make sense to shift some of the mixin
traits from `DataSourceReader` to `DataReaderFactory`, so that they can be
applied per split.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
