Github user rdblue commented on the issue:
https://github.com/apache/spark/pull/21230
@cloud-fan, I opened #21262 that is similar to this, but does pushdown when
converting to a physical plan. You might like that as an alternative because it
cleans up `DataSourceV2Relation` quite a bit and adds `output` to the case
class arguments like other relations.
The drawback to that approach that I had forgotten about is that it breaks
`computeStats` because that runs on the optimized plan (but this affects all
the other code paths as well).
Up to you how to continue with this work, I just think we should consider
the other approach since it solves a few problems. And `computeStats` is
something we should update to work on physical plans anyway, right? Just let me
know how you want to move forward. If you want to pull that commit into this
PR, I'll close the other one.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
