rdblue commented on issue #26150: [SPARK-29501][SQL] SupportsPushDownRequiredColumns should report pruned schema immediately URL: https://github.com/apache/spark/pull/26150#issuecomment-543928231 @cloud-fan, I don't think that removing this method is possible without correctness issues. The read schema may be impacted by filters. For example, if I project column `a` and filter by column `b` then the read schema can be `(a, b)` or just `a`. So the read schema can't be fully determined when the method to push the projection is called. Why not just create a scan and use the `readSchema` from the scan, but still pass the builder into v1?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
