cloud-fan commented on a change in pull request #33822:
URL: https://github.com/apache/spark/pull/33822#discussion_r696414946
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala
##########
@@ -409,7 +409,7 @@ object DataSourceStrategy
pushedFilters.toSet,
handledFilters,
None,
- scanBuilder(requestedColumns, candidatePredicates, pushedFilters),
+ scanBuilder(requestedColumns, candidatePredicates,
handledFilters.toSeq),
Review comment:
I think this change is correct but is pretty risky. DS v1 has been there
for many years and it's possible that some v1 sources forget to override
`unhandledFilters`.
Before this PR, it's not a big deal if v1 sources forget to override
`unhandledFilters`. Filter pushdown still works, only the EXPLAIN result will
be inaccurate.
I don't think we can make this change now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
