zml1206 commented on issue #7261: URL: https://github.com/apache/incubator-gluten/issues/7261#issuecomment-2357619290
Can we determine whether expression supports offload? @zhztheplayer There are three layers of filters here. The first layer is pushedDownFilters of FileSourceScanExec, which is also used by vanilla spark scan. The second layer is the dataFilters of FileSourceScanExec, which is used to generate the pushedDownFilters of FileSourceScanExec, as well as the filters used after the current offload. The third layer is the condition of the filter. In theory, the filter condition contains dataFilters, and dataFilters contains pushedDownFilters. The current implementation will push down dataFilters or condition filters. Simply rewrite dataFilters to pushedDownFilters may cause dataFilters to be pushed down before, but only pushedDownFilters can be pushed down. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
