malinjawi commented on PR #12218: URL: https://github.com/apache/gluten/pull/12218#issuecomment-5069790691
Thanks @felipepessoto, I double-checked the combined Delta CI. The 13 reported regressions all had the same root cause: `DeltaCDFScanStrategy` expands the CDF relation during physical planning, after Spark's normal optimizer pass, so the reconstructed filter stayed above the CDF union and never reached the Delta file scans. Row correctness was unaffected, but this was a real predicate-pushdown regression rather than a baseline issue. I fixed it in d740b0da2 by optimizing the resolved replacement plan before handing it back to physical planning. I also added a regression assertion that verifies the `id` predicate is present in `DeltaScanTransformer.dataFilters`. The change passes Spotless and clean test-compiles for Spark 3.3, 3.4, 3.5, 4.0, and 4.1. Could you please rerun the combined Delta CI against the updated head? It should specifically clear the filter-pushdown failures from run 29772714951. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
