felipepessoto opened a new issue, #12753: URL: https://github.com/apache/gluten/issues/12753
### Description Gluten's file-source scan transformers (`FileSourceScanExecTransformerBase` and its subclasses: `FileSourceScanExecTransformer`, `DeltaScanTransformer`, `HudiScanTransformer`) push **every** filter conjunct into the native scan via `PushDownFilterToScan` and evaluate them as exact row-level filters. As a result the paired `FilterExecTransformer` becomes a no-op (`FilterExecTransformerBase.isNoop`) and no separate post-scan `Filter` is needed. Spark marks exactly this situation with a leading `*` on a scan's `PushedFilters` metadata — the `RowDataSourceScanExec` convention for a filter the source fully evaluates itself. Gluten, however, inherits `FileSourceScanLike`'s rendering, which leaves `PushedFilters` **unmarked**: ``` PushedFilters: [IsNotNull(id), LessThan(id,5)] <- Gluten today PushedFilters: [*IsNotNull(id), *LessThan(id,5)] <- what the state actually is ``` So the plan output under-reports what the native scan actually does. This is purely a plan-rendering gap — the filters are already fully pushed down (present in the scan's `dataFilters` / Substrait `NativeFilters`); only the `*` marker is missing. This also surfaces when running upstream engines' plan-string assertions against Gluten (e.g. Delta's CDF filter-pushdown tests assert the `*`-marked form). ### Expected behavior Each fully-pushed filter entry in a `FileSourceScanExecTransformer` scan's `PushedFilters` should be rendered with a leading `*`, matching Spark's `RowDataSourceScanExec` convention. Vanilla `FileSourceScanExec` nodes (e.g. the AQE Initial Plan) must remain unmarked, since they still have a real `Filter` above them. ### Notes The change lives in the common `gluten-substrait` module (hence `[CORE]`). The observable golden updates are Velox-only (TPC-H / TPC-DS / gluten-tpch plan-stability). Addressed by apache/gluten#12626. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
