Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24596 )
Change subject: IMPALA-15191: Fix _row_id virtual column in USING clause ...................................................................... IMPALA-15191: Fix _row_id virtual column in USING clause _row_id and _last_updated_sequence_number are Iceberg V3 syntactic-sugar virtual columns that are expanded during analysis to COALESCE() expressions by the mandatory IcebergVirtualColumnRewriteRule, so they never reach the planner or backend. That rewrite runs only once, between the first analyze() and the re-analysis that follows expr rewriting. A JOIN ... USING (col) clause, however, is re-generated into an on-clause on every analysis pass (TableRef.analyzeJoin()), and reset() discards the previously built on-clause when a USING clause is present. As a result the rewritten COALESCE() predicate produced in the first pass was thrown away during re-analysis and replaced by a raw equi-predicate referencing the ICEBERG_ROW_ID virtual column. That column then reached the scan node, where it hit DCHECK(false) in HdfsScanPlanNode. This patch expands the syntactic-sugar virtual columns into their COALESCE() equivalents at the point where the USING clause builds the equi-join predicate, instead of relying on the one-shot expr rewrite. Testing: * New e2e regression tests Change-Id: Ia1aef8e4030dc7ea25f38ca78219cde217bc3d84 Assisted-by: Claude Opus 4.8 (1M context) Reviewed-on: http://gerrit.cloudera.org:8080/24596 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/analysis/TableRef.java A testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-row-lineage-join.test M testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-row-lineage.test M tests/query_test/test_iceberg.py 4 files changed, 128 insertions(+), 2 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/24596 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia1aef8e4030dc7ea25f38ca78219cde217bc3d84 Gerrit-Change-Number: 24596 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
