Hello Peter Rozsa, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24596
to look at the new patch set (#2).
Change subject: IMPALA-15191: Fix _row_id virtual column in USING clause
......................................................................
IMPALA-15191: Fix _row_id virtual column in USING clause
_row_id and _last_updated_sequence_number are Iceberg V3 syntactic-sugar
virtual columns that are expanded during analysis to COALESCE()
expressions by the mandatory IcebergVirtualColumnRewriteRule, so they
never reach the planner or backend.
That rewrite runs only once, between the first analyze() and the
re-analysis that follows expr rewriting. A JOIN ... USING (col) clause,
however, is re-generated into an on-clause on every analysis pass
(TableRef.analyzeJoin()), and reset() discards the previously built
on-clause when a USING clause is present. As a result the rewritten
COALESCE() predicate produced in the first pass was thrown away during
re-analysis and replaced by a raw equi-predicate referencing the
ICEBERG_ROW_ID virtual column. That column then reached the scan node,
where it hit DCHECK(false) in HdfsScanPlanNode.
This patch expands the syntactic-sugar virtual columns into their
COALESCE() equivalents at the point where the USING clause builds the
equi-join predicate, instead of relying on the one-shot expr rewrite.
Testing:
* New e2e regression tests
Change-Id: Ia1aef8e4030dc7ea25f38ca78219cde217bc3d84
Assisted-by: Claude Opus 4.8 (1M context)
---
M fe/src/main/java/org/apache/impala/analysis/TableRef.java
A
testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-row-lineage-join.test
M
testdata/workloads/functional-query/queries/QueryTest/iceberg-v3-row-lineage.test
M tests/query_test/test_iceberg.py
4 files changed, 128 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/24596/2
--
To view, visit http://gerrit.cloudera.org:8080/24596
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia1aef8e4030dc7ea25f38ca78219cde217bc3d84
Gerrit-Change-Number: 24596
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Peter Rozsa <[email protected]>
