[
https://issues.apache.org/jira/browse/IMPALA-14952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18080182#comment-18080182
]
ASF subversion and git services commented on IMPALA-14952:
----------------------------------------------------------
Commit a5f22d0b976098b8e50b695d2a030aff06209f32 in impala's branch
refs/heads/master from Peter Rozsa
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=a5f22d0b9 ]
IMPALA-14952: Fix non-clustering column listing for Iceberg V3 tables
This change fixes 'collectExprs' in HdfsTableSink that refers to
non-clustering columns as a sublist in the registered target
expressions. Hidden columns added for Iceberg V3 tables are listed as
non-clustering columns and it causes index out-of-bound errors during
the subList call.
Tests
- new test added to test_lineage custom cluster test.
Change-Id: I341b46737431f090f99c8324a37ba682c45c0df0
Reviewed-on: http://gerrit.cloudera.org:8080/24277
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> IndexOutOfBoundsException in Iceberg Insert if lineage tracking is enabled
> --------------------------------------------------------------------------
>
> Key: IMPALA-14952
> URL: https://issues.apache.org/jira/browse/IMPALA-14952
> Project: IMPALA
> Issue Type: Bug
> Reporter: Peter Rozsa
> Assignee: Peter Rozsa
> Priority: Major
>
> IndexOutOfBoundsException is thrown on INSERT to Iceberg v3 tables if lineage
> tracking is enabled for Impala, either by --lineage_event_log_dir, or if a
> lineage hook is installed.
> The issue stems from the newly added hidden columns and the column label
> index calculation.
> Error:
> java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: toIndex = 3
> at
> org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2459)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2126)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:175)
> Caused by: java.lang.IndexOutOfBoundsException: toIndex = 3
> at
> java.base/java.util.AbstractList.subListRangeCheck(AbstractList.java:507)
> at java.base/java.util.ArrayList.subList(ArrayList.java:1108)
> at
> org.apache.impala.planner.HdfsTableSink.collectExprs(HdfsTableSink.java:392)
> at
> org.apache.impala.planner.Planner.createPlanFragments(Planner.java:281)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:315)
> at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2064)
> at
> org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:3424)
> at
> org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:3198)
> at
> org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2590)
> at
> org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2436)
> ... 2 more
>
> Repro:
> {code:java}
> create table a(id int) stored as iceberg tblproperties('format-version'='3');
> insert into a values(1); {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
