[
https://issues.apache.org/jira/browse/IMPALA-15072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noémi Pap-Takács resolved IMPALA-15072.
---------------------------------------
Fix Version/s: Impala 5.0.0
Resolution: Fixed
> Exclude OPTIMIZE TABLE maintenance operation from lineage graph
> ---------------------------------------------------------------
>
> Key: IMPALA-15072
> URL: https://issues.apache.org/jira/browse/IMPALA-15072
> Project: IMPALA
> Issue Type: Bug
> Reporter: Noémi Pap-Takács
> Assignee: Noémi Pap-Takács
> Priority: Major
> Fix For: Impala 5.0.0
>
>
> Running OPTIMIZE TABLE on Iceberg V3 tables migrated from V2 throws an
> IllegalStateException in ColumnLineageGraph:
> {noformat}
> java.lang.RuntimeException: java.lang.IllegalStateException at
> org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2473)
> at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2140)
> at
> org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:180)
> Caused by: java.lang.IllegalStateException at
> com.google.common.base.Preconditions.checkState(Preconditions.java:497) at
> org.apache.impala.analysis.ColumnLineageGraph.computeProjectionDependencies(ColumnLineageGraph.java:637)
> at
> org.apache.impala.analysis.ColumnLineageGraph.computeLineageGraph(ColumnLineageGraph.java:607)
> at org.apache.impala.planner.Planner.createPlanFragments(Planner.java:281)
> at org.apache.impala.planner.Planner.createPlans(Planner.java:314) at
> org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2078){noformat}
> The Precondition check
> {code:java}
> Preconditions.checkState(resultExprs.size() ==
> targetColumnLabels_.size());{code}
> fails because V3 tables have 2 hidden columns: {{_file_row_id}} and
> {{_file_last_updated_sequence_number.}}
> 'targetColumnLabels_' contains them but 'resultExprs' filters out hidden
> columns.
>
> Proposed solution:
> We could exclude OPTIMIZE TABLE from the lineage graph, since it is a
> maintenance operation that unnecessarily pollutes the graph.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
