[
https://issues.apache.org/jira/browse/IMPALA-13515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe McDonnell updated IMPALA-13515:
-----------------------------------
Description:
When using the Calcite planner, selecting from an ORC table gets an
IllegalStateException at this location:
{noformat}
int totalCols = getRowType().getFieldNames().size();
...
HdfsTable table = ((CalciteTable) getTable()).getHdfsTable();
Preconditions.checkState(totalCols == table.getColumns().size());
<-------{noformat}
Adding some logging to print getRowType().getFieldNames() and
table.getColumns(), I get:
{noformat}
I1105 10:19:42.841647 3295737 ImpalaHdfsScanRel.java:114]
c849f4ccbd5f3498:125f063e00000000] getRowType().getFieldNames(): [a, b]
I1105 10:19:42.842378 3295737 ImpalaHdfsScanRel.java:115]
c849f4ccbd5f3498:125f063e00000000] table.getColumns(): [Column{name_=row__id,
type_=org.apache.impala.catalog.StructType@db1430c5, comment_=,
stats=ColumnStats{avgSize_=-1.0, avgSerializedSize_=-1.0, maxSize_=-1,
numDistinct_=-1, numNulls_=-1, numTrues=-1, numFalses=-1, lowValue=-1,
highValue=-1}, position_=0}, Column{name_=a, type_=STRING, comment_=null,
stats=ColumnStats{avgSize_=-1.0, avgSerializedSize_=-1.0, maxSize_=-1,
numDistinct_=-1, numNulls_=-1, numTrues=-1, numFalses=-1, lowValue=-1,
highValue=-1}, position_=1}, Column{name_=b, type_=STRING, comment_=null,
stats=ColumnStats{avgSize_=-1.0, avgSerializedSize_=-1.0, maxSize_=-1,
numDistinct_=-1, numNulls_=-1, numTrues=-1, numFalses=-1, lowValue=-1,
highValue=-1}, position_=2}]{noformat}
The table.getColumns() include an extra "row__id" column that Calcite doesn't
know about.
> ORC tables hit IllegalStateException due to "row__id" column
> ------------------------------------------------------------
>
> Key: IMPALA-13515
> URL: https://issues.apache.org/jira/browse/IMPALA-13515
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Affects Versions: Impala 4.5.0
> Reporter: Joe McDonnell
> Priority: Major
>
> When using the Calcite planner, selecting from an ORC table gets an
> IllegalStateException at this location:
> {noformat}
> int totalCols = getRowType().getFieldNames().size();
> ...
> HdfsTable table = ((CalciteTable) getTable()).getHdfsTable();
> Preconditions.checkState(totalCols == table.getColumns().size());
> <-------{noformat}
> Adding some logging to print getRowType().getFieldNames() and
> table.getColumns(), I get:
> {noformat}
> I1105 10:19:42.841647 3295737 ImpalaHdfsScanRel.java:114]
> c849f4ccbd5f3498:125f063e00000000] getRowType().getFieldNames(): [a, b]
> I1105 10:19:42.842378 3295737 ImpalaHdfsScanRel.java:115]
> c849f4ccbd5f3498:125f063e00000000] table.getColumns(): [Column{name_=row__id,
> type_=org.apache.impala.catalog.StructType@db1430c5, comment_=,
> stats=ColumnStats{avgSize_=-1.0, avgSerializedSize_=-1.0, maxSize_=-1,
> numDistinct_=-1, numNulls_=-1, numTrues=-1, numFalses=-1, lowValue=-1,
> highValue=-1}, position_=0}, Column{name_=a, type_=STRING, comment_=null,
> stats=ColumnStats{avgSize_=-1.0, avgSerializedSize_=-1.0, maxSize_=-1,
> numDistinct_=-1, numNulls_=-1, numTrues=-1, numFalses=-1, lowValue=-1,
> highValue=-1}, position_=1}, Column{name_=b, type_=STRING, comment_=null,
> stats=ColumnStats{avgSize_=-1.0, avgSerializedSize_=-1.0, maxSize_=-1,
> numDistinct_=-1, numNulls_=-1, numTrues=-1, numFalses=-1, lowValue=-1,
> highValue=-1}, position_=2}]{noformat}
> The table.getColumns() include an extra "row__id" column that Calcite doesn't
> know about.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
