qphien commented on pull request #2171:
URL: https://github.com/apache/iceberg/pull/2171#issuecomment-769742755
```
public void testScanTable() throws IOException {
shell.setHiveSessionValue("hive.vectorized.execution.enabled", "false");
testTables.createTable(shell, "customers",
HiveIcebergStorageHandlerTestUtils.CUSTOMER_SCHEMA, fileFormat,
HiveIcebergStorageHandlerTestUtils.CUSTOMER_RECORDS);
// Adding the ORDER BY clause will cause Hive to spawn a local MR job
this time.
List<Object[]> descRows =
shell.executeStatement("SELECT last_name, customer_id FROM
default.customers ORDER BY customer_id DESC");
Assert.assertEquals(3, descRows.size());
Assert.assertArrayEquals(new Object[] {"Pink", 2L}, descRows.get(0));
Assert.assertArrayEquals(new Object[] {"Green", 1L}, descRows.get(1));
Assert.assertArrayEquals(new Object[] {"Brown", 0L}, descRows.get(2));
}
```
With vectorization disabled, this modified `testScanTable` test still throws
the same exception due to missing `hive.io.file.readcolumn.names` in
`MapOperator.jonConf`:
```
Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
at org.apache.iceberg.data.GenericRecord.get(GenericRecord.java:114)
at
org.apache.iceberg.mr.hive.serde.objectinspector.IcebergRecordObjectInspector.getStructFieldData(IcebergRecordObjectInspector.java:75)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:95)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:80)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:68)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:88)
... 25 more
```
We can add `conf.set("tez.mrreader.config.update.properties",
"hive.io.file.readcolumn.names")` in `HiveIcebergSerde.initialize()` or some
other place with TEZ-4248 to solve this problem.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
