pvary opened a new pull request #2254:
URL: https://github.com/apache/iceberg/pull/2254
When Hive query `SELECT * from date_test WHERE d_date='1998-02-19'` contains
a date literal as a predicated Iceberg filter fails with the following
exception:
```
Caused by: org.apache.hive.service.cli.HiveSQLException:
java.io.IOException: java.lang.IllegalStateException: Not an instance of
java.lang.Integer: 2130-01-20
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:499)
at
org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:307)
at
org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:878)
at
org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:559)
at
org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:551)
at
org.apache.iceberg.mr.hive.TestHiveShell.executeStatement(TestHiveShell.java:143)
... 62 more
Caused by: java.io.IOException: java.lang.IllegalStateException: Not an
instance of java.lang.Integer: 2130-01-20
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:521)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:428)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:147)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:2208)
at
org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:494)
... 67 more
Caused by: java.lang.IllegalStateException: Not an instance of
java.lang.Integer: 2130-01-20
at org.apache.iceberg.data.GenericRecord.get(GenericRecord.java:123)
at
org.apache.iceberg.Accessors$PositionAccessor.get(Accessors.java:71)
at
org.apache.iceberg.Accessors$PositionAccessor.get(Accessors.java:58)
at
org.apache.iceberg.expressions.BoundReference.eval(BoundReference.java:39)
at
org.apache.iceberg.expressions.Evaluator$EvalVisitor.eq(Evaluator.java:132)
at
org.apache.iceberg.expressions.Evaluator$EvalVisitor.eq(Evaluator.java:52)
at
org.apache.iceberg.expressions.ExpressionVisitors$BoundVisitor.predicate(ExpressionVisitors.java:249)
at
org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:346)
at
org.apache.iceberg.expressions.Evaluator$EvalVisitor.eval(Evaluator.java:57)
at
org.apache.iceberg.expressions.Evaluator$EvalVisitor.access$100(Evaluator.java:52)
at org.apache.iceberg.expressions.Evaluator.eval(Evaluator.java:49)
at
org.apache.iceberg.mr.mapreduce.IcebergInputFormat$IcebergRecordReader.lambda$applyResidualFiltering$0(IcebergInputFormat.java:288)
at
org.apache.iceberg.io.CloseableIterable$3.shouldKeep(CloseableIterable.java:82)
at
org.apache.iceberg.io.FilterIterator.advance(FilterIterator.java:67)
at
org.apache.iceberg.io.FilterIterator.hasNext(FilterIterator.java:50)
at
org.apache.iceberg.mr.mapreduce.IcebergInputFormat$IcebergRecordReader.nextKeyValue(IcebergInputFormat.java:202)
at
org.apache.iceberg.mr.mapred.MapredIcebergInputFormat$MapredIcebergRecordReader.next(MapredIcebergInputFormat.java:104)
at
org.apache.iceberg.mr.mapred.MapredIcebergInputFormat$MapredIcebergRecordReader.next(MapredIcebergInputFormat.java:81)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:488)
... 71 more
```
There are 2 issue:
1. Literal conversion is problematic as Hive uses Date with local timezone,
so we have to fix the conversion
2. When comparing the Record with the literal `GenericRecord.get()` expect
the value to be as an Integer, but it gets LocalDate instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
