Naresh P R created HIVE-22178:
---------------------------------
Summary: Parquet FilterPredicate throws CastException after
SchemaEvolution.
Key: HIVE-22178
URL: https://issues.apache.org/jira/browse/HIVE-22178
Project: Hive
Issue Type: Bug
Affects Versions: 3.1.0
Reporter: Naresh P R
Assignee: Naresh P R
Below are the repro steps.
{code:java}
create table parq_test(age int, name string) stored as parquet;
insert into parq_test values(1, 'aaaa');
alter table parq_test change age age string;
insert into parq_test values('b', 'bbbb');
select * from parq_test where age='b';{code}
Exception thrown after changing column datatype is below
{code:java}
Caused by: java.lang.IllegalArgumentException: FilterPredicate column: age's
declared type (org.apache.parquet.io.api.Binary) does not match the schema
found in file metadata. Column age is of type: INT32
Valid types for this column are: [class java.lang.Integer]
at
org.apache.parquet.filter2.predicate.ValidTypeMap.assertTypeValid(ValidTypeMap.java:126)
at
org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumn(SchemaCompatibilityValidator.java:181)
at
org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validateColumnFilterPredicate(SchemaCompatibilityValidator.java:151)
at
org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:85)
at
org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.visit(SchemaCompatibilityValidator.java:58)
at org.apache.parquet.filter2.predicate.Operators$Eq.accept(Operators.java:181)
at
org.apache.parquet.filter2.predicate.SchemaCompatibilityValidator.validate(SchemaCompatibilityValidator.java:63)
at
org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:92)
at
org.apache.parquet.filter2.compat.RowGroupFilter.visit(RowGroupFilter.java:43)
at
org.apache.parquet.filter2.compat.FilterCompat$FilterPredicateCompat.accept(FilterCompat.java:137)
at
org.apache.parquet.filter2.compat.RowGroupFilter.filterRowGroups(RowGroupFilter.java:64)
at
org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:111)
at
org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.<init>(VectorizedParquetRecordReader.java:147)
... 31 more{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
