[ https://issues.apache.org/jira/browse/HIVE-19016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16517769#comment-16517769 ]
Vihang Karajgaonkar commented on HIVE-19016: -------------------------------------------- Thanks [~mmccline] for the patch. I was wondering if it makes sense to make the patch more generic so that any file format can expose the types which are not supported for vectorization. For instance define a method in the {{VectorizedInputFormatInterface}} to return {{true}} or {{false}} when given a list of TypeInfos depending on whether the types are supported or not. +1 (pending tests) > Vectorization and Parquet: Disable vectorization for nested complex types > ------------------------------------------------------------------------- > > Key: HIVE-19016 > URL: https://issues.apache.org/jira/browse/HIVE-19016 > Project: Hive > Issue Type: Bug > Components: Hive > Affects Versions: 3.0.0 > Reporter: Matt McCline > Assignee: Matt McCline > Priority: Critical > Attachments: HIVE-19016.01.patch > > > Original title: Vectorization and Parquet: When vectorized, > parquet_nested_complex.q produces RuntimeException: Unsupported type used > > Adding "SET hive.vectorized.execution.enabled=true;" to > parquet_nested_complex.q triggers this call stack: > {noformat} > Caused by: java.lang.RuntimeException: Unsupported type used in > list:array<array<array<array<array<array<array<array<array<array<array<array<array<array<array<array<array<array<array<array<array<array<int>>>>>>>>>>>>>>>>>>>>>> > at > org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.checkListColumnSupport(VectorizedParquetRecordReader.java:589) > ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.buildVectorizedParquetReader(VectorizedParquetRecordReader.java:525) > ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.checkEndOfRowGroup(VectorizedParquetRecordReader.java:440) > ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.nextBatch(VectorizedParquetRecordReader.java:401) > ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.next(VectorizedParquetRecordReader.java:353) > ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedParquetRecordReader.next(VectorizedParquetRecordReader.java:92) > ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] > at > org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360) > ~[hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT] > {noformat} > FYI: [~vihangk1] -- This message was sent by Atlassian JIRA (v7.6.3#76005)