[ https://issues.apache.org/jira/browse/FLINK-31811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17712937#comment-17712937 ]
Krzysztof Chmielewski commented on FLINK-31811: ----------------------------------------------- Hi [~jirawech.s] ??Could you share me reproducible code??? The code is attached to the issue I've created https://issues.apache.org/jira/browse/FLINK-31197 > Unsupported complex data type for Flink SQL > ------------------------------------------- > > Key: FLINK-31811 > URL: https://issues.apache.org/jira/browse/FLINK-31811 > Project: Flink > Issue Type: Bug > Components: Connectors / FileSystem > Affects Versions: 1.16.1 > Reporter: jirawech.s > Priority: Major > > I found this issue when I tried to write data on local filesystem using Flink > SQL > {code:java} > 19:51:32,966 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph > [] - compact-operator (1/4) > (4f2a09b638c786f74262c675d248afd9_80fe6c4f32f605d447b391cdb16cc1ff_0_4) > switched from RUNNING to FAILED on 69ed2306-371b-4bfc-a98e-bf75fb41748f @ > localhost (dataPort=-1). > java.lang.IndexOutOfBoundsException: Index: 1, Size: 1 > at java.util.ArrayList.rangeCheck(ArrayList.java:659) ~[?:1.8.0_301] > at java.util.ArrayList.get(ArrayList.java:435) ~[?:1.8.0_301] > at org.apache.parquet.schema.GroupType.getType(GroupType.java:216) > ~[parquet-column-1.12.2.jar:1.12.2] > at > org.apache.flink.formats.parquet.vector.ParquetSplitReaderUtil.createWritableColumnVector(ParquetSplitReaderUtil.java:523) > ~[flink-parquet-1.16.1.jar:1.16.1] > at > org.apache.flink.formats.parquet.vector.ParquetSplitReaderUtil.createWritableColumnVector(ParquetSplitReaderUtil.java:503) > ~[flink-parquet-1.16.1.jar:1.16.1] > at > org.apache.flink.formats.parquet.ParquetVectorizedInputFormat.createWritableVectors(ParquetVectorizedInputFormat.java:281) > ~[flink-parquet-1.16.1.jar:1.16.1] > at > org.apache.flink.formats.parquet.ParquetVectorizedInputFormat.createReaderBatch(ParquetVectorizedInputFormat.java:270) > ~[flink-parquet-1.16.1.jar:1.16.1] > at > org.apache.flink.formats.parquet.ParquetVectorizedInputFormat.createPoolOfBatches(ParquetVectorizedInputFormat.java:260) > ~[flink-parquet-1.16.1.jar:1.16.1] > {code} > What i tried to do is writing complex data type to parquet file > Here is the schema of sink table. The problematic data type is > ARRAY<MAP<STRING,STRING>> > {code:java} > CREATE TEMPORARY TABLE local_table ( > `user_id` STRING, `order_id` STRING, `amount` INT, `restaurant_id` STRING, > `experiment` ARRAY<MAP<STRING,STRING>>, `dt` STRING > ) PARTITIONED BY (`dt`) WITH ( > 'connector'='filesystem', > 'path'='file:///tmp/test_hadoop_write', > 'format'='parquet', > 'auto-compaction'='true', > 'sink.partition-commit.policy.kind'='success-file' > ) {code} > PS. It is used to work in Flink version 1.15.1 > -- This message was sent by Atlassian Jira (v8.20.10#820010)