[
https://issues.apache.org/jira/browse/FLINK-22271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17320879#comment-17320879
]
Kurt Young commented on FLINK-22271:
------------------------------------
This is by design for now, but I think it's also useful if we can make the
schema handling to be more flexible in the future. cc [~jark]
> FlinkSQL Read Hive(parquet file) field does not exist
> ------------------------------------------------------
>
> Key: FLINK-22271
> URL: https://issues.apache.org/jira/browse/FLINK-22271
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
> Affects Versions: 1.12.2
> Reporter: moran
> Priority: Major
>
> Create a parquet table format student insert the data for each field is not
> empty, FlinkSQL can query the table, if you add a field after query error.
> 1.Step:
> CREATE TABLE tmp.student (
> name STRING,
> age INT
> )
> STORED AS PARQUET;
> insert into table tmp.student values ("java", 12);
> FlinkSQL can read the table at this point.
> 2.Step:
> alter table tmp.student add columns(update_time timestamp);
> Query error after adding field update_time.
> error:
> java.lang.IllegalArgumentException: update_time does not exist
> at
> org.apache.flink.hive.shaded.formats.parquet.ParquetVectorizedInputFormat.clipParquetSchema(ParquetVectorizedInputFormat.java:193)
> at
> org.apache.flink.hive.shaded.formats.parquet.ParquetVectorizedInputFormat.createReader(ParquetVectorizedInputFormat.java:120)
> at
> org.apache.flink.hive.shaded.formats.parquet.ParquetVectorizedInputFormat.createReader(ParquetVectorizedInputFormat.java:73)
> at
> org.apache.flink.connectors.hive.read.HiveBulkFormatAdapter.createReader(HiveBulkFormatAdapter.java:108)
> at
> org.apache.flink.connectors.hive.read.HiveBulkFormatAdapter.createReader(HiveBulkFormatAdapter.java:63)
> at
> org.apache.flink.connector.file.src.impl.FileSourceSplitReader.checkSplitOrStartNext(FileSourceSplitReader.java:112)
> at
> org.apache.flink.connector.file.src.impl.FileSourceSplitReader.fetch(FileSourceSplitReader.java:65)
> at
> org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:56)
> at
> org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:138)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
