[
https://issues.apache.org/jira/browse/HUDI-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-1453:
---------------------------------
Labels: pull-request-available (was: )
> Throw Exception when input data schema is not equal to the hoodie table schema
> ------------------------------------------------------------------------------
>
> Key: HUDI-1453
> URL: https://issues.apache.org/jira/browse/HUDI-1453
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Writer Core
> Reporter: pengzhiwei
> Assignee: pengzhiwei
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.7.0
>
>
> The hoodie table *h0's* schema is :
> {code:java}
> (id long, price double){code}
> when I write *dataframe* to *h0* with the follow schema:
> {code:java}
> (id long, price int){code}
> An Exception is threw as follow:
> {code:java}
> at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:136)at
> org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:136) at
> org.apache.hudi.common.util.ParquetReaderIterator.hasNext(ParquetReaderIterator.java:49)
> at
> org.apache.hudi.common.util.queue.IteratorBasedQueueProducer.produce(IteratorBasedQueueProducer.java:45)
> at
> org.apache.hudi.common.util.queue.BoundedInMemoryExecutor.lambda$null$0(BoundedInMemoryExecutor.java:102)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ... 4
> moreCaused by: java.lang.UnsupportedOperationException:
> org.apache.parquet.avro.AvroConverters$FieldIntegerConverter at
> org.apache.parquet.io.api.PrimitiveConverter.addDouble(PrimitiveConverter.java:84)
> at
> org.apache.parquet.column.impl.ColumnReaderImpl$2$2.writeValue(ColumnReaderImpl.java:228)
> at
> org.apache.parquet.column.impl.ColumnReaderImpl.writeCurrentValueToConverter(ColumnReaderImpl.java:367)
> at
> org.apache.parquet.io.RecordReaderImplementation.read(RecordReaderImplementation.java:406)
> at
> org.apache.parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:226)
> ... 11 more
> {code}
> I have enable the *AVRO_SCHEMA_VALIDATE,* it ** can pass the schema
> validate in *HoodieTable#validateUpsertSchema,* so it is right to write the
> "int" data to the "double" field in hoodie.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
