zhangxinjian123 edited a comment on issue #829: Structured Streaming read kafka
URL: https://github.com/apache/incubator-hudi/issues/829#issuecomment-521086833
 
 
   first json:
   ```
   
{"database":"zhang","table":"hao","type":"insert","ts":1555984612,"xid":90202,"commit":true,"data":{"id":7,"Name":"zhang"}}
   ```
   I get 7 and Zhang as values by parsing JSON to get ID and name as schemas, 
respectively. This is writable
   
   second json:
   ```
   
{"database":"zhang","table":"haow","type":"insert","ts":1555984612,"xid":90202,"commit":true,"data":{"id":6,"Name":"zhang","Age":13,"code":19}}
   ```
   
   ```
   57947 [pool-17-thread-2] ERROR 
com.uber.hoodie.common.util.queue.BoundedInMemoryExecutor  - error consuming 
records
   java.lang.ArrayIndexOutOfBoundsException: 10
        at org.apache.avro.generic.GenericData$Record.get(GenericData.java:212)
        at org.apache.avro.generic.GenericData.getField(GenericData.java:697)
        at 
org.apache.parquet.avro.AvroWriteSupport.writeRecordFields(AvroWriteSupport.java:188)
        at 
org.apache.parquet.avro.AvroWriteSupport.write(AvroWriteSupport.java:165)
        at 
org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:128)
        at org.apache.parquet.hadoop.ParquetWriter.write(ParquetWriter.java:299)
        at 
com.uber.hoodie.io.storage.HoodieParquetWriter.writeAvro(HoodieParquetWriter.java:103)
        at 
com.uber.hoodie.io.HoodieMergeHandle.write(HoodieMergeHandle.java:292)
        at 
com.uber.hoodie.table.HoodieCopyOnWriteTable$UpdateHandler.consumeOneRecord(HoodieCopyOnWriteTable.java:452)
        at 
com.uber.hoodie.table.HoodieCopyOnWriteTable$UpdateHandler.consumeOneRecord(HoodieCopyOnWriteTable.java:442)
        at 
com.uber.hoodie.common.util.queue.BoundedInMemoryQueueConsumer.consume(BoundedInMemoryQueueConsumer.java:38)
        at 
com.uber.hoodie.common.util.queue.BoundedInMemoryExecutor.lambda$null$2(BoundedInMemoryExecutor.java:126)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
   57979 [pool-17-thread-1] ERROR 
com.uber.hoodie.common.util.queue.BoundedInMemoryExecutor  - error producing 
records
   com.uber.hoodie.exception.HoodieException: operation has failed
        at 
com.uber.hoodie.common.util.queue.BoundedInMemoryQueue.throwExceptionIfFailed(BoundedInMemoryQueue.java:234)
        at 
com.uber.hoodie.common.util.queue.BoundedInMemoryQueue.insertRecord(BoundedInMemoryQueue.java:171)
        at 
com.uber.hoodie.common.util.queue.IteratorBasedQueueProducer.produce(IteratorBasedQueueProducer.java:45)
        at 
com.uber.hoodie.common.util.queue.BoundedInMemoryExecutor.lambda$null$0(BoundedInMemoryExecutor.java:96)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   ```
   
   At first I wrote correctly, but this error occurs when there are more new 
fields than the first one.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

Reply via email to