[jira] [Assigned] (PARQUET-1656) Schema change results in exception - java.lang.ClassCastException

2019-12-20 Thread Xinli Shang (Jira) 


 [ 
https://issues.apache.org/jira/browse/PARQUET-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xinli Shang reassigned PARQUET-1656:


 External issue ID: PARQUET-1681
External issue URL: https://issues.apache.org/jira/browse/PARQUET-1681
  Assignee: Xinli Shang

> Schema change  results in exception - java.lang.ClassCastException
> --
>
> Key: PARQUET-1656
> URL: https://issues.apache.org/jira/browse/PARQUET-1656
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-avro
>Affects Versions: 1.8.1, 1.12.0
> Environment: Hoodie/Parquet/Avro
> Parquet-1.8.1
> Avro-1.7.6
>Reporter: Balajee Nagasubramaniam
>Assignee: Xinli Shang
>Priority: Major
>  Labels: Parquet, avro
>
> Following exception was seen with parquet 1.8.1 (and in parquet 1.12.0, when 
> trying to reproduce it).
> Exception in thread "main" java.lang.ClassCastException: optional binary 
> phone_number (STRING) is not a group
> at 
> com.uber.komondor.shaded.org.apache.parquet.schema.Type.asGroupType(Type.java:250)
> at 
> com.uber.komondor.shaded.org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:279)
> at 
> com.uber.komondor.shaded.org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:232)
> at 
> com.uber.komondor.shaded.org.apache.parquet.avro.AvroRecordConverter.access$100(AvroRecordConverter.java:78)
> at 
> org.apache.parquet.avro.AvroRecordConverter$AvroCollectionConverter$ElementConverter.(AvroRecordConverter.java:536)
> at 
> org.apache.parquet.avro.AvroRecordConverter$AvroCollectionConverter.(AvroRecordConverter.java:486)
> at 
> org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:289)
> at 
> org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:141)
> at 
> org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:279)
> at 
> org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:141)
> at 
> org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:95)
> at 
> org.apache.parquet.avro.AvroRecordMaterializer.(AvroRecordMaterializer.java:33)
> at 
> org.apache.parquet.avro.AvroReadSupport.prepareForRead(AvroReadSupport.java:138)
> at 
> org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:183)
> at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:156)
> at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:135)
> at 
> util.ParquetToAvroSchemaConverter$.convert(ParquetToAvroSchemaConverter.scala:46)
> at 
> util.ParquetToAvroSchemaConverter$.main(ParquetToAvroSchemaConverter.scala:20)
> at util.ParquetToAvroSchemaConverter.main(ParquetToAvroSchemaConverter.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Original exception was triggered by the following schema change.
> Schema Before change:
>  {
> "default": null,
> "name": "master_cluster",
> "type": [
> "null",
> {
> "fields": [
> {
> "name": "uuid",
> "type": "string"
> },
> {
> "name": "namespace",
> "type": "string"
> },
> {
> "name": "version",
> "type": "long"
> }
> ],
> "name": "master_cluster",
> "type": "record"
> }
> ]
> },
> After schema change:
> {
> "default": null,
> "name": "master_cluster",
> "type": [
> "null",
> {
>

[jira] [Commented] (PARQUET-1656) Schema change results in exception - java.lang.ClassCastException

2019-12-20 Thread Xinli Shang (Jira) 


[ 
https://issues.apache.org/jira/browse/PARQUET-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17001328#comment-17001328
 ] 

Xinli Shang commented on PARQUET-1656:
--

https://issues.apache.org/jira/browse/PARQUET-1681

 

> Schema change  results in exception - java.lang.ClassCastException
> --
>
> Key: PARQUET-1656
> URL: https://issues.apache.org/jira/browse/PARQUET-1656
> Project: Parquet
>  Issue Type: Bug
>  Components: parquet-avro
>Affects Versions: 1.8.1, 1.12.0
> Environment: Hoodie/Parquet/Avro
> Parquet-1.8.1
> Avro-1.7.6
>Reporter: Balajee Nagasubramaniam
>Priority: Major
>  Labels: Parquet, avro
>
> Following exception was seen with parquet 1.8.1 (and in parquet 1.12.0, when 
> trying to reproduce it).
> Exception in thread "main" java.lang.ClassCastException: optional binary 
> phone_number (STRING) is not a group
> at 
> com.uber.komondor.shaded.org.apache.parquet.schema.Type.asGroupType(Type.java:250)
> at 
> com.uber.komondor.shaded.org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:279)
> at 
> com.uber.komondor.shaded.org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:232)
> at 
> com.uber.komondor.shaded.org.apache.parquet.avro.AvroRecordConverter.access$100(AvroRecordConverter.java:78)
> at 
> org.apache.parquet.avro.AvroRecordConverter$AvroCollectionConverter$ElementConverter.(AvroRecordConverter.java:536)
> at 
> org.apache.parquet.avro.AvroRecordConverter$AvroCollectionConverter.(AvroRecordConverter.java:486)
> at 
> org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:289)
> at 
> org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:141)
> at 
> org.apache.parquet.avro.AvroRecordConverter.newConverter(AvroRecordConverter.java:279)
> at 
> org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:141)
> at 
> org.apache.parquet.avro.AvroRecordConverter.(AvroRecordConverter.java:95)
> at 
> org.apache.parquet.avro.AvroRecordMaterializer.(AvroRecordMaterializer.java:33)
> at 
> org.apache.parquet.avro.AvroReadSupport.prepareForRead(AvroReadSupport.java:138)
> at 
> org.apache.parquet.hadoop.InternalParquetRecordReader.initialize(InternalParquetRecordReader.java:183)
> at org.apache.parquet.hadoop.ParquetReader.initReader(ParquetReader.java:156)
> at org.apache.parquet.hadoop.ParquetReader.read(ParquetReader.java:135)
> at 
> util.ParquetToAvroSchemaConverter$.convert(ParquetToAvroSchemaConverter.scala:46)
> at 
> util.ParquetToAvroSchemaConverter$.main(ParquetToAvroSchemaConverter.scala:20)
> at util.ParquetToAvroSchemaConverter.main(ParquetToAvroSchemaConverter.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Original exception was triggered by the following schema change.
> Schema Before change:
>  {
> "default": null,
> "name": "master_cluster",
> "type": [
> "null",
> {
> "fields": [
> {
> "name": "uuid",
> "type": "string"
> },
> {
> "name": "namespace",
> "type": "string"
> },
> {
> "name": "version",
> "type": "long"
> }
> ],
> "name": "master_cluster",
> "type": "record"
> }
> ]
> },
> After schema change:
> {
> "default": null,
> "name": "master_cluster",
> "type": [
> "null",
> {
> "fields": [
> {
>