[
https://issues.apache.org/jira/browse/PARQUET-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483835#comment-16483835
]
ASF GitHub Bot commented on PARQUET-1305:
-----------------------------------------
nandorKollar opened a new pull request #483: PARQUET-1305: Backward
incompatible change introduced in 1.8
URL: https://github.com/apache/parquet-mr/pull/483
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Backward incompatible change introduced in 1.8
> ----------------------------------------------
>
> Key: PARQUET-1305
> URL: https://issues.apache.org/jira/browse/PARQUET-1305
> Project: Parquet
> Issue Type: Bug
> Reporter: Nandor Kollar
> Assignee: Nandor Kollar
> Priority: Major
>
> ThriftSchemaConverter#convert(StructType struct) introduced a backward
> incompatible logic in Parquet 1.8, If StructType's [deprecated
> constructor|https://github.com/apache/parquet-mr/blob/master/parquet-thrift/src/main/java/org/apache/parquet/thrift/struct/ThriftType.java#L225]
> is used to create StructTypes, and this convert method is used, then the
> client receives the following exception:
> {code}
> org.apache.parquet.ShouldNeverHappenException: Encountered UNKNOWN
> StructOrUnionType
> at
> org.apache.parquet.thrift.ThriftSchemaConvertVisitor.isUnion(ThriftSchemaConvertVisitor.java:342)
> at
> org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:218)
> at
> org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:74)
> at
> org.apache.parquet.thrift.struct.ThriftType$StructType.accept(ThriftType.java:269)
> at
> org.apache.parquet.thrift.ThriftSchemaConvertVisitor.convert(ThriftSchemaConvertVisitor.java:93)
> at
> org.apache.parquet.thrift.ThriftSchemaConverter.convert(ThriftSchemaConverter.java:75)
> {code}
> To illustrate the issue, here's a test case, which passes on older versions,
> but fails on master:
> {code}
> @Test
> public void testIncompatibleThriftConverterChange() {
> ThriftSchemaConverter converter = new ThriftSchemaConverter();
> ThriftType.StructType structType = new ThriftType.StructType(
> asList(
> new ThriftField("a", (short)1, REQUIRED, new ThriftType.StringType()),
> new ThriftField("b", (short) 2, REQUIRED, new ThriftType.StringType())
> )
> );
> converter.convert(structType);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
