Nandor Kollar created PARQUET-1305:
--------------------------------------
Summary: Backward incompatible change introduced in 1.8
Key: PARQUET-1305
URL: https://issues.apache.org/jira/browse/PARQUET-1305
Project: Parquet
Issue Type: Bug
Reporter: Nandor Kollar
Assignee: Nandor Kollar
ThriftSchemaConverter#convert(StructType struct) introduced a backward
incompatible logic in Parquet 1.8, If StructType's [deprecated
constructor|https://github.com/apache/parquet-mr/blob/master/parquet-thrift/src/main/java/org/apache/parquet/thrift/struct/ThriftType.java#L225]
is used to create StructTypes, and this convert method is used, then the
client receives the following exception:
{code}
org.apache.parquet.ShouldNeverHappenException: Encountered UNKNOWN
StructOrUnionType
at
org.apache.parquet.thrift.ThriftSchemaConvertVisitor.isUnion(ThriftSchemaConvertVisitor.java:342)
at
org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:218)
at
org.apache.parquet.thrift.ThriftSchemaConvertVisitor.visit(ThriftSchemaConvertVisitor.java:74)
at
org.apache.parquet.thrift.struct.ThriftType$StructType.accept(ThriftType.java:269)
at
org.apache.parquet.thrift.ThriftSchemaConvertVisitor.convert(ThriftSchemaConvertVisitor.java:93)
at
org.apache.parquet.thrift.ThriftSchemaConverter.convert(ThriftSchemaConverter.java:75)
{code}
To illustrate the issue, here's a test case, which passes on older versions,
but fails on master:
{code}
@Test
public void testIncompatibleThriftConverterChange() {
ThriftSchemaConverter converter = new ThriftSchemaConverter();
ThriftType.StructType structType = new ThriftType.StructType(
asList(
new ThriftField("a", (short)1, REQUIRED, new ThriftType.StringType()),
new ThriftField("b", (short) 2, REQUIRED, new ThriftType.StringType())
)
);
converter.convert(structType);
}
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
