clairemcginty commented on code in PR #3753:
URL: https://github.com/apache/parquet-java/pull/3753#discussion_r3926710988
##########
parquet-avro/src/main/java/org/apache/parquet/avro/AvroReadSupport.java:
##########
@@ -230,4 +250,39 @@ private GenericData getDataModel(ParquetConfiguration
conf, Schema schema) {
return ReflectionUtils.newInstance(suppClass,
ConfigurationUtil.createHadoopConfiguration(conf))
.get();
}
+
+ private static boolean writesNewListStructure(MessageType schema) {
+ return Boolean.TRUE.equals(allListStructuresAreThreeLevel(schema));
+ }
+
+ // Given a Parquet schema, return true only if the schema:
+ // - contains one or more List fields
+ // - encodes every List field using 3-level list structure
+ private static Boolean allListStructuresAreThreeLevel(Type type) {
+ if (type.isPrimitive()) {
+ return null;
+ }
+ GroupType group = type.asGroupType();
+ if (group.getLogicalTypeAnnotation() instanceof
LogicalTypeAnnotation.ListLogicalTypeAnnotation) {
+ if (group.getFieldCount() != 1) {
+ return false;
+ }
+ Type repeated = group.getType(0);
+ return !repeated.isPrimitive()
Review Comment:
added a few more test cases
[here](https://github.com/apache/parquet-java/pull/3753/changes/bf9d6b4a8322d4d4805623f62afb1680876dc30a)
and the auto-detect correctly returns false for those cases! lmk if you had
any other cases in mind...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
