clairemcginty commented on code in PR #1078:
URL: https://github.com/apache/parquet-mr/pull/1078#discussion_r1175253810
##########
parquet-avro/src/main/java/org/apache/parquet/avro/AvroWriteSupport.java:
##########
@@ -400,9 +400,18 @@ private Binary fromAvroString(Object value) {
return Binary.fromCharSequence(value.toString());
}
- private static GenericData getDataModel(Configuration conf) {
+ private static GenericData getDataModel(Configuration conf, Schema schema) {
+ if (conf.get(AVRO_DATA_SUPPLIER) == null) {
+ final GenericData modelForSchema =
AvroRecordConverter.getModelForSchema(schema);
Review Comment:
From where it's called in the code, `schema` should never be null -- but I
guess it would be safest to add a null-check here to future-proof it a bit.
Added!
##########
parquet-avro/src/main/java/org/apache/parquet/avro/AvroReadSupport.java:
##########
@@ -149,12 +149,22 @@ private static <T> RecordMaterializer<T>
newCompatMaterializer(
parquetSchema, avroSchema, model);
}
- private GenericData getDataModel(Configuration conf) {
+ private GenericData getDataModel(Configuration conf, Schema schema) {
if (model != null) {
return model;
}
+
+ if (conf.get(AVRO_DATA_SUPPLIER) == null) {
+ final GenericData modelForSchema =
AvroRecordConverter.getModelForSchema(schema);
+
+ if (modelForSchema != null) {
+ return modelForSchema;
+ }
+ }
+
Class<? extends AvroDataSupplier> suppClass = conf.getClass(
- AVRO_DATA_SUPPLIER, SpecificDataSupplier.class,
AvroDataSupplier.class);
- return ReflectionUtils.newInstance(suppClass, conf).get();
+ AVRO_DATA_SUPPLIER, SpecificDataSupplier.class, AvroDataSupplier.class);
+
+ return ReflectionUtils.newInstance(suppClass, conf).get();
Review Comment:
done, sorry about that!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
