martin-traverse commented on code in PR #718:
URL: https://github.com/apache/arrow-java/pull/718#discussion_r2050413561
##########
adapter/avro/src/main/java/org/apache/arrow/adapter/avro/AvroToArrowUtils.java:
##########
@@ -277,11 +333,17 @@ private static Consumer createConsumer(
break;
case BYTES:
if (logicalType instanceof LogicalTypes.Decimal) {
- arrowType = createDecimalArrowType((LogicalTypes.Decimal)
logicalType);
+ LogicalTypes.Decimal decimalType = (LogicalTypes.Decimal)
logicalType;
+ arrowType = createDecimalArrowType(decimalType, schema);
fieldType =
new FieldType(nullable, arrowType, /* dictionary= */ null,
getMetaData(schema));
vector = createVector(consumerVector, fieldType, name, allocator);
- consumer = new
AvroDecimalConsumer.BytesDecimalConsumer((DecimalVector) vector);
+ if (decimalType.getPrecision() <= 38) {
Review Comment:
For FIXED decimals I am using the fixedSize to choose the decimal type. For
BYTES decimals yes they would just come back as the smaller type if the
precision fits.
I used FIXED as the default output for decimals in the producers, because it
is closer to the Arrow representation, but on reflection Avro as a format is
very focused on keeping data compact, maybe BYTES makes more sense. It seems to
be the default choice in Avro. Do you think we should go with that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
