+dev maybe someone can answer on this
On Thu, Apr 2, 2020 at 1:07 PM <[email protected]> wrote:
> Hello,
>
> I have a question on a specific design decision in Avro. I have a schema
> with a "logicalType=decimal" field. When using SpecificDatumReader to
> deserialize it, the field will get correctly deserialized as BigDecimal,
> because the set of Converters contains the BigDecimalConversion.
>
> When converting with a GenericDatumReader, the set of converters is empty.
> Is there a reason why it's empty? Why are the default converters not
> included?
>
> When reading the field with a GenericDatumReader, the converters set is
> provided by the GenericData object. So if I provide a GenericData with the
> converters, it will get converted to BigDecimal. If GenericData is not
> provided in the GenericDatumReader's constructor, I will get a ByteBuffer.
>
> Sample code below:
>
> car.avsc:
> {
> "type": "record",
> "namespace": "com.schwarzenegger",
> "name": "Car",
> "fields": [
> { "name": "model", "type": "string" },
> { "name": "engineCode", "type": { "type": "bytes",
> "logicalType": "decimal", "precision": 8, "scale": 0 } }
> ]
> }
>
> Test.java:
> public class Test {
>
> public static void main(String[] args) throws Exception {
> Schema schema = new Schema.Parser().parse( ... );
> System.out.println("LogicalType = " +
> schema.getField("engineCode").schema().getLogicalType());
>
> GenericData.Record record = null;
> try (FileInputStream payloadInputStream = new FileInputStream(new
> File("C:\\Temp\\car.txt"))) {
> GenericData genericData = new GenericData();
> genericData.addLogicalTypeConversion(new
> Conversions.DecimalConversion());
> GenericDatumReader genericReader = new
> GenericDatumReader(schema, schema, genericData);
> record = (GenericData.Record) genericReader.read(null,
> DecoderFactory.get().binaryDecoder(payloadInputStream, null));
> }
> Object engineCode =
> record.get(record.getSchema().getField("engineCode").pos());
> System.out.println(String.format("code = %s, class = %s",
> engineCode, engineCode.getClass().getName()));
> }
> }
>
> This will print out:
>
> LogicalType = org.apache.avro.LogicalTypes$Decimal@f8
> code = 12345678, class = java.math.BigDecimal
>
> If I remove that Genericdata part and create the GenericDatumReader without
> it, then I will get backa ByteBuffer because the conversions set is empty.
>
> Is there a reason why that is? If not, can we modify Avro and add the
> default conversions to the GenericDatumReader?
>
> In GenericDatumReader this is the relevant code:
>
> ****************
> protected Object read(Object old, Schema expected, ResolvingDecoder in)
> throws IOException {
> Object datum = this.readWithoutConversion(old, expected, in);
> LogicalType logicalType = expected.getLogicalType();
> if (logicalType != null) {
> Conversion<?> conversion =
> this.getData().getConversionFor(logicalType);
> if (conversion != null) {
> return this.convert(datum, expected, logicalType,
> conversion);
> }
> }
>
> return datum;
> }
>
> public Conversion<Object> getConversionFor(LogicalType logicalType) {
> return logicalType == null ? null :
> (Conversion)this.conversions.get(logicalType.getName());
> }
> ****************
>
> Thanks,
> Csaba
>
>
>
