Hello,

I am working on a project that aims at converting Mainframe data to Avro records (https://github.com/legsem/legstar.avro).

Mainframe data often contains Decimal types. For these, I would like the corresponding Avro records to expose BigDecimal fields.

Initially, I followed the recommendation here: http://avro.apache.org/docs/1.7.7/spec.html#Decimal. My schema contains for instance:

    {
      "name":"transactionAmount",
      "type":{
        "type":"bytes",
        "logicalType":"decimal",
        "precision":7,
        "scale":2
      }
    }

This works fine but the Avro Specific record produced by the SpecificCompiler exposes a ByteBuffer for that field.

  @Deprecated public java.nio.ByteBuffer transactionAmount;

Not what I want.

I tried this alternative:

    {
      "name":"transactionAmount",
      "type":{
        "type":"string",
        "java-class":"java.math.BigDecimal",
        "logicalType":"decimal",
        "precision":7,
        "scale":2
      }

Now the SchemaCompiler produces the result I need:

  @Deprecated public java.math.BigDecimal transactionAmount;

There are 2 problems though:

1. It is less efficient to serialize/deserialize a BigDecimal from a string rather then the 2's complement.

2. The Specific Record obtained this way cannot be populated using a deep copy from a Generic Record.

To clarify the second point:

When I convert the mainframe data I do something like:

        GenericRecord genericRecord = new GenericData.Record(schema);
        ... populate genericRecord from Mainframe data ...
        return (D) SpecificData.get().deepCopy(schema, genericRecord);

This fails with :
java.lang.ClassCastException: java.lang.String cannot be cast to java.math.BigDecimal at legstar.avro.test.specific.cusdat.Transaction.put(Transaction.java:47) at org.apache.avro.generic.GenericData.setField(GenericData.java:573) at org.apache.avro.generic.GenericData.setField(GenericData.java:590) at org.apache.avro.generic.GenericData.deepCopy(GenericData.java:972) at org.apache.avro.generic.GenericData.deepCopy(GenericData.java:926) at org.apache.avro.generic.GenericData.deepCopy(GenericData.java:970) at org.apache.avro.generic.GenericData.deepCopy(GenericData.java:970)


This is because the code in the Specific record assumes the value received is already a BigDecimal

    case 1: transactionAmount = (java.math.BigDecimal)value$; break;

In other words, the java-class trick produces the right interface for Specific classes but the internal data types are not consistent with the GenericRecord derived from the same schema.

So my question is: what would be a better approach for Specific classes to expose BigDecimal fields?

Reply via email to