Hello,
I am working on a project that aims at converting Mainframe data to Avro
records (https://github.com/legsem/legstar.avro).
Mainframe data often contains Decimal types. For these, I would like the
corresponding Avro records to expose BigDecimal fields.
Initially, I followed the recommendation here:
http://avro.apache.org/docs/1.7.7/spec.html#Decimal. My schema contains
for instance:
{
"name":"transactionAmount",
"type":{
"type":"bytes",
"logicalType":"decimal",
"precision":7,
"scale":2
}
}
This works fine but the Avro Specific record produced by the
SpecificCompiler exposes a ByteBuffer for that field.
@Deprecated public java.nio.ByteBuffer transactionAmount;
Not what I want.
I tried this alternative:
{
"name":"transactionAmount",
"type":{
"type":"string",
"java-class":"java.math.BigDecimal",
"logicalType":"decimal",
"precision":7,
"scale":2
}
Now the SchemaCompiler produces the result I need:
@Deprecated public java.math.BigDecimal transactionAmount;
There are 2 problems though:
1. It is less efficient to serialize/deserialize a BigDecimal from a
string rather then the 2's complement.
2. The Specific Record obtained this way cannot be populated using a
deep copy from a Generic Record.
To clarify the second point:
When I convert the mainframe data I do something like:
GenericRecord genericRecord = new GenericData.Record(schema);
... populate genericRecord from Mainframe data ...
return (D) SpecificData.get().deepCopy(schema, genericRecord);
This fails with :
java.lang.ClassCastException: java.lang.String cannot be cast
to java.math.BigDecimal
at
legstar.avro.test.specific.cusdat.Transaction.put(Transaction.java:47)
at
org.apache.avro.generic.GenericData.setField(GenericData.java:573)
at
org.apache.avro.generic.GenericData.setField(GenericData.java:590)
at
org.apache.avro.generic.GenericData.deepCopy(GenericData.java:972)
at
org.apache.avro.generic.GenericData.deepCopy(GenericData.java:926)
at
org.apache.avro.generic.GenericData.deepCopy(GenericData.java:970)
at
org.apache.avro.generic.GenericData.deepCopy(GenericData.java:970)
This is because the code in the Specific record assumes the value
received is already a BigDecimal
case 1: transactionAmount = (java.math.BigDecimal)value$; break;
In other words, the java-class trick produces the right interface for
Specific classes but the internal data types are not consistent with the
GenericRecord derived from the same schema.
So my question is: what would be a better approach for Specific classes
to expose BigDecimal fields?