[
https://issues.apache.org/jira/browse/AVRO-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16516822#comment-16516822
]
ASF GitHub Bot commented on AVRO-2078:
--------------------------------------
nandorKollar commented on issue #247: AVRO-2078: Avro does not enforce schema
resolution rules for Decimal …
URL: https://github.com/apache/avro/pull/247#issuecomment-398326638
Hey @big-andy-coates ! I didn't yet merge this PR, because the logical type
schema resolution rules are not well defined, and there was some debate how we
should handle it. The spec mentions an evolution rule for Decimal, but other
types are not mentioned at all, therefore no evolution rule is defined for
example Time -> Timestamp promotion. Is it allowed? How should Avro interpret
the value? Right now if one uses generated classes, the methods will return
wrong values, because the in memory representation is incorrect!
The current implementation doesn't conform with the specification, because
it omits changes in the decimal parameters, but I'm not sure if we should
change the spec, or get this PR merged. I found a discussion on AVRO-1721
(actually you're the last one who commented it), which is about similar topic,
and looks like there was no consensus about the correct approach. Could you
please open this topic for discussion on the dev list?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Avro does not enforce schema resolution rules for Decimal type
> --------------------------------------------------------------
>
> Key: AVRO-2078
> URL: https://issues.apache.org/jira/browse/AVRO-2078
> Project: Avro
> Issue Type: Bug
> Affects Versions: 1.8.2
> Reporter: Anthony Hsu
> Assignee: Nandor Kollar
> Priority: Major
> Attachments: dec.avro
>
>
> According to http://avro.apache.org/docs/1.8.2/spec.html#Decimal
> bq. For the purposes of schema resolution, two schemas that are {{decimal}}
> logical types _match_ if their scales and precisions match.
> This is not enforced.
> I wrote a file with (precision 5, scale 2) and tried to read it with a reader
> schema with (precision 3, scale 1). I expected an AvroTypeException to be
> thrown, but none was thrown.
> Test data file attached. The code to read it is:
> {noformat:title=ReadDecimal.java}
> import java.io.File;
> import org.apache.avro.Schema;
> import org.apache.avro.file.DataFileReader;
> import org.apache.avro.generic.GenericDatumReader;
> import org.apache.avro.generic.GenericRecord;
> import org.apache.avro.io.DatumReader;
> public class ReadDecimal {
> public static void main(String[] args) throws Exception {
> Schema schema = new Schema.Parser().parse("{\n" + " \"type\" :
> \"record\",\n" + " \"name\" : \"some_schema\",\n"
> + " \"namespace\" : \"com.howdy\",\n" + " \"fields\" : [ {\n" + "
> \"name\" : \"name\",\n"
> + " \"type\" : \"string\"\n" + " }, {\n" + " \"name\" :
> \"value\",\n" + " \"type\" : {\n"
> + " \"type\" : \"bytes\",\n" + " \"logicalType\" :
> \"decimal\",\n" + " \"precision\" : 3,\n"
> + " \"scale\" : 1\n" + " }\n" + " } ]\n" + "}");
> DatumReader<GenericRecord> datumReader = new GenericDatumReader<>(schema);
> // dec.avro has precision 5, scale 2
> DataFileReader<GenericRecord> dataFileReader = new DataFileReader<>(
> new File("/tmp/dec.avro"), datumReader);
> GenericRecord foo = null;
> while (dataFileReader.hasNext()) {
> foo = dataFileReader.next(foo); // AvroTypeException expected due to
> change in scale/precision but none occurs
> }
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
