[jira] [Updated] (AVRO-2078) Avro does not enforce schema resolution rules for Decimal type
[ https://issues.apache.org/jira/browse/AVRO-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thiruvalluvan M. G. updated AVRO-2078: -- Component/s: java > Avro does not enforce schema resolution rules for Decimal type > -- > > Key: AVRO-2078 > URL: https://issues.apache.org/jira/browse/AVRO-2078 > Project: Apache Avro > Issue Type: Bug > Components: java >Affects Versions: 1.8.2 >Reporter: Anthony Hsu >Assignee: Nandor Kollar >Priority: Major > Attachments: dec.avro > > > According to http://avro.apache.org/docs/1.8.2/spec.html#Decimal > bq. For the purposes of schema resolution, two schemas that are {{decimal}} > logical types _match_ if their scales and precisions match. > This is not enforced. > I wrote a file with (precision 5, scale 2) and tried to read it with a reader > schema with (precision 3, scale 1). I expected an AvroTypeException to be > thrown, but none was thrown. > Test data file attached. The code to read it is: > {noformat:title=ReadDecimal.java} > import java.io.File; > import org.apache.avro.Schema; > import org.apache.avro.file.DataFileReader; > import org.apache.avro.generic.GenericDatumReader; > import org.apache.avro.generic.GenericRecord; > import org.apache.avro.io.DatumReader; > public class ReadDecimal { > public static void main(String[] args) throws Exception { > Schema schema = new Schema.Parser().parse("{\n" + " \"type\" : > \"record\",\n" + " \"name\" : \"some_schema\",\n" > + " \"namespace\" : \"com.howdy\",\n" + " \"fields\" : [ {\n" + " > \"name\" : \"name\",\n" > + "\"type\" : \"string\"\n" + " }, {\n" + "\"name\" : > \"value\",\n" + "\"type\" : {\n" > + " \"type\" : \"bytes\",\n" + " \"logicalType\" : > \"decimal\",\n" + " \"precision\" : 3,\n" > + " \"scale\" : 1\n" + "}\n" + " } ]\n" + "}"); > DatumReader datumReader = new GenericDatumReader<>(schema); > // dec.avro has precision 5, scale 2 > DataFileReader dataFileReader = new DataFileReader<>( > new File("/tmp/dec.avro"), datumReader); > GenericRecord foo = null; > while (dataFileReader.hasNext()) { > foo = dataFileReader.next(foo); // AvroTypeException expected due to > change in scale/precision but none occurs > } > } > } > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AVRO-2078) Avro does not enforce schema resolution rules for Decimal type
[ https://issues.apache.org/jira/browse/AVRO-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated AVRO-2078: Fix Version/s: (was: 1.8.2) > Avro does not enforce schema resolution rules for Decimal type > -- > > Key: AVRO-2078 > URL: https://issues.apache.org/jira/browse/AVRO-2078 > Project: Avro > Issue Type: Bug >Affects Versions: 1.8.2 >Reporter: Anthony Hsu >Assignee: Nandor Kollar > Attachments: dec.avro > > > According to http://avro.apache.org/docs/1.8.2/spec.html#Decimal > bq. For the purposes of schema resolution, two schemas that are {{decimal}} > logical types _match_ if their scales and precisions match. > This is not enforced. > I wrote a file with (precision 5, scale 2) and tried to read it with a reader > schema with (precision 3, scale 1). I expected an AvroTypeException to be > thrown, but none was thrown. > Test data file attached. The code to read it is: > {noformat:title=ReadDecimal.java} > import java.io.File; > import org.apache.avro.Schema; > import org.apache.avro.file.DataFileReader; > import org.apache.avro.generic.GenericDatumReader; > import org.apache.avro.generic.GenericRecord; > import org.apache.avro.io.DatumReader; > public class ReadDecimal { > public static void main(String[] args) throws Exception { > Schema schema = new Schema.Parser().parse("{\n" + " \"type\" : > \"record\",\n" + " \"name\" : \"some_schema\",\n" > + " \"namespace\" : \"com.howdy\",\n" + " \"fields\" : [ {\n" + " > \"name\" : \"name\",\n" > + "\"type\" : \"string\"\n" + " }, {\n" + "\"name\" : > \"value\",\n" + "\"type\" : {\n" > + " \"type\" : \"bytes\",\n" + " \"logicalType\" : > \"decimal\",\n" + " \"precision\" : 3,\n" > + " \"scale\" : 1\n" + "}\n" + " } ]\n" + "}"); > DatumReader datumReader = new GenericDatumReader<>(schema); > // dec.avro has precision 5, scale 2 > DataFileReader dataFileReader = new DataFileReader<>( > new File("/tmp/dec.avro"), datumReader); > GenericRecord foo = null; > while (dataFileReader.hasNext()) { > foo = dataFileReader.next(foo); // AvroTypeException expected due to > change in scale/precision but none occurs > } > } > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2078) Avro does not enforce schema resolution rules for Decimal type
[ https://issues.apache.org/jira/browse/AVRO-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated AVRO-2078: Affects Version/s: 1.8.2 > Avro does not enforce schema resolution rules for Decimal type > -- > > Key: AVRO-2078 > URL: https://issues.apache.org/jira/browse/AVRO-2078 > Project: Avro > Issue Type: Bug >Affects Versions: 1.8.2 >Reporter: Anthony Hsu >Assignee: Nandor Kollar > Fix For: 1.8.2 > > Attachments: dec.avro > > > According to http://avro.apache.org/docs/1.8.2/spec.html#Decimal > bq. For the purposes of schema resolution, two schemas that are {{decimal}} > logical types _match_ if their scales and precisions match. > This is not enforced. > I wrote a file with (precision 5, scale 2) and tried to read it with a reader > schema with (precision 3, scale 1). I expected an AvroTypeException to be > thrown, but none was thrown. > Test data file attached. The code to read it is: > {noformat:title=ReadDecimal.java} > import java.io.File; > import org.apache.avro.Schema; > import org.apache.avro.file.DataFileReader; > import org.apache.avro.generic.GenericDatumReader; > import org.apache.avro.generic.GenericRecord; > import org.apache.avro.io.DatumReader; > public class ReadDecimal { > public static void main(String[] args) throws Exception { > Schema schema = new Schema.Parser().parse("{\n" + " \"type\" : > \"record\",\n" + " \"name\" : \"some_schema\",\n" > + " \"namespace\" : \"com.howdy\",\n" + " \"fields\" : [ {\n" + " > \"name\" : \"name\",\n" > + "\"type\" : \"string\"\n" + " }, {\n" + "\"name\" : > \"value\",\n" + "\"type\" : {\n" > + " \"type\" : \"bytes\",\n" + " \"logicalType\" : > \"decimal\",\n" + " \"precision\" : 3,\n" > + " \"scale\" : 1\n" + "}\n" + " } ]\n" + "}"); > DatumReader datumReader = new GenericDatumReader<>(schema); > // dec.avro has precision 5, scale 2 > DataFileReader dataFileReader = new DataFileReader<>( > new File("/tmp/dec.avro"), datumReader); > GenericRecord foo = null; > while (dataFileReader.hasNext()) { > foo = dataFileReader.next(foo); // AvroTypeException expected due to > change in scale/precision but none occurs > } > } > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2078) Avro does not enforce schema resolution rules for Decimal type
[ https://issues.apache.org/jira/browse/AVRO-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandor Kollar updated AVRO-2078: Assignee: Nandor Kollar Status: Patch Available (was: Open) > Avro does not enforce schema resolution rules for Decimal type > -- > > Key: AVRO-2078 > URL: https://issues.apache.org/jira/browse/AVRO-2078 > Project: Avro > Issue Type: Bug >Reporter: Anthony Hsu >Assignee: Nandor Kollar > Fix For: 1.8.2 > > Attachments: dec.avro > > > According to http://avro.apache.org/docs/1.8.2/spec.html#Decimal > bq. For the purposes of schema resolution, two schemas that are {{decimal}} > logical types _match_ if their scales and precisions match. > This is not enforced. > I wrote a file with (precision 5, scale 2) and tried to read it with a reader > schema with (precision 3, scale 1). I expected an AvroTypeException to be > thrown, but none was thrown. > Test data file attached. The code to read it is: > {noformat:title=ReadDecimal.java} > import java.io.File; > import org.apache.avro.Schema; > import org.apache.avro.file.DataFileReader; > import org.apache.avro.generic.GenericDatumReader; > import org.apache.avro.generic.GenericRecord; > import org.apache.avro.io.DatumReader; > public class ReadDecimal { > public static void main(String[] args) throws Exception { > Schema schema = new Schema.Parser().parse("{\n" + " \"type\" : > \"record\",\n" + " \"name\" : \"some_schema\",\n" > + " \"namespace\" : \"com.howdy\",\n" + " \"fields\" : [ {\n" + " > \"name\" : \"name\",\n" > + "\"type\" : \"string\"\n" + " }, {\n" + "\"name\" : > \"value\",\n" + "\"type\" : {\n" > + " \"type\" : \"bytes\",\n" + " \"logicalType\" : > \"decimal\",\n" + " \"precision\" : 3,\n" > + " \"scale\" : 1\n" + "}\n" + " } ]\n" + "}"); > DatumReader datumReader = new GenericDatumReader<>(schema); > // dec.avro has precision 5, scale 2 > DataFileReader dataFileReader = new DataFileReader<>( > new File("/tmp/dec.avro"), datumReader); > GenericRecord foo = null; > while (dataFileReader.hasNext()) { > foo = dataFileReader.next(foo); // AvroTypeException expected due to > change in scale/precision but none occurs > } > } > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AVRO-2078) Avro does not enforce schema resolution rules for Decimal type
[ https://issues.apache.org/jira/browse/AVRO-2078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Hsu updated AVRO-2078: -- Attachment: dec.avro > Avro does not enforce schema resolution rules for Decimal type > -- > > Key: AVRO-2078 > URL: https://issues.apache.org/jira/browse/AVRO-2078 > Project: Avro > Issue Type: Bug >Reporter: Anthony Hsu > Fix For: 1.8.2 > > Attachments: dec.avro > > > According to http://avro.apache.org/docs/1.8.2/spec.html#Decimal > bq. For the purposes of schema resolution, two schemas that are {{decimal}} > logical types _match_ if their scales and precisions match. > This is not enforced. > I wrote a file with (precision 5, scale 2) and tried to read it with a reader > schema with (precision 3, scale 1). I expected an AvroTypeException to be > thrown, but none was thrown. > Test data file attached. The code to read it is: > {noformat:title=ReadDecimal.java} > import java.io.File; > import org.apache.avro.Schema; > import org.apache.avro.file.DataFileReader; > import org.apache.avro.generic.GenericDatumReader; > import org.apache.avro.generic.GenericRecord; > import org.apache.avro.io.DatumReader; > public class ReadDecimal { > public static void main(String[] args) throws Exception { > Schema schema = new Schema.Parser().parse("{\n" + " \"type\" : > \"record\",\n" + " \"name\" : \"some_schema\",\n" > + " \"namespace\" : \"com.howdy\",\n" + " \"fields\" : [ {\n" + " > \"name\" : \"name\",\n" > + "\"type\" : \"string\"\n" + " }, {\n" + "\"name\" : > \"value\",\n" + "\"type\" : {\n" > + " \"type\" : \"bytes\",\n" + " \"logicalType\" : > \"decimal\",\n" + " \"precision\" : 3,\n" > + " \"scale\" : 1\n" + "}\n" + " } ]\n" + "}"); > DatumReader datumReader = new GenericDatumReader<>(schema); > // dec.avro has precision 5, scale 2 > DataFileReader dataFileReader = new DataFileReader<>( > new File("/tmp/dec.avro"), datumReader); > GenericRecord foo = null; > while (dataFileReader.hasNext()) { > foo = dataFileReader.next(foo); // AvroTypeException expected due to > change in scale/precision but none occurs > } > } > } > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)