[ https://issues.apache.org/jira/browse/AVRO-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540421#comment-16540421 ]
Andy Coates commented on AVRO-2164: ----------------------------------- [~rdblue], if we go with always deserializing using the write schema's scale then we'll avoid data corruption. How would it work if read/write schema's had mix of logical and non-logical types? ||Write Schema||Read Schema||Action|| |Logical|Logical with same scale|Read with specified scale| |Logical|Logical with different scale|Read with scale specified in write schema| |Logical|Not logical|Read a bytes| |Not logical|Logical|Read as decimal using read schema scale???| > Make Decimal a first class type. > -------------------------------- > > Key: AVRO-2164 > URL: https://issues.apache.org/jira/browse/AVRO-2164 > Project: Avro > Issue Type: Improvement > Components: logical types > Affects Versions: 1.8.2 > Reporter: Andy Coates > Priority: Major > > I'd be interested to hear the communities thoughts on making decimal a first > class type. > The current logical type encodes a decimal into a _bytes_ or _fixed_. This > encoding does not include any information about the scale, i.e. this encoding > is lossy. > There are open issues around the compatibility / evolvability of schemas > containing decimal logical types, (e.g. AVRO-2078 & AVRO-1721), that mean > reading data that was previously written with a different scale will result > in data corruption. > If these issues were fixed, with suitable compatibility checks put in place, > this would then make it impossible to evolve an Avro schema where the scale > needs to be changed. This inability to evolve the scale is very restrictive, > and can result in high overhead for organizations that _need_ to change the > scale, i.e. they may potentially need to copy their entire data set, > deserializing with the old scale and re-serializing with the new. > If _decimal_ were promoted to a first class type, this would allow the scale > to be captured in the serialized form, allow for schema evolution support. -- This message was sent by Atlassian JIRA (v7.6.3#76005)