I think the rule is that if you want bi-directional compatibility, give every field a default value.
Doug On Mon, Oct 22, 2018 at 2:13 PM Jesse Anderson <[email protected]> wrote: > In the schema resolution of the Avro spec, it doesn't really talk about > the effect of aliases on schema resolution. So, aliases should be avoided > for forward compatibility, but are fine for backward compatibility? If you > want full compatibility, you should think long and hard about your field > names. > > On Mon, Oct 22, 2018 at 2:05 PM Doug Cutting <[email protected]> wrote: > >> Despite the change in field name, they have the same binary format, so >> you can read one and write the other in either order. If you're actually >> adding or removing fields or re-ordering fields then it probably would not >> work. >> >> Doug >> >> On Mon, Oct 22, 2018 at 1:57 PM Jesse Anderson <[email protected]> >> wrote: >> >>> I was trying to get V2 writer with V1 reader. >>> >>> This V2 schema will not work because of suit alias: >>> { >>> "namespace": "model", >>> "type": "record", >>> "name": "SimpleCardV2", >>> "aliases": ["SimpleCard"], >>> "fields": [ >>> { >>> "name": "cardsuit", >>> "type": "string", >>> "doc" : "The suit of the card", >>> * "aliases": ["suit"]* >>> } >>> ] >>> } >>> >>> This V2 schema will work because suit is no longer aliased: >>> { >>> "namespace": "model", >>> "type": "record", >>> "name": "SimpleCardV2", >>> "aliases": ["SimpleCard"], >>> "fields": [ >>> { >>> "name": "suit", >>> "type": "string", >>> "doc" : "The suit of the card" >>> } >>> ] >>> } >>> >>> Even with suit no longer aliased, the schema change doesn't pass the >>> compatibility checks, but does pass serializing and deserializing them back >>> into SpecificRecord objects. >>> >>> Thanks, >>> >>> Jesse >>> >>> On Mon, Oct 22, 2018 at 1:46 PM Doug Cutting <[email protected]> wrote: >>> >>>> Aliases in the reader's schema are used to adapt the writer's schema. >>>> Aliases in the writer's schema are not considered. So your V2 can read V1 >>>> data, but not vice versa. Is that what you're asking? >>>> >>>> Thanks, >>>> >>>> Doug >>>> >>>> On Mon, Oct 22, 2018 at 1:01 PM Jesse Anderson < >>>> [email protected]> wrote: >>>> >>>>> The email subject should have been "Alias with Forward Compatibility." >>>>> Are using aliases forward incompatible? I'm using Avro 1.8.2. >>>>> >>>>> Using Hotels.com's Avro compatibility checker, I get: >>>>> com.hotels.avro.compatibility.SchemaCompatibilityException: >>>>> Compatibility type 'CAN_BE_READ_BY' does not hold between 1 schema(s) in >>>>> the chronology because: Schema[0] has incompatibilities: ['NAME_MISMATCH: >>>>> expected: model.SimpleCardV2' at >>>>> '/name','READER_FIELD_MISSING_DEFAULT_VALUE: suit' at '/fields/0']. >>>>> >>>>> Thanks, >>>>> >>>>> Jesse >>>>> >>>>> On Fri, Oct 19, 2018 at 4:17 PM Jesse Anderson < >>>>> [email protected]> wrote: >>>>> >>>>>> Is adding an alias a backwards incompatible change? I have two >>>>>> schemas. I'm going from V2 to V1 and getting an error that the field is >>>>>> missing. Am I missing something again? >>>>>> >>>>>> Here is V1: >>>>>> { >>>>>> "namespace": "model", >>>>>> "type": "record", >>>>>> "name": "SimpleCard", >>>>>> "fields": [ >>>>>> { >>>>>> "name": "suit", >>>>>> "type": "string", >>>>>> "doc" : "The suit of the card" >>>>>> } >>>>>> ] >>>>>> } >>>>>> >>>>>> Here is V2: >>>>>> { >>>>>> "namespace": "model", >>>>>> "type": "record", >>>>>> "name": "SimpleCardV2", >>>>>> "aliases": ["SimpleCard"], >>>>>> "fields": [ >>>>>> { >>>>>> "name": "cardsuit", >>>>>> "type": "string", >>>>>> "doc" : "The suit of the card", >>>>>> "aliases": ["suit"] >>>>>> } >>>>>> ] >>>>>> } >>>>>> >>>>>> Here is the stack trace: >>>>>> org.apache.avro.AvroTypeException: Found model.SimpleCardV2, >>>>>> expecting model.SimpleCard, missing required field suit >>>>>> at >>>>>> org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:292) >>>>>> at org.apache.avro.io.parsing.Parser.advance(Parser.java:88) >>>>>> at >>>>>> org.apache.avro.io.ResolvingDecoder.readFieldOrder(ResolvingDecoder.java:130) >>>>>> at >>>>>> org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:215) >>>>>> at >>>>>> org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:175) >>>>>> at >>>>>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:153) >>>>>> at >>>>>> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:145) >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Jesse >>>>>> >>>>>
