[
https://issues.apache.org/jira/browse/AVRO-3027?focusedWorklogId=647994&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-647994
]
ASF GitHub Bot logged work on AVRO-3027:
----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Sep/21 14:42
Start Date: 08/Sep/21 14:42
Worklog Time Spent: 10m
Work Description: ycherkes edited a comment on pull request #1103:
URL: https://github.com/apache/avro/pull/1103#issuecomment-915247553
> This looks like a good candidate feature for the 1.11.0 major release --
the "soft match" behavior is wrong (I think) but it happens in Java, and the C#
SDK is essentially unchanged if the flag isn't turned on. Can some C# eyes take
a look at this?
I think, it's not the only compatibility issue.
For example, we can't find the Avro schema in the schema registry by using
the output of SomeSpecificRecord.Schema.ToString() method call.
Where SomeSpecificRecord is a class pre-generated by avrogen tool.
This call returns the schema string that doesn't match the original schema.
(misses top-level doc property).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 647994)
Time Spent: 0.5h (was: 20m)
> .net consumer Backward compatibility issue.
> --------------------------------------------
>
> Key: AVRO-3027
> URL: https://issues.apache.org/jira/browse/AVRO-3027
> Project: Apache Avro
> Issue Type: Bug
> Components: csharp, java
> Affects Versions: 1.10.1
> Environment: .Net core 3.1
> Reporter: Yevhen Cherkes
> Priority: Critical
> Labels: pull-request-available
> Attachments: CompatibilityTests.zip
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> We are facing the following exception on the .net core consumer's side:
> (*Java*-based consumers work with no issues for this particular case.)
> {code:java}
> No matching schema for
> {"type":"record","name":"InnerType1","namespace":"com.mycorp.mynamespace","fields":[
> {"name":"value","type":"string"}
> ]} in
> {"type":["null",{"type":"record","name":"InnerType2","namespace":"com.mycorp.mynamespace","fields":[
> {"name":"value","type":"string"}
> ]}]}
> at Avro.Generic.DefaultReader.findBranch(UnionSchema us, Schema s)
> at Avro.Generic.DefaultReader.ReadUnion(Object reuse, UnionSchema
> writerSchema, Schema readerSchema, Decoder d)
> at Avro.Generic.DefaultReader.Read(Object reuse, Schema writerSchema, Schema
> readerSchema, Decoder d)
> at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema
> writerSchema, Schema readerSchema, Decoder dec)
> {code}
> We have 2 versions of the schema on a topic:
> Schema v1:
> {code:json}
> {
> "fields": [
> {
> "name": "InnerComplexValue",
> "type": [
> "null",
> {
> "fields": [
> {
> "name": "value",
> "type": "string"
> }
> ],
> "name": "InnerType1",
> "namespace": "com.mycorp.mynamespace",
> "type": "record"
> }
> ]
> }
> ],
> "name": "RootType",
> "namespace": "com.mycorp.mynamespace",
> "type": "record"
> }
> {code}
> Schema v2:
> {code:json}
> {
> "fields": [
> {
> "name": "InnerComplexValue",
> "type": [
> "null",
> {
> "fields": [
> {
> "name": "value",
> "type": "string"
> }
> ],
> "name": "InnerType2",
> "namespace": "com.mycorp.mynamespace",
> "type": "record"
> }
> ]
> }
> ],
> "name": "RootType",
> "namespace": "com.mycorp.mynamespace",
> "type": "record"
> }
> {code}
> InnerType1 -> InnerType2 is the only change.
> The schema for a topic is configured with *Backward* compatibility.
> An updated version of a Schema is saved with no errors.
> Then we generated a specific record RootType with avrogen tool.
> Reproduction code:
>
> {code:c#}
> var base64 = "AhR0ZXN0IHZhbHVl";
> using var stream = new MemoryStream(Convert.FromBase64String(base64));
> var schema_v1 =
> Schema.Parse("{\"type\":\"record\",\"name\":\"RootType\",\"namespace\":\"com.mycorp.mynamespace\",\"fields\":"
> +
>
> "[{\"name\":\"InnerComplexValue\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"InnerType1\""
> +
>
> ",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}]}]}");
> var schema_v2 =
> Schema.Parse("{\"type\":\"record\",\"name\":\"RootType\",\"namespace\":\"com.mycorp.mynamespace\",\"fields\":"
> +
>
> "[{\"name\":\"InnerComplexValue\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"InnerType2\""
> +
>
> ",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}]}]}");
> var specificRecord = Deserialize<RootType>(stream, schema_v1, schema_v2);
> {code}
> Full sources see in attachments.
> Method Deserialize was taken from the
> [SpecificTests.cs#L408-L417|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/test/Specific/SpecificTests.cs#L408-L417]
>
> Probably, the reason is in a different Schema resolution code:
> java:
> [https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/Resolver.java#L647-L659]
> .net:
> [https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/RecordSchema.cs#L310-L312]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
