[
https://issues.apache.org/jira/browse/AVRO-3027?focusedWorklogId=803948&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-803948
]
ASF GitHub Bot logged work on AVRO-3027:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Aug/22 12:33
Start Date: 26/Aug/22 12:33
Worklog Time Spent: 10m
Work Description: KalleOlaviNiemitalo commented on code in PR #1103:
URL: https://github.com/apache/avro/pull/1103#discussion_r955995155
##########
lang/csharp/src/apache/main/Schema/SchemaConfiguration.cs:
##########
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * https://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+namespace Avro
+{
+ /// <summary>
+ /// Schema configuration
+ /// </summary>
+ public static class SchemaConfiguration
+ {
+ /// <summary>
+ /// Enables 'soft-match' algorithm for compatibility java and .net
consumers
+ /// see
https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/Resolver.java#L640
+ /// </summary>
+ public static bool UseSoftMatch { get; set; }
Review Comment:
I don't like making this a static property. I'd prefer adding a
SchemaConfiguration parameter in Schema.CanRead, if it is necessary to support
an undocumented match algorithm. But it would be even better to fix the schemas
so that they use the documented "aliases" feature. That way, the compatiblity
would work the same in all programming languages.
Issue Time Tracking
-------------------
Worklog Id: (was: 803948)
Time Spent: 50m (was: 40m)
> .net consumer Backward compatibility issue.
> --------------------------------------------
>
> Key: AVRO-3027
> URL: https://issues.apache.org/jira/browse/AVRO-3027
> Project: Apache Avro
> Issue Type: Bug
> Components: csharp, java
> Affects Versions: 1.10.1
> Environment: .Net core 3.1
> Reporter: Yevhen Cherkes
> Priority: Critical
> Labels: pull-request-available
> Attachments: CompatibilityTests.zip
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We are facing the following exception on the .net core consumer's side:
> (*Java*-based consumers work with no issues for this particular case.)
> {code:java}
> No matching schema for
> {"type":"record","name":"InnerType1","namespace":"com.mycorp.mynamespace","fields":[
> {"name":"value","type":"string"}
> ]} in
> {"type":["null",{"type":"record","name":"InnerType2","namespace":"com.mycorp.mynamespace","fields":[
> {"name":"value","type":"string"}
> ]}]}
> at Avro.Generic.DefaultReader.findBranch(UnionSchema us, Schema s)
> at Avro.Generic.DefaultReader.ReadUnion(Object reuse, UnionSchema
> writerSchema, Schema readerSchema, Decoder d)
> at Avro.Generic.DefaultReader.Read(Object reuse, Schema writerSchema, Schema
> readerSchema, Decoder d)
> at Avro.Specific.SpecificDefaultReader.ReadRecord(Object reuse, RecordSchema
> writerSchema, Schema readerSchema, Decoder dec)
> {code}
> We have 2 versions of the schema on a topic:
> Schema v1:
> {code:json}
> {
> "fields": [
> {
> "name": "InnerComplexValue",
> "type": [
> "null",
> {
> "fields": [
> {
> "name": "value",
> "type": "string"
> }
> ],
> "name": "InnerType1",
> "namespace": "com.mycorp.mynamespace",
> "type": "record"
> }
> ]
> }
> ],
> "name": "RootType",
> "namespace": "com.mycorp.mynamespace",
> "type": "record"
> }
> {code}
> Schema v2:
> {code:json}
> {
> "fields": [
> {
> "name": "InnerComplexValue",
> "type": [
> "null",
> {
> "fields": [
> {
> "name": "value",
> "type": "string"
> }
> ],
> "name": "InnerType2",
> "namespace": "com.mycorp.mynamespace",
> "type": "record"
> }
> ]
> }
> ],
> "name": "RootType",
> "namespace": "com.mycorp.mynamespace",
> "type": "record"
> }
> {code}
> InnerType1 -> InnerType2 is the only change.
> The schema for a topic is configured with *Backward* compatibility.
> An updated version of a Schema is saved with no errors.
> Then we generated a specific record RootType with avrogen tool.
> Reproduction code:
>
> {code:c#}
> var base64 = "AhR0ZXN0IHZhbHVl";
> using var stream = new MemoryStream(Convert.FromBase64String(base64));
> var schema_v1 =
> Schema.Parse("{\"type\":\"record\",\"name\":\"RootType\",\"namespace\":\"com.mycorp.mynamespace\",\"fields\":"
> +
>
> "[{\"name\":\"InnerComplexValue\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"InnerType1\""
> +
>
> ",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}]}]}");
> var schema_v2 =
> Schema.Parse("{\"type\":\"record\",\"name\":\"RootType\",\"namespace\":\"com.mycorp.mynamespace\",\"fields\":"
> +
>
> "[{\"name\":\"InnerComplexValue\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"InnerType2\""
> +
>
> ",\"fields\":[{\"name\":\"value\",\"type\":\"string\"}]}]}]}");
> var specificRecord = Deserialize<RootType>(stream, schema_v1, schema_v2);
> {code}
> Full sources see in attachments.
> Method Deserialize was taken from the
> [SpecificTests.cs#L408-L417|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/test/Specific/SpecificTests.cs#L408-L417]
>
> Probably, the reason is in a different Schema resolution code:
> java:
> [https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/Resolver.java#L647-L659]
> .net:
> [https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/RecordSchema.cs#L310-L312]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
