Dave Kliczbor created AVRO-3969:
-----------------------------------
Summary: [C#] Serializing a Map/Dictionary using
SpecificDefaultWriter and JsonEncoder fails with AvroTypeException
Key: AVRO-3969
URL: https://issues.apache.org/jira/browse/AVRO-3969
Project: Apache Avro
Issue Type: Bug
Components: csharp
Affects Versions: 1.11.3
Environment: We believe the environment to not matter here, but as
there is a box for that here... :)
* Windows 11
* .NET 8.0
* C# 12
* Apache.Avro 1.11.3 (released via nuget)
Reporter: Dave Kliczbor
Hello,
We think we identified a bug in the SpecificDefaultWriter implementation of the
Apache.Avro C# library (as of Release 1.11.3, also main branch as of
2024-04-03).
h4. Reproduction of the problem:
The following C# code throws an {{{}AvroTypeException("Attempt to process an
array-start when a map-start was expected"){}}}:
{code:java}
var schema = MapSchema.CreateMap(PrimitiveSchema.Create(Schema.Type.String));
var ms = new MemoryStream();
Encoder enc = new JsonEncoder(schema, ms, false);
var writer = new SpecificDefaultWriter(schema);
writer.Write(new Dictionary<string, string>(), enc);
enc.Flush();{code}
We expect this code to not throw up.
The same code with {{BinaryEncoder(schema, ms)}} instead of
{{JsonEncoder(schema, ms, false)}} works as expected: we get a Avro message
byte array without an Exception being thrown.
h4. Preliminary Solution Analysis:
We believe the root cause to be in this 13 years old code:
[https://github.com/apache/avro/blob/8cb32f05d50382e5a5e97d6d45b080bef0c0e6a5/lang/csharp/src/apache/main/Specific/SpecificWriter.cs#L152]
{code:java}
protected override void WriteMap(MapSchema schema, object value,
Encoder encoder)
{
var map = value as System.Collections.IDictionary;
if (map == null)
throw new AvroTypeException("Map does not implement non-generic
IDictionary");
encoder.WriteArrayStart(); // <----- This is L.152
encoder.SetItemCount(map.Count);
foreach (System.Collections.DictionaryEntry de in map)
{
encoder.StartItem();
encoder.WriteString(de.Key as string);
Write(schema.ValueSchema, de.Value, encoder);
}
encoder.WriteMapEnd();
} {code}
In method {{{}SpecificDefaultWriter.WriteMap(MapSchema, object, Encoder){}}},
the call {{encoder.WriteArrayStart()}} is used. Which is in stark contrast to
the usage of {{encoder.WriteMapEnd()}} later and also matches the Exception
message above (using the BinaryEncoder works without problems, as
{{WriteArrayStart}} and {{WriteMapStart }}are identical empty methods there).
Therefore, we believe that {{encoder.WriteMapStart();}} is the intended call in
L.152.
h4. Additional information:
Here’s our test case (code using xUnit/FluentAssertions and Apache.Avro 1.11.3)
that fails with the bug present:
{code:java}
[Fact]
public void AvroLibraryJsonEncoderDoesNotChokeOnMaps()
{
Action action = () =>
{
var schema =
MapSchema.CreateMap(PrimitiveSchema.Create(Schema.Type.String));
var ms = new MemoryStream();
Encoder enc = new JsonEncoder(schema, ms, false);
var writer = new SpecificDefaultWriter(schema);
writer.Write(new Dictionary<string, string>(), enc);
enc.Flush();
};
action.Should().NotThrow<AvroTypeException>();
}{code}
Our workaround is to use this derived SpecificDefaultWriter class instead of
the original from the Apache.Avro C# library. This works for our purposes, but
up to now, we are mere users of the library and do not know yet whether this
might break something else.
{code:java}
public class SpecificDefaultWriterWithWriteMapToJsonBugfix(Schema schema) :
SpecificDefaultWriter(schema) {
protected override void WriteMap(MapSchema schema, object value, Encoder
encoder)
{
if (!(value is IDictionary dictionary))
throw new AvroTypeException("Map does not implement non-generic
IDictionary");
encoder.WriteMapStart();
encoder.SetItemCount((long)dictionary.Count);
foreach (DictionaryEntry dictionaryEntry in dictionary)
{
encoder.StartItem();
encoder.WriteString(dictionaryEntry.Key as string);
this.Write(schema.ValueSchema, dictionaryEntry.Value, encoder);
}
encoder.WriteMapEnd();
}
}
{code}
Cheers!
Dave Kliczbor (Materna SE)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
