[
https://issues.apache.org/jira/browse/AVRO-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shawn Wu updated AVRO-3182:
---------------------------
Description:
Version: 1.10.2
When I use Avro in C#, I found Schema.ToString() for union schema will add an
extra *type* property in JSON string.
[Schema.ToString() in
C#|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/Schema.cs#L272]
However, the correct union schema is a JSON array.
This is my test code in C#:
{code:java}
var columnCountSchema =
Schema.Parse("{\"type\":\"record\",\"name\":\"columnCount\",\"namespace\":\"avro.test.header\",\"fields\":[{\"name\":\"columnCount\",\"type\":\"int\"}]}");
var errorMessageSchema =
Schema.Parse("{\"type\":\"record\",\"name\":\"errorMessage\",\"namespace\":\"avro.test.header\",\"fields\":[{\"name\":\"errorMessage\",\"type\":[\"string\",\"null\"]}]}");
var schema = Schema.Parse($"[{columnCountSchema},{errorMessageSchema}]");
Console.WriteLine(schema.ToString());{code}
The output:
{code:java}
{"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}
{code}
When I use Avro in C# to parse this union schema JSON string, it is correct.
[Schema.ParseJson() in
C#|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/Schema.cs#L203]
But when I use Avro in Java to parse the union schema in Stream from C#, there
is the following error message:
{code:java}
org.apache.avro.SchemaParseException: No type:
{"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}org.apache.avro.SchemaParseException:
No type:
{"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}
at org.apache.avro.Schema.getRequiredText(Schema.java:1784) at
org.apache.avro.Schema.parse(Schema.java:1637) at
org.apache.avro.Schema$Parser.parse(Schema.java:1425) at
org.apache.avro.Schema$Parser.parse(Schema.java:1413) at
org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:130) at
org.apache.avro.file.DataFileStream.<init>(DataFileStream.java:90) at
avro.AvroService.read(AvroService.java:68) at
avro.AvroService$1.onResponse(AvroService.java:144) at
okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
I think this format of union schema should be supported in Java.
was:
Version: 1.10.2
When I use Avro in C#, I found Schema.ToString() for union schema will add an
extra *type* property in JSON string.
[Schema.ToString() in
C#|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/Schema.cs#L272]
However, the correct union schema is a JSON array.
This is my test code in C#:
{code:java}
var columnCountSchema =
Schema.Parse("{\"type\":\"record\",\"name\":\"columnCount\",\"namespace\":\"avro.test.header\",\"fields\":[{\"name\":\"columnCount\",\"type\":\"int\"}]}");
var errorMessageSchema =
Schema.Parse("{\"type\":\"record\",\"name\":\"errorMessage\",\"namespace\":\"avro.test.header\",\"fields\":[{\"name\":\"errorMessage\",\"type\":[\"string\",\"null\"]}]}");
var schema = Schema.Parse($"[{columnCountSchema},{errorMessageSchema}]");
Console.WriteLine(schema.ToString());{code}
The output:
{code:java}
{"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}
{code}
When I use Avro in C# to parse this union schema JSON string, it is correct.
[Schema.ParseJson() in
C#|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/Schema.cs#L203]
But when I use Avro in Java to parse the union schema in Stream from C#, there
is the following error message:
{code:java}
org.apache.avro.SchemaParseException: No type:
{"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}org.apache.avro.SchemaParseException:
No type:
{"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}
at org.apache.avro.Schema.getRequiredText(Schema.java:1784) at
org.apache.avro.Schema.parse(Schema.java:1637) at
org.apache.avro.Schema$Parser.parse(Schema.java:1425) at
org.apache.avro.Schema$Parser.parse(Schema.java:1413) at
org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:130) at
org.apache.avro.file.DataFileStream.<init>(DataFileStream.java:90) at
avro.AvroService.read(AvroService.java:68) at
avro.AvroService$1.onResponse(AvroService.java:144) at
okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
I think this format of union schema should be supported in Java.
{code:java}
{code}
> Avro in Java cannot parse the union schema in stream from C#
> ------------------------------------------------------------
>
> Key: AVRO-3182
> URL: https://issues.apache.org/jira/browse/AVRO-3182
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.10.2
> Reporter: Shawn Wu
> Priority: Critical
>
> Version: 1.10.2
>
> When I use Avro in C#, I found Schema.ToString() for union schema will add an
> extra *type* property in JSON string.
> [Schema.ToString() in
> C#|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/Schema.cs#L272]
> However, the correct union schema is a JSON array.
> This is my test code in C#:
> {code:java}
> var columnCountSchema =
> Schema.Parse("{\"type\":\"record\",\"name\":\"columnCount\",\"namespace\":\"avro.test.header\",\"fields\":[{\"name\":\"columnCount\",\"type\":\"int\"}]}");
> var errorMessageSchema =
> Schema.Parse("{\"type\":\"record\",\"name\":\"errorMessage\",\"namespace\":\"avro.test.header\",\"fields\":[{\"name\":\"errorMessage\",\"type\":[\"string\",\"null\"]}]}");
> var schema = Schema.Parse($"[{columnCountSchema},{errorMessageSchema}]");
> Console.WriteLine(schema.ToString());{code}
> The output:
> {code:java}
> {"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}
> {code}
>
> When I use Avro in C# to parse this union schema JSON string, it is correct.
> [Schema.ParseJson() in
> C#|https://github.com/apache/avro/blob/master/lang/csharp/src/apache/main/Schema/Schema.cs#L203]
> But when I use Avro in Java to parse the union schema in Stream from C#,
> there is the following error message:
> {code:java}
> org.apache.avro.SchemaParseException: No type:
> {"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}org.apache.avro.SchemaParseException:
> No type:
> {"type":[{"type":"record","name":"columnCount","namespace":"avro.test.header","fields":[{"name":"columnCount","type":"int"}]},{"type":"record","name":"errorMessage","namespace":"avro.test.header","fields":[{"name":"errorMessage","type":["string","null"]}]}]}
> at org.apache.avro.Schema.getRequiredText(Schema.java:1784) at
> org.apache.avro.Schema.parse(Schema.java:1637) at
> org.apache.avro.Schema$Parser.parse(Schema.java:1425) at
> org.apache.avro.Schema$Parser.parse(Schema.java:1413) at
> org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:130) at
> org.apache.avro.file.DataFileStream.<init>(DataFileStream.java:90) at
> avro.AvroService.read(AvroService.java:68) at
> avro.AvroService$1.onResponse(AvroService.java:144) at
> okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519) at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> I think this format of union schema should be supported in Java.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
