Jinge Dai created AVRO-2831:
-------------------------------
Summary: Resolver cannot find reader/writer schema difference of
fields in Union
Key: AVRO-2831
URL: https://issues.apache.org/jira/browse/AVRO-2831
Project: Apache Avro
Issue Type: Bug
Components: java
Affects Versions: 1.9.2, 1.9.1, 1.9.0
Reporter: Jinge Dai
When avro java library deserializes and tries to resolve fields in a Union, it
will use the writer schema if the reader field type is same as writer's, but it
ignores the properties and logical type.
{code:java}
private static boolean unionEquiv(Schema write, Schema read, Map<SeenPair,
Boolean> seen) {
final Schema.Type wt = write.getType();
if (wt != read.getType()) {
return false;
}
{code}
However, I have hit two issues:
# https://issues.apache.org/jira/browse/AVRO-2702 If producer is not java, the
string field the strings of the inner record will be deserialized to
org.apache.avro.util.Utf8 and then get error
{code:java}
Caused by: java.lang.ClassCastException: class org.apache.avro.util.Utf8 cannot
be cast to class java.lang.String{code}
2. I am using a built-in-house C# lib on the producer side, and it builds Date
field schema as INT type without logicalTyple, this is bug on my side. However,
AVRO 1.8.2 is able to handle it by using reader schema, whereas 1.9.2 gets
error
{code:java}
Caused by: java.lang.IllegalArgumentException: Parameters cannot be null!
Parameter values:[-129, "int", null,
org.apache.avro.data.TimeConversions$DateConversion@d3449b1]
{code}
The reason is, although the consumer side generates the class properly and
sets the field to be converted to logical type, but the convertToLogicalType
failed because the writer schema is used and the logical type is null
{code:java}
public static Object convertToLogicalType(Object datum, Schema schema,
LogicalType type, Conversion<?> conversion) {
if (datum == null) {
return null;
}
if (schema == null || type == null || conversion == null) {
throw new IllegalArgumentException("Parameters cannot be null! Parameter
values:"
+ Arrays.deepToString(new Object[] { datum, schema, type, conversion
}));
}
{code}
I am wondering if we should check type and properties in "unionEquiv" so that
we can use the reader schema if there is difference. This complies the AVRO spec
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
