[
https://issues.apache.org/jira/browse/AVRO-3224?focusedWorklogId=704513&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-704513
]
ASF GitHub Bot logged work on AVRO-3224:
----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Jan/22 12:05
Start Date: 06/Jan/22 12:05
Worklog Time Spent: 10m
Work Description: liu-du commented on a change in pull request #1355:
URL: https://github.com/apache/avro/pull/1355#discussion_r779481230
##########
File path:
lang/java/avro/src/main/java/org/apache/avro/reflect/ReflectDatumReader.java
##########
@@ -287,6 +288,26 @@ protected void readField(Object record, Field field,
Object oldDatum, ResolvingD
return;
}
}
+
+ if (field.schema().isUnion()) {
Review comment:
I like the idea of adding target type property to the schema and
overriding the `GenericDatumReader#read` method in `ReflectDatumReader`. I
tried it by adding a property `SpecificData.CLASS_PROP` (java-class) indicating
java class for logical types (it seems CLASS_PROP is the conventional property
name used to indicate java class in the code base). It broke a few unit tests
but they're easy to fix.
It almost worked but when I tried fixing the last broken unit test
`TestReflectLogicalTypes.testReflectedSchemaLocalDateTime`, I got the following
exception:
Although I was able to add java-class property to schema with a logical
type, it seems some code assumes logical types should not be used with
java-class. PS: The code changes are here, though I'm no longer sure we should
go this way:
https://github.com/liu-du/avro/commit/6320aae25b6f3c8f10d48b6b4e0fd31d03ebd1bd .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 704513)
Time Spent: 1h 40m (was: 1.5h)
> Wrong deserialisation when using ReflectData.AllowNull with custom
> LogicalType conversions
> ------------------------------------------------------------------------------------------
>
> Key: AVRO-3224
> URL: https://issues.apache.org/jira/browse/AVRO-3224
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.10.2
> Reporter: Du Liu
> Priority: Minor
> Labels: pull-request-available
> Attachments: ReflectData.AllowNull.get doesnt work.png,
> ReflectData.get works.png, class_and_conversion.png, tests.txt
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> If a class contains both a java.time.LocalDate field and a java.sql.Date
> field, serialising an instance of the class using ReflectData.*AllowNull*
> with two conversions that both convert to avro date logical type (e.g. one
> converts java.time.LocalDate to avro date, the other converts java.sql.Date
> to avro date) is done correctly, but when deserialising the data back, the
> LocalDate field of the deserialised instance incorrectly contains a
> java.sql.Date (or the Date field incorrectly contains a LocalDate depending
> on which conversions is added last).
> This only happens when using ReflectData.*AllowNull*.get(). ReflectData.get()
> can correctly deserialised data back. I think this issue affects other types
> as long as two java types are mapped to the same avro logical type.
> A working example (using ReflectData.get()):
> !ReflectData.get works.png|width=887,height=379!
>
> A not working example (using ReflectData.AllowNull.get()):
> !ReflectData.AllowNull.get doesnt work.png|width=831,height=469!
> The class and conversion:
> !class_and_conversion.png|width=798,height=687!
>
> These tests are included in the attached patch file: [^tests.txt]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
