[jira] [Created] (NIFI-3861) AvroReader logical date type conversion does not work with nullable type

[Koji Kawamura (JIRA)]

Koji Kawamura created NIFI-3861:
-----------------------------------

             Summary: AvroReader logical date type conversion does not work 
with nullable type
                 Key: NIFI-3861
                 URL: https://issues.apache.org/jira/browse/NIFI-3861
             Project: Apache NiFi
          Issue Type: Bug
          Components: Extensions
            Reporter: Koji Kawamura
            Assignee: Koji Kawamura


A Date field can be defined as logical 'date' type in Avro schema like this, 
the value is stored as int physically, but logically annotated this is a date. 
The stored value represents days after Unix epoch (1970-01-01):

{code}
{"name":"date","type":{"type":"int","logicalType":"date"}}
{code}

When AvroReader reads the logical date value, it converts the int back to 
java.sql.Date at 
[AvroTypeUtil.normalizeValue|https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-extension-utils/nifi-record-utils/nifi-avro-record-utils/src/main/java/org/apache/nifi/avro/AvroTypeUtil.java#L441]
 method. However, if the field is also a nullable (union with null), the 
conversion is skipped and the original int is returned.

Then the int will be treated as milliseconds from Unix epoch by other 
RecordWriter such as JsonRecordSetWriter.

Here is a reproducible flow:

{code: title=input.json}
{"date": "2017-05-10", "time": "19:55:34"}
{code}

{code}
input.json
 -> ConvertRecord (JsonTreeReader/AvroRecordSetWriter)
 -> ConvertRecord (AvroRecordReader/JsonRecordSetWriter): this avro reader 
fails to convert with a nullable logical date.
{code}

{code: title=result-not_null}
[{"date":"2017-05-10","time":"19:55:34"}]
{code}

{code: title=result-nullable}
[{"date":"1970-01-01","time":"19:55:34"}]
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)