[
https://issues.apache.org/jira/browse/NIFI-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17933079#comment-17933079
]
Daniel Stieglitz commented on NIFI-14331:
-----------------------------------------
I realized there was a simpler fix and that is not to perform the deepCopy on
the incoming jsonNode which was preventing the overall JSON from being updated
(i.e. removing all undefined parts of the JSON).
> Unknown embedded fields not dropped by JSON Writer as expected by specified
> schema
> ----------------------------------------------------------------------------------
>
> Key: NIFI-14331
> URL: https://issues.apache.org/jira/browse/NIFI-14331
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 2.2.0
> Reporter: Daniel Stieglitz
> Assignee: Daniel Stieglitz
> Priority: Major
> Attachments: convertRecordResults.json, person.avsc,
> person_dropfield.json
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> NIFI-13843 was aimed to eliminate any fields found in the JSON which were not
> defined in a specifed Avro schema. While that fix seems to have solved the
> issue for top level items it did not solve the issue for an undefined key
> within a defined object and for an undefined key in a defined object for an
> array. Attached are the person.avsc Avro schema and the person_dropfield.json
> which includes undefined top level fields such as single key value pair
> ("undefinedKey"), array ("undefinedScalarArray"), object ("undefinedObject")
> and object array ("undefinedObjectArray"). It also includes undefined field
> ("undefinedKeyInObject") inside the defined "name" top level object and an
> undefined field ("undefinedKeyInObject") in a "job" object found in the
> "jobs" array. The result after calling ConvertRecord can be seen in the
> attached convertRecordResults.json. Note fields "undefinedKey",
> "undefinedScalarArray", "undefinedObject" and "undefinedObjectArray" all get
> dropped while fields "undefinedKeyInObject" still exist in the "name" object
> and the "job" object inside the "jobs" array.
> Currently this behavior is seen in both ConvertRecord and MergeRecord when
> both are configured with a JsonTreeReader and JsonRecordSetWriter.
> It is interesting to note this behavior is seen in NIFI 1.28.1 only for
> MergeRecord while ConvertRecord drops all unknown fields.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
