Dariusz Seweryn created NIFI-14753:
--------------------------------------
Summary: JsonRecordSetWriter leaves a FlowFile in inconsistent
state after trying to write an incompatible schema record
Key: NIFI-14753
URL: https://issues.apache.org/jira/browse/NIFI-14753
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 2.4.0
Reporter: Dariusz Seweryn
h1. Context
Consider a JsonRecordSetWriter for schema ["Int Field Name": INT].
Writing a record with ["Int Field Name": Long.MAX_VALUE] throws an
IllegalTypeConversionException — this is expected. Unfortunately the FlowFile
is then left with content that is not a valid JSON:
{ "Int Field Name" }
h1. IllegalTypeConversionException handling
This situation has the following consequences for exception handling apart from
handling the offending record in the processor that tried to write the record:
The same malformed record will most probably surface/cause a warning log in a
following processor which can be surprising without the knowledge of this
behavior in JsonRecordSetWriter (unless all other processors assume that
Malformed Records may appear in consumed FlowFiles and silently drop such
records). {_}This can lead to unnecessary work being wasted by people triaging
such warning logs{_}.
h1. Ideal solution
JsonRecordSetWriter in case of IllegalTypeConversionException does not write
any data to the FlowFile.
Such approach would need to buffer all writes to the FlowFile until full record
is processed, executing them afterwards. This may impact performance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
