[
https://issues.apache.org/jira/browse/NIFI-8130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275444#comment-17275444
]
Matt Burgess commented on NIFI-8130:
------------------------------------
Is there anything in the logs before/after the "already marked for transfer"
entry? Trying to figure out how a flow file can get transferred and then
something goes wrong (where we'd try to also send it to failure)
> PutDatabaseRecord after MergeRecord randomly hangs forcing to discard the
> whole queue
> -------------------------------------------------------------------------------------
>
> Key: NIFI-8130
> URL: https://issues.apache.org/jira/browse/NIFI-8130
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Alessandro D'Armiento
> Priority: Major
> Attachments: Screenshot 2021-01-11 at 11.34.06.png, Screenshot
> 2021-01-11 at 11.38.45.png
>
>
> This bug is hard to replicate as it happens randomly.
> In the following (common) configuration, in which multiple records are merged
> and then sent to a PutDatabaseRecord, it happens sometimes that a specific
> FlowFile cause the PutDatabaseRecord to fail with `FlowFileHandlingException:
> FlowFile already marked for transfer`
> !Screenshot 2021-01-11 at 11.38.45.png!
> !Screenshot 2021-01-11 at 11.34.06.png!
> In case of such an event, the processor remains stuck trying to process that
> specific FlowFile (i.e. it is not routed to the failure relationship). This
> forces the user to empty the whole queue in order to continue, which causes
> data loss.
> I noticed the following:
> * The issue is bound with the FlowFile: the same FlowFile will make multiple
> processors to fail with the same error.
> * Creating a new FlowFile with the same content (i.e. publishing the
> FlowFile on a Kafka queue and consuming it right after) doesn't solve the
> issue, and the FlowFile will raise the error again once sent to the
> PutDatabaseRecord
> * This error happened to me only when using the PutDatabaseRecord after a
> MergeRecord (in order to batch multiple records in a single DB transaction).
> * This issue was already raised in the [Cloudera Community
> Forum|https://community.cloudera.com/t5/Support-Questions/quot-is-already-marked-for-transfer-quot-in/td-p/236588],
> alas, without any answer.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
