[
https://issues.apache.org/jira/browse/NIFI-8130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alessandro D'Armiento updated NIFI-8130:
----------------------------------------
Description:
This bug is hard to replicate as it happens randomly.
In the following (common) configuration, in which multiple records are merged
and then sent to a PutDatabaseRecord, it happens sometimes that a specific
FlowFile cause the PutDatabaseRecord to fail with `FlowFileHandlingException:
FlowFile already marked for transfer`
!Screenshot 2021-01-11 at 11.38.45.png!
!Screenshot 2021-01-11 at 11.34.06.png!
In case of such an event, the processor remains stuck trying to process that
specific FlowFile (i.e. it is not routed to the failure relationship). This
forces the user to empty the whole queue in order to continue, which causes
data loss.
I noticed the following:
* The issue is bound with the FlowFile: the same FlowFile will make multiple
processors to fail with the same error.
* Creating a new FlowFile with the same content (i.e. publishing the FlowFile
on a Kafka queue and consuming it right after) doesn't solve the issue, and the
FlowFile will raise the error again once sent to the PutDatabaseRecord
* This error happened to me only when using the PutDatabaseRecord after a
MergeRecord (in order to batch multiple records in a single DB transaction).
* This issue was already raised in the [Cloudera Community
Forum|https://community.cloudera.com/t5/Support-Questions/quot-is-already-marked-for-transfer-quot-in/td-p/236588],
alas, without any answer.
was:
This bug is hard to replicate as it happens randomly.
In the following (common) configuration, in which multiple records are merged
and then sent to a PutDatabaseRecord, it happens sometimes that a specific
FlowFile cause the PutDatabaseRecord to fail with `FlowFileHandlingException:
FlowFile already marked for transfer`
!Screenshot 2021-01-11 at 11.38.45.png!
!Screenshot 2021-01-11 at 11.34.06.png!
In case of such an event, the processor remains stuck trying to process that
specific FlowFile (i.e. it is not routed to the failure relationship). This
forces the user to empty the whole queue in order to continue, which causes
data loss.
I noticed the following:
* The issue is bound with the FlowFile: the same FlowFile will make multiple
processors to fail with the same error.
* Creating a new FlowFile with the same content (i.e. publishing the FlowFile
on a Kafka queue and consuming it right after) doesn't solve the issue, and the
FlowFile will raise the error again once sent to the PutDatabaseRecord
* This error happened to me only when using the PutDatabaseRecord after a
MergeRecord (in order to batch multiple records in a single DB transaction).
* This issue was already raised in the [Cloudera Community
Forum|https://community.cloudera.com/t5/Support-Questions/quot-is-already-marked-for-transfer-quot-in/td-p/236588],
alas, without any answer.
> PutDatabaseRecord after MergeRecord randomly hangs forcing to discard the
> whole queue
> -------------------------------------------------------------------------------------
>
> Key: NIFI-8130
> URL: https://issues.apache.org/jira/browse/NIFI-8130
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Alessandro D'Armiento
> Priority: Major
> Attachments: Screenshot 2021-01-11 at 11.34.06.png, Screenshot
> 2021-01-11 at 11.38.45.png
>
>
> This bug is hard to replicate as it happens randomly.
> In the following (common) configuration, in which multiple records are merged
> and then sent to a PutDatabaseRecord, it happens sometimes that a specific
> FlowFile cause the PutDatabaseRecord to fail with `FlowFileHandlingException:
> FlowFile already marked for transfer`
> !Screenshot 2021-01-11 at 11.38.45.png!
> !Screenshot 2021-01-11 at 11.34.06.png!
> In case of such an event, the processor remains stuck trying to process that
> specific FlowFile (i.e. it is not routed to the failure relationship). This
> forces the user to empty the whole queue in order to continue, which causes
> data loss.
> I noticed the following:
> * The issue is bound with the FlowFile: the same FlowFile will make multiple
> processors to fail with the same error.
> * Creating a new FlowFile with the same content (i.e. publishing the
> FlowFile on a Kafka queue and consuming it right after) doesn't solve the
> issue, and the FlowFile will raise the error again once sent to the
> PutDatabaseRecord
> * This error happened to me only when using the PutDatabaseRecord after a
> MergeRecord (in order to batch multiple records in a single DB transaction).
> * This issue was already raised in the [Cloudera Community
> Forum|https://community.cloudera.com/t5/Support-Questions/quot-is-already-marked-for-transfer-quot-in/td-p/236588],
> alas, without any answer.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
