[
https://issues.apache.org/jira/browse/NIFI-7773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmytro Kulyk updated NIFI-7773:
-------------------------------
Attachment: Sportsbook (2).json
Description:
Using `JoltTransformJSON` to transform kafka JSON messages before making an
update on Postgres DB
In some rare cases messages (received from same topic partition) are "mixing"
up an order within a node
Therefore updates are invoked in wrong order, affecting on data consistency
and correctness
*Provenance* logs for some found cases:
!image-2020-08-28-13-06-03-269.png|width=580,height=382!
!image-2020-08-28-13-06-20-964.png|width=571,height=327!
*Flow* (mentioned "flow pipeline" marked with yellow):
!image-2020-08-28-13-07-39-001.png|width=567,height=303!
Flow code has been attached
was:
Using `JoltTransformJSON` to transform kafka JSON messages before making an
update on Postgres DB
In some rare cases messages (received from same topic partition) are "mixing"
up an order within a node
Therefore updates are invoked in wrong order, affecting on data consistency and
correctness
Provenance logs for some found cases:
!image-2020-08-28-13-06-03-269.png!
!image-2020-08-28-13-06-20-964.png!
Flow:
!image-2020-08-28-13-07-39-001.png!
> JoltTransformJSON mixing up flow files order
> --------------------------------------------
>
> Key: NIFI-7773
> URL: https://issues.apache.org/jira/browse/NIFI-7773
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.11.4
> Environment: Ubuntu 16 x64, 3 nodes cluster
> Reporter: Dmytro Kulyk
> Priority: Critical
> Labels: jolt, order
> Attachments: Sportsbook (2).json, image-2020-08-28-13-06-03-269.png,
> image-2020-08-28-13-06-20-964.png, image-2020-08-28-13-07-39-001.png
>
>
> Using `JoltTransformJSON` to transform kafka JSON messages before making an
> update on Postgres DB
> In some rare cases messages (received from same topic partition) are "mixing"
> up an order within a node
> Therefore updates are invoked in wrong order, affecting on data consistency
> and correctness
> *Provenance* logs for some found cases:
> !image-2020-08-28-13-06-03-269.png|width=580,height=382!
> !image-2020-08-28-13-06-20-964.png|width=571,height=327!
> *Flow* (mentioned "flow pipeline" marked with yellow):
> !image-2020-08-28-13-07-39-001.png|width=567,height=303!
> Flow code has been attached
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
