turcsanyip commented on pull request #5634: URL: https://github.com/apache/nifi/pull/5634#issuecomment-1006994817
@markap14 Thanks for implementing the changes in the processors. The merge processors now work properly both in traditional and stateless NiFi with simple flows. However, there is still an issue in the stateless framework related to the execution order of the processors which becomes apparent when you have multiple (at least 3) processors before the merge processor. In this case, the expected order of execution would be: - Proc1 - Proc2 - Proc3 - Merge (more input needed) - Proc1 - Proc2 - Proc3 - Merge (more input needed) - Proc1 - ... But the actual order is: - Proc1 - Proc2 - Proc3 - Merge (more input needed) - Proc1 - Proc2 - Merge (no new input as Proc3 has not called yet, outputs the single item received in the 1st round) - Proc3 (called in the wrong order) The root cause seems to be that although `AsynchronousCommitTracker` registers the ready components in the right order: https://github.com/apache/nifi/blob/56593ad12f40bbacd4627436eadc38a24f58ee16/nifi-stateless/nifi-stateless-bundle/nifi-stateless-engine/src/main/java/org/apache/nifi/stateless/session/AsynchronousCommitTracker.java#L47-L57 `StandardStatelessFlowCurrent` iterates over an earlier (and potentially outdated) snapshot of the ready components: https://github.com/apache/nifi/blob/56593ad12f40bbacd4627436eadc38a24f58ee16/nifi-stateless/nifi-stateless-bundle/nifi-stateless-engine/src/main/java/org/apache/nifi/stateless/flow/StandardStatelessFlowCurrent.java#L72-L92 (The set and/or order of the ready components may be changed in `triggerWhileReady()` within the for loop) My suggested solution would be to implement `Connectable getNextReady()` in `AsynchronousCommitTracker` instead of `List<Connectable> getReady()` and to call it from `StandardStatelessFlowCurrent` (and to remove the for loop). What is your opinion about it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
