[ https://issues.apache.org/jira/browse/NIFI-190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joseph Gresock updated NIFI-190: -------------------------------- Description: Our team has developed a set of processors for the following use case: * Format A needs to be sent to Endpoint A * Format B needs to be sent to Endpoint B, but should not proceed until A has reached Endpoint A. We most commonly have this restriction when Endpoint B requires some output of Endpoint A. The proposed Wait/Notify processors enable this functionality: * Wait: routes files to the 'wait' relationship until a matching Release Signal Identifier is found in the distributed map cache. Then routes them to 'success' (unless they have expired) * Notify: stores a Release Signal Identifier in the distributed map cache, optionally with attributes to copy to the outgoing matching Wait flow files. An example: Wait is configured with Release Signal Attribute = "$\{myId}". Its 'wait' relationship routes back onto itself. flowFile 1 \{ myId : "123" } comes into Wait processor Wait checks the distributed cache map for "123", doesn't find it, and is routed to the 'wait' relationship Notify is configured with Release Signal Attribute = "$\{myId}" flowFile 2 \{ myId : "123" } comes in to Notify processor Notify puts an entry in the map for "123" with any other attributes from flowFile2 Next time flowFile 1 is processed by Wait... Finds an entry for "123" Removes that entry from the map Copies attributes to flowFile 1 Sends flowFile 1 out the success relationship Notify will optionally cache attributes in the distributed map, as determined by a regex property. This is what allows the output of Endpoint A to pass to Endpoint B, above. Wait also allows conflicting attributes from the cache to either be replaced or kept, depending on property configuration. was: Our team has developed a set of processors for the following use case: * Format A needs to be sent to Endpoint A * Format B needs to be sent to Endpoint B, but should not proceed until A has reached Endpoint A. We most commonly have this restriction when Endpoint B requires some output of Endpoint A. The proposed Wait/Notify processors enable this functionality: * Wait: routes files to the 'wait' relationship until a matching Release Signal Identifier is found in the distributed map cache. Then routes them to 'success' (unless they have expired) * Notify: stores a Release Signal Identifier in the distributed map cache, optionally with attributes to copy to the outgoing matching Wait flow files. An example: Wait is configured with Release Signal Attribute = "${myId}". Its 'wait' relationship routes back onto itself. flowFile 1 { myId : "123" } comes into Wait processor Wait checks the distributed cache map for "123", doesn't find it, and is routed to the 'wait' relationship Notify is configured with Release Signal Attribute = "${myId}" flowFile 2 { myId : "123" } comes in to Notify processor Notify puts an entry in the map for "123" with any other attributes from flowFile2 Next time flowFile 1 is processed by Wait... Finds an entry for "123" Removes that entry from the map Copies attributes to flowFile 1 Sends flowFile 1 out the success relationship Notify will optionally cache attributes in the distributed map, as determined by a regex property. This is what allows the output of Endpoint A to pass to Endpoint B, above. Wait also allows conflicting attributes from the cache to either be replaced or kept, depending on property configuration. > Wait/Notify processors > ---------------------- > > Key: NIFI-190 > URL: https://issues.apache.org/jira/browse/NIFI-190 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions > Reporter: Joseph Gresock > Assignee: Joseph Gresock > Priority: Minor > Fix For: 1.2.0 > > Attachments: Wait_Notify_template.xml > > > Our team has developed a set of processors for the following use case: > * Format A needs to be sent to Endpoint A > * Format B needs to be sent to Endpoint B, but should not proceed until A has > reached Endpoint A. We most commonly have this restriction when Endpoint B > requires some output of Endpoint A. > The proposed Wait/Notify processors enable this functionality: > * Wait: routes files to the 'wait' relationship until a matching Release > Signal Identifier is found in the distributed map cache. Then routes them to > 'success' (unless they have expired) > * Notify: stores a Release Signal Identifier in the distributed map cache, > optionally with attributes to copy to the outgoing matching Wait flow files. > An example: > Wait is configured with Release Signal Attribute = "$\{myId}". Its 'wait' > relationship routes back onto itself. > flowFile 1 \{ myId : "123" } > comes into Wait processor > Wait checks the distributed cache map for "123", doesn't find it, and is > routed to the 'wait' relationship > Notify is configured with Release Signal Attribute = "$\{myId}" > flowFile 2 \{ myId : "123" } > comes in to Notify processor > Notify puts an entry in the map for "123" with any other attributes from > flowFile2 > Next time flowFile 1 is processed by Wait... > Finds an entry for "123" > Removes that entry from the map > Copies attributes to flowFile 1 > Sends flowFile 1 out the success relationship > Notify will optionally cache attributes in the distributed map, as determined > by a regex property. This is what allows the output of Endpoint A to pass to > Endpoint B, above. Wait also allows conflicting attributes from the cache to > either be replaced or kept, depending on property configuration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)