Mark Payne created NIFI-2220:
--------------------------------
Summary: PutAzureEventHub should send events asynchronously and
then wait for completion to improve efficiency
Key: NIFI-2220
URL: https://issues.apache.org/jira/browse/NIFI-2220
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Affects Versions: 1.0.0
Reporter: Mark Payne
In the PutAzureEventHub processor, we send each individual FlowFile
synchronously as an individual message. We can improve efficiency by sending a
batch of some user-configurable number (default to say 100) by calling send()
and stashing away the Future that it provides and then, after sending all of
the FlowFiles in the batch calling get() on the futures. This allows us to be
sending Event #2, 3, 4, etc. while we are still waiting on the acknowledgement
for Event #1.
We would have to ensure that we are able to keep together the FlowFile with the
Event that was sent so that if any fails to send, we can properly route it to
'failure'.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
