[
https://issues.apache.org/jira/browse/NIFI-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joe Witt resolved NIFI-6998.
----------------------------
Resolution: Fixed
Thanks for the Contrib Martin.
Thanks for the reviews Pierre and Shayne.
+1 merged to master after local build verification and travis all clear on 8
and 11.
> PutAzureEventHub to utilize Batch, Properties & Partitioning key
> ----------------------------------------------------------------
>
> Key: NIFI-6998
> URL: https://issues.apache.org/jira/browse/NIFI-6998
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.8.0
> Reporter: Martin Šimek
> Priority: Major
> Fix For: 1.11.0
>
> Attachments: EH_01_flow.png, EH_02_flowFile.png,
> EH_03_processor_settings.png, EH_04_EventHub.png, EH_05_receivedMessage.png
>
> Original Estimate: 72h
> Time Spent: 1h 40m
> Remaining Estimate: 70h 20m
>
> Current implementation of _PutAzureEventHub_ is too simplistic. It sends one
> flow file at a time synchronously which poses a significant bottlenec. It
> doesn't lift attributes into the message's _User defined properties_ section.
> And most important problem is that you can't specify the _partitioning key_
> which plays critical role in a delivery consistency across multiple
> partitions in case there are more partitions. Without partitioning key
> multiple use-cases can't choose NiFi as an option.
> Aim of this improvement is to consume up to specified amount of flow files,
> lift all _attributes_ of each flow file into _User defined properties_
> section of each message and let processor to use one of those attributes as a
> partitioning key.
> Each flow file in a batch is sent asynchronously which is way more faster
> then processor wait for all confirmations (or errors) and based on the result
> messages are transfered to success or failure output.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
