[
https://issues.apache.org/jira/browse/NIFI-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17014851#comment-17014851
]
ASF subversion and git services commented on NIFI-6998:
-------------------------------------------------------
Commit 2601f722b3230a640e86c83320a75b86288c0e60 in nifi's branch
refs/heads/master from Martin Šimek
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=2601f72 ]
NIFI-6998 This closes #3972. Batch & Partitioning key
+ Asynchronous sending in limited batches
+ Property to determine attribute name carrying partitioning key
+ Maximum batch size property
+ Carrier object
- Unit test fakery
NIFI-6998 Attributes to User Defined properties
NIFI-6998 Unit tests
NIFI-6998 Review corrections
+ Interruption propagation (& test)
+ Final carrier members
+ Unnecessary generic modifiers removed from generic container
NIFI-6998 checkstyle corrections
+ Tabs to spaces, trailnig spaces
+ Absolute Imports
+ Braces locations
NIFI-6998 imports & license
Signed-off-by: Joe Witt <[email protected]>
> PutAzureEventHub to utilize Batch, Properties & Partitioning key
> ----------------------------------------------------------------
>
> Key: NIFI-6998
> URL: https://issues.apache.org/jira/browse/NIFI-6998
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 1.8.0
> Reporter: Martin Šimek
> Priority: Major
> Attachments: EH_01_flow.png, EH_02_flowFile.png,
> EH_03_processor_settings.png, EH_04_EventHub.png, EH_05_receivedMessage.png
>
> Original Estimate: 72h
> Time Spent: 1h 40m
> Remaining Estimate: 70h 20m
>
> Current implementation of _PutAzureEventHub_ is too simplistic. It sends one
> flow file at a time synchronously which poses a significant bottlenec. It
> doesn't lift attributes into the message's _User defined properties_ section.
> And most important problem is that you can't specify the _partitioning key_
> which plays critical role in a delivery consistency across multiple
> partitions in case there are more partitions. Without partitioning key
> multiple use-cases can't choose NiFi as an option.
> Aim of this improvement is to consume up to specified amount of flow files,
> lift all _attributes_ of each flow file into _User defined properties_
> section of each message and let processor to use one of those attributes as a
> partitioning key.
> Each flow file in a batch is sent asynchronously which is way more faster
> then processor wait for all confirmations (or errors) and based on the result
> messages are transfered to success or failure output.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
