[ 
https://issues.apache.org/jira/browse/NIFI-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Handermann updated NIFI-16605:
------------------------------------
    Status: Patch Available  (was: Open)

> Add per-event end position attributes to ConsumeAzureEventHub
> -------------------------------------------------------------
>
>                 Key: NIFI-16605
>                 URL: https://issues.apache.org/jira/browse/NIFI-16605
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: David Handermann
>            Assignee: David Handermann
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> ConsumeAzureEventHub currently writes {{eventhub.enqueued.timestamp}}, 
> {{eventhub.offset}}, and {{eventhub.sequence}} from Azure 
> {{LastEnqueuedEventProperties}}, which describe the newest event in the 
> partition rather than the consumed event. This change should keep those 
> existing attributes unchanged and add 
> {{eventhub.event.end.enqueued.timestamp}}, {{eventhub.event.end.offset}}, and 
> {{eventhub.event.end.sequence}} from EventData. For one FlowFile per event, 
> those names come from that event. For a record-oriented batch, they come from 
> the last successfully parsed event in the batch. Parse-failure FlowFiles 
> should receive a dedicated attribute map so they do not share state with the 
> combined success FlowFile.
> The processor documentation already describes the current names as applying 
> to the consumed message, and GetAzureEventHub already writes per-event 
> positions, so ConsumeAzureEventHub has no stable way to expose an event’s own 
> sequence, offset, or enqueue time. Using the {{eventhub.event.end}} prefix 
> for both output modes keeps a single attribute contract, preserves existing 
> flows that depend on last-enqueued partition values, and provides the last 
> consumed event’s position for ordering, correlation, and checkpointing.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to