David Handermann created NIFI-16605:
---------------------------------------

             Summary: Add per-event end position attributes to 
ConsumeAzureEventHub
                 Key: NIFI-16605
                 URL: https://issues.apache.org/jira/browse/NIFI-16605
             Project: Apache NiFi
          Issue Type: Improvement
          Components: Extensions
            Reporter: David Handermann
            Assignee: David Handermann


ConsumeAzureEventHub currently writes {{eventhub.enqueued.timestamp}}, 
{{eventhub.offset}}, and {{eventhub.sequence}} from Azure 
{{LastEnqueuedEventProperties}}, which describe the newest event in the 
partition rather than the consumed event. This change should keep those 
existing attributes unchanged and add 
{{eventhub.event.end.enqueued.timestamp}}, {{eventhub.event.end.offset}}, and 
{{eventhub.event.end.sequence}} from EventData. For one FlowFile per event, 
those names come from that event. For a record-oriented batch, they come from 
the last successfully parsed event in the batch. Parse-failure FlowFiles should 
receive a dedicated attribute map so they do not share state with the combined 
success FlowFile.

The processor documentation already describes the current names as applying to 
the consumed message, and GetAzureEventHub already writes per-event positions, 
so ConsumeAzureEventHub has no stable way to expose an event’s own sequence, 
offset, or enqueue time. Using the {{eventhub.event.end}} prefix for both 
output modes keeps a single attribute contract, preserves existing flows that 
depend on last-enqueued partition values, and provides the last consumed 
event’s position for ordering, correlation, and checkpointing.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to