David Handermann created NIFI-16605:
---------------------------------------
Summary: Add per-event end position attributes to
ConsumeAzureEventHub
Key: NIFI-16605
URL: https://issues.apache.org/jira/browse/NIFI-16605
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: David Handermann
Assignee: David Handermann
ConsumeAzureEventHub currently writes {{eventhub.enqueued.timestamp}},
{{eventhub.offset}}, and {{eventhub.sequence}} from Azure
{{LastEnqueuedEventProperties}}, which describe the newest event in the
partition rather than the consumed event. This change should keep those
existing attributes unchanged and add
{{eventhub.event.end.enqueued.timestamp}}, {{eventhub.event.end.offset}}, and
{{eventhub.event.end.sequence}} from EventData. For one FlowFile per event,
those names come from that event. For a record-oriented batch, they come from
the last successfully parsed event in the batch. Parse-failure FlowFiles should
receive a dedicated attribute map so they do not share state with the combined
success FlowFile.
The processor documentation already describes the current names as applying to
the consumed message, and GetAzureEventHub already writes per-event positions,
so ConsumeAzureEventHub has no stable way to expose an event’s own sequence,
offset, or enqueue time. Using the {{eventhub.event.end}} prefix for both
output modes keeps a single attribute contract, preserves existing flows that
depend on last-enqueued partition values, and provides the last consumed
event’s position for ordering, correlation, and checkpointing.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)