Peter Turcsanyi created NIFI-12306:
--------------------------------------
Summary: ConsumeAzureEventHub should not log partition ownership
changes at error level
Key: NIFI-12306
URL: https://issues.apache.org/jira/browse/NIFI-12306
Project: Apache NiFi
Issue Type: Improvement
Reporter: Peter Turcsanyi
Assignee: Peter Turcsanyi
Azure Event Hub client instances run a load balancing algorithm which checks
the distribution of the partitions among the consumer group members and can
rebalance the partition assignments.
This algorithm runs independently on each client node and if the node owns less
partitions than the fair share (typically after start-up), it "steals" a
partition from another node.
The other node only recognizes it via the broken connection (2 consumers from
the same consumer group cannot consume the same partition at the same time) so
basically it is an error event on that side. It seems it is by design and will
not change in the Azure client lib.
However, we can handle this type of "error" differently on the NiFi side and do
not log it at error level but only info. The current error bulletin is quite
confusing for the user.
Error message:
{code:java}
2023-11-01 13:19:46,613 ERROR [partition-pump-1-22]
o.a.n.p.a.eventhub.ConsumeAzureEventHub
ConsumeAzureEventHub[id=7d57cb75-018b-1000-0000-000042a1ec17] Receive Events
failed Namespace [turcsanyi-eventhub-ns.servicebus.windows.net] Event Hub
[turcsanyi-eventhub-2] Consumer Group [$default] Partition [1]
com.azure.core.amqp.exception.AmqpException: New receiver
'7661792b-acf1-49b8-9474-2ac06ebc22d9' with higher epoch of '0' is created
hence current receiver '8bec0a38-2d58-46d1-ac37-4cb80bd7ba29' with epoch '0' is
getting disconnected. If you are recreating the receiver, make sure a higher
epoch is used. TrackingId:ddb15716000068bf0010179a6542425c_G13S3_B13S1,
SystemTracker:turcsanyi-eventhub-ns:eventhub:turcsanyi-eventhub-2~16383|$default,
Timestamp:2023-11-01T12:19:46, errorContext[NAMESPACE:
turcsanyi-eventhub-ns.servicebus.windows.net. ERROR CONTEXT: N/A, PATH:
turcsanyi-eventhub-2/ConsumerGroups/$default/Partitions/1, REFERENCE_ID:
1_b1a39c_1698841180105, LINK_CREDIT: 300]
at
com.azure.core.amqp.implementation.ExceptionUtil.toException(ExceptionUtil.java:85)
at
com.azure.core.amqp.implementation.handler.LinkHandler.handleRemoteLinkClosed(LinkHandler.java:124)
...
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
