[
https://issues.apache.org/jira/browse/KAFKA-21104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-21104:
-----------------------------------
Description:
The onPartitionsAssigned callback runs after the new assignment has been fully
applied on the client (both Classic and Consumer protocols), and with the
current behaviour it's just a notification triggered only once with the added
partitions, letting the app know that an assignment was applied.
If the onPartitionsAssigned callback fails, the error is propagated to the app
on the consumer.poll, but it's never retried with those partitions (retried
with empty assignment). Internally, the partitions are successfully assigned to
the consumer, but remain unfetchable due to the callback failure.
This would be a behaviour change for both consumer, so this task is only to
think about potential improvements here. Note that the onPartitionsRevoked
behaves differently, it is retried on every poll if it fails, receiving the
same partitions as parameters (the consumer remains rebalancing/reconciling, so
it's all triggered again on a next poll)
was:
The onPartitionsAssigned callback runs after the new assignment has been fully
applied on the client (both Classic and Consumer protocols), and with the
current behaviour it's just a notification triggered only once, letting the app
know that an assignment was applied.
If the onPartitionsAssigned callback fails, the error is propagated to the app
on the consumer.poll, but it's never retried. Internally, the partitions are
successfully assigned to the consumer, but remain unfetchable due to the
callback failure.
This would be a behaviour change for both consumer, so this task is only to
think about potential improvements here. Note that the onPartitionsRevoked
behaves differently, it is retried on every poll if it fails (the consumer
remains rebalancing/reconciling, so it's all triggered again on a next poll)
> Consider supporting retries on onPartitionsAssigned callback on failure
> -----------------------------------------------------------------------
>
> Key: KAFKA-21104
> URL: https://issues.apache.org/jira/browse/KAFKA-21104
> Project: Kafka
> Issue Type: Improvement
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Major
>
> The onPartitionsAssigned callback runs after the new assignment has been
> fully applied on the client (both Classic and Consumer protocols), and with
> the current behaviour it's just a notification triggered only once with the
> added partitions, letting the app know that an assignment was applied.
> If the onPartitionsAssigned callback fails, the error is propagated to the
> app on the consumer.poll, but it's never retried with those partitions
> (retried with empty assignment). Internally, the partitions are successfully
> assigned to the consumer, but remain unfetchable due to the callback failure.
> This would be a behaviour change for both consumer, so this task is only to
> think about potential improvements here. Note that the onPartitionsRevoked
> behaves differently, it is retried on every poll if it fails, receiving the
> same partitions as parameters (the consumer remains rebalancing/reconciling,
> so it's all triggered again on a next poll)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
