[
https://issues.apache.org/jira/browse/KAFKA-19044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lucas Brutschy updated KAFKA-19044:
-----------------------------------
Description:
A heartbeat might be sent to the group coordinator, claiming to own tasks that
we do not know about. This may happen for a variety of reasons. We need some
logic to handle those requests. In KIP-1071, we propose to return
`INVALID_REQUEST` error whenever this happens, effectively letting the clients
crash. This is safe, and may be a valid approach for the first version of the
protocol.
This behavior will, however, make topology updates impossible. [~cadonna]
proposed to only check that owned tasks match our set of expected tasks if the
topology epochs between the group and the client match. The aim of this ticket
is to implement a check and a behavior for the first version of the protocol,
which is to always return `INVALID_REQUEST` if an unknown task is sent to the
group coordinator.
We can relax this constraint once we allow topology updating with topology
epochs.
was:
A heartbeat might be sent to the group coordinator, claiming to own tasks that
we do not know about. This may happen for a variety of reasons. We need some
logic to handle those requests. In KIP-1071, we propose to return
`INVALID_REQUEST` error whenever this happens, effectively letting the clients
crash. This is safe, and may be a valid approach for the first version of the
protocol.
This behavior will, however, make topology updates impossible. [~cadonna]
proposed to only check that owned tasks match our set of expected tasks if the
topology epochs between the group and the client match.
Long term, I am wondering if it isn't a better course of action to just ignore
tasks that we do not expect. After all, the client is just saying what he
thinks he owns. If we'd just ignore those tasks, the next target assignment
would not include the tasks, so the client would automatically be instructed to
revoke whatever tasks it thinks it currently has.
The aim of this ticket is to implement a check and a behavior for the first
version of the protocol, and create follow-up ticket(s) for any extensions for
partition scaling and topology updating that we may want to add.
> Handle tasks that are not present in the current topology
> ---------------------------------------------------------
>
> Key: KAFKA-19044
> URL: https://issues.apache.org/jira/browse/KAFKA-19044
> Project: Kafka
> Issue Type: Sub-task
> Components: streams
> Reporter: Lucas Brutschy
> Priority: Major
>
> A heartbeat might be sent to the group coordinator, claiming to own tasks
> that we do not know about. This may happen for a variety of reasons. We need
> some logic to handle those requests. In KIP-1071, we propose to return
> `INVALID_REQUEST` error whenever this happens, effectively letting the
> clients crash. This is safe, and may be a valid approach for the first
> version of the protocol.
> This behavior will, however, make topology updates impossible. [~cadonna]
> proposed to only check that owned tasks match our set of expected tasks if
> the topology epochs between the group and the client match. The aim of this
> ticket is to implement a check and a behavior for the first version of the
> protocol, which is to always return `INVALID_REQUEST` if an unknown task is
> sent to the group coordinator.
> We can relax this constraint once we allow topology updating with topology
> epochs.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
