Lucas Brutschy created KAFKA-21370:
--------------------------------------
Summary: Return STREAMS_INVALID_TOPOLOGY_EPOCH instead of
INVALID_REQUEST for topology epoch mismatches in streams groups
Key: KAFKA-21370
URL: https://issues.apache.org/jira/browse/KAFKA-21370
Project: Kafka
Issue Type: Improvement
Components: streams
Affects Versions: 4.2.2, 4.4.0, 4.3.1
Reporter: Lucas Brutschy
Assignee: Lucas Brutschy
KIP-1071 defines the error code STREAMS_INVALID_TOPOLOGY_EPOCH (131) for a
member that sends a topology that doesn't match the group's topology at the
same epoch. The error code and StreamsInvalidTopologyEpochException exist, and
StreamsGroupHeartbeatResponse lists the error as supported, but the group
coordinator never returns it.
In GroupMetadataManager#maybeUpdateTopology, when the group already has a
topology and the request carries a topology with an equal or higher epoch that
differs from the stored one, the coordinator throws
InvalidRequestException("Topology updates are not supported yet."). The
same-epoch case should return STREAMS_INVALID_TOPOLOGY_EPOCH. It means two
instances of the application are running different topologies under the same
epoch, and a specific error helps users diagnose it. The higher-epoch case is
an actual topology update, which is not yet supported, and can keep returning
INVALID_REQUEST.
The KIP-1331 topology description push has the same problem.
GroupMetadataManager#validateStreamsGroupTopologyDescriptionUpdate throws
InvalidRequestException when the pushed epoch doesn't match the group's current
topology epoch. It should throw StreamsInvalidTopologyEpochException, and
StreamsGroupTopologyDescriptionUpdateResponse.json should list
STREAMS_INVALID_TOPOLOGY_EPOCH among its supported errors. The mismatch can
happen in normal operation, when the epoch is bumped between the heartbeat that
requested the push and the push itself. So the client should log it below warn
level.
Clients need no protocol changes and behave the same way.
StreamsGroupHeartbeatRequestManager already treats
STREAMS_INVALID_TOPOLOGY_EPOCH as fatal, just like INVALID_REQUEST.
StreamsGroupTopologyDescriptionRequestManager handles it in its default branch,
which drops the push without retrying, the same as for INVALID_REQUEST; we
should add an explicit case there. GroupCoordinatorService already maps the
exception to code 131 through Errors.forException.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
