heldergr opened a new issue, #18067:
URL: https://github.com/apache/iceberg/issues/18067
### Query engine
_No response_
### Question
I have encountered an issue with the Iceberg connector for Kafka Connect
that I haven't been able to figure out how to resolve.
When I start the connector from a completely clean state (with the Kafka
Connect cluster configuration, connector status, offsets, and the
control/source topics all empty), it works as expected.
However, if the task encounters an error and stops working, I cannot get it
to recover afterward. Even restarting the Kafka Connect cluster (running in a
Kubernetes pod) does not resolve the issue. The task continues to log the
following messages:
```text
[2026-09-11 14:03:52,943] INFO [connector-name|task-0] [Consumer
clientId=45e600cb-3e05-4461-97b7-e88e4cbe7237,
groupId=group-id-prefix779a2a31-e1c0-44d7-a0e7-1cd777609fa4] JoinGroup failed:
The coordinator is not aware of this member. Need to re-join the group. Sent
generation was Generation{generationId=-1,
memberId='45e600cb-3e05-4461-97b7-e88e4cbe7237-1f4d28a5-c66a-4a92-93fd-2fe4bb187ae8',
protocol='null'}
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator:689)
[2026-09-11 14:03:52,943] INFO [connector-name|task-0] [Consumer
clientId=45e600cb-3e05-4461-97b7-e88e4cbe7237,
groupId=group-id-prefix779a2a31-e1c0-44d7-a0e7-1cd777609fa4] Resetting
generation and member id due to: encountered UNKNOWN_MEMBER_ID from JOIN_GROUP
response (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator:1062)
[2026-09-11 14:03:52,943] INFO [connector-name|task-0] [Consumer
clientId=45e600cb-3e05-4461-97b7-e88e4cbe7237,
groupId=group-id-prefix779a2a31-e1c0-44d7-a0e7-1cd777609fa4] Request joining
group due to: encountered UNKNOWN_MEMBER_ID from JOIN_GROUP response
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator:1109)
[2026-09-11 14:03:52,943] INFO [connector-name|task-0] [Consumer
clientId=45e600cb-3e05-4461-97b7-e88e4cbe7237,
groupId=group-id-prefix779a2a31-e1c0-44d7-a0e7-1cd777609fa4] Request joining
group due to: rebalance failed due to 'The coordinator is not aware of this
member.' (UnknownMemberIdException)
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator:1109)
[2026-09-11 14:03:52,944] INFO [connector-name|task-0] [Consumer
clientId=45e600cb-3e05-4461-97b7-e88e4cbe7237,
groupId=group-id-prefix779a2a31-e1c0-44d7-a0e7-1cd777609fa4] (Re-)joining group
(org.apache.kafka.clients.consumer.internals.ConsumerCoordinator:611)
```
The connector only starts working again if I remove it from the cluster and
clear all the Kafka Connect internal topics involved.
### Environment
* **Iceberg Kafka Connect:** 1.11.0
* **Catalog:** Nessie
* **Table storage:** Dell ECS
* **Deployment:** Kubernetes
* **Connectivity:** I have verified connectivity between all deployed
components, and there don't appear to be any connectivity issues.
### Steps to reproduce
1. Start Kafka Connect and the Iceberg connector from a clean state.
2. Let the connector process messages normally.
3. Cause the task to encounter an error and stop.
4. Restart the Kafka Connect pod/cluster.
5. The task does not recover and continuously logs
`UnknownMemberIdException` / `The coordinator is not aware of this member`.
6. The connector only starts working again after removing it and clearing
the Kafka Connect internal topics.
Has anyone encountered this behavior before?
Is there any additional Kafka Connect state or configuration that I should
clear or reset to allow the task to recover without deleting the internal
topics?
Any suggestions on what could cause the consumer group to remain in this
state would be greatly appreciated.
Thanks in advance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
