Harshit Pedamkar created KAFKA-18233:
----------------------------------------
Summary: Mirrormaker 2 All Worker tasks graceful shutdown fails
after which the consumer group offsets are replicated .
Key: KAFKA-18233
URL: https://issues.apache.org/jira/browse/KAFKA-18233
Project: Kafka
Issue Type: Bug
Components: mirrormaker
Affects Versions: 3.5.1
Reporter: Harshit Pedamkar
Fix For: 3.5.1
Attachments: connect-distributed.properties, connect.log,
connect.log.2024-12-13-07, mm2-cpc-cust-repl-policy.json,
mm2-hbc-cust-repl-policy.json, mm2-msc-cust-repl-policy.json
Hi,
It is noticed during load the Checkpoint Connectors graceful shutdown fails.
After which the consumer group offsets are not replicates from source to target.
We have a distributed mirrormaker2 container running. Logs are attached.
Logs snippets:
[2024-12-13 10:31:53,594] ERROR [mm2-cpc|worker] Connector 'mm2-cpc' failed to
properly shut down, has become unresponsive, and may be consuming external
resources. Correct the configuration for this connector or remove the
connector. After fixing the connector, it may be necessary to restart this
worker to release any consumed resources.
(org.apache.kafka.connect.runtime.Worker:437)
[2024-12-13 10:31:58,818] ERROR [mm2-cpc|worker] Connector 'mm2-cpc' failed to
properly shut down, has become unresponsive, and may be consuming external
resources. Correct the configuration for this connector or remove the
connector. After fixing the connector, it may be necessary to restart this
worker to release any consumed resources.
(org.apache.kafka.connect.runtime.Worker:437)
[2024-12-13 10:32:03,928] ERROR [mm2-cpc|worker] Connector 'mm2-cpc' failed to
properly shut down, has become unresponsive, and may be consuming external
resources. Correct the configuration for this connector or remove the
connector. After fixing the connector, it may be necessary to restart this
worker to release any consumed resources.
(org.apache.kafka.connect.runtime.Worker:437)
[2024-12-13 10:32:30,444] ERROR [mm2-cpc|task-30] Graceful stop of task
mm2-cpc-30 failed. (org.apache.kafka.connect.runtime.Worker:1029)
[2024-12-13 10:32:30,447] ERROR [mm2-cpc|task-27] Graceful stop of task
mm2-cpc-27 failed. (org.apache.kafka.connect.runtime.Worker:1029)
[2024-12-13 10:32:30,449] ERROR [mm2-cpc|task-33] Graceful stop of task
mm2-cpc-33 failed. (org.apache.kafka.connect.runtime.Worker:1029)
[2024-12-13 10:32:30,452] ERROR [mm2-cpc|task-36] Graceful stop of task
mm2-cpc-36 failed. (org.apache.kafka.connect.runtime.Worker:1029)
[2024-12-13 10:32:30,455] ERROR [mm2-cpc|task-42] Graceful stop of task
mm2-cpc-42 failed. (org.apache.kafka.connect.runtime.Worker:1029)
We have mirrormaker with the configuration. config files attached.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
