[
https://issues.apache.org/jira/browse/KAFKA-13392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yevgeniy Korin updated KAFKA-13392:
-----------------------------------
Description:
*Scenario when we faced with this issue:*
One of three brokers is down. Add another (fourth) broker and try to reassign
partitions using '--bootstrap-server'
option.
*What's failed:*
{code:java}
/opt/kafka/bin/kafka-reassign-partitions.sh --bootstrap-server
xxx.xxx.xxx.xxx:9092 --reassignment-json-file
/tmp/reassignment-20211021130718.json --throttle 100000000 --execute{code}
failed with
{code:java}
Error: org.apache.kafka.common.errors.TimeoutException:
Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TimeoutException:
Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
at
org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at
org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at
org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at
org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at
kafka.admin.ReassignPartitionsCommand$.modifyInterBrokerThrottle(ReassignPartitionsCommand.scala:1435)
at
kafka.admin.ReassignPartitionsCommand$.modifyReassignmentThrottle(ReassignPartitionsCommand.scala:1412)
at
kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:974)
at
kafka.admin.ReassignPartitionsCommand$.handleAction(ReassignPartitionsCommand.scala:255)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:216)
at kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: org.apache.kafka.common.errors.TimeoutException:
Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting
for a node assignment. Call: incrementalAlterConfigs{code}
*Expected behavio**:*
partition reassignment process started.
*Workaround:*
Trigger partition reassignment process using '--zookeeper' option:
{code:java}
/opt/kafka/bin/kafka-reassign-partitions.sh --zookeeper
zookeeper.my.company:2181/kafka-cluster --reassignment-json-file
/tmp/reassignment-20211021130718.json --throttle 100000000 --execute{code}
*Additional info:*
We are able to trigger partition reassignment using '--bootstrap-server'
option with no exceptions when all four brokers are alive.
was:
*Scenario when we faced with this issue:*
One of three brokers is down. Add another (fourth) broker and try to reassign
partitions using `--bootstrap-server` option.
*What's failed:*
`/opt/kafka/bin/kafka-reassign-partitions.sh --bootstrap-server
xxx.xxx.xxx.xxx:9092 --reassignment-json-file
/tmp/reassignment-20211021130718.json --throttle 100000000 --execute` failed
with
Error: org.apache.kafka.common.errors.TimeoutException:
Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TimeoutException:
Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
at
org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at
org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at
org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at
org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at
kafka.admin.ReassignPartitionsCommand$.modifyInterBrokerThrottle(ReassignPartitionsCommand.scala:1435)
at
kafka.admin.ReassignPartitionsCommand$.modifyReassignmentThrottle(ReassignPartitionsCommand.scala:1412)
at
kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:974)
at
kafka.admin.ReassignPartitionsCommand$.handleAction(ReassignPartitionsCommand.scala:255)
at
kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:216)
at
kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
Caused by: org.apache.kafka.common.errors.TimeoutException:
Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting
for a node assignment. Call: incrementalAlterConfigs
*Expected behavio**:*
partition reassignment process started.
*Workaround:*
Trigger partition reassignment process using `–zookeeper` option:
/opt/kafka/bin/kafka-reassign-partitions.sh --zookeeper
zookeeper.my.company:2181/kafka-cluster --reassignment-json-file
/tmp/reassignment-20211021130718.json --throttle 100000000 --execute
Additional info:
We were able to trigger partition reassignment using `--bootstrap-server`
option when all four brokers are alive.
> Timeout Exception triggering reassign partitions with --bootstrap-server
> option
> -------------------------------------------------------------------------------
>
> Key: KAFKA-13392
> URL: https://issues.apache.org/jira/browse/KAFKA-13392
> Project: Kafka
> Issue Type: Bug
> Components: admin
> Affects Versions: 2.8.0
> Reporter: Yevgeniy Korin
> Priority: Minor
>
> *Scenario when we faced with this issue:*
> One of three brokers is down. Add another (fourth) broker and try to
> reassign partitions using '--bootstrap-server'
> option.
> *What's failed:*
> {code:java}
> /opt/kafka/bin/kafka-reassign-partitions.sh --bootstrap-server
> xxx.xxx.xxx.xxx:9092 --reassignment-json-file
> /tmp/reassignment-20211021130718.json --throttle 100000000 --execute{code}
> failed with
> {code:java}
> Error: org.apache.kafka.common.errors.TimeoutException:
> Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
> nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
> java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.TimeoutException:
> Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
> nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
> at
> kafka.admin.ReassignPartitionsCommand$.modifyInterBrokerThrottle(ReassignPartitionsCommand.scala:1435)
> at
> kafka.admin.ReassignPartitionsCommand$.modifyReassignmentThrottle(ReassignPartitionsCommand.scala:1412)
> at
> kafka.admin.ReassignPartitionsCommand$.executeAssignment(ReassignPartitionsCommand.scala:974)
> at
> kafka.admin.ReassignPartitionsCommand$.handleAction(ReassignPartitionsCommand.scala:255)
> at
> kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:216)
> at
> kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)
> Caused by: org.apache.kafka.common.errors.TimeoutException:
> Call(callName=incrementalAlterConfigs, deadlineMs=1634811369255, tries=1,
> nextAllowedTryMs=1634811369356) timed out at 1634811369256 after 1 attempt(s)
> Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out
> waiting for a node assignment. Call: incrementalAlterConfigs{code}
> *Expected behavio**:*
> partition reassignment process started.
> *Workaround:*
> Trigger partition reassignment process using '--zookeeper' option:
> {code:java}
> /opt/kafka/bin/kafka-reassign-partitions.sh --zookeeper
> zookeeper.my.company:2181/kafka-cluster --reassignment-json-file
> /tmp/reassignment-20211021130718.json --throttle 100000000 --execute{code}
> *Additional info:*
> We are able to trigger partition reassignment using '--bootstrap-server'
> option with no exceptions when all four brokers are alive.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
