[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[ https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927262#comment-16927262 ] ASF GitHub Bot commented on KAFKA-8345: --- cmccabe commented on pull request #7128: KAFKA-8345 (KIP-455): Controller and KafkaApi changes (part 3/4) URL: https://github.com/apache/kafka/pull/7128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create an Administrative API for Replica Reassignment > - > > Key: KAFKA-8345 > URL: https://issues.apache.org/jira/browse/KAFKA-8345 > Project: Kafka > Issue Type: Improvement >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe >Priority: Major > > Create an Administrative API for Replica Reassignment, as discussed in KIP-455 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[ https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911553#comment-16911553 ] ASF GitHub Bot commented on KAFKA-8345: --- stanislavkozlovski commented on pull request #7041: KAFKA-8345: Add an Admin API for partition reassignment (KIP-455) URL: https://github.com/apache/kafka/pull/7041 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create an Administrative API for Replica Reassignment > - > > Key: KAFKA-8345 > URL: https://issues.apache.org/jira/browse/KAFKA-8345 > Project: Kafka > Issue Type: Improvement >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe >Priority: Major > > Create an Administrative API for Replica Reassignment, as discussed in KIP-455 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[ https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907409#comment-16907409 ] ASF GitHub Bot commented on KAFKA-8345: --- cmccabe commented on pull request #7120: KAFKA-8345: KIP-455: Admin API changes (Part 2) URL: https://github.com/apache/kafka/pull/7120 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create an Administrative API for Replica Reassignment > - > > Key: KAFKA-8345 > URL: https://issues.apache.org/jira/browse/KAFKA-8345 > Project: Kafka > Issue Type: Improvement >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe >Priority: Major > > Create an Administrative API for Replica Reassignment, as discussed in KIP-455 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[ https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16895602#comment-16895602 ] ASF GitHub Bot commented on KAFKA-8345: --- cmccabe commented on pull request #7114: KAFKA-8345: KIP-455 Protocol changes (part 1) URL: https://github.com/apache/kafka/pull/7114 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create an Administrative API for Replica Reassignment > - > > Key: KAFKA-8345 > URL: https://issues.apache.org/jira/browse/KAFKA-8345 > Project: Kafka > Issue Type: Improvement >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe >Priority: Major > > Create an Administrative API for Replica Reassignment, as discussed in KIP-455 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[ https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16895527#comment-16895527 ] ASF GitHub Bot commented on KAFKA-8345: --- stanislavkozlovski commented on pull request #7128: KAFKA-8345 (KIP-455): Controller and KafkaApi changes (part 3/4) URL: https://github.com/apache/kafka/pull/7128 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create an Administrative API for Replica Reassignment > - > > Key: KAFKA-8345 > URL: https://issues.apache.org/jira/browse/KAFKA-8345 > Project: Kafka > Issue Type: Improvement >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe >Priority: Major > > Create an Administrative API for Replica Reassignment, as discussed in KIP-455 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[ https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16893995#comment-16893995 ] ASF GitHub Bot commented on KAFKA-8345: --- stanislavkozlovski commented on pull request #7120: KAFKA-8345: Admin API changes (Part 2) URL: https://github.com/apache/kafka/pull/7120 This implements the needed AdminClient APIs, implementation and test Previous PR: https://github.com/apache/kafka/pull/7114 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create an Administrative API for Replica Reassignment > - > > Key: KAFKA-8345 > URL: https://issues.apache.org/jira/browse/KAFKA-8345 > Project: Kafka > Issue Type: Improvement >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe >Priority: Major > > Create an Administrative API for Replica Reassignment, as discussed in KIP-455 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[ https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892868#comment-16892868 ] ASF GitHub Bot commented on KAFKA-8345: --- stanislavkozlovski commented on pull request #7114: KAFKA-8345: Protocol changes (part 1) URL: https://github.com/apache/kafka/pull/7114 This PR implements parts of the changes needed for [KIP-455: Create an Administrative API for Replica Reassignment](https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A+Create+an+Administrative+API+for+Replica+Reassignment) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create an Administrative API for Replica Reassignment > - > > Key: KAFKA-8345 > URL: https://issues.apache.org/jira/browse/KAFKA-8345 > Project: Kafka > Issue Type: Improvement >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe >Priority: Major > > Create an Administrative API for Replica Reassignment, as discussed in KIP-455 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[
https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16887206#comment-16887206
]
Andrew Olson commented on KAFKA-8345:
-
[~cmccabe] [~enether] Can you verify that this could be use to just change the
replica order so that a different broker becomes the preferred leader?
We have an admin script (shown below) that demotes a select broker from being
the leader for any partitions. This is a use case that could possibly use this
new API, if it's supported.
{noformat}
# usage:
# 1. On a Kafka broker node find all partitions with a broker id as first
replica making it the preferred leader
# export BROKER_ID=
# export KAFKA_ZOOKEEPER=$(awk -F= '/zookeeper.connect/{print $2}'
/opt/kafka/config/server.properties)
# /opt/kafka/bin/kafka-topics.sh --zookeeper ${KAFKA_ZOOKEEPER} --describe |
grep "Replicas: ${BROKER_ID}," | awk '{print $2,$4,$8}' >
kafka_topics_output.txt
# 2. Download and run this script to move first replica to end of replica list
making it a follower by default
# ruby demote_kafka_broker.rb > reorder_replicas.json
# 3. Execute the replica order reassignment
# /opt/kafka/bin/kafka-reassign-partitions.sh --execute --zookeeper
${KAFKA_ZOOKEEPER} --manual-assignment-json-file reorder_replicas.json
# 4. Verify the change was executed as expected
# /opt/kafka/bin/kafka-topics.sh --zookeeper ${KAFKA_ZOOKEEPER} --describe
require 'json'
topics = []
File.open("kafka_topics_output.txt", "r") do |f|
f.each_line do |line|
parts = line.split(' ')
t = {}
t['topic'] = parts[0]
t['partition'] = parts[1].to_i
t['replicas'] = parts[2].split(',').map {|r| r.to_i }
t['replicas'] << t['replicas'][0]
t['replicas'].delete_at(0)
topics << t
end
end
p = {}
p['partitions'] = topics
p['version'] = 1
puts JSON.pretty_generate(p)
{noformat}
> Create an Administrative API for Replica Reassignment
> -
>
> Key: KAFKA-8345
> URL: https://issues.apache.org/jira/browse/KAFKA-8345
> Project: Kafka
> Issue Type: Improvement
>Reporter: Colin P. McCabe
>Assignee: Colin P. McCabe
>Priority: Major
>
> Create an Administrative API for Replica Reassignment, as discussed in KIP-455
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (KAFKA-8345) Create an Administrative API for Replica Reassignment
[ https://issues.apache.org/jira/browse/KAFKA-8345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880195#comment-16880195 ] ASF GitHub Bot commented on KAFKA-8345: --- stanislavkozlovski commented on pull request #7041: KAFKA-8345: Add an Admin API for partition reassignment (KIP-455) URL: https://github.com/apache/kafka/pull/7041 This PR implements the changes needed for [KIP-455: Create an Administrative API for Replica Reassignment](https://cwiki.apache.org/confluence/display/KAFKA/KIP-455%3A+Create+an+Administrative+API+for+Replica+Reassignment) The changes here are heavily influenced by https://github.com/apache/kafka/pull/6955. We synced offline with Colin and decided I'll continue the work here. There was some missing stuff from the existing commits there, so I'll be re-creating them, instead of cherry-picking. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create an Administrative API for Replica Reassignment > - > > Key: KAFKA-8345 > URL: https://issues.apache.org/jira/browse/KAFKA-8345 > Project: Kafka > Issue Type: Improvement >Reporter: Colin P. McCabe >Assignee: Colin P. McCabe >Priority: Major > > Create an Administrative API for Replica Reassignment, as discussed in KIP-455 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
