[jira] [Commented] (KAFKA-4782) change chroot for a kafka instance
[ https://issues.apache.org/jira/browse/KAFKA-4782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874650#comment-15874650 ] Jayesh Thakrar commented on KAFKA-4782: --- A good question and not an uncommon situation. However I don't know if there is a generic tool/api to "move" a znode or set of znodes around. I think you may have to write a custom program to do the task. If you do, it could be a good contribution back to the Zookeeper community that will be useful to many beyond just the Kafka project. > change chroot for a kafka instance > -- > > Key: KAFKA-4782 > URL: https://issues.apache.org/jira/browse/KAFKA-4782 > Project: Kafka > Issue Type: Improvement >Reporter: polarbear > > Hello > I am a beginner of the kafka. > Our kafka is "kafka_2.11-0.10.0.1.jar.asc", should be 0.10.0.1. > chroot is default "/". > Our user ask us to change the chroot from "/" to "/kafka". > I change the parameter zookeeper.connect to add /kafka, but we cannot find > the old topic in "/kafka", so I think it should be a new instance. > Which tool can we use to migrate the topic from the old "/" to new "/kafka"? > Thank you -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh
[ https://issues.apache.org/jira/browse/KAFKA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayesh Thakrar updated KAFKA-3663: -- Reviewer: Gwen Shapira Added [~gwenshap] as reviewer. > Proposal for a kafka broker command - kafka-brokers.sh > -- > > Key: KAFKA-3663 > URL: https://issues.apache.org/jira/browse/KAFKA-3663 > Project: Kafka > Issue Type: Improvement > Components: admin >Reporter: Jayesh Thakrar > > This is a proposal for an admin tool - say, kafka-brokers.sh to provide > broker related useful information. See > https://cwiki.apache.org/confluence/display/KAFKA/KIP-59%3A+Proposal+for+a+kafka+broker+command > for details. > The kafka-brokers.sh command mimics the kafka-topic.sh command, but provides > details by broker rather than by topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh
[ https://issues.apache.org/jira/browse/KAFKA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayesh Thakrar updated KAFKA-3663: -- Status: Open (was: Patch Available) Patch not created yet, hence not avaiable. > Proposal for a kafka broker command - kafka-brokers.sh > -- > > Key: KAFKA-3663 > URL: https://issues.apache.org/jira/browse/KAFKA-3663 > Project: Kafka > Issue Type: Improvement > Components: admin >Reporter: Jayesh Thakrar > > This is a proposal for an admin tool - say, kafka-brokers.sh to provide > broker related useful information. See > https://cwiki.apache.org/confluence/display/KAFKA/KIP-59%3A+Proposal+for+a+kafka+broker+command > for details. > The kafka-brokers.sh command mimics the kafka-topic.sh command, but provides > details by broker rather than by topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh
[ https://issues.apache.org/jira/browse/KAFKA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15360809#comment-15360809 ] Jayesh Thakrar edited comment on KAFKA-3663 at 7/4/16 4:04 AM: --- Patch not created yet, hence not available. was (Author: jthakrar): Patch not created yet, hence not avaiable. > Proposal for a kafka broker command - kafka-brokers.sh > -- > > Key: KAFKA-3663 > URL: https://issues.apache.org/jira/browse/KAFKA-3663 > Project: Kafka > Issue Type: Improvement > Components: admin >Reporter: Jayesh Thakrar > > This is a proposal for an admin tool - say, kafka-brokers.sh to provide > broker related useful information. See > https://cwiki.apache.org/confluence/display/KAFKA/KIP-59%3A+Proposal+for+a+kafka+broker+command > for details. > The kafka-brokers.sh command mimics the kafka-topic.sh command, but provides > details by broker rather than by topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh
[ https://issues.apache.org/jira/browse/KAFKA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayesh Thakrar updated KAFKA-3663: -- Status: Patch Available (was: Open) Pull request - https://github.com/apache/kafka/pull/1539 > Proposal for a kafka broker command - kafka-brokers.sh > -- > > Key: KAFKA-3663 > URL: https://issues.apache.org/jira/browse/KAFKA-3663 > Project: Kafka > Issue Type: Improvement > Components: admin >Reporter: Jayesh Thakrar > > This is a proposal for an admin tool - say, kafka-brokers.sh to provide > broker related useful information. See > https://cwiki.apache.org/confluence/display/KAFKA/KIP-59%3A+Proposal+for+a+kafka+broker+command > for details. > The kafka-brokers.sh command mimics the kafka-topic.sh command, but provides > details by broker rather than by topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh
[ https://issues.apache.org/jira/browse/KAFKA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15345240#comment-15345240 ] Jayesh Thakrar commented on KAFKA-3663: --- I think I need some pointers/help here. Apparently the pull request shows a Jenkins error, although I verified that the build works. Here's the link to the pull request - https://github.com/apache/kafka/pull/1539 > Proposal for a kafka broker command - kafka-brokers.sh > -- > > Key: KAFKA-3663 > URL: https://issues.apache.org/jira/browse/KAFKA-3663 > Project: Kafka > Issue Type: Improvement > Components: admin >Reporter: Jayesh Thakrar > > This is a proposal for an admin tool - say, kafka-brokers.sh to provide > broker related useful information. See > https://cwiki.apache.org/confluence/display/KAFKA/KIP-59%3A+Proposal+for+a+kafka+broker+command > for details. > The kafka-brokers.sh command mimics the kafka-topic.sh command, but provides > details by broker rather than by topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh
[ https://issues.apache.org/jira/browse/KAFKA-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayesh Thakrar updated KAFKA-3663: -- Description: This is a proposal for an admin tool - say, kafka-brokers.sh to provide broker related useful information. See https://cwiki.apache.org/confluence/display/KAFKA/KIP-59%3A+Proposal+for+a+kafka+broker+command for details. The kafka-brokers.sh command mimics the kafka-topic.sh command, but provides details by broker rather than by topic. was: This is a proposal for an admin tool - say, kafka-brokers.sh to provide broker related useful information. Note that I could not see an option to create a child page at https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals. So here are the details for the proposals. *Motivation* Some of the key succcess factor for Kafka's success are its performant architecture and operational simplicity. This is further complemented with a set of commandline tools and utilities for managing topics as well as testing/stress-testing. However currently Kafka lacks commands/tools to get a cluster and broker overview. Although it should be mentioned that Kafka does expose cluster information via API and broker metrics via JMX. *Proposed Change* This KIP is for a command, say kafka-brokers.sh that provides useful cluster and broker information. The command will essentially provide the following pieces of information: * Cluster Overview Information ** Controller Broker Id (and version/epoch information) ** Broker Count ** Total Topic Count ** Total Partition Count * Broker Information ** Broker Id ** Rack Id ** Hostname ** Endpoints (protocol, port) ** JMX port ** Topic count ** All partition count ** Leader partition count ** Under-replicated partition count ** Topic partitions (Name, *partition-) - An asterisk would indicate that the broker is the leader for that partition. A hyphen/negative sign would indicates that the partition is not in-sync. As you can see, the above information provides a view of the cluster and brokers that complements kafka-topics.sh. This command can be further evolved to do more things like: - Drain one or more brokers for decommissioning. This feature would allow distributing off all partitions of a list of brokers to other brokers in the cluster (need to be cognizant of rack configuration). - Onboard one or more new brokers. This feature would do the reverse of removing brokers and allow controlled distribution of partition to the new brokers. - Obtain JMX metrics from a broker (see KIP for its usage). - See KAFKA-3649 for other features > Proposal for a kafka broker command - kafka-brokers.sh > -- > > Key: KAFKA-3663 > URL: https://issues.apache.org/jira/browse/KAFKA-3663 > Project: Kafka > Issue Type: Improvement > Components: admin >Reporter: Jayesh Thakrar > > This is a proposal for an admin tool - say, kafka-brokers.sh to provide > broker related useful information. See > https://cwiki.apache.org/confluence/display/KAFKA/KIP-59%3A+Proposal+for+a+kafka+broker+command > for details. > The kafka-brokers.sh command mimics the kafka-topic.sh command, but provides > details by broker rather than by topic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (KAFKA-3663) Proposal for a kafka broker command - kafka-brokers.sh
Jayesh Thakrar created KAFKA-3663: - Summary: Proposal for a kafka broker command - kafka-brokers.sh Key: KAFKA-3663 URL: https://issues.apache.org/jira/browse/KAFKA-3663 Project: Kafka Issue Type: Improvement Components: admin Reporter: Jayesh Thakrar This is a proposal for an admin tool - say, kafka-brokers.sh to provide broker related useful information. Note that I could not see an option to create a child page at https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals. So here are the details for the proposals. *Motivation* Some of the key succcess factor for Kafka's success are its performant architecture and operational simplicity. This is further complemented with a set of commandline tools and utilities for managing topics as well as testing/stress-testing. However currently Kafka lacks commands/tools to get a cluster and broker overview. Although it should be mentioned that Kafka does expose cluster information via API and broker metrics via JMX. *Proposed Change* This KIP is for a command, say kafka-brokers.sh that provides useful cluster and broker information. The command will essentially provide the following pieces of information: * Cluster Overview Information ** Controller Broker Id (and version/epoch information) ** Broker Count ** Total Topic Count ** Total Partition Count * Broker Information ** Broker Id ** Rack Id ** Hostname ** Endpoints (protocol, port) ** JMX port ** Topic count ** All partition count ** Leader partition count ** Under-replicated partition count ** Topic partitions (Name, *partition-) - An asterisk would indicate that the broker is the leader for that partition. A hyphen/negative sign would indicates that the partition is not in-sync. As you can see, the above information provides a view of the cluster and brokers that complements kafka-topics.sh. This command can be further evolved to do more things like: - Drain one or more brokers for decommissioning. This feature would allow distributing off all partitions of a list of brokers to other brokers in the cluster (need to be cognizant of rack configuration). - Onboard one or more new brokers. This feature would do the reverse of removing brokers and allow controlled distribution of partition to the new brokers. - Obtain JMX metrics from a broker (see KIP for its usage). - See KAFKA-3649 for other features -- This message was sent by Atlassian JIRA (v6.3.4#6332)
