Re: Kafka Command Line Shell
Thanks Joe. I will read the wiki page. On Tue, Nov 11, 2014 at 11:47 PM, Joe Stein joe.st...@stealth.ly wrote: I started writing this up on the wiki https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements Instead of starting a new thread I figure just continue this one I started. I also added another (important) component for centralized management of configuration as global level much like we have topic level. These global configuration would be overridden (perhaps not all) from the server.properties on start (so like in case one broker needs a different port, sure). One concern I have is that using RQ/RP wire protocol to the controller instead of the current way (via ZK admin path) may expose concurrency on the admin requests, which may not be supported yet. Guozhang, take a look at the diagram how I am thinking of this it would be a new handle request that will execute the tools pretty much how they are today. My thinking is maybe to-do one at a time (so TopicCommand first I think) and have what the TopicCommand is doing happen on server and send the RQ/RP to the client but execute on the server. If there is something not supported we will of course have to deal with that and implement it for sure. Once we get one working end to end I think adding the rest will be (more or less) concise iterations to get it done. I added your concern to the wiki under the gotchas section. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / On Mon, Oct 20, 2014 at 2:15 AM, Guozhang Wang wangg...@gmail.com wrote: One concern I have is that using RQ/RP wire protocol to the controller instead of the current way (via ZK admin path) may expose concurrency on the admin requests, which may not be supported yet. Some initial discussion about this is on KAFKA-1305. Guozhang On Sun, Oct 19, 2014 at 1:55 PM, Joe Stein joe.st...@stealth.ly wrote: Maybe we should add some AdminMessage RQ/RP wire protocol structure(s) and let the controller handle it? We could then build the CLI and Shell in the project both as useful tools and samples for others. Making a http interface should be simple after KAFKA-1494 is done which all client libraries could offer. I will update the design tonight/tomorrow and should be able to have someone starting to work on it this week. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop / On Oct 19, 2014 1:21 PM, Harsha ka...@harsha.io wrote: +1 for Web Api On Sat, Oct 18, 2014, at 11:48 PM, Glen Mazza wrote: Apache Karaf has been doing this for quite a few years, albeit in Java not Scala. Still, their coding approach to creating a CLI probably captures many lessons learned over that time. Glen On 10/17/2014 08:03 PM, Joe Stein wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / -- -- Guozhang -- -- Guozhang
[jira] [Updated] (KAFKA-1694) kafka command line shell
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1694: - Description: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements was: There are a lot of different kafka tools. Right now I think 5-6 of them are being used commonly. I was thinking we could start by taking https://cwiki.apache.org/confluence/display/KAFKA/System+Tools and https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools and exposing them in a plugin type way for a command line shell interface. It would be both: command line and shell. so kafka -b brokerlist -a reasign-partition status would run from the cli and kafka shell -b brokerlist kafkadescribe; ... kafka-topics.sh --describe kafkaset topic_security['pci','profile','dss'] = true ...etc An important item is that folks that are using existing tools we should have an easy api type way so they can keep doing that and get benefit too. This interface should also have some monitoring stats too, e.g visualize in text the consumer lag trending between offset committed and log end offset. kafkause topic name; kafkastats; kafka command line shell Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Priority: Critical Fix For: 0.8.3 https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: Kafka Command Line Shell
I started writing this up on the wiki https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Command+Line+and+Related+Improvements Instead of starting a new thread I figure just continue this one I started. I also added another (important) component for centralized management of configuration as global level much like we have topic level. These global configuration would be overridden (perhaps not all) from the server.properties on start (so like in case one broker needs a different port, sure). One concern I have is that using RQ/RP wire protocol to the controller instead of the current way (via ZK admin path) may expose concurrency on the admin requests, which may not be supported yet. Guozhang, take a look at the diagram how I am thinking of this it would be a new handle request that will execute the tools pretty much how they are today. My thinking is maybe to-do one at a time (so TopicCommand first I think) and have what the TopicCommand is doing happen on server and send the RQ/RP to the client but execute on the server. If there is something not supported we will of course have to deal with that and implement it for sure. Once we get one working end to end I think adding the rest will be (more or less) concise iterations to get it done. I added your concern to the wiki under the gotchas section. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / On Mon, Oct 20, 2014 at 2:15 AM, Guozhang Wang wangg...@gmail.com wrote: One concern I have is that using RQ/RP wire protocol to the controller instead of the current way (via ZK admin path) may expose concurrency on the admin requests, which may not be supported yet. Some initial discussion about this is on KAFKA-1305. Guozhang On Sun, Oct 19, 2014 at 1:55 PM, Joe Stein joe.st...@stealth.ly wrote: Maybe we should add some AdminMessage RQ/RP wire protocol structure(s) and let the controller handle it? We could then build the CLI and Shell in the project both as useful tools and samples for others. Making a http interface should be simple after KAFKA-1494 is done which all client libraries could offer. I will update the design tonight/tomorrow and should be able to have someone starting to work on it this week. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop / On Oct 19, 2014 1:21 PM, Harsha ka...@harsha.io wrote: +1 for Web Api On Sat, Oct 18, 2014, at 11:48 PM, Glen Mazza wrote: Apache Karaf has been doing this for quite a few years, albeit in Java not Scala. Still, their coding approach to creating a CLI probably captures many lessons learned over that time. Glen On 10/17/2014 08:03 PM, Joe Stein wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop / -- -- Guozhang
Re: Kafka Command Line Shell
Apache Karaf has been doing this for quite a few years, albeit in Java not Scala. Still, their coding approach to creating a CLI probably captures many lessons learned over that time. Glen On 10/17/2014 08:03 PM, Joe Stein wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
Re: Kafka Command Line Shell
+1 for Web Api On Sat, Oct 18, 2014, at 11:48 PM, Glen Mazza wrote: Apache Karaf has been doing this for quite a few years, albeit in Java not Scala. Still, their coding approach to creating a CLI probably captures many lessons learned over that time. Glen On 10/17/2014 08:03 PM, Joe Stein wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
Re: Kafka Command Line Shell
Maybe we should add some AdminMessage RQ/RP wire protocol structure(s) and let the controller handle it? We could then build the CLI and Shell in the project both as useful tools and samples for others. Making a http interface should be simple after KAFKA-1494 is done which all client libraries could offer. I will update the design tonight/tomorrow and should be able to have someone starting to work on it this week. /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop / On Oct 19, 2014 1:21 PM, Harsha ka...@harsha.io wrote: +1 for Web Api On Sat, Oct 18, 2014, at 11:48 PM, Glen Mazza wrote: Apache Karaf has been doing this for quite a few years, albeit in Java not Scala. Still, their coding approach to creating a CLI probably captures many lessons learned over that time. Glen On 10/17/2014 08:03 PM, Joe Stein wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
Kafka Command Line Shell
Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
Re: Kafka Command Line Shell
Joe I think this is great! On Fri, Oct 17, 2014 at 5:03 PM, Joe Stein joe.st...@stealth.ly wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
Re: Kafka Command Line Shell
We've been talking about this a little internally as well. What about the idea of presenting all the admin functions through a web API interface (restful or not) complete with authentication? That would make it much easier for creating structure around Kafka without having to layer commands on top of each other. I'm not a big fan of the language specific interfaces, because they tend to complicate trying to integrate with larger systems. Consider something like AWS or Azure, where it would be much easier if there is an API interface like that. -Todd On Oct 17, 2014, at 5:03 PM, Joe Stein joe.st...@stealth.ly wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
Re: Kafka Command Line Shell
+1 It would definitely be useful to have a CLI. We had a cursory discussion on this in the past [1] but it would be useful to have a full proposal describing everything the CLI should provide. [1] http://grokbase.com/t/kafka/dev/1435tr3pfc/command-line-tools On Fri, Oct 17, 2014 at 05:12:16PM -0700, Todd Palino wrote: We've been talking about this a little internally as well. What about the idea of presenting all the admin functions through a web API interface (restful or not) complete with authentication? That would make it much easier for creating structure around Kafka without having to layer commands on top of each other. I'm not a big fan of the language specific interfaces, because they tend to complicate trying to integrate with larger systems. Consider something like AWS or Azure, where it would be much easier if there is an API interface like that. -Todd On Oct 17, 2014, at 5:03 PM, Joe Stein joe.st...@stealth.ly wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
Re: Kafka Command Line Shell
Absolutely. My suggestion of an HTTP interface was in addition to a CLI. I think the CLI should can the HTTP interface underneath to keep it simple. -Todd On Oct 17, 2014, at 6:24 PM, Joel Koshy jjkosh...@gmail.com wrote: +1 It would definitely be useful to have a CLI. We had a cursory discussion on this in the past [1] but it would be useful to have a full proposal describing everything the CLI should provide. [1] http://grokbase.com/t/kafka/dev/1435tr3pfc/command-line-tools On Fri, Oct 17, 2014 at 05:12:16PM -0700, Todd Palino wrote: We've been talking about this a little internally as well. What about the idea of presenting all the admin functions through a web API interface (restful or not) complete with authentication? That would make it much easier for creating structure around Kafka without having to layer commands on top of each other. I'm not a big fan of the language specific interfaces, because they tend to complicate trying to integrate with larger systems. Consider something like AWS or Azure, where it would be much easier if there is an API interface like that. -Todd On Oct 17, 2014, at 5:03 PM, Joe Stein joe.st...@stealth.ly wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
Re: Kafka Command Line Shell
+1 to http interface and cli. Http layer will make it easier to integrate with gui like Hue. Gwen — Sent from Mailbox On Fri, Oct 17, 2014 at 10:14 PM, Todd Palino tpal...@gmail.com wrote: Absolutely. My suggestion of an HTTP interface was in addition to a CLI. I think the CLI should can the HTTP interface underneath to keep it simple. -Todd On Oct 17, 2014, at 6:24 PM, Joel Koshy jjkosh...@gmail.com wrote: +1 It would definitely be useful to have a CLI. We had a cursory discussion on this in the past [1] but it would be useful to have a full proposal describing everything the CLI should provide. [1] http://grokbase.com/t/kafka/dev/1435tr3pfc/command-line-tools On Fri, Oct 17, 2014 at 05:12:16PM -0700, Todd Palino wrote: We've been talking about this a little internally as well. What about the idea of presenting all the admin functions through a web API interface (restful or not) complete with authentication? That would make it much easier for creating structure around Kafka without having to layer commands on top of each other. I'm not a big fan of the language specific interfaces, because they tend to complicate trying to integrate with larger systems. Consider something like AWS or Azure, where it would be much easier if there is an API interface like that. -Todd On Oct 17, 2014, at 5:03 PM, Joe Stein joe.st...@stealth.ly wrote: Hi, I have been thinking about the ease of use for operations with Kafka. We have lots of tools doing a lot of different things and they are all kind of in different places. So, what I was thinking is to have a single interface for our tooling https://issues.apache.org/jira/browse/KAFKA-1694 This would manifest itself in two ways 1) a command line interface 2) a repl We would have one entry point centrally for all Kafka commands. kafka CMD ARGS kafka createTopic --brokerList etc, kafka reassignPartition --brokerList etc, or execute and run the shell kafka --brokerList localhost kafkause topicName; kafkaset acl='label'; I was thinking that all calls would be initialized through --brokerList and the broker can tell the KafkaCommandTool what server to connect to for MetaData. Thoughts? Tomatoes? /*** Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop http://www.twitter.com/allthingshadoop /
[jira] [Created] (KAFKA-1694) kafka command line shell
Joe Stein created KAFKA-1694: Summary: kafka command line shell Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Priority: Critical Fix For: 0.8.3 There are a lot of different kafka tools. Right now I think 5-6 of them are being used commonly. I was thinking we could start by taking https://cwiki.apache.org/confluence/display/KAFKA/System+Tools and https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools and exposing them in a plugin type way for a command line shell interface. It would be both: command line and shell. so kafka -b brokerlist -a reasign-partition status would run from the cli and kafka shell -b brokerlist kafkakafkadescribe; ... kafka-topics.sh --describe kafkaset topic_security['pci','profile','dss'] = true ...etc An important item is that folks that are using existing tools we should have an easy api type way so they can keep doing that and get benefit too. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (KAFKA-1694) kafka command line shell
[ https://issues.apache.org/jira/browse/KAFKA-1694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joe Stein updated KAFKA-1694: - Description: There are a lot of different kafka tools. Right now I think 5-6 of them are being used commonly. I was thinking we could start by taking https://cwiki.apache.org/confluence/display/KAFKA/System+Tools and https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools and exposing them in a plugin type way for a command line shell interface. It would be both: command line and shell. so kafka -b brokerlist -a reasign-partition status would run from the cli and kafka shell -b brokerlist kafkadescribe; ... kafka-topics.sh --describe kafkaset topic_security['pci','profile','dss'] = true ...etc An important item is that folks that are using existing tools we should have an easy api type way so they can keep doing that and get benefit too. This interface should also have some monitoring stats too, e.g visualize in text the consumer lag trending between offset committed and log end offset. kafkause topic name; kafkastats; was: There are a lot of different kafka tools. Right now I think 5-6 of them are being used commonly. I was thinking we could start by taking https://cwiki.apache.org/confluence/display/KAFKA/System+Tools and https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools and exposing them in a plugin type way for a command line shell interface. It would be both: command line and shell. so kafka -b brokerlist -a reasign-partition status would run from the cli and kafka shell -b brokerlist kafkakafkadescribe; ... kafka-topics.sh --describe kafkaset topic_security['pci','profile','dss'] = true ...etc An important item is that folks that are using existing tools we should have an easy api type way so they can keep doing that and get benefit too. This interface should also have some monitoring stats too, some updates to the cu kafka command line shell Key: KAFKA-1694 URL: https://issues.apache.org/jira/browse/KAFKA-1694 Project: Kafka Issue Type: Bug Reporter: Joe Stein Priority: Critical Fix For: 0.8.3 There are a lot of different kafka tools. Right now I think 5-6 of them are being used commonly. I was thinking we could start by taking https://cwiki.apache.org/confluence/display/KAFKA/System+Tools and https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools and exposing them in a plugin type way for a command line shell interface. It would be both: command line and shell. so kafka -b brokerlist -a reasign-partition status would run from the cli and kafka shell -b brokerlist kafkadescribe; ... kafka-topics.sh --describe kafkaset topic_security['pci','profile','dss'] = true ...etc An important item is that folks that are using existing tools we should have an easy api type way so they can keep doing that and get benefit too. This interface should also have some monitoring stats too, e.g visualize in text the consumer lag trending between offset committed and log end offset. kafkause topic name; kafkastats; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
