Hi Colin, thanks! While I personally don't like the current command line tools I realistically think we'll be stuck with them for a while yet, so a cleanup might make sense. So I'll start looking into that.
Regarding a central entrypoint, that would indeed be brilliant and I'd love to work on that, but I currently have enough other open things to look at, so I won't draw that one to myself as well for now. But I'll keep it in mind for when some time frees up :) Best regards, Sönke Colin McCabe <cmcc...@apache.org> schrieb am Di., 7. Mai 2019, 00:56: > On Mon, May 6, 2019, at 10:21, Sönke Liebau wrote: > > Hi Colin, > > > > it was my intention to keep the structure of the commands mostly intact > > while doing the refactoring - if that is possible, have not really > checked > > yet to be honest. > > > > But what I wanted to try and do is recreate the current parsing with > > argparse as much as possible. And in the process simply adding synonyms, > > for example make the kafka-console-producer understand a > > bootstrap-parameter in addition to broker-list. > > There is a bit of custom logic about which parameters go together etc. in > > the current classes, so output may look different here and there, but in > > principle I do believe that it should be possible to recreate the current > > structure. > > Sounds like a good idea. Thanks for the clarification. > > > > > If there is an appetite for a new, hadoop-like entrypoint anyway, then > all > > of this might be "wasted" effort, or rather effort better spent though, > you > > are right. > > I don't think anyone is working on a new entry point right now -- or if > they are, they haven't said anything yet :) > > I just wanted to mention it as a possible approach in case you wanted to > do a bigger project. > > best, > Colin > > > > > Best regards, > > Sönke > > > > > > > > On Mon, May 6, 2019 at 7:13 PM Colin McCabe <cmcc...@apache.org> wrote: > > > > > Hi Sönke, > > > > > > #2 is a bit tough because people have come to rely on the way the > commands > > > are structured right now. > > > > > > If we want to make big changes, it might be easier just to create a > > > separate tool and deprecate the old one(s). One thing we've talked > about > > > doing in the past is creating a single entry point for all the tool > > > functionality, kind of like hadoop did with the "hadoop" command Or > git > > > with the "git" command, etc. Then we could deprecate the standalone > > > commands and remove them after enough time had passed-- kind of like > the > > > old consumer. > > > > > > On the other hand, a more incremental change would be standardizing > flags > > > a bit. So for example, at least setting it up so that there is a > standard > > > way of supplying bootstrap brokers, etc. We could keep the old flags > > > around for a while as variants to ease the transition. > > > > > > best, > > > Colin > > > > > > > > > On Sun, May 5, 2019, at 00:54, Sönke Liebau wrote: > > > > Hi Colin, > > > > > > > > I totally agree! Especially the differently named bootstrap server > > > options > > > > have been annoying me a long time. > > > > > > > > I'd propose a two-step approach: > > > > 1. Add new default options objects similar to CommandLineUtils and > > > > CommandDefaultOptions (based on argparse4j) but in the clients > project, > > > as > > > > this is referenced by all command line tools as far as I can tell > > > > 2. Refactor tools one by one to use these new helper classes (and > thus > > > > argparse) and add standardized synonyms for parameters as necessary > > > > > > > > I think for step 1 we can get away with no KIP, as this doesn't > change > > > any > > > > public interfaces or behavior. > > > > Step 2 probably needs a KIP as we are adding new parameters? We can > pick > > > up > > > > KIP-14 again for that I think. A lot of work has been done on that > > > already. > > > > > > > > Does that sound useful to everybody? > > > > > > > > Best regards, > > > > Sönke > > > > > > > > > > > > On Thu, Apr 18, 2019 at 1:44 AM Colin McCabe <cmcc...@apache.org> > wrote: > > > > > > > > > If we are going to standardize on one argument parsing library, it > > > should > > > > > certainly be argparse4j, I think. > > > > > argparse4j is simply a better argument parsing library with > support > > > for > > > > > more features. One example is mutually exclusive options. > argparse4j > > > > > supports this with MutuallyExclusiveGroup. jopt doesn't support > this, > > > so > > > > > when it is needed, we have to add extra code to manually check that > > > > > mutually exclusive options are not set. > > > > > > > > > > argparse4j also has subcommands. If you want something like "git > add" > > > > > with some set of flags, and "git remove" with another, you can do > this > > > with > > > > > argparse4j, but not with jopt. This would be very helpful for > > > clearing up > > > > > confusion in a lot of our shell scripts which have accumulated > dozens > > > of > > > > > arguments, most of which are only relevant to a very specific > > > operation. > > > > > But you just can't do it with jopt. > > > > > > > > > > Just to give an example, argparse4j with subcommands would allow > you to > > > > > run something like ./kafka-topics.sh list --help and get just > options > > > that > > > > > were relevant for listing topics, not the full dozens of options > that > > > might > > > > > relate to adding topics, removing them, etc. > > > > > > > > > > To be honest, though, what would help users the most is > standardizing > > > the > > > > > option flags across tools. We should have a standard way of > specifying > > > > > bootstrap brokers, for example. (We can continue to support the > old > > > > > synonyms for a while, of course.) > > > > > > > > > > best, > > > > > Colin > > > > > > > > > > > > > > > On Wed, Apr 17, 2019, at 08:56, Guozhang Wang wrote: > > > > > > I took another look at the PR itself and I think it would be > great to > > > > > have > > > > > > this cleanup too -- I cannot remember at the beginning why we > > > gradually > > > > > > moved to different mechanism (argparse4j) for different cmds, if > > > there's > > > > > no > > > > > > rationales behind it we should just make them consistent. > > > > > > > > > > > > Thanks for driving this! > > > > > > > > > > > > Guozhang > > > > > > > > > > > > On Wed, Apr 17, 2019 at 7:19 AM Ryanne Dolan < > ryannedo...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > Sönke, I'd find this very helpful. It's annoying to keep track > of > > > which > > > > > > > commands use which form -- I always seem to guess wrong. > > > > > > > > > > > > > > Though I don't think there is any reason to deprecate existing > > > forms, > > > > > e.g. > > > > > > > consumer.config vs consumer-config. I think it's perfectly > > > reasonable > > > > > to > > > > > > > have multiple spellings of the same arguments. I don't really > see a > > > > > > > downside to keeping the aliases around indefinitely. > > > > > > > > > > > > > > Ryanne > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Apr 17, 2019, 7:07 AM Sönke Liebau > > > > > > > <soenke.lie...@opencore.com.invalid> wrote: > > > > > > > > > > > > > > > Hi everybody, > > > > > > > > > > > > > > > > Jason and I were recently discussing command line argument > > > parsing on > > > > > > > > KAFKA-8131 (or rather the related pull request) [1]. > > > > > > > > > > > > > > > > Command line tools and their arguments are somewhat diverse > at > > > the > > > > > > > moment. > > > > > > > > Most of the tools use joptsimple for argument parsing, some > newer > > > > > java > > > > > > > > tools use argparse4j instead and some tools use nothing at > all. > > > > > > > > I've looked for a reason as to why there are two libraries > being > > > > > used, > > > > > > > but > > > > > > > > couldn't really find anything. Paolo brought up the same > > > question on > > > > > the > > > > > > > > mailing list a while back [7], but got no response either. > > > > > > > > Does anybody know why this is the case? > > > > > > > > > > > > > > > > This results in no central place to add universal parameters > like > > > > > help > > > > > > > and > > > > > > > > version, as well as the help output looking different between > > > some > > > > > of the > > > > > > > > tools. > > > > > > > > Also, there are a number of parameters that should be > renamed to > > > > > adhere > > > > > > > to > > > > > > > > defaults. > > > > > > > > > > > > > > > > There have been a few discussions and initiatives around > this in > > > the > > > > > > > past. > > > > > > > > Just of the top of my head (and a 5 minute jira search) there > > > are: > > > > > > > > - KIP-14 [2] > > > > > > > > - KAFKA-2111 [3] > > > > > > > > - KIP-316 [4] > > > > > > > > - KAFKA-1292 [5] > > > > > > > > - KAFKA-3530 [6] > > > > > > > > - and probably many more > > > > > > > > > > > > > > > > Would people generally be in favor of revisiting this topic? > > > > > > > > > > > > > > > > What I'd propose to do is: > > > > > > > > - comb through jira and KIPs, clean up old stuff and creae a > new > > > > > umbrella > > > > > > > > issue to track this (maybe reuse KIP-4 as well) > > > > > > > > - agree on one library for parsing command line arguments > (don't > > > care > > > > > > > which > > > > > > > > one, but two is one too many I think) > > > > > > > > - refactor tools to use one library and default way of > argument > > > > > parsing > > > > > > > > with central help and version parameter > > > > > > > > - add aliases for options that should be renamed according to > > > KIP-4 > > > > > (and > > > > > > > > maybe others) so that both new and old work for a while, > > > deprecate > > > > > old > > > > > > > > parameters for a cycle or two and then remove them > > > > > > > > > > > > > > > > I'll shut up now and see if people would consider this > useful or > > > > > have any > > > > > > > > other input :) > > > > > > > > > > > > > > > > Best regards, > > > > > > > > Sönke > > > > > > > > > > > > > > > > [1] > > > https://github.com/apache/kafka/pull/6481#discussion_r273773003 > > > > > > > > <https://issues.apache.org/jira/browse/KAFKA-8131> > > > > > > > > [2] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-14+-+Tools+Standardization > > > > > > > > [3] https://issues.apache.org/jira/browse/KAFKA-2111 > > > > > > > > [4] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-316%3A+Command-line+overrides+for+ConnectDistributed+worker+properties > > > > > > > > [5] https://issues.apache.org/jira/browse/KAFKA-1292 > > > > > > > > [6] https://issues.apache.org/jira/browse/KAFKA-3530 > > > > > > > > [7] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://sematext.com/opensee/m/Kafka/uyzND10ObP01p77VS?subj=From+Scala+to+Java+based+tools+joptsimple+vs+argparse4j > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > -- Guozhang > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Sönke Liebau > > > > Partner > > > > Tel. +49 179 7940878 > > > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany > > > > > > > > > > > > > -- > > Sönke Liebau > > Partner > > Tel. +49 179 7940878 > > OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany > > >
