Re: [Babel-users] babeld interface
Hi Juliusz, On Thu, Mar 15, 2018 at 12:37:55AM +0100, Juliusz Chroboczek wrote: > https://github.com/christf/libbabelhelper When you feel you're ready, please provide me with a one-line description, and I'll add a link to the Babel page. I just pushed a set of improvements to the library after being inspired by your lexer. The approach supports using blocking and non-blocking alike. I wanted to be able to parse the data in one go, which is why there are so many different levels of indentation in the central parsing function. Ideas for improvements are appreciated. Cheers Christof -- () ascii ribbon campaign - against html e-mail /\ against proprietary attachments signature.asc Description: PGP signature ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] babeld interface
> https://github.com/christf/libbabelhelper When you feel you're ready, please provide me with a one-line description, and I'll add a link to the Babel page. ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] babeld interface
No, parsing input using string.split is always incorrect. You really want to write a proper lexer. The lexer I posted above is 70 lines of C, including full error checking. In Python, it's probably 15 to 20 lines. Is it really worth switching to a bloated and unreadable format in order to avoid 20 lines of code in each client? Well, it is of course completely up to you where you want to set the barrier to entry for interacting with babeld. And if you want to set that at "being able to write a lexer", well, that is a tradeoff. Just don't be surprised when people try to do sscanf or string.split-style parsing anyway, and subsequently complain about the interface. But then I guess you do like educating people, so maybe that is all well and good... ;) Well, this is what libbabelhelper [1] is for. At the moment it is this horrible sscanf-thing because at the time of writing I did not know any better. But since I use it from two daemons already, I think I might just put the new shiny babel lexer into it and see where this will take us. So if the interface remains as it is, we can still hand users a tool to use to parse it properly. [1] https://github.com/christf/libbabelhelper Christof -- () ascii ribbon campaign - against html e-mail /\ against proprietary attachments signature.asc Description: PGP signature ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] babeld interface
Juliusz Chroboczek writes: >> In this example, parsing the custom format in Python would be something like: > > No, parsing input using string.split is always incorrect. You really > want to write a proper lexer. > > The lexer I posted above is 70 lines of C, including full error > checking. In Python, it's probably 15 to 20 lines. Is it really worth > switching to a bloated and unreadable format in order to avoid 20 > lines of code in each client? Well, it is of course completely up to you where you want to set the barrier to entry for interacting with babeld. And if you want to set that at "being able to write a lexer", well, that is a tradeoff. Just don't be surprised when people try to do sscanf or string.split-style parsing anyway, and subsequently complain about the interface. But then I guess you do like educating people, so maybe that is all well and good... ;) -Toke ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] babeld interface
> The main benefit of a well-known format is that the requirement for a > client goes from easy parsing code to *no* parsing code. Right. So either we add a dependency on JSON to babeld, or we add a trivial parser to every program that consumes babeld's input. > In this example, parsing the custom format in Python would be something like: No, parsing input using string.split is always incorrect. You really want to write a proper lexer. The lexer I posted above is 70 lines of C, including full error checking. In Python, it's probably 15 to 20 lines. Is it really worth switching to a bloated and unreadable format in order to avoid 20 lines of code in each client? The problem that JSON aims to solve is not very complicated, and JSON doesn't solve it particularly well. -- Juliusz ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] babeld interface
> I agree that the current format is very easy to parse for a human, which > is a benefit. I am struggling to have a machine parse this though. You need a lexer. Here's a simple lexer for you: https://www.irif.fr/~jch/software/babel/babel-lexer.c -- Juliusz ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] babeld interface
Juliusz Chroboczek writes: >> Nils and I wrote such a daemon, called mmfd. It is able to forward >> multicast packets in the whole network. To learn about the topology and >> the relevant neighbours, it queries babeld: mmfd is listening via >> "monitor" on the babeld socket. > > Good. > >> What do you think of providing the same data over json to make it better >> parsable? > > The format of the monitoring interface is well defined : it's a series of > lines of the form > > keyword id key value key value ... > > where each key/value is either space separated, or a string within double > quotes. > > This format is extensible (we can add new keywords and new keys at any > time), and it's also easy to parse by a human (Antonin and I have just > spent much of today telnetting by hand into Babel's monitoring > interface). So I'm not convinced that switching to JSON would have any > benefits. The main benefit of a well-known format is that the requirement for a client goes from easy parsing code to *no* parsing code. In this example, parsing the custom format in Python would be something like: data = {} for line in input: keyword, id, rest = line.split(" ", 3) data[keyword] = {'id': id} parts = rest.split(" ") # wait, this doesn't deal with quoted values for key, val in zip(parts[::2], parts[1::2]): data[keyword][key] = val vs data = json.loads(input) Also, if you pretty-print the json it is quite human-readable; try e.g. `cat datafile.json | python -m json.tool` -Toke ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] babeld interface
What do you think of providing the same data over json to make it better parsable? The format of the monitoring interface is well defined : it's a series of lines of the form keyword id key value key value ... where each key/value is either space separated, or a string within double quotes. This format is extensible (we can add new keywords and new keys at any time), and it's also easy to parse by a human (Antonin and I have just spent much of today telnetting by hand into Babel's monitoring interface). So I'm not convinced that switching to JSON would have any benefits. Please let me know if you disagree, I'm not fundamentally opposed to adding a new command "monitor-json" if you think it's desirable, but I'd rather avoid the added complexity. I agree that the current format is very easy to parse for a human, which is a benefit. I am struggling to have a machine parse this though. Just from the switch for babeld-1.8.0 to master a new field got introduced for neighbour lines. This caused my sscanf-based line-parser to break: int n = sscanf(line, "%ms neighbour %*x address %ms if %ms " "reach %x ureach %*x rxcost %d txcost %d cost %d", &action, &address_str, &ifname, &reach, &rxcost, &txcost, &cost); The whole code for the parsing is in a library that lives here: https://github.com/christf/libbabelhelper Maybe I am just not parsing this the easy but the hard way. So before going into the benefits of json, maybe we can see if there is a manufactoring error with my own approach in parsing this. So instead of running sscanf like above, I guess I could strtok the line and just match for known keywords and extract their values, however blanks in values would be a problem to this approach. Do we really need to deal with blanks in values? Christof -- () ascii ribbon campaign - against html e-mail /\ against proprietary attachments signature.asc Description: PGP signature ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users
Re: [Babel-users] babeld interface
> Nils and I wrote such a daemon, called mmfd. It is able to forward > multicast packets in the whole network. To learn about the topology and > the relevant neighbours, it queries babeld: mmfd is listening via > "monitor" on the babeld socket. Good. > What do you think of providing the same data over json to make it better > parsable? The format of the monitoring interface is well defined : it's a series of lines of the form keyword id key value key value ... where each key/value is either space separated, or a string within double quotes. This format is extensible (we can add new keywords and new keys at any time), and it's also easy to parse by a human (Antonin and I have just spent much of today telnetting by hand into Babel's monitoring interface). So I'm not convinced that switching to JSON would have any benefits. Please let me know if you disagree, I'm not fundamentally opposed to adding a new command "monitor-json" if you think it's desirable, but I'd rather avoid the added complexity. -- Juliusz ___ Babel-users mailing list Babel-users@lists.alioth.debian.org http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/babel-users