On Sunday, 27.12.2015 at 11:06, Antti Kantee wrote: > On 21/12/15 14:25, Martin Lucina wrote: > >Given that the existing JSON format was just something that "fell out of > >the implementation", part (1) involved not only refactoring the > >specification, but also doing a complete rototill of the rumprun_config() > >implementation to match. > > > >This work is now feature-complete (i.e. it does everything the existing > >configuration implementation does) and ready for review on the > >'mato-wip-rumprun-config' branch on Github: > > > >https://github.com/rumpkernel/rumprun/tree/mato-wip-rumprun-config > > I'm still in slow-start xmas recovery mode, but to expedite the process of > getting the config revamp done, I'll point out what on first skim looks like > the big problem, and can do a more thorough review next(ish) week.
Thanks. > First, out of curiosity, IIRC a month or two ago you said that you didn't > want json as the human-interface config format, since it's somehow > difficult/unfashionable/etc. I don't particularly care what the format is > as long as it's something standard -- the union of opinions is that all > formats suck -- but what made you settle on json after all? I had several out-of-band conversations with people about config formats over the last month or so, including people who manage upwards of 10^5 systems whose opinion I trust. The alternatives discussed were JSON, YAML, XML, or "custom UNIX shell-script/Cisco/other style". Given the goals of: 1. human editability 2. widest possible software interoperability 3. standard, extensible format which will last us for years 4. straightforward and lightweight to parse in C with few dependencies Goals 2 and 3 rule out anything in the "custom" department. Goal 4 rules out (most of) XML. JSON gives the best possible compromise between 1 and 2. YAML is more human-friendly (supports comments, less syntax) but has less existing implementations and is more heavyweight to parse. Also, other projects I've worked on in the past couple of months have lead me to realise that in the long run, the majority of users will probably be using rumprun as part of some other higher-level stack, so human-friendly config files are less important than I originally thought. > Now, you've replaced the working json parser which was running below libc > with another json parser which is running on top of libc. What was the > motivation for replacing one json parser with another? No matter what the > answer, the result is not acceptable, since it effectively makes it > impossible to support kernelonly-mode config with the same parser. Jsmn is not a good enough json parser to write a config parser which can handle arbitrarily json structures easily. In fact, it (jsmn) is technically only a tokeniser, so in order to handle unknown-but-valid data in a robust fashion I'd have had to hand-code a bunch of state machines for each particular data structure. I tried this for several days, and the resulting code was both hard to adapt to changes in the JSON structure and totally unmaintaintable. Hence, I went looking for a real parser which builds a proper parse tree, and found the one in jsoncvt. This works very well for our purposes -- e.g. it does not build any kind of custom data structures, and the resulting config code is easy to follow. Regarding the dependency on libc interfaces, rumprun_base (or at least config.c) already depends on libc interfaces today -- I wasn't aware of any documented requirement for it not to. > Generally speaking, the bits of the config which make sense for kernonly > (e.g. network address configuration) should be written on top of > bmk/rumpkernel where at all feasible, not libc. That rewrite is on one hand > a separate change, but on the other hand it must not be torpedoed as part of > a config format revamp. Fair point, however I was not aware of what your plans were with kernonly mode as it is still documented as "experimental", much less that "supporting kernonly" was a requirement for config.c. The number of interfaces needed by the new json parser is not huge (mainly str* and ctype, the latter of which could be dropped if we explicitly specify the config as ASCII) so it could be adapted to work under bmk directly. Martin
