Re: [kbuild-devel] RFC: kernel config: new dependency syntax
[I wrote]
> I've come up with syntax I think I'm happy with.
Thank you one and all for all the discussion and suggestions for
improvement on my proposals. I've incorporated a lot of feedback.
I know I promised to try and come up with a working prototype
including some Config.in files, but some of the Menuconfig bits were
harder than they looked, and I ran out of time tonight - I'm about to
leave town for a couple of days. My Menuconfig currently doesn't
work, and I haven't converted any Config.in files yet. Meanwhile,
here is the documentation portion. Critiques are more than welcome,
both for the proposal and for its documentation..
Peter
--- 2.5.31/Documentation/kbuild/config-language.txt 2002-06-09 00:29:17.0
-0500
+++ 2.5.31w/Documentation/kbuild/config-language.txt2002-08-16 21:18:16.0
+-0500
@@ -84,8 +84,10 @@
to generate dependencies on individual CONFIG_* symbols instead of
making one massive dependency on include/linux/autoconf.h.
-A /dep/ is a dependency. Syntactically, it is a /word/. At run
-time, a /dep/ must evaluate to "y", "m", "n", or "".
+A /dep/ is a dependency list. Syntactically, it is zero or more
+/word/s separated by whitespace. When evaluated, a /dep/ is
+reduced to a single character "y", "m" or "n". See the following
+section for details.
An /expr/ is a bash-like expression using the operators
'=', '!=', '-a', '-o', and '!'.
@@ -116,12 +118,12 @@
Dependent statements:
-dep_bool/prompt/ /symbol/ /dep/ ...
-dep_mbool /prompt/ /symbol/ /dep/ ...
-dep_hex /prompt/ /symbol/ /word/ /dep/ ...
-dep_int /prompt/ /symbol/ /word/ /dep/ ...
-dep_string /prompt/ /symbol/ /word/ /dep/ ...
-dep_tristate/prompt/ /symbol/ /dep/ ...
+dep_bool/prompt/ /symbol/ /dep/
+dep_mbool /prompt/ /symbol/ /dep/
+dep_hex /prompt/ /symbol/ /word/ /dep/
+dep_int /prompt/ /symbol/ /word/ /dep/
+dep_string /prompt/ /symbol/ /word/ /dep/
+dep_tristate/prompt/ /symbol/ /dep/
Unset statement:
@@ -132,7 +134,22 @@
choice /prompt/ /word/ /word/
nchoice /prompt/ /symbol/ /prompt/ /symbol/ ...
-If statements:
+New-style if blocks:
+
+if_dep /dep/
+ /statement/
+ ...
+fi_dep
+
+if_dep /dep/
+ /statement/
+ ...
+else_dep
+ /statement/
+ ...
+fi_dep
+
+Old-style if blocks (deprecated):
if [ /expr/ ] ; then
/statement/
@@ -161,6 +178,69 @@
+=== Dependency lists
+
+A dependency list evaluates to a single character "y", "m" or "n".
+Individual words in the list can include:
+
+The word "or" divides a list into two sub-lists, one on either
+side of the "or". The sub-lists are evaluated independently and
+then combined, as described below.
+
+A /symbol/ is expanded to the current value of the symbol, or the
+empty string if the symbol is unset. In the latter case, if
+/symbol/ appears in a word by itself, the word is ignored. This
+is for backward compatibility and may change in the future.
+
+If a /symbol/ has a "$" prefix, that prefix is ignored, for
+backward compatibility. This may become a syntax error in the
+future.
+
+A word with a suffix of "=X", where X is a quoted or unquoted
+string, expands to "y" if the rest of the word evaluates to X, or
+"n" otherwise.
+
+A word prefixed by "!" expands to a transformation of the word:
+"!y" becomes "n"; "!n" or "!" become "y"; "!m" becomes "m". The
+"!" prefix is expanded only after expansion of /symbol/s and the
+"=" suffix.
+
+Once each word is expanded as described above, it should have a value
+of "y", "m" or "n", or the empty value "", which is discarded. If the
+dependency list has multiple words, the words are reduced to a single
+word as follows:
+
+If any word is "n", the value of the list becomes "n".
+
+Otherwise, if any word is "m", the value of the list becomes "m".
+
+Otherwise, the value of the list becomes "y".
+
+If there are multiple sub-lists separated by "or" words, each sub-list
+is reduced to a single word as described above, then they are combined
+as follows:
+
+ If any sub-list is "y", the final result is "y".
+
+ Otherwise, if any sub-list is "m", the final result is "m".
+
+ Otherwise, if all sub-lists are "n", the final result is "n".
+
+Finally, an empty /dep/ evaluates to "y".
+
+If you think in terms of Boolean algebra, most of the above rules make
+sense if you think of each primitive value ("y", "m" and "n") as two
+bits: "y"=11, "m"=01, "n"=00. Adjacent words are implicitly ANDed
+together, and the "or" statement, with lower precedence, performs an
+OR between lists of words. The "!" operator does not quite fit
Re: [kbuild-devel] RFC: kernel config: new dependency syntax
Hi,
On Fri, 16 Aug 2002, Peter Samuelson wrote:
> Basically the current discussion revolves around the best way to
> evolve the config language to make it more suitable for its purpose.
> This is of course in contrast to what ESR and you have tried, which is
> to replace the whole thing.
I haven't given up yet. :)
> * The current 'if' statement is really ugly and unintuitive,
Is that problem really that big? I think most kernel hackers can deal with
that. Of course this needs fixing, but there are other more important
problems. Actually this one of the simple problems my converter can solve
automatically.
> * Current 'if' semantics are hard to get right in many common cases.
> For example, a guard variable may be either unset ("") or False
> ("n"). If you want something to depend on the guard variable, you
> have to cover both cases with AND or OR. Many people do not think
> to do this.
Many people don't have to think about this, because most dependencies are
quite simple. (This problem also becomes irrelevant with the new parser.)
> None of that answers your real question: why not use either CML2 or
> your work? Answer: evolution, not revolution. You say you have tried
> hard to keep the existing rulebase functionally identical (something
> Eric, of course, did not do). Fine, but so did Keith Owens when he
> wrote kbuild 2.5, and look where that got him.
It's not really the same. CML2 and kbuild 2.5 both have the same problem -
the "fix all problems at once + a little bit more" syndrome.
Kai instead concentrated on the most important problem - correct
dependencies. Most of the surgery was concentrated on Rules.make and the
changes to the Makefiles were not only cleanups, but helped to fix the
main problem.
What is now the main problem with the config system? The shell based
dependency syntax? It's an annoying problem, but not really that serious.
The biggest problem are still two completely different parsers, which
limit the language syntax. To solve this problem we need to drop one or
both parsers. The shell parsers are the cause of the language
limitations, which force us to keep the syntax imperative and makes it
difficult to add new information. Nobody wants to touch xconfig.
So the only solution I see is to drop both parsers. You can evolutionize
as much as you want, but it doesn't help to solve the main problem. On the
other hand as soon as the main problem is solved, lots of the small
problems can be solved for free.
> I'm taking pretty much the same approach: removing warts from the
> current config system, one by one, to see how clean it can get.
> Perhaps eventually someone will be able to merge in a completely
> different system, but if so, as Greg has said, what we are doing now
> will lay the ground-work for such a change, by removing a lot of
> kludges from the rulebase.
Try to make a list of all the problems, try to prioritize them and
estimate the needed changes to rulebase and parsers. Now try to minimize
the amount of work needed to solve all the problems.
You currently ignore the xconfig problem, which seem to make most of the
problems easily solvable, but we possibly end up with a broken X
interface. The new parser with the new frontends is the largest amount of
the work, which needs to be done anyway and once it's done, it will solve
a lot of the small problems automatically.
I don't think that making multiple large syntax changes is a good idea.
Learning a new syntax is already annoying, but having to do it multiple
times will really annoy a lot of people. Making some small controlled
extensions or small rulebase fixes are not the problem, but you should
really try to avoid changes which require massive changes to all config
files.
bye, Roman
---
This sf.net email is sponsored by: OSDN - Tired of that same old
cell phone? Get a new here for FREE!
https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390
___
kbuild-devel mailing list
[EMAIL PROTECTED]
https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] RFC: kernel config: new dependency syntax
[Roman Zippel]
> I really hate to spoil the fun
(:
> but could someone explain to me, why this is necessary? What
> problems does that fix?
It's not necessary, technically - it doesn't directly fix any bugs.
It falls under "cleanup", and as such, it is supposed to make bugs
harder to write and easier to spot. The problems it solves are
problems of readability and awkwardness in the current [Cc]onfig.in
corpus.
Basically the current discussion revolves around the best way to
evolve the config language to make it more suitable for its purpose.
This is of course in contrast to what ESR and you have tried, which is
to replace the whole thing.
My main goal is to make it easier to write Config.in files, by making
the syntax and semantics less awkward. Certainly one side effect is
that it will be possible to do automatic "(EXPERIMENTAL)" and
"(OBSOLETE)" tags, but that has never been the primary intent.
* The current 'if' statement is really ugly and unintuitive, unless
you are comparing it to Bourne shell syntax, in which case it is
merely ugly and *somewhat* unintuitive. (It is still unintuitive
because it offers only a subset of Bourne shell functionality, and
that subset is arbitrarily defined.) If you use Bourne shell syntax
outside the config language spec, your file may break subtly in
Configure or Menuconfig, or spectacularly in xconfig (since xconfig
doesn't use the shell). Since most people don't test their changes
on all three parsers, they may not notice.
* Current 'if' semantics are hard to get right in many common cases.
For example, a guard variable may be either unset ("") or False
("n"). If you want something to depend on the guard variable, you
have to cover both cases with AND or OR. Many people do not think
to do this.
I am trying to design a replacement statement which *does* cover
common cases easily, while not losing any of the flexibility we have
now. The current state of my proposal gives quite a bit *more*
functionality than we have now - some config files have the
potential to become considerably simpler / shorter.
* By having *one* standard syntax for dependency declarations, the
whole language spec is more consistent and, incidentally, easier to
write a parser for. (The fact that *human* parsing is easier as
well is a related but separate advantage.)
None of that answers your real question: why not use either CML2 or
your work? Answer: evolution, not revolution. You say you have tried
hard to keep the existing rulebase functionally identical (something
Eric, of course, did not do). Fine, but so did Keith Owens when he
wrote kbuild 2.5, and look where that got him. I have looked through
the kbuild2.5 makefiles, and there were many places where Keith
deliberately enforced things that looked completely arbitrary (link
order in particular) merely for the sake of having an apples-to-
apples comparison.
So what did Linus do? He refused to take Keith's patches directly,
but instead accepted Kai's incremental fixups to the current build
system, which is nothing like Keith's system. Kai, in turn, made so
many tweaks to the current system that it now resembles - in
functionality if not in implementation - kbuild2.5. (Yes, the current
build system is missing some features, but considering that there have
been no radical changes, it's pretty impressive what Kai has managed
to wring out of it.)
I'm taking pretty much the same approach: removing warts from the
current config system, one by one, to see how clean it can get.
Perhaps eventually someone will be able to merge in a completely
different system, but if so, as Greg has said, what we are doing now
will lay the ground-work for such a change, by removing a lot of
kludges from the rulebase.
> The only reason I've seen so far is to make the generation of
> "(EXPERIMENTAL)" possible. Is this really that important to risk new
> problems?
I happen to think the risk of new problems is relatively low. My
experience with Menuconfig so far is that the types of features I am
adding are relatively straightforward, and it's surprisingly easy to
verify that things are all working correctly. I suppose xconfig will
be the usual nightmare - perhaps Andrzej (the de facto xconfig
maintainer) will help out when the time comes - but Configure and
Menuconfig have presented no major problems so far.
There *is* the risk of adding bugs when changing the rulebase, but
those will be isolated to individual rule files, and should be easy to
debug if and when they happen.
Of course, Menuconfig is probably the easiest of the three, since it
explicitly requires bash (Configure, OTOH, works with any /bin/sh), so
I don't have to worry much about shell compatibility. I guess it
wouldn't hurt to install bash 1.14 somewhere so I can make sure things
will work on Red Hat 6.0. An old bash bug bit me once before, in
Linux 2.2.19pre
Peter
---
This
Re: [kbuild-devel] RFC: kernel config: new dependency syntax
Hi, On Fri, 16 Aug 2002, Kai Germaschewski wrote: > Oh well, I think the only way to find out if all that is really a good > idea is to try, convert some config.in's and look at the result. I really hate to spoil the fun, but could someone explain to me, why this is necessary? What problems does that fix? The only reason I've seen so far is to make the generation of "(EXPERIMENTAL)" possible. Is this really that important to risk new problems? bye, Roman --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390 ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] RFC: kernel config: new dependency syntax
[Kai Germaschewski] > Seeing it from that point of view, it may actually turn into > something which can agree with much more easily. Great! I've been hoping for your support - not only because I respect your judgment, but also because Linus takes patches from you. (: > Maybe it actually suffices to declare if_dep, fi_dep as "virtually > adds the listed arguments to each statement in the enclosed > region". Maybe there would be a better name for that, then if_dep, > OTOH using if_dep makes it quite clear what else_dep does. OK.. > Make every statement accepts dependencies (and depreciate the dep_ > prefixes, since they become unnecessary). There really isn't any > reason for the difference between bool and dep_bool right now, it > only leads to confusion and mistakes. I've proposed this in the past - well, at least the part about making bool == dep_bool or dep_mbool, and tristate == dep_tristate. I was going to raise the issue again, after the if_dep debate settled down some. > I didn't look into like choice statements, but I'd hope it's > possible to add dependencies to them, too, for consistency. I agree. Actually, if we're changing 'choice' anyway, it should be redesigned. Status quo takes three arguments: choice 'prompt string' 'choice-text-1 CONFIG_CHOICE1 choice-text-2 CONFIG_CHOICE2' 'choice-text-default' This is bad because it pretty much requires multi-line strings, and because the choice texts cannot have spaces in them. In my opinion it should take a variable number of args: choice 'prompt string' 'choice text 1' CONFIG_CHOICE1 'choice text 2' CONFIG_CHOICE2 CONFIG_CHOICE_DEFAULT I suppose you could then extend it with dependencies by putting those at the end. That makes for minor parsing problems, since you have to detect which arguments are which - basically I guess you look for two CONFIG_* in a row. Alternatively, we could make the parsing job a little less fragile by using an explicit keyword like 'dep': choice 'prompt' 'text 1' CONFIG_1 'text 2' CONFIG_2 'text 3' CONFIG_3 CONFIG_1 dep CONFIG_ISA !CONFIG_ARM Translation: CONFIG_1 is the default, but the prompt is suppressed if CONFIG_ISA is unset, or if CONFIG_ARM is set. Eh? > Extend the normal dep_* dependencies (now just called *) to accept > the same kind of dep lines as if_dep, i.e. !, =[ymn], !=[ymn], 'or', > 'and'. Already in my patches. Actually, I don't have an 'and' (Greg asked about this too), for two reasons. First, adjacent words are implicitly ANDed together anyway, so there's no need. Second, since OR takes low precedence and the implicit AND takes high precedence, having an explicit 'and' keyword would generate confusion about its precedence with regards to 'or'. I already rejected trying to do any sort of grouping - mostly because the shell parsers would choke on ( ) and nothing else looks as nice - so, since there is no way to group things, the precedence table needs to be as simple and clear as possible. I think my current proposal fits the bill. > Also, when doing that conversion, I think it may be worthwhile to get rid > of the distinction between mbool and bool by something like, changing > > dep_mbool 'CAPI2.0 filesystem support' CONFIG_ISDN_CAPI_CAPIFS >$CONFIG_ISDN_CAPI_CAPI20 > > to > > bool 'CAPI2.0 filesystem support' CONFIG_ISDN_CAPI_CAPIFS' CONFIG_ISDN_CAPI!=n > > I think a sensible default would be m -> y in this case (like > mbool) That's the current default for if_dep already. I do not know if it is worthwhile to add a "!=" operator to complement the "=" operator, but perhaps it is, if only because people will expect it. I'll go ahead and add it to my Menuconfig patch, at least for now. It should be noted that in many cases, this whole distinction is unimportant, because the guard symbol (for example CONFIG_ISA) is already boolean. > I'd be interested to see if these statements are actually sufficient > to express all we've got today. [...] > Oh well, I think the only way to find out if all that is really a > good idea is to try, convert some config.in's and look at the > result. Right. I'll try to find the time to convert a few large Config.in / config.in files today, and post a patch (along with my current version of Menuconfig). Speaking of which - would it help anyone evaluate this stuff if I produced a patched Configure to complement the patched Menuconfig? I haven't been hacking on Configure, because for my purposes, stuff is easier to test with Menuconfig. (I added a line to init/Config.in that does "source test.in", so I can easily put tests in $(TOPDIR)/test.in. Works great.) > (The CONFIG_FOO=n case would still run through the statements > included by if_dep / fi_dep, automatically setting all vars to 'n', > since one of their deps is 'n') Reasonable people can disagree on this one. '' today means the user has never seen the question, so you would be changing existing semantics. I for one am fine with that, but others will probably argu
Re: [kbuild-devel] RFC: kernel config: new dependency syntax
On Thu, 15 Aug 2002, Peter Samuelson wrote: > The more I think about it, the more I think the default if_dep should > do the m-restricting thing. That way: > > dep_bool FOO1 BAR BAZ > dep_mbool FOO2 BAR BAZ > dep_tristate FOO3 BAR BAZ > > is exactly equivalent to > > if_dep BAR BAZ > bool FOO1 > mbool FOO2 > tristate FOO3 > fi_dep > Of course, that requires the invention of 'mbool'. But I do believe > it covers most if not all common cases. I guess in this case, though, > it's still an open question whether 'define_bool' should be immune > from the M effect (like 'mbool') or not. Seeing it from that point of view, it may actually turn into something which can agree with much more easily. Maybe it actually suffices to declare if_dep, fi_dep as "virtually adds the listed arguments to each statement in the enclosed region". Maybe there would be a better name for that, then if_dep, OTOH using if_dep makes it quite clear what else_dep does. Then, there would be some more steps, which could be easily done: Make every statement accepts dependencies (and depreciate the dep_ prefixes, since they become unnecessary). There really isn't any reason for the difference between bool and dep_bool right now, it only leads to confusion and mistakes. I didn't look into like choice statements, but I'd hope it's possible to add dependencies to them, too, for consistency. Extend the normal dep_* dependencies (now just called *) to accept the same kind of dep lines as if_dep, i.e. !, =[ymn], !=[ymn], 'or', 'and'. Also, when doing that conversion, I think it may be worthwhile to get rid of the distinction between mbool and bool by something like, changing dep_mbool 'CAPI2.0 filesystem support' CONFIG_ISDN_CAPI_CAPIFS $CONFIG_ISDN_CAPI_CAPI20 to bool 'CAPI2.0 filesystem support' CONFIG_ISDN_CAPI_CAPIFS' CONFIG_ISDN_CAPI!=n and dep_bool ' AVM A1 (Fritz)' CONFIG_HISAX_AVM_A1 $CONFIG_ISA to bool ' AVM A1 (Fritz)' CONFIG_HISAX_AVM_A1 CONFIG_ISA=y I think a sensible default would be m -> y in this case (like mbool), which would allow to handle 95% of the cases with just bool 'CAPI2.0 filesystem support' CONFIG_ISDN_CAPI_CAPIFS' CONFIG_ISDN_CAPI bool ' AVM A1 (Fritz)' CONFIG_HISAX_AVM_A1 CONFIG_ISA (both cases are equivalent to the above, CONFIG_ISA is a bool) If the real old "dep_bool" is needed, it can be written as "CONFIG_FOO=y", in most cases "CONFIG_FOO!=n" <=> "CONFIG_FOO" will be what's needed. I'd be interested to see if these statements are actually sufficient to express all we've got today. A nice side effect of the above definition of if_dep would be that bool '...' CONFIG_FOO if [ "$CONFIG_FOO" = "y" ]; then bool '...' CONFIG_BAR [...] else define_bool CONFIG_BAR n fi can turn into just bool '...' CONFIG_FOO if_dep CONFIG_FOO bool '...' CONFIG_BAR [...] fi_dep (The CONFIG_FOO=n case would still run through the statements included by if_dep / fi_dep, automatically setting all vars to 'n', since one of their deps is 'n') OTOH, one of the problems I can see already is that bool '...' CONFIG_FOO if [ "$CONFIG_FOO" = "y" ]; then bool '...' CONFIG_BAR [...] else define_bool CONFIG_BAR y fi doesn't translate well. (Hacks I can see would be: bool '...' CONFIG_FOO if_dep CONFIG_FOO bool 'not ...' CONFIG_NOT_BAR [...] fi_dep define_bool CONFIG_BAR !CONFIG_NOT_BAR or bool '...' CONFIG_FOO if_dep CONFIG_FOO bool '...' CONFIG_FOO_BAR [...] fi_dep define_bool CONFIG_BAR CONFIG_FOO_BAR or !CONFIG_FOO both not particularly nice, but if there's only like one or two cases like this in the current config.in's, it may be acceptable) Oh well, I think the only way to find out if all that is really a good idea is to try, convert some config.in's and look at the result. --Kai --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390 ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] Re: [patch] config language dep_* enhancements
Hi, On Fri, 16 Aug 2002, Greg Banks wrote: > The easy targets being done now are mostly things that I believe would need > to be done regardless of the eventual strategy, be it a) do nothing b) make > the existing system suck less c) replace the parsers and keep the rules > d) replace everything. For any of these strategies to be successful you would > need to start with a clean clear and consistent rules corpus. The problem here is one should consider, how all these little changes will help to solve the big problems. Do they allow to more easily fix the big problems or have these changes to be dumped again? Most of the suggestions I've seen so far fix problems, which either can be either fixed automatically or which don't exists anymore, once we switch to a new syntax/parser. That's the reason I ask to understand the whole picture, so we can judge whether a change is really necessary or not. > Remember how people were complaining that ESR couldn't prove that the CML2 > rules corpus did the same things as the CML1 rules corpus? One of the > reasons was that the CML1 rules corpus is so screwed that's its impossible > for either a human or a machine to figure out what was supposed to happen > and whether what was actually happening was deliberate. I can't give you a mathematical proof, but I tried very hard to keep the behaviour the same. Unless I made mistake the rules are almost exactly the same. Most of the CML1 rules are usable, there are only very few cases which need manual fixing. I can't guarantee that where won't be any surprises, but they should be easily fixable in the new system. (Unless ESR I don't insist that my rulebase is correct or perfect, so I'm open to suggestion/changes. :) ) > This is why I'm not talking about replacing shell based parsers yet. First > we need to get a rules corpus for which it is possible to create a parser > which can parse cleanly, consistently, and correctly. Most of these problems can actually be fixed without syntax changes. Something that can't be sanely fixed this way are recursive dependencies, which I think are not worth fixing with the old parsers, but which are easily fixable with the new syntax. If you want to fix logical errors in the rulebase, they will be more easily fixable with the new tools. For the X interface I'm planning some debug options, which e.g. allow you to see the complete dependencies of every symbol. bye, Roman --- This sf.net email is sponsored by: OSDN - Tired of that same old cell phone? Get a new here for FREE! https://www.inphonic.com/r.asp?r=sourceforge1&refcode1=vs3390 ___ kbuild-devel mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/kbuild-devel
