[kbuild-devel] RFC - kconfig date must die
IOW: can we switch the default of CONFIG_NODATESTAMP ? Clearly, somebody has already thought about stripping the date from .configs, but it seems to me that this has been forgotten, and warrants a fresh look. - I searched http://marc.theaimsgroup.com/?l=kbuild-develr=1b=200604w=2 for CONFIG_NODATESTAMP, got *no* hits. How long ago was this added ? Since it has been so long, I'll state some obvious benefits of dropping the date: - its the date of the last 'save' or make oldconfig (true even w.o changes), - not the last real change made, compiled, installed, tested. - ls -l would tell the same, - cvs/../git would tell *much* more. IE its usually a meaningless difference If we drop it, then we get: - less noise when comparing configs - each config is unique and fingerprintable. - kernel gets the configure-print, builtin, as /proc/config.md5 or something. - this fingerprint is orthogonal to CONFIG_MODULE_SRCVERSION_ALL This has Many Implications - 'known' configs Each kernel release 'creates' a bunch of known configs (A*S*M of them) A - all the arches S - each arch's sub-arches M - make-targets: allnoconfig, allyesconfig, allmodconfig, etc With date gone, these are trivially reproducible, byte for byte. Bug reports with attached config-vs-x86-64-all-defaults.diff become meaningful, and arguably better than straight attachments (since theyre shorter, and refer to a well understood (and completely repeatable) standard). 'make standard-configs' This notional target would generate a set of M standard configs, and name them appropriately: config-$M foreach $M (@maketarget) Just recently on LKML, Adrian Bunk rightly complained about a bug-report with a partial .config, since its tedious to reproduce the problem, and OP noted he didnt want to 'spam' the list. If the OP had seen a config-alldefaults file in his build-directory, he would likely have sent a diff against it, and Adrian would have a way to reproduce it. Blue-Sky / Garden-Path As releases happen, these reference configs could be stored to SCM, with references to previous releases. The changes that are tracked this way are the cumulative results of all the accepted/committed work. - QA (extra hand-wavery here) When bugs are filed, the dmesg should/will contain a config-md5. The md5 will instantly tell us if its a vanilla config, or tinkered and tweaked one. Suppose a kernel gets out the door with a missing kconfig dependency, if the 1st 5 bug-reports have a config.diff-vs-$MyArch-alldefaults, it should help streamline/automate the corrective procedures, which Id describe currently as: Andrew Morton emails X about his problem. (perhaps described currently as: MM emails X) config.diffs that expose missing dependencies can become regression tests. If configs are permuted and (re)-generated, fingerprints can be taken, along with a status (randomly-generated). If/when these configs are compile-tested, errs also become regression tests. - make config-print gather status: the configureprint, installed and running status, whatevers relevant, and send to [EMAIL PROTECTED] -rcX and -mm trees could conceivably benefit from information pumped back by interested hackers (esp if its easy as 'make qa') -- a config-spectrum-analyser Even though the md5 doesnt say much about what features are being configured into a submitters kernel, it is enough to 'histogram'. Doing so would show us the distribution of configurations being tested over a given period. WRT test coverage, a very spiky 'config-spectrum-analysis' would support the view that in -MM, all CONFIG_ items should default to [YM], so at least the code gets built. Extreme spikiness might highlight the need for a kind of [EMAIL PROTECTED] project, which just distributes a permute-and-test task amongst available machines. Sounds too much like PR fodder, but... Ill stop here, ( I may have floated too many half-described ideas already) What do we need to do to bury the date ? Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ kbuild-devel mailing list kbuild-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] RFC - kconfig date must die
On Mon, Jun 26, 2006 at 04:04:24PM -0600, Jim Cromie wrote: ... Since it has been so long, I'll state some obvious benefits of dropping the date: ... If we drop it, then we get: - less noise when comparing configs - each config is unique and fingerprintable. - kernel gets the configure-print, builtin, as /proc/config.md5 or something. - this fingerprint is orthogonal to CONFIG_MODULE_SRCVERSION_ALL I don't see the big advantage of any of them. This has Many Implications - 'known' configs Each kernel release 'creates' a bunch of known configs (A*S*M of them) A - all the arches S - each arch's sub-arches M - make-targets: allnoconfig, allyesconfig, allmodconfig, etc With date gone, these are trivially reproducible, byte for byte. Bug reports with attached config-vs-x86-64-all-defaults.diff become meaningful, and arguably better than straight attachments (since theyre shorter, and refer to a well understood (and completely repeatable) standard). There are defconfigs for this purpose. Since diff's aren't always shorter (they can even be significantely larger) than the file. At most times the .config alone is the important thing to read, only in some rare cases the diff compared to another .config is the interesting thing. And in the latter case, don't assume the person requiring it was too dumb to use diff himself. 'make standard-configs' This notional target would generate a set of M standard configs, and name them appropriately: config-$M foreach $M (@maketarget) Already present for ages, called defconfig. Just recently on LKML, Adrian Bunk rightly complained about a bug-report with a partial .config, since its tedious to reproduce the problem, and OP noted he didnt want to 'spam' the list. If the OP had seen a config-alldefaults file in his build-directory, he would likely have sent a diff against it, and Adrian would have a way to reproduce it. The size is not an issue. And if it was, bzip2 would beat your proposal easily for this purpose. Blue-Sky / Garden-Path As releases happen, these reference configs could be stored to SCM, with references to previous releases. The changes that are tracked this way are the cumulative results of all the accepted/committed work. Call them defconfig, and it's already implemented. - QA (extra hand-wavery here) When bugs are filed, the dmesg should/will contain a config-md5. The md5 will instantly tell us if its a vanilla config, or tinkered and tweaked one. Using a vanilla config wouldn't make sense for anyone. Suppose a kernel gets out the door with a missing kconfig dependency, if the 1st 5 bug-reports have a config.diff-vs-$MyArch-alldefaults, it should help streamline/automate the corrective procedures, ... Wrong. diff's of .config's are harder to read than .config's. config.diffs that expose missing dependencies can become regression tests. ... These are usually bugs that don't occur again. -- a config-spectrum-analyser Even though the md5 doesnt say much about what features are being configured into a submitters kernel, it is enough to 'histogram'. Doing so would show us the distribution of configurations being tested over a given period. ... With the exception of distribution kernels, the probability that someone else uses exactly the same .config I'm using is nearly zero, since the number of possible different .config's on a given architecture is something in the order of 2^1000 or one (american) billion. Extreme spikiness might highlight the need for a kind of [EMAIL PROTECTED] project, which just distributes a permute-and-test task amongst available machines. Sounds too much like PR fodder, but... ... Testing kernels is an important topic, but it can't work the way you think of it. cu Adrian -- Is there not promise of rain? Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. Only a promise, Lao Er said. Pearl S. Buck - Dragon Seed Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ kbuild-devel mailing list kbuild-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] RFC - kconfig date must die
Adrian Bunk wrote: On Mon, Jun 26, 2006 at 04:04:24PM -0600, Jim Cromie wrote: ... Since it has been so long, I'll state some obvious benefits of dropping the date: ... If we drop it, then we get: - less noise when comparing configs - each config is unique and fingerprintable. - kernel gets the configure-print, builtin, as /proc/config.md5 or something. - this fingerprint is orthogonal to CONFIG_MODULE_SRCVERSION_ALL I don't see the big advantage of any of them. 'big' would be an overstatement, but this following property is un-good 1019 cp .config config-ok 1020 make oldconfig 1021 diff .config config-ok $ diff .config config-ok 4c4 # Mon Jun 26 19:29:21 2006 --- # Sun Jun 25 07:50:27 2006 granted, its little more than a speed-bump, but we like smooth roads.. This has Many Implications - 'known' configs Each kernel release 'creates' a bunch of known configs (A*S*M of them) A - all the arches S - each arch's sub-arches M - make-targets: allnoconfig, allyesconfig, allmodconfig, etc With date gone, these are trivially reproducible, byte for byte. Bug reports with attached config-vs-x86-64-all-defaults.diff become meaningful, and arguably better than straight attachments (since theyre shorter, and refer to a well understood (and completely repeatable) standard). There are defconfigs for this purpose. Since diff's aren't always shorter (they can even be significantely larger) than the file. That suggests they diff'd against the wrong thing. diffing vs allmodconfig will generally be *far* better than vs allnoconfig. At most times the .config alone is the important thing to read, only in some rare cases the diff compared to another .config is the interesting thing. And in the latter case, don't assume the person requiring it was too dumb to use diff himself. 'dumb' wasnt what I was thinking, rather the opposite. In the case mentioned below, the OP thought about what to send. That he was 'wrong' suggests it will happen again. I could easily have done the same. I'll not pretend to have much practice at debugging a .config, I always start by looking at a diff against a good .config to see where I screwed up. 'make standard-configs' This notional target would generate a set of M standard configs, and name them appropriately: config-$M foreach $M (@maketarget) Already present for ages, called defconfig. ack. I guess nobody who reports a problem sends a config-vs-defconfig.diff, So I have to doubt my inferences, but.. - Just recently on LKML, Adrian Bunk rightly complained about a bug-report with a partial .config, since its tedious to reproduce the problem, and OP noted he didnt want to 'spam' the list. If the OP had seen a config-alldefaults file in his build-directory, he would likely have sent a diff against it, and Adrian would have a way to reproduce it. The size is not an issue. And if it was, bzip2 would beat your proposal easily for this purpose. bzips are not readable as is, and folks with knowledge to recognize broken-ness may not have the inclination to unzip and see. Size was an issue for the OP, so he sent a fragment that he thought was the salient info. If he had several 'defconfigs', he could have diff'd them all, just to see which was the smallest, and sent that. This focus on size is unfortunate; its only an approximation of the real differences that may or may-not be a factor in the problem Blue-Sky / Garden-Path As releases happen, these reference configs could be stored to SCM, with references to previous releases. The changes that are tracked this way are the cumulative results of all the accepted/committed work. Call them defconfig, and it's already implemented. - QA (extra hand-wavery here) When bugs are filed, the dmesg should/will contain a config-md5. The md5 will instantly tell us if its a vanilla config, or tinkered and tweaked one. Using a vanilla config wouldn't make sense for anyone. OK. I'll infer this means that a vanilla config shouldnt be expected to boot, and therefore that its not an ideal basis for comparing others, I guess you mean that I dont mean to be obtuse, but what is the value of a vanilla config Suppose a kernel gets out the door with a missing kconfig dependency, if the 1st 5 bug-reports have a config.diff-vs-$MyArch-alldefaults, it should help streamline/automate the corrective procedures, ... Wrong. diff's of .config's are harder to read than .config's. .. line for line. but if the diff is 3 dozen lines instead of 1528 lines, Id think it an improvement. config.diffs that expose missing dependencies can become regression tests. ... These are usually bugs that don't occur again. Ack. Still, missing dependencies isnt the only purpose. Some bugs will only manifest in unusual configs (which contributes to their persistence) -- a
Re: [kbuild-devel] RFC - kconfig date must die
On Tue, Jun 27, 2006 at 01:53:02PM -0600, Jim Cromie wrote: Adrian Bunk wrote: On Mon, Jun 26, 2006 at 04:04:24PM -0600, Jim Cromie wrote: ... Since it has been so long, I'll state some obvious benefits of dropping the date: ... If we drop it, then we get: - less noise when comparing configs - each config is unique and fingerprintable. - kernel gets the configure-print, builtin, as /proc/config.md5 or something. - this fingerprint is orthogonal to CONFIG_MODULE_SRCVERSION_ALL I don't see the big advantage of any of them. 'big' would be an overstatement, but this following property is un-good 1019 cp .config config-ok 1020 make oldconfig 1021 diff .config config-ok $ diff .config config-ok 4c4 # Mon Jun 26 19:29:21 2006 --- # Sun Jun 25 07:50:27 2006 granted, its little more than a speed-bump, but we like smooth roads.. I'm getting the impression you are searching for non-existing problems... ... - 'known' configs Each kernel release 'creates' a bunch of known configs (A*S*M of them) A - all the arches S - each arch's sub-arches M - make-targets: allnoconfig, allyesconfig, allmodconfig, etc With date gone, these are trivially reproducible, byte for byte. Bug reports with attached config-vs-x86-64-all-defaults.diff become meaningful, and arguably better than straight attachments (since theyre shorter, and refer to a well understood (and completely repeatable) standard). There are defconfigs for this purpose. Since diff's aren't always shorter (they can even be significantely larger) than the file. That suggests they diff'd against the wrong thing. diffing vs allmodconfig will generally be *far* better than vs allnoconfig. It seems you don't know what you are talking about. With my .config and kernel 2.6.16.22: 35146 .config 67963 .config-allmodconfig 97572 diff-allmodconfig-myconfig Yes, the diff has nearly three times the size of the .config ... QED ... Just recently on LKML, Adrian Bunk rightly complained about a bug-report with a partial .config, since its tedious to reproduce the problem, and OP noted he didnt want to 'spam' the list. If the OP had seen a config-alldefaults file in his build-directory, he would likely have sent a diff against it, and Adrian would have a way to reproduce it. The size is not an issue. And if it was, bzip2 would beat your proposal easily for this purpose. bzips are not readable as is, and folks with knowledge to recognize broken-ness may not have the inclination to unzip and see. Size was an issue for the OP, so he sent a fragment that he thought was the salient info. If he had several 'defconfigs', he could have diff'd them all, just to see which was the smallest, and sent that. This focus on size is unfortunate; its only an approximation of the real differences that may or may-not be a factor in the problem The best solution is simply to send the .config . All your suggestions make things more complicated for no good reason. ... -- a config-spectrum-analyser Even though the md5 doesnt say much about what features are being configured into a submitters kernel, it is enough to 'histogram'. Doing so would show us the distribution of configurations being tested over a given period. ... With the exception of distribution kernels, the probability that someone else uses exactly the same .config I'm using is nearly zero, since the number of possible different .config's on a given architecture is something in the order of 2^1000 or one (american) billion. yes - but most of them are probably precluded by the kconfig rules; hundreds of random configs will collapse to the same result once run thru 'make oldconfig'. This reduction seems mildly interesting in itself, for some value of interesting. ... No, these .config's will not collapse. thanks Jim Cromie cu Adrian -- Is there not promise of rain? Ling Tan asked suddenly out of the darkness. There had been need of rain for many days. Only a promise, Lao Er said. Pearl S. Buck - Dragon Seed Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ kbuild-devel mailing list kbuild-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kbuild-devel
Re: [kbuild-devel] RFC - kconfig date must die
On Mon, Jun 26, 2006 at 04:04:24PM -0600, Jim Cromie wrote: IOW: can we switch the default of CONFIG_NODATESTAMP ? Clearly, somebody has already thought about stripping the date from .configs, but it seems to me that this has been forgotten, and warrants a fresh look. - I searched http://marc.theaimsgroup.com/?l=kbuild-develr=1b=200604w=2 for CONFIG_NODATESTAMP, got *no* hits. How long ago was this added ? Try searching linux-kernel. And at linux-kernel try to locate: [rfc] Compressing those annoying .configs which is a nice tool for reporting your .config compressed and readable. That said it is hard to realise what real-life problem you try to solve. Sam Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ kbuild-devel mailing list kbuild-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kbuild-devel
