Re: Can version bump up to 9.99.100?
Date:Fri, 23 Sep 2022 22:57:52 -0400 From:"David H. Gutteridge" Message-ID: | Sometimes it's necessary to test for when a feature was added in a | -current release, and there's no simple or precise way to do it, as | you've noted. If a feature was added sometime in xx.yy.zz, then a test | might (retroactively) be expressed with zz+1 as the floor. Yes, I know. But if a feature is added then there ought also be some better way added to test that that has happened, not the kernel version. If that doesn't happen, people (that's us) should complain, and make it happen. For kernel changes, testing the kernel version has at least some kind of rationale behind it, though the x.99.abi bump scheme doesn't generally fit well, as the abi bumps almost never happen for new features - there could be months (or longer) pass before the abi value is altered. When the change isn't to the kernel, but relates to something changed in userland, testing the kernel version is 100% useless (and often wrong). (And yes, that includes anything that uname outputs, or any other kernel supplied information that wasn't previously set by userland). The really hard case is where a kernel bug (eg: an ioctl not working correctly) is fixed, which has been worked around, but no longer needs to be afterwards. An example might be the O_NONBLOCK on pty master devices that just got changed. This kind of thing will generally not require a kernel abi bump (and could also happen in x.n (like 9.2 over 9.1) as well, and appear in x.n_STABLE along the way) and very often requires no other changes other than to the code in the kernel source file(s) concerned, so there really is nothing to test. An idea might be to add some new sysctl var (for kernel changes only) that gets bumped far more frequently (every time something new is added, or some user noticeable bug gets fixed (ie: not whitespace, KNF, spelling errors in comments, changes to printf output) and is perhaps reset whenever a __NetBSD_Version__ bump occurs (and this would apply in both stable and HEAD versions) so that there is a more precise way to test for this kind of thing (including looking to see if running a version that had a bug temporarily imported, by knowing (with hindsight) the value of this var before the bug was added, and when it was later fixed). Alternatively, perhaps only reset for new netbsd-N branches (so 10.0 would start again at 0, as soon as branched, and HEAD which will become 10.99.0 will also revert to 0 at that time) and otherwise simply both climb (an unbroken sequence through 10.1 10.2 ... 10.17 ...) until we branch -11 where 11.0 would start at 0, and 11.99.0 would as well (but 10.whatever would just keep climbing, until EOL). This ought to be a potentially BIG number (something we can never run out of in any practical situation), but just a simple integer, so it probably would not, ever go past 6 or 7 digits, but we don't want to ever worry about the possibility of overflow so an int64_t or similar, a bump should just be a ++. It also ought live in a source file of its own, depending only on the header file which declares it for the sysctl routines that export it (in some other file, where that is doesn't matter) so changing its value is cheap, and no-one will be reluctant to do it. Cheap particularly wrt the cost of builds after it is altered - unlike __NetBSD_Version__ which, being in param.h, causes almost every file in the kernel to need to be recompiled when it is altered, even though almost nothing in the kernel cares in the slightest what value it has (just a few things here and there). This tiny (3 line - one comment, one #include, one var decl - plus copyright noise) file could even be MD, so the version numbers for one port don't affect others (a change to an x86 private function, need not show up as a change that is visible on sparc systems) - or we could have 2 variables, one MI for changes that affect everyone, and one MD, for the others, and return both of them from one sysctl. Or this might just be too much complexity, a single number shared by everything would also work, and is what I think I'd prefer. Kernel commits would then note what this value has been bumped to for the change (when the change is one that requires it - and the rule should be: if in doubt, bump it, it is cheap). How to test new features / fixes for userland, I'll leave to others to ponder. (also, here, this is tech-kern). kre
Re: Can version bump up to 9.99.100?
On Sat, 2022-09-17 at 02:50 +0700, Robert Elz wrote: > Date: Fri, 16 Sep 2022 12:59:24 -0400 > From: "David H. Gutteridge" > Message-ID: > > | So there will be information loss there, at minimum. Whether that > ends > | up being significant at some point, I guess we can't say. > > I would hope not. That is, I am assuming (but don't know pkgsrc well > enough to be sure) that OPSYS_VERSION gets used for some kind of > feature > test. That's OK (not the ideal method - but sometimes it is the only > practical one) for major, or even minor version comparisons. It isn't > for > the 3rd field (xx) in N.99.xx for NetBSD. That field is not changed > for feature additions, so some N.99.xx may have a particular feature, > and others not, but is changed for internal ABI alterations (which > don't > necessarily affect what is visible by applications in any way at all). > > Note also that this value is never changed (in the NetBSD N.99.xx > case) > because of changes that occur to anything outside the kernel - so it > can > never safely be used to test what version of some application or > library > function might be installed. Never. > > If pkgsrc (or pkgsrc packages) are using this sensibly, then limiting > OPSYS_VERSION at 09 for all future __NetBSD_Version__ values > 9.99.x > where x >= 100 should be safe, as nothing should ever care about those > final 2 digits. > > That's "if". Sometimes it's necessary to test for when a feature was added in a -current release, and there's no simple or precise way to do it, as you've noted. If a feature was added sometime in xx.yy.zz, then a test might (retroactively) be expressed with zz+1 as the floor. (An example is some of the ARM compatibility handling in mk/java-vm.mk.) That's not perfect, but it's the reality of pkgsrc. > > kre > > ps: the issue I was concerned about more would occur when the kernel > version info gets embedded in a package version, and other similar > things. Yes, well, there's pkgtools/osabi, for example, which is all about that, and is arguably abused to conflate both kernel and userland (including X11) state at present. This is a bit of a mess in pkgsrc, IMO. It uses a different variable (OS_VERSION), which is expressed as a string output of uname -r, and so shouldn't be impacted here (I haven't checked). Regards, Dave
Re: Can version bump up to 9.99.100?
Hi, On 2022/09/17 19:34, Robert Elz wrote: Date:Fri, 16 Sep 2022 23:46:59 + From:David Holland Message-ID: | While it's possible that some of | these may exist, it's unlikely that there are many of them or that | they appear anywhere especially important. That's all encouraging, and yet more reason to bump the version soon (as required) so we have the 9.99.1xx series around long enough for any issues to be found and fixed, before we're back to 10.0 and 10.99.1 with a whole different set of issues to fix. I commit the patch and bump up to 9.99.100 now. Thank you for your advise! Thanks, -- // Internet Initiative Japan Inc. Device Engineering Section, Product Division, Technology Unit Kengo NAKAHARA
Re: Can version bump up to 9.99.100?
Date:Fri, 16 Sep 2022 23:46:59 + From:David Holland Message-ID: | While it's possible that some of | these may exist, it's unlikely that there are many of them or that | they appear anywhere especially important. That's all encouraging, and yet more reason to bump the version soon (as required) so we have the 9.99.1xx series around long enough for any issues to be found and fixed, before we're back to 10.0 and 10.99.1 with a whole different set of issues to fix. kre
Re: Can version bump up to 9.99.100?
David Holland writes: > On Fri, Sep 16, 2022 at 07:00:23PM +0700, Robert Elz wrote: > > That is, except for in pkgsrc, which is why I still > > have a (very mild) concern about that one - it actually compares the > > version numbers using its (until it gets changed) "Dewey" comparison > > routines, and for those, 9.99.100 is uncharted territory. > > No, it's not, pkgsrc-Dewey is well defined on arbitrarily large > numbers. In fact, that's in some sense the whole point of it relative > to using fixed-width fields. And, surely we had 9.99.9 and 9.99.10. The third digit is no more special than the second. It's just that it happens less often so the problem of arguably incorrectly written two-digit patterns is more likely than for that to happen with one. It's not reasonable to constrain a normal process because other bugs might exist. signature.asc Description: PGP signature
Re: Can version bump up to 9.99.100?
On Fri, Sep 16, 2022 at 07:00:23PM +0700, Robert Elz wrote: > That is, except for in pkgsrc, which is why I still > have a (very mild) concern about that one - it actually compares the > version numbers using its (until it gets changed) "Dewey" comparison > routines, and for those, 9.99.100 is uncharted territory. No, it's not, pkgsrc-Dewey is well defined on arbitrarily large numbers. In fact, that's in some sense the whole point of it relative to using fixed-width fields. The only problem that might arise is that someone might have used a glob pattern of the form NetBSD-9.99.[7-9]* that will unexpectedly stop working. These would appear because make doesn't know how to do pkgsrc-Dewey computations internally. While it's possible that some of these may exist, it's unlikely that there are many of them or that they appear anywhere especially important. (Patterns of the form NetBSD-[6-9]* do appear and people have been hunting those down lately.) -- David A. Holland dholl...@netbsd.org
Re: Can version bump up to 9.99.100?
Date:Fri, 16 Sep 2022 12:59:24 -0400 From:"David H. Gutteridge" Message-ID: | So there will be information loss there, at minimum. Whether that ends | up being significant at some point, I guess we can't say. I would hope not. That is, I am assuming (but don't know pkgsrc well enough to be sure) that OPSYS_VERSION gets used for some kind of feature test. That's OK (not the ideal method - but sometimes it is the only practical one) for major, or even minor version comparisons. It isn't for the 3rd field (xx) in N.99.xx for NetBSD. That field is not changed for feature additions, so some N.99.xx may have a particular feature, and others not, but is changed for internal ABI alterations (which don't necessarily affect what is visible by applications in any way at all). Note also that this value is never changed (in the NetBSD N.99.xx case) because of changes that occur to anything outside the kernel - so it can never safely be used to test what version of some application or library function might be installed. Never. If pkgsrc (or pkgsrc packages) are using this sensibly, then limiting OPSYS_VERSION at 09 for all future __NetBSD_Version__ values 9.99.x where x >= 100 should be safe, as nothing should ever care about those final 2 digits. That's "if". kre ps: the issue I was concerned about more would occur when the kernel version info gets embedded in a package version, and other similar things.
Re: Can version bump up to 9.99.100?
On Fri, 16 Sep 2022 at 19:00:23 +0700, Robert Elz wrote:
[...] That is, except for in pkgsrc, which is why I still
have a (very mild) concern about that one - it actually compares the
version numbers using its (until it gets changed) "Dewey" comparison
routines, and for those, 9.99.100 is uncharted territory.
One wrinkle is that the current definition of OPSYS_VERSION (in
pkgsrc/mk/bsd.prefs.mk) does this (wrapped for formatting):
_OPSYS_VERSION_CMD= ${UNAME} -r | \
awk -F. '{major=int($$1); minor=int($$2);
if (minor>=100) minor=99; patch=int($$3);
if (patch>=100) patch=99; printf "%02d%02d%02d",
major, minor, patch}'
So there will be information loss there, at minimum. Whether that ends
up being significant at some point, I guess we can't say. Someone could
always re-implement something different for NetBSD (meaning all the
existing NetBSD-specific uses of OPSYS_VERSION would be adjusted).
(There are 135 instances of OPSYS_VERSION in pkgsrc presently. I don't
know offhand how many are NetBSD-specific, that's harder to count, but
it's a significant subset, I believe.)
Regards,
Dave
Re: Can version bump up to 9.99.100?
I test module loading with just bumped up src. After build.sh modules && build.sh installmodules, that works fine. # uname -r 9.99.100 # ls -dl /stand/amd64/9.99.100/modules/tprof* drwxr-xr-x 2 root wheel 512 Sep 16 17:45 /stand/amd64/9.99.100/modules/tprof drwxr-xr-x 2 root wheel 512 Sep 16 17:46 /stand/amd64/9.99.100/modules/tprof_x86 # modstat | grep tprof # modload tprof_x86 # echo $? 0 # modstat | grep tprof tprof driver filesys a14957 - tprof_x86 driver filesys -02010 tprof Excellent - looks good. ++--+--+ | Paul Goyette | PGP Key fingerprint: | E-mail addresses:| | (Retired) | FA29 0E3B 35AF E8AE 6651 | p...@whooppee.com| | Software Developer | 0786 F758 55DE 53BA 7731 | pgoye...@netbsd.org | | & Network Engineer | | pgoyett...@gmail.com | ++--+--+
Re: Can version bump up to 9.99.100?
Date:Thu, 15 Sep 2022 23:46:45 -0700 (PDT) From:Paul Goyette Message-ID: | The human-oriented version is used as part of the path to modules | directory. Need to make sure that the modules set is properly | populated, That much I had tested. | and that module loads find them in the directory. but that I did not - but Kengo NAKAHARA now has, so that's all good, I really couldn't see how there would be a problem here (but testing it was good) as it is all just strings (the only comparisons are of the binary blobs). That is, except for in pkgsrc, which is why I still have a (very mild) concern about that one - it actually compares the version numbers using its (until it gets changed) "Dewey" comparison routines, and for those, 9.99.100 is uncharted territory. kre
Re: Can version bump up to 9.99.100?
Hi, Thank you for your comment. On 2022/09/16 15:46, Paul Goyette wrote: On Fri, 16 Sep 2022, Robert Elz wrote: Date: Fri, 16 Sep 2022 11:10:30 +0900 From: Kengo NAKAHARA Message-ID: <90c3c46e-6668-9644-70c3-0eab2cf1c...@iij.ad.jp> | Hmm, I will test kernel module building before commit. Sorry, I wasn't clear - I build everything (modules included) - I just never actually load any modules, so I haven't tested them (my kernels have the MODULAR option disabled). I cannot imagine an issue, as internally everything just uses __NetBSD_Version__ as a 32 bit (ordered) blob - the breakdown into 9.99.100 type strings is just for us humans (and pkgsrc). Not entirely true. The human-oriented version is used as part of the path to modules directory. Need to make sure that the modules set is properly populated, and that module loads find them in the directory. I test module loading with just bumped up src. After build.sh modules && build.sh installmodules, that works fine. # uname -r 9.99.100 # ls -dl /stand/amd64/9.99.100/modules/tprof* drwxr-xr-x 2 root wheel 512 Sep 16 17:45 /stand/amd64/9.99.100/modules/tprof drwxr-xr-x 2 root wheel 512 Sep 16 17:46 /stand/amd64/9.99.100/modules/tprof_x86 # modstat | grep tprof # modload tprof_x86 # echo $? 0 # modstat | grep tprof tprof driver filesys a14957 - tprof_x86 driver filesys -02010 tprof Thanks, -- // Internet Initiative Japan Inc. Device Engineering Section, Product Division, Technology Unit Kengo NAKAHARA
Re: Can version bump up to 9.99.100?
On Fri, 16 Sep 2022, Robert Elz wrote: Date:Fri, 16 Sep 2022 11:10:30 +0900 From:Kengo NAKAHARA Message-ID: <90c3c46e-6668-9644-70c3-0eab2cf1c...@iij.ad.jp> | Hmm, I will test kernel module building before commit. Sorry, I wasn't clear - I build everything (modules included) - I just never actually load any modules, so I haven't tested them (my kernels have the MODULAR option disabled). I cannot imagine an issue, as internally everything just uses __NetBSD_Version__ as a 32 bit (ordered) blob - the breakdown into 9.99.100 type strings is just for us humans (and pkgsrc). Not entirely true. The human-oriented version is used as part of the path to modules directory. Need to make sure that the modules set is properly populated, and that module loads find them in the directory. ++--+--+ | Paul Goyette | PGP Key fingerprint: | E-mail addresses:| | (Retired) | FA29 0E3B 35AF E8AE 6651 | p...@whooppee.com| | Software Developer | 0786 F758 55DE 53BA 7731 | pgoye...@netbsd.org | | & Network Engineer | | pgoyett...@gmail.com | ++--+--+
Re: Can version bump up to 9.99.100?
Date:Fri, 16 Sep 2022 11:10:30 +0900 From:Kengo NAKAHARA Message-ID: <90c3c46e-6668-9644-70c3-0eab2cf1c...@iij.ad.jp> | Hmm, I will test kernel module building before commit. Sorry, I wasn't clear - I build everything (modules included) - I just never actually load any modules, so I haven't tested them (my kernels have the MODULAR option disabled). I cannot imagine an issue, as internally everything just uses __NetBSD_Version__ as a 32 bit (ordered) blob - the breakdown into 9.99.100 type strings is just for us humans (and pkgsrc). kre
Re: Can version bump up to 9.99.100?
Hi, Thank you for your detailed comments! On 2022/09/15 20:09, Robert Elz wrote: Date:Thu, 15 Sep 2022 17:08:52 +0900 From:Kengo NAKAHARA Message-ID: <279eae4e-79f4-39c0-5279-83d5738b6...@iij.ad.jp> | Can version bump up to 9.99.100? Is there anything wrong? It can. There are no issues with the base system (incl xsrc) I have tested this in the past, it all just works - and that we were going to need it sometime before the -10 branch has been obvious for a while. Two things I did not test were kernel modules (since I never use them) which I highly doubt will give any problem, and should be a trivial fix in the unlikely event there is an issue; Hmm, I will test kernel module building before commit. And pkgsrc, because when I tested I needed to revert to the then current version number (.97 at the time I think), and that reversion would do things to some pkgsrc version numbers that it should not be required to deal with. I don't have enough of a handle on the latter to guess, but if something in pkgsrc breaks, this will provide the motivation to fix it, which otherwise might never happen. In case any change is required to the parts of it in base, getting that done before the -10 branch would be good. So just do it. I see. I will commit it early next week. Thanks, -- // Internet Initiative Japan Inc. Device Engineering Section, Product Division, Technology Unit Kengo NAKAHARA
Re: Can version bump up to 9.99.100?
Date:Thu, 15 Sep 2022 17:08:52 +0900 From:Kengo NAKAHARA Message-ID: <279eae4e-79f4-39c0-5279-83d5738b6...@iij.ad.jp> | Can version bump up to 9.99.100? Is there anything wrong? It can. There are no issues with the base system (incl xsrc) I have tested this in the past, it all just works - and that we were going to need it sometime before the -10 branch has been obvious for a while. Two things I did not test were kernel modules (since I never use them) which I highly doubt will give any problem, and should be a trivial fix in the unlikely event there is an issue; And pkgsrc, because when I tested I needed to revert to the then current version number (.97 at the time I think), and that reversion would do things to some pkgsrc version numbers that it should not be required to deal with. I don't have enough of a handle on the latter to guess, but if something in pkgsrc breaks, this will provide the motivation to fix it, which otherwise might never happen. In case any change is required to the parts of it in base, getting that done before the -10 branch would be good. So just do it. kre
Can version bump up to 9.99.100?
Hi, I want to commit some fixes which need to bump up. E.g. fix to be left inconsistent routes https://github.com/knakahara/netbsd-src/commit/0ce8e1114d42e51a1ea88f42b1d942563c8312d6 Can version bump up to 9.99.100? Is there anything wrong? Thanks, -- // Internet Initiative Japan Inc. Device Engineering Section, Product Division, Technology Unit Kengo NAKAHARA
