bug#53801: [PATCH] tests: have printf(1) use octal numbers for POSIX conformance
Hi Brian, On Mon, 7 Feb 2022 08:52:29 -0800 "Brian C. Lane" wrote: > On Sat, Feb 05, 2022 at 10:44:29AM +, Kerin Millar wrote: > > Hello, > > > > The attached patch rectifies some test failures that were reported at > > https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, > > POSIX does not require for the printf utility to support hexadecimal > > notation. > > > > -- > > Kerin Millar > > Thanks for the patch. I'll have to ponder this for a bit. So the issue > is showing up in Gentoo because it uses dash, right? The issue I have > with the change is that it then obscures what's being written. Everyone > knows what 0x55aa is :) I'd prefer a solution that continues to use hex > so that future users don't have to figure out what's going on. In fact, Gentoo provides bash as a default sh(1) implementation, by which I mean that /bin/sh begins life as being a symlink to bash. Still, that's neither here nor there. The policy of the distro is that the user may select another sh(1) provider, as long as it is POSIX-conforming. Dash is merely one such example of a shell that implements the Shell Command Language specification correctly and in full. Should an issue arise as a consequence of /bin/sh being any conforming implementation other than bash, it is rightly treated as a bug. Often, patches are proposed and the more responsible contributors try to get them upstreamed so that everyone benefits. Of course, there exists other distributions where /bin/sh is some other implementation by default. Essentially, this is one of those cases where one cannot have one's cake and eat it. By beginning a script with a shebang that reads #!/bin/sh, it is implied - in no uncertain terms - that the entirety of the script should run correctly in any conforming implementation of the Shell Command Language. In almost all modern implementations, the printf utility is implemented as a builtin and their authors are in no way obligated to implement anything above and beyond the syntax described by https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html. While bash is - at heart - a valid implementation of the Shell Command Language, it supports a great many extensions and does not suppress them, even when running in its "posix" mode. In fact, this is not the first time this issue has come up; see "t0100-print.sh" and "t0251-gpt-unicode.sh". I understand that most developers will find hex notation easier to read, which is why my patch also adjusts the surrounding commentary to that end. In my view, the alternatives are worse:- a) changing the shebangs to #!/bin/bash, rendering bash a hard requirement for running the affected tests (merely for a printf) b) combining arithmetic expansion with printf Regarding (b), below is an example. # Write the two magic bytes magic=$(printf '\\%03o' $(( 0x55 )) $(( 0x0A ))) printf "$magic" | ... Not only is this more expensive, one might then need to be cautious as to the potential matter of unintentionally injecting the results of expansions into the format string, owing to the double-quoted context. I certainly wouldn't consider it to be any easier to digest than the following. # Write the two magic bytes, 0x55 0xAA. printf '\125\252' | ... Incidentally, octal is trivial to convert to peruse as hex using only standard utilities. $ printf '\125\252' | od -An -t x1 55 aa Regards, -- Kerin Millar
bug#53801: [PATCH] tests: have printf(1) use octal numbers for POSIX conformance
On Mon, 7 Feb 2022 09:10:12 -0800 "Brian C. Lane" wrote: > On Mon, Feb 07, 2022 at 08:52:33AM -0800, Brian C. Lane wrote: > > On Sat, Feb 05, 2022 at 10:44:29AM +, Kerin Millar wrote: > > > Hello, > > > > > > The attached patch rectifies some test failures that were reported at > > > https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, > > > POSIX does not require for the printf utility to support hexadecimal > > > notation. > > > > > > -- > > > Kerin Millar > > > > Thanks for the patch. I'll have to ponder this for a bit. So the issue > > is showing up in Gentoo because it uses dash, right? The issue I have > > with the change is that it then obscures what's being written. Everyone > > knows what 0x55aa is :) I'd prefer a solution that continues to use hex > > so that future users don't have to figure out what's going on. > > Actually, on Fedora at least, printf comes from coreutils so the shell > shouldn't matter at all. The tests are single quoted to prevent shell > expansion, so maybe dash is using a builtin printf that's different? $ bash -c 'type printf' printf is a shell builtin Yes, it's also a builtin in dash (it is rare these days for printf to ever be invoked in its capacity as a binary). As explained by my prior message, no implementation of the POSIX shell and utilities (XCU) is under any obligation to support GNU-specific features. Therein lies the rub. Octal sequences are guaranteed to work. The problem is a great many Linux developers only ever test things in bash, even where their shebangs purport to target sh. -- Kerin Millar
bug#53801: [PATCH] tests: have printf(1) use octal numbers for POSIX conformance
On Mon, Feb 07, 2022 at 08:52:33AM -0800, Brian C. Lane wrote: > On Sat, Feb 05, 2022 at 10:44:29AM +, Kerin Millar wrote: > > Hello, > > > > The attached patch rectifies some test failures that were reported at > > https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, > > POSIX does not require for the printf utility to support hexadecimal > > notation. > > > > -- > > Kerin Millar > > Thanks for the patch. I'll have to ponder this for a bit. So the issue > is showing up in Gentoo because it uses dash, right? The issue I have > with the change is that it then obscures what's being written. Everyone > knows what 0x55aa is :) I'd prefer a solution that continues to use hex > so that future users don't have to figure out what's going on. Actually, on Fedora at least, printf comes from coreutils so the shell shouldn't matter at all. The tests are single quoted to prevent shell expansion, so maybe dash is using a builtin printf that's different? Brian -- Brian C. Lane (PST8PDT) - weldr.io - lorax - parted - pykickstart
bug#53801: [PATCH] tests: have printf(1) use octal numbers for POSIX conformance
On Sat, Feb 05, 2022 at 10:44:29AM +, Kerin Millar wrote: > Hello, > > The attached patch rectifies some test failures that were reported at > https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, > POSIX does not require for the printf utility to support hexadecimal notation. > > -- > Kerin Millar Thanks for the patch. I'll have to ponder this for a bit. So the issue is showing up in Gentoo because it uses dash, right? The issue I have with the change is that it then obscures what's being written. Everyone knows what 0x55aa is :) I'd prefer a solution that continues to use hex so that future users don't have to figure out what's going on. Brian -- Brian C. Lane (PST8PDT) - weldr.io - lorax - parted - pykickstart
bug#53801: [PATCH] tests: have printf(1) use octal numbers for POSIX conformance
Hello, The attached patch rectifies some test failures that were reported at https://bugs.gentoo.org/774363. Whether it be a builtin of sh(1) or not, POSIX does not require for the printf utility to support hexadecimal notation. -- Kerin Millar 0001-tests-have-printf-1-use-octal-numbers-for-POSIX-conf.patch Description: Binary data
