On 6/28/25 21:42, Andy Chu wrote:
(Coding is easy. Figuring out what it's supposed to DO has always been
the hard part.)
I agree!
And to repeat, that's what I'm saying OSH has already done. It's the
most bash-compatible shell, by far -
https://pages.oils.pub/spec-compat/2025-06-26/renamed-tmp/spec/compat/PASSING.html
So I think sharing some labor rather than producing 4 different shells
is a good idea ... e.g. in particular OSH has some holes on the
interactive side, and we need to test it building real distros.
I.E. "Stop working on your project, come work on my project".
And we have a budget of 50 K euros for the next 6-12 months. I
realized that reducing test cases from real distro shell scripts
(Alpine, Nix) is now a bottleneck. That is, we have plenty of
isolated tests, but it's better to PRIORITIZE those by what actually
happens in the wild.
I have SO MANY TESTS, all of which I isolated to a minimal reproduction
sequence but didn't bother to wire up into the automated regression test
script because I have enough tests that don't pass already.
I mentioned needing to dig through my blog, I've also got multiple text
files full of boring stuff like:
$ thingy ()
> ^C
$ echo (
bash: syntax error near unexpected token `newline'
$ function
bash: syntax error near unexpected token `newline'
$ function name () {
> }
bash: syntax error near unexpected token `}'
$ function name () { ;}
bash: syntax error near unexpected token `;'
$ if ; then; echo hello; fi
bash: syntax error near unexpected token `;'
$ func (); { echo hello; }
bash: syntax error near unexpected token `;'
$ if true |
> then
bash: syntax error
echo one && if true; then echo hello; fi
one
hello
Basically, if I needed to run it from the command line to figure out
what bash would do with a given input, I cut and pasted it to a file. If
I needed to know, the test suite needs to know. And then I ask Chet
about some of the really weird ones that make no sense like "why does
if; syntax error but if\n prompts for line continuation", and sometimes
he "fixes" a corner case, and then TEST_HOST has version skew and I step
away from the keyboard...
But yay more tests. I'm always happy to find more tests. I'm trying to
organize the tests I have into some sort of coherent order, and I need
to do a close reading of both posix-2024 and the bash man page at some
point to ensure that every concrete statement they make has a test, but
that's... later. :)
It would be great if there was something like the old automated posix
test suite that stopped mattering because they charged an arm and a leg
for it so the Linux community went "up yours" and everybody stopped
caring what Posix said after Linux's famous "212% annually growth" in
1998 because all the Java devs flooeded over (which I wrote about a bit
in http://www.catb.org/esr/writings/taoup/html/ch02s03.html and probably
more in a blog entry)...
Sigh, I'm pretty sure Linux Journal had an article on Linus's rejection
of the offer of explicit Posix conformance testing for Linux, published
back in the 1990s when it was happening, but Google no longer indexes
their archives. https://liw.fi/linux-news/issue03/ was back before Posix
noticed Linux existed, and the later interviews don't mention posix
because it was irrelevant. By the time Red Hat had its IPO, FIPS-151-2
had been withdrawn, ala
https://www.opengroup.org/testing/fips/general_info.html and
https://www.federalregister.gov/documents/2000/02/25/00-4512/announcing-approval-of-withdrawal-of-thirty-three-federal-information-processing-standards-fips
Anyway, tests are nice but rigid tests aren't nice. Testing is a whole
discipline of its own, and mostly what you're testing for is that your
behavior is consistent, intentional, and expected. "Right" is...
complicated.
Andy
Rob
_______________________________________________
Toybox mailing list
Toybox@lists.landley.net
http://lists.landley.net/listinfo.cgi/toybox-landley.net
