Re: ksh: documented substitution behavior contradicts actual behavior
On Sun, Oct 16, 2022 at 11:48:35AM +0100, cho...@jtan.com wrote:
> Kastus Shchuka writes:
> > On Sat, Oct 15, 2022 at 11:42:17PM -0300, Lucas de Sena wrote:
> > > Hi,
> > >
> > > After trying to split a string into fields delimited with colons and
> > > spaces, I found this bug in how ksh(1) does substitution. The actual
> > > behavior contradicts what other shells like bash and mksh do and also
> > > contradicts its own manual.
> > >
> > > Running the following on other shells (say, bash) prints "/foo/bar/".
> > > This command splits the string " foo : bar " into two fields: "foo"
> > > and "bar", considering colon and space as delimiters.
> > >
> > > echo " foo : bar " | {
> > > IFS=": "
> > > read -r a b
> > > printf -- "/%s/%s/\n" "$a" "$b"
> > > }
> > >
> > > However, running the same command in OpenBSD ksh(1) (or sh(1)) splits
> > > the string into "foo" and ": bar".
> >
> > This is because the last parameter (b) is a concatenation of two fields.
> > Parsing
> > is done properly if you add c to the read command:
> >
> > + echo foo : bar
> > + IFS=:
> > + read -r a b c
> > + printf -- /%s/%s/%s/\n foo bar
> > /foo//bar/
> >
> >
> > >
> > > The manual ksh(1) provides the following, similar example:
> > >
> > > > Example: If IFS is set to “:”, and VAR is set to
> > > > “A:B::D”, the substitution for $VAR
> > > > results in four fields: ‘A’, ‘B’, ‘’ (an empty field), and ‘D’.
> > > > Note that if the IFS parameter is set to the NULL string, no field
> > > > splitting is done; if the parameter is unset, the default value of
> > > > space, tab, and newline is used.
> > >
> > > Let's try it:
> > >
> > > echo " A : B::D" | {
> > > IFS=" :"
> > > read -r arg1 arg2 arg3 arg4
> > > printf -- '1st: "%s"\n' "$arg1"
> > > printf -- '2nd: "%s"\n' "$arg2"
> > > printf -- '3rd: "%s"\n' "$arg3"
> > > printf -- '4th: "%s"\n' "$arg4"
> > > }
> > >
> > > bash(1) splits the line into the following fields:
> > >
> > > 1st: "A"
> > > 2nd: "B"
> > > 3rd: ""
> > > 4th: "D"
> > >
> > > This is actually the expected output, as described in the manual.
> > >
> > > However, running the same command in OpenBSD ksh, prints this:
> > >
> > > 1st: "A"
> > > 2nd: ""
> > > 3rd: "B"
> > > 4th: ":D"
> > >
> > > A completelly different thing.
> > > The same occurs with OpenBSD sh(1).
> >
> > What you observe is the result of the next paragraph in the man page
> > after the example you quoted:
> >
> > Also, note that the field splitting applies only to the immediate
> > result
> > of the substitution. Using the previous example, the substitution for
> > $VAR:E results in the fields: `A', `B', `', and `D:E', not `A', `B',
> > `',
> > `D', and `E'. This behavior is POSIX compliant, but incompatible with
> > some other shell implementations which do field splitting on the word
> > which contained the substitution or use IFS as a general whitespace
> > delimiter.
>
> Actually you need to look further into the manual since the word
> splitting is performed not by parameter substitution but by read:
>
> Reads a line of input from the standard input, separates the line
> into fields using the IFS parameter (see Substitution above), and
> assigns each field to the specified parameters.
>
> This is why adding an extra variable to read above (and here) makes
> it capture the remainder of the string.
>
> For example:
>
> $ alias dump="perl -MData::Dumper -e 'print Dumper @ARGV'"
> $ dump a b c
> $VAR1 = 'a';
> $VAR2 = 'b';
> $VAR3 = 'c';
>
> $ ( IFS=:; dump $PATH )
> $VAR1 = '/bin';
> $VAR2 = '/sbin';
> $VAR3 = '/usr/bin';
> $VAR4 = '/usr/sbin';
> $VAR5 = '/usr/X11R6/bin';
> $VAR6 = '/usr/local/bin';
> $VAR7 = '/usr/local/sbin';
> $VAR8 = '/usr/games';
>
> So given $X:
>
> $ X=' A : B::D'
>
> Parameter substitution:
>
> $ ( IFS=' :'; dump $X )
> $VAR1 = 'A';
> $VAR2 = 'B';
> $VAR3 = '';
> $VAR4 = 'D';
>
> Similarly:
>
> $ fn() { dump "$@" ); fn $X
> $VAR1 = 'A';
> $VAR2 = ':';
> $VAR3 = 'B::D';
>
> $ fn() { dump "$@"; }; ( IFS=' :'; fn $X )
> $VAR1 = 'A';
> $VAR2 = 'B';
> $VAR3 = '';
> $VAR4 = 'D';
>
> read substitution:
>
> $ echo "$X" | ( IFS=' :'; read a1 a2 a3; dump "$a1" "$a2" "$a3" )
> $VAR1 = 'A';
> $VAR2 = '';
> $VAR3 = 'B::D';
>
> $ echo "$X" | ( IFS=' :'; read a1 a2 a3 a4; dump "$a1" "$a2" "$a3"
> "$a4" )
> $VAR1 = 'A';
> $VAR2 = '';
> $VAR3 = 'B';
> $VAR4 = ':D';
>
> $ echo "$X" | ( IFS=' :'; read a1 a2 a3 a4 a5; dump "$a1" "$a2" "$a3"
> "$a4" "$a5" )
> $VAR1 = 'A';
> $VAR2 = '';
> $VAR3 = 'B';
> $VAR4 = '';
> $VAR5 = 'D';
>
> It does look like
Re: sndio and bit perfect playback
Hi Misc, As the OP, i hope you will excuse the top-posting. Thanks for all the feedback and exciting discussions. Context/Backstory.. i was using that laptop as its the only system i have with a remaining working cd/dvd device, which just happened to have OpenBSD installed. The 'cdio cdrip' tooling was awesome and just worked, adding a few missing physical_2_digital albumns to my collection. (if only cdio cddbinfo was fully functional). this allows me to rip a cd, and listen back to it. So with that i guess i could run sndio with flags matching to cd bit rate of 44100k. this will be bit-perfect for my purpose, and make the dac light up green. I'll test this later this week. -SUB --- Original Message --- On Sunday, October 16th, 2022 at 4:22 PM, Alexandre Ratchov wrote: > > Sure, there is measurable aliasing. My questioning is: is it audible > in 99% of the cases? > > Remains the question, how to handle the 1% remaining. For now, > resampling off-line is the least annoying, IMHO. > > AFAIK, sample rate changes in the kernel are OK, at least in uaudio > driver. > > Now it's all in sndiod (or how to bypass sndiod to get bit-perfect > audio)
Re: ksh: documented substitution behavior contradicts actual behavior
Kastus Shchuka writes:
> On Sat, Oct 15, 2022 at 11:42:17PM -0300, Lucas de Sena wrote:
> > Hi,
> >
> > After trying to split a string into fields delimited with colons and
> > spaces, I found this bug in how ksh(1) does substitution. The actual
> > behavior contradicts what other shells like bash and mksh do and also
> > contradicts its own manual.
> >
> > Running the following on other shells (say, bash) prints "/foo/bar/".
> > This command splits the string " foo : bar " into two fields: "foo"
> > and "bar", considering colon and space as delimiters.
> >
> > echo " foo : bar " | {
> > IFS=": "
> > read -r a b
> > printf -- "/%s/%s/\n" "$a" "$b"
> > }
> >
> > However, running the same command in OpenBSD ksh(1) (or sh(1)) splits
> > the string into "foo" and ": bar".
>
> This is because the last parameter (b) is a concatenation of two fields.
> Parsing
> is done properly if you add c to the read command:
>
> + echo foo : bar
> + IFS=:
> + read -r a b c
> + printf -- /%s/%s/%s/\n foo bar
> /foo//bar/
>
>
> >
> > The manual ksh(1) provides the following, similar example:
> >
> > > Example: If IFS is set to “:”, and VAR is set to
> > > “A:B::D”, the substitution for $VAR
> > > results in four fields: ‘A’, ‘B’, ‘’ (an empty field), and ‘D’.
> > > Note that if the IFS parameter is set to the NULL string, no field
> > > splitting is done; if the parameter is unset, the default value of
> > > space, tab, and newline is used.
> >
> > Let's try it:
> >
> > echo " A : B::D" | {
> > IFS=" :"
> > read -r arg1 arg2 arg3 arg4
> > printf -- '1st: "%s"\n' "$arg1"
> > printf -- '2nd: "%s"\n' "$arg2"
> > printf -- '3rd: "%s"\n' "$arg3"
> > printf -- '4th: "%s"\n' "$arg4"
> > }
> >
> > bash(1) splits the line into the following fields:
> >
> > 1st: "A"
> > 2nd: "B"
> > 3rd: ""
> > 4th: "D"
> >
> > This is actually the expected output, as described in the manual.
> >
> > However, running the same command in OpenBSD ksh, prints this:
> >
> > 1st: "A"
> > 2nd: ""
> > 3rd: "B"
> > 4th: ":D"
> >
> > A completelly different thing.
> > The same occurs with OpenBSD sh(1).
>
> What you observe is the result of the next paragraph in the man page
> after the example you quoted:
>
> Also, note that the field splitting applies only to the immediate result
> of the substitution. Using the previous example, the substitution for
> $VAR:E results in the fields: `A', `B', `', and `D:E', not `A', `B', `',
> `D', and `E'. This behavior is POSIX compliant, but incompatible with
> some other shell implementations which do field splitting on the word
> which contained the substitution or use IFS as a general whitespace
> delimiter.
Actually you need to look further into the manual since the word
splitting is performed not by parameter substitution but by read:
Reads a line of input from the standard input, separates the line
into fields using the IFS parameter (see Substitution above), and
assigns each field to the specified parameters.
This is why adding an extra variable to read above (and here) makes
it capture the remainder of the string.
For example:
$ alias dump="perl -MData::Dumper -e 'print Dumper @ARGV'"
$ dump a b c
$VAR1 = 'a';
$VAR2 = 'b';
$VAR3 = 'c';
$ ( IFS=:; dump $PATH )
$VAR1 = '/bin';
$VAR2 = '/sbin';
$VAR3 = '/usr/bin';
$VAR4 = '/usr/sbin';
$VAR5 = '/usr/X11R6/bin';
$VAR6 = '/usr/local/bin';
$VAR7 = '/usr/local/sbin';
$VAR8 = '/usr/games';
So given $X:
$ X=' A : B::D'
Parameter substitution:
$ ( IFS=' :'; dump $X )
$VAR1 = 'A';
$VAR2 = 'B';
$VAR3 = '';
$VAR4 = 'D';
Similarly:
$ fn() { dump "$@" ); fn $X
$VAR1 = 'A';
$VAR2 = ':';
$VAR3 = 'B::D';
$ fn() { dump "$@"; }; ( IFS=' :'; fn $X )
$VAR1 = 'A';
$VAR2 = 'B';
$VAR3 = '';
$VAR4 = 'D';
read substitution:
$ echo "$X" | ( IFS=' :'; read a1 a2 a3; dump "$a1" "$a2" "$a3" )
$VAR1 = 'A';
$VAR2 = '';
$VAR3 = 'B::D';
$ echo "$X" | ( IFS=' :'; read a1 a2 a3 a4; dump "$a1" "$a2" "$a3"
"$a4" )
$VAR1 = 'A';
$VAR2 = '';
$VAR3 = 'B';
$VAR4 = ':D';
$ echo "$X" | ( IFS=' :'; read a1 a2 a3 a4 a5; dump "$a1" "$a2" "$a3"
"$a4" "$a5" )
$VAR1 = 'A';
$VAR2 = '';
$VAR3 = 'B';
$VAR4 = '';
$VAR5 = 'D';
It does look like read, which uses its own expansion routine, has
a bug: a2/VAR2 should be 'B' (or 'B::D') not ''.
Matthew
Re: ksh: documented substitution behavior contradicts actual behavior
On Sat, Oct 15, 2022 at 11:42:17PM -0300, Lucas de Sena wrote:
> Hi,
>
> After trying to split a string into fields delimited with colons and
> spaces, I found this bug in how ksh(1) does substitution. The actual
> behavior contradicts what other shells like bash and mksh do and also
> contradicts its own manual.
>
> Running the following on other shells (say, bash) prints "/foo/bar/".
> This command splits the string " foo : bar " into two fields: "foo"
> and "bar", considering colon and space as delimiters.
>
> echo " foo : bar " | {
> IFS=": "
> read -r a b
> printf -- "/%s/%s/\n" "$a" "$b"
> }
>
> However, running the same command in OpenBSD ksh(1) (or sh(1)) splits
> the string into "foo" and ": bar".
This is because the last parameter (b) is a concatenation of two fields.
Parsing
is done properly if you add c to the read command:
+ echo foo : bar
+ IFS=:
+ read -r a b c
+ printf -- /%s/%s/%s/\n foo bar
/foo//bar/
>
> The manual ksh(1) provides the following, similar example:
>
> > Example: If IFS is set to “:”, and VAR is set to
> > “A:B::D”, the substitution for $VAR
> > results in four fields: ‘A’, ‘B’, ‘’ (an empty field), and ‘D’.
> > Note that if the IFS parameter is set to the NULL string, no field
> > splitting is done; if the parameter is unset, the default value of
> > space, tab, and newline is used.
>
> Let's try it:
>
> echo " A : B::D" | {
> IFS=" :"
> read -r arg1 arg2 arg3 arg4
> printf -- '1st: "%s"\n' "$arg1"
> printf -- '2nd: "%s"\n' "$arg2"
> printf -- '3rd: "%s"\n' "$arg3"
> printf -- '4th: "%s"\n' "$arg4"
> }
>
> bash(1) splits the line into the following fields:
>
> 1st: "A"
> 2nd: "B"
> 3rd: ""
> 4th: "D"
>
> This is actually the expected output, as described in the manual.
>
> However, running the same command in OpenBSD ksh, prints this:
>
> 1st: "A"
> 2nd: ""
> 3rd: "B"
> 4th: ":D"
>
> A completelly different thing.
> The same occurs with OpenBSD sh(1).
What you observe is the result of the next paragraph in the man page
after the example you quoted:
Also, note that the field splitting applies only to the immediate result
of the substitution. Using the previous example, the substitution for
$VAR:E results in the fields: `A', `B', `', and `D:E', not `A', `B', `',
`D', and `E'. This behavior is POSIX compliant, but incompatible with
some other shell implementations which do field splitting on the word
which contained the substitution or use IFS as a general whitespace
delimiter.
>
> I could not understand how OpenBSD does the spliting, but the way it
> does is clearly a bug: it does not only contradicts its own manual,
> but also differs from other implementations.
I do not see contradictions in the man page. It does say that behavior
is incompatible with other shells.
>
> Thank you,
> Lucas de Sena.
>
Re: sndio and bit perfect playback
On Fri, Oct 14, 2022 at 04:01:45PM -0400, Geoff Steckel wrote: > > > > I did simple A/B tests with music from CDs and my ears couldn't hear > > the aliasing noise. Try it. > Good a/b >x< tests for audio require extreme care to get accurate results. > Simple sine sweeps don't show IM distortion well. > In most cases numerically equal amounts of IM distortion are far more easily > noticed than harmonic distortions or simple noise (white, pink, etc.) Sure, there is measurable aliasing. My questioning is: is it audible in 99% of the cases? Remains the question, how to handle the 1% remaining. For now, resampling off-line is the least annoying, IMHO. > > Sometimes you just don't want to think about it (ex., when you debug > > audio stuff), so resampling off-line (or switching the device rate) > > still makes sense in certain cases. > This is the classic "why would you ever want to do that?" > "Just as good" is an opinion. > Other OSs can and do provide controls which allow setting the device > sample rate to whatever the device can do. > This user wants that to work. > > This means compiling a kernel with AUDIO_DEBUG Real Soon Now > and inserting a few more DPRINTFs. AFAIK, sample rate changes in the kernel are OK, at least in uaudio driver. Now it's all in sndiod (or how to bypass sndiod to get bit-perfect audio)
Re: sndio and bit perfect playback
On Sat, Oct 15, 2022 at 10:03:52PM +0200, Åke Nordin wrote: > On 10/14/22 11:21, Alexandre Ratchov wrote: > > Here are the measures of the aliasing noise using sine sweeps. Check > > the figure for the 44.1kHz to 48kH conversion, the sndiod column: > > > > https://arverb.com/pub/src/ > > Those are interesting results, indeed. Is there a write-up about the > testing method somewhere where I can read how to reproduce such tests? > Sorry, no detailed write-up, but I did the aliasing measurement roughly as follows. Example for 44.1kHz to 48kHz: First generate a sine sweep at 44.1kHz, up to half the sample rate: sox -V -r 44100 -n sweep-44100.aif synth 30 sin 0+22050 gain -6 the gain must be slightly reduced to avoid filter over-shoots which would clip and show the clipping distortion. Then, resample the file to 48kHz using aucat (it uses the same algorithm as sndiod): aucat -n -i sweep-44100.aif -r 48000 -c 0:0 -o sweep-44100-48000.aif And plot the spectrogram: sox sweep-44100-48000.aif -n spectrogram -o sweep-44100-48000.png sxiv sweep-44100-48000.png The column with the "linear" method is obtained by resampling using the old aucat version. The column with the "offline" method is obtained by resampling with sox instead of aucat. Let me know if you need more information or have suggestions.
