Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Brian Inglis]

On 2017-03-02 11:45, Eric Blake wrote:
> On 03/02/2017 11:28 AM, Brian Inglis wrote:
>> summary of checkbashisms -f run on 140 POSIX shell scripts as 
>> identified by file in my Cygwin 64 /bin/ with 784 packages (about
>> 20% of cygwin-pkg-maint, 8% of setup.ini) installed, 70 possible
>> issues (some from wrapped scripts because of -f):
> 
> Looking through some of the output, I already spot a few interesting
> things:
> 
>> possible bashism in /bin/autoconf-2.69 line 55 ($BASH_SOMETHING):
>> if test -z "$BASH_VERSION$ZSH_VERSION" \
>> && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then
> 
> Not a problem - the script is probing whether it needs to do
> something based on being bash or not, but is written with all shells
> in mind.
> 
>> error: /bin/autoconf-2.69: Unterminated quoted string found, EOF
>> reached. Wanted: <'>, opened in line 429
> 
> bug in checkbashisms; looks like it happens in several places in the
> output

Because of -f checks shell wrappers which it would otherwise just ignore 
- may not be helpful - but -px (POSIX + eXtra) may be more useful, as 
checkbashisms defaults to lintian checks where Debian Policy 10.4 
overrides POSIX.
Also running with -f on bash scripts and checking for exit 4 identifies 
candidates for changing shebang from bash to sh.

>> possible bashism in /bin/autopoint line 57 ($RANDOM):
>> tmp=$TMPDIR/gt$$-$RANDOM
 
> Not a problem - the script is using $RANDOM for an additional level
> of safety, but works safely even when $RANDOM is the empty string (as
> on dash).

Those are where I suggested mktemp would be a better solution to provide 
a random string - most scripts using this open multiple temp files and 
are part of or used by autotools which may be run on MPs.

>> possible bashism in /bin/bzgrep line 66 (${parm/?/pat[/str]}):
>> j=${i//\\/}
> 
> First real problem.  Except that 'grep j= /bin/bzgrep' has no hits on my
> system with bzip2-1.0.6-2 installed; what version was this problem in?

Cygwin patch release 1.0.6-3 installed 2017-02-24 but unchanged since 2007: 

$ ls -ghlo bzip2-1.0.6-3.src/bzip2-1.0.6/bzgrep
-rw-r--r--+ 1 1.7K Jan  2  2007 bzip2-1.0.6-3.src/bzip2-1.0.6/bzgrep

$ file /bin/bz{cmp,diff,less,more,*grep} # scripts from /etc/setup/bzip2.lst.gz
/bin/bzcmp:   symbolic link to bzdiff
/bin/bzdiff:  POSIX shell script, ASCII text executable
/bin/bzless:  symbolic link to bzmore
/bin/bzmore:  POSIX shell script, ASCII text executable
/bin/bzegrep: symbolic link to bzgrep
/bin/bzfgrep: symbolic link to bzgrep
/bin/bzgrep:  POSIX shell script, ASCII text executable
$ fgrep -C9 j= bzip2-1.0.6-3.src/bzip2-1.0.6/bzgrep
for i do
  if test -f "$i"; then :; else if test -f "$i.bz2"; then i="$i.bz2"; fi; fi
  if test $list -eq 1; then
bzip2 -cdfq "$i" | $grep $opt "$pat" 2>&1 > /dev/null && echo $i
r=$?
  elif test $# -eq 1 -o $silent -eq 1; then
bzip2 -cdfq "$i" | $grep $opt "$pat"
r=$?
  else
j=${i//\\/}
j=${j//|/\\|}
j=${j//&/\\&}
j=`printf "%s" "$j" | tr '\n' ' '`
bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${j}:|"
r=$?
  fi
  test "$r" -ne 0 && res="$r"
done
exit $res

Judgement call whether to change shebang or spawn sed or other?

Debian release is 1.0.6-7, bzgrep dated 2014, and bash expressions 
are gone, thanks to lintian and changes made to support more newer 
grep options.

>> possible bashism in /bin/messagebus-config line 26 (should be '.',
>> not 'source'):
>> source ${CSIH_SCRIPT}
> 
> Ah - now we get to an example file that is Cygwin-only (part of the 
> Cygwin port of dbus, but reading that file makes it obvious that it
> is not from upstream) - where changing /bin/sh will cause breakage
> that we can't rely on Debian to have caught first.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Eric Blake]

On 03/02/2017 11:28 AM, Brian Inglis wrote:
> summary of checkbashisms -f run on 140 POSIX shell scripts as 
> identified by file in my Cygwin 64 /bin/ with 784 packages (about 20% 
> of cygwin-pkg-maint, 8% of setup.ini) installed, 70 possible issues 
> (some from wrapped scripts because of -f): 

Looking through some of the output, I already spot a few interesting things:

> possible bashism in /bin/autoconf-2.69 line 55 ($BASH_SOMETHING):
> if test -z "$BASH_VERSION$ZSH_VERSION" \
> && (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then

Not a problem - the script is probing whether it needs to do something
based on being bash or not, but is written with all shells in mind.

> error: /bin/autoconf-2.69: Unterminated quoted string found, EOF reached. 
> Wanted: <'>, opened in line 429

bug in checkbashisms; looks like it happens in several places in the output

> possible bashism in /bin/autopoint line 57 ($RANDOM):
> tmp=$TMPDIR/gt$$-$RANDOM

Not a problem - the script is using $RANDOM for an additional level of
safety, but works safely even when $RANDOM is the empty string (as on dash).

> possible bashism in /bin/bzgrep line 66 (${parm/?/pat[/str]}):
> j=${i//\\/}

First real problem.  Except that 'grep j= /bin/bzgrep' has no hits on my
system with bzip2-1.0.6-2 installed; what version was this problem in?

> possible bashism in /bin/messagebus-config line 26 (should be '.', not 
> 'source'):
> source ${CSIH_SCRIPT}

Ah - now we get to an example file that is Cygwin-only (part of the
Cygwin port of dbus, but reading that file makes it obvious that it is
not from upstream) - where changing /bin/sh will cause breakage that we
can't rely on Debian to have caught first.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Achim Gratz]

Eric Blake writes:
> To reiterate my answer in different terms:
>
> If you can convince Fedora to switch /bin/sh to dash, then I will
> immediately follow in Cygwin.  Until then, I'm worried that there are
> enough scripts in the wild that use bashisms and will therefore break if
> /bin/sh is not bash, even though that number has reduced somewhat since
> Debian made their switch.  Trying to make Cygwin the guinea pig, instead
> of Fedora, is going about it backwards (you WANT the change to be done
> in a place where there is plenty of manpower to deal with the fallout,
> and Fedora has more manpower than Cygwin).

Ack.

I want to put forward that we should get the postinstall scripts moved
from bash to dash as a first step even then.

> I'm still toying with the idea of doing a test release of both bash and
> dash that flips /bin/sh between them; but I'm still stuck on the problem
> that a user MUST upgrade (or downgrade) both packages in tandem, or else
> risk being left without a /bin/sh at all.  Help would be appreciated in
> figuring out the problem (telling me that "dash is faster than bash" is
> not help, nor is telling me that "portable shell scripts don't care if
> /bin/sh is bash or dash" - I already know those points. What I don't
> know is how many non-portable scripts are out there, so how much
> breakage would I be causing by forcing those non-portable scripts to
> deal with their non-portability, and how to minimize the even-worse
> breakage of an upgrade scenario that leaves no /bin/sh at all).

I'd test that in production.  I don't really need handholding
w.r.t. update and potential downgrade process, so if you'd put the two
packages somewhere into an unofficial repo that would work for me.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Brian Inglis]

On 2017-03-02 07:29, Eric Blake wrote:
> On 03/02/2017 07:36 AM, Marco Atzeri wrote:
>> On 02/03/2017 13:36, Steven Penny wrote:
>>> On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote:
 Then you haven't been paying attention. And I didn't even
 attempt to make an argument one way or the other, except to say
 stop arguing. The horse is dead.
>>>
>>> Perhaps you could link to a constructive, concrete idea against
>>> the change that someone has made besides Eric. Even better, you
>>> could post your own constructive idea; surely you havent emailed
>>> twice now with nothing constructive to add?
>>
>> He was constructive, but you seems biased in understanding the answer.
> 
> To reiterate my answer in different terms:
> 
> If you can convince Fedora to switch /bin/sh to dash, then I will
> immediately follow in Cygwin.  Until then, I'm worried that there are
> enough scripts in the wild that use bashisms and will therefore break if
> /bin/sh is not bash, even though that number has reduced somewhat since
> Debian made their switch.  Trying to make Cygwin the guinea pig, instead
> of Fedora, is going about it backwards (you WANT the change to be done
> in a place where there is plenty of manpower to deal with the fallout,
> and Fedora has more manpower than Cygwin).

I can see distros like *BSD or Debian making decisions like that or 
systemd, but not vendors like RH or Canonical: too little upside with 
too much downside for paying customers.

> I'm still toying with the idea of doing a test release of both bash
> and dash that flips /bin/sh between them; but I'm still stuck on the
> problem that a user MUST upgrade (or downgrade) both packages in
> tandem, or else risk being left without a /bin/sh at all. Help would
> be appreciated in figuring out the problem (telling me that "dash is
> faster than bash" is not help, nor is telling me that "portable shell
> scripts don't care if /bin/sh is bash or dash" - I already know those
> points. What I don't know is how many non-portable scripts are out
> there, so how much breakage would I be causing by forcing those
> non-portable scripts to deal with their non-portability, and how to
> minimize the even-worse breakage of an upgrade scenario that leaves
> no /bin/sh at all).

summary of checkbashisms -f run on 140 POSIX shell scripts as 
identified by file in my Cygwin 64 /bin/ with 784 packages (about 20% 
of cygwin-pkg-maint, 8% of setup.ini) installed, 70 possible issues 
(some from wrapped scripts because of -f): 
 14 /bin/cronbug
  7 /bin/procmail-config
  5 /bin/ca-legacy
  4 /bin/pcdovtoppm
  3 /bin/ppmshadow
  3 /bin/ppmfade
  3 /bin/fig2ps2tex
  3 /bin/bzgrep
  2 /bin/ppmrainbow
  2 /bin/pnmquant
  2 /bin/pnmflip
  2 /bin/install-catalog
  1 /bin/zgrep
  1 /bin/zdiff
  1 /bin/xzdiff
  1 /bin/updatedb
  1 /bin/texi2dvi
  1 /bin/ppmquant
  1 /bin/pnmquantall
  1 /bin/pic2graph
  1 /bin/pgmcrater
  1 /bin/pdfroff
  1 /bin/messagebus-config
  1 /bin/gzexe
  1 /bin/grap2graph
  1 /bin/gettextize
  1 /bin/gettext.sh
  1 /bin/eqn2graph
  1 /bin/clisp-link
  1 /bin/ccmakedep
  1 /bin/autopoint
  1 /bin/autoconf-2.69

with diagnostics (some from wrapped scripts because of -f):
 16 echo -e
 12 '((' should be '$(('
  7 alternative test command ([[ foo ]] should be [ foo ])
  7 $RANDOM
  6 unsafe echo with backslash
  5 'function' is useless
  4 type
  4 bash arrays, ${name[0|*|@]}
  3 ${parm/?/pat[/str]}
  3 $_
  1 should be '.', not 'source'
  1 $BASH_SOMETHING
  1 $(OS|MACH)TYPE

So about 20% of packages have scripts installed, with potential issues 
ranging from 5-12*70 == 350-840.
Some potential fixes are obvious from my sample (attached, hopefully 
as inline text which will pass thru the filters): 

- change these scripts to shebang #!/bin/bash 
- replace echo -e with printf in most cases
- replace [[ ... ]] with [ ... ] or case if patterns are used 
- strip leading function 
- use  mktemp to eliminate $RANDOM

but most require judgement in application.

Generated scripts like libtool{,ize} may require complex patches submitted 
upstream to fix what may be sh dependent code generation or execution; not 
included in the attached sample, as file categorized these as:
/bin/libtool:POSIX shell script, ASCII text executable, with very long 
lines, with escape sequences
/bin/libtoolize: POSIX shell script, ASCII text executable, with very long 
lines, with escape sequences

> Hmm, maybe I could create a NEW package, 'sh', which packages /bin/sh as
> however I want it (probably bash to begin with, to at least give people
> time to upgrade and pick up the packaging change before also having to
> deal with any shell changing). New releases of both bash and dash would
> depend on the new package, to guarantee that if you upgrade one shell,
> you pick up the dependency.  And by not 

RE: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Nellis, Kenneth (Conduent)]

From: Eric Blake 
> ...
> Until then, I'm worried that there are
> enough scripts in the wild that use bashisms and will therefore break if
> /bin/sh is not bash, even though that number has reduced somewhat since
> Debian made their switch.
> ...

I was thinking of testing my scripts by changing sh to be a link to 
dash, but ran into the following:

$ which sh bash dash
/usr/bin/sh
/usr/bin/bash
/usr/bin/dash
$ ls -ld /usr/bin/*sh
lrwxrwxrwx 1 knellis Domain Users8 Dec  1 11:38 /usr/bin/csh -> tcsh.exe
-rwxr-xr-x 1 knellis Domain Users 4287 Jan 15 07:53 /usr/bin/instmodsh
lrwxrwxrwx 1 knellis Domain Users   12 May 26  2015 /usr/bin/tclsh -> 
tclsh8.5.exe
$

Why don't I see sh, bash, or dash?
But then I tried:

$ ls -ld /usr/bin/*sh.exe
-rwxr-xr-x 2 knellis Domain Users 100883 Feb 17 08:47 /usr/bin/ash.exe
-rwxr-xr-x 2 knellis Domain Users 739859 Jan 27 14:13 /usr/bin/bash.exe
-rwxr-xr-x 2 knellis Domain Users 100883 Feb 17 08:47 /usr/bin/dash.exe
-rwxr-xr-x 1 knellis Domain Users  11795 Aug 18  2013 /usr/bin/rsh.exe
-rwxr-xr-x 2 knellis Domain Users 739859 Jan 27 14:13 /usr/bin/sh.exe
-rwxr-xr-x 1 knellis Domain Users 650771 Dec 20 06:57 /usr/bin/ssh.exe
-rwxr-xr-x 1 knellis Domain Users 372755 Nov 24 11:07 /usr/bin/tcsh.exe
$

Another unexpected result from Cygwin's .exe magic.

--Ken Nellis

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Brian Inglis]

On 2017-03-02 06:36, Marco Atzeri wrote:
> On 02/03/2017 13:36, Steven Penny wrote:
>> On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote:
>>> Then you haven't been paying attention. And I didn't even attempt
>>> to make an argument one way or the other, except to say stop
>>> arguing. The horse is dead.
>>
>> Perhaps you could link to a constructive, concrete idea against
>> the change that someone has made besides Eric. Even better, you
>> could post your own constructive idea; surely you havent emailed
>> twice now with nothing constructive to add?
> 
> He was constructive, but you seems biased in understanding the
> answer.
> 
> ~ 70% of (weighted*) package managers rejected the proposal
> already here:
> 
> https://cygwin.com/ml/cygwin/2017-02/msg00310.html
> https://cygwin.com/ml/cygwin/2017-02/msg00295.html
> 
> And I agree with them; we have no bandwith/resources
> to handle it.

That makes it over 75% and we can appreciate your efforts and 
viewpoint. I will just have to continue using my Cygwin setup check 
and fixup profile script.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Eric Blake]

On 03/02/2017 07:36 AM, Marco Atzeri wrote:
> On 02/03/2017 13:36, Steven Penny wrote:
>> On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote:
>>> Then you haven't been paying attention.
>>> And I didn't even attempt to make an argument one way or the other,=20
>>> except to say stop arguing. The horse is dead.=
>>
>> Perhaps you could link to a constructive, concrete idea against the
>> change that
>> someone has made besides Eric. Even better, you could post your own
>> constructive
>> idea; surely you havent emailed twice now with nothing constructive to
>> add?
> 
> He was constructive, but you seems biased in understanding the answer.

To reiterate my answer in different terms:

If you can convince Fedora to switch /bin/sh to dash, then I will
immediately follow in Cygwin.  Until then, I'm worried that there are
enough scripts in the wild that use bashisms and will therefore break if
/bin/sh is not bash, even though that number has reduced somewhat since
Debian made their switch.  Trying to make Cygwin the guinea pig, instead
of Fedora, is going about it backwards (you WANT the change to be done
in a place where there is plenty of manpower to deal with the fallout,
and Fedora has more manpower than Cygwin).

I'm still toying with the idea of doing a test release of both bash and
dash that flips /bin/sh between them; but I'm still stuck on the problem
that a user MUST upgrade (or downgrade) both packages in tandem, or else
risk being left without a /bin/sh at all.  Help would be appreciated in
figuring out the problem (telling me that "dash is faster than bash" is
not help, nor is telling me that "portable shell scripts don't care if
/bin/sh is bash or dash" - I already know those points. What I don't
know is how many non-portable scripts are out there, so how much
breakage would I be causing by forcing those non-portable scripts to
deal with their non-portability, and how to minimize the even-worse
breakage of an upgrade scenario that leaves no /bin/sh at all).

Hmm, maybe I could create a NEW package, 'sh', which packages /bin/sh as
however I want it (probably bash to begin with, to at least give people
time to upgrade and pick up the packaging change before also having to
deal with any shell changing). New releases of both bash and dash would
depend on the new package, to guarantee that if you upgrade one shell,
you pick up the dependency.  And by not having /bin/sh in either the
bash or dash package, then we would at least avoid the current situation
where upgrading/downgrading in the wrong order could leave a user
without /bin/sh at all.  You might still be in a situation where the
wrong version of the 'sh' package leaves you with an outdated version of
a shell, or the wrong flavor in relation to the current distro choice,
but that's less of a problem than having no /bin/sh at all.  In fact,
having a separate 'sh' package may make it even easier to pick which
shell flavor you prefer (if I always keep the 'curr' and 'test' versions
pointed to different shells, then you make the choice of which sh flavor
you want by which version you install).

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Marco Atzeri]

On 02/03/2017 13:36, Steven Penny wrote:

On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote:

Then you haven't been paying attention.
And I didn't even attempt to make an argument one way or the other,=20
except to say stop arguing. The horse is dead.=


Perhaps you could link to a constructive, concrete idea against the
change that
someone has made besides Eric. Even better, you could post your own
constructive
idea; surely you havent emailed twice now with nothing constructive to add?


He was constructive, but you seems biased in understanding the answer.

~ 70% of (weighted*) package managers rejected the proposal
already here:

https://cygwin.com/ml/cygwin/2017-02/msg00310.html
https://cygwin.com/ml/cygwin/2017-02/msg00295.html

And I agree with them; we have no bandwith/resources
to handle it.

Regards
Marco


* https://www.cygwin.com/cygwin-pkg-maint

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Wed, 1 Mar 2017 23:31:24, Vince Rice wrote:

Then you haven't been paying attention.
And I didn't even attempt to make an argument one way or the other,=20
except to say stop arguing. The horse is dead.=


Perhaps you could link to a constructive, concrete idea against the change that
someone has made besides Eric. Even better, you could post your own constructive
idea; surely you havent emailed twice now with nothing constructive to add?


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Vince Rice]

> On Mar 1, 2017, at 11:27 PM, Steven Penny wrote:
> 
> On Wed, 1 Mar 2017 21:46:38, Vince Rice wrote:
>> There are valid reasons (which several others have made) not to force others
>> to use it.
> 
> Have they though? I have not seen anyone save Eric (including you) make a 
> valid
> argument beyond "I like it the old way".

Then you haven't been paying attention.
And I didn't even attempt to make an argument one way or the other, 
except to say stop arguing. The horse is dead.
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Wed, 1 Mar 2017 21:46:38, Vince Rice wrote:

There are valid reasons (which several others have made) not to force others
to use it.


Have they though? I have not seen anyone save Eric (including you) make a valid
argument beyond "I like it the old way".


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Vince Rice]

> On Mar 1, 2017, at 6:22 PM, Steven Penny  wrote:
> 
> On Wed, 1 Mar 2017 09:42:27, cyg Simple wrote:
>> Oh but it is.  We're discussing the bike shed named /bin/sh.
>> One of the color names is bash the other is dash; it's still the same
>> bike shed.
> 
> I see, when you realize your argument does not hold water, you resort to name
> calling. Here are concrete arguments:
> 
> 1. Debian uses Dash as /bin/sh
> 2. Ubuntu uses Dash as /bin/sh
> 3. The POSIX standard defines sh, and by definition Dash is closer to this 
> than
>  Bash
> 4. Dash is 3-5 times faster than Bash
> 5. While it might be upsetting to some users, _they_ have made an error in
>  assuming Bash is /bin/sh, it would not be our error in switching to a shell
>  that is closer to POSIX sh
> 
> Now, perhaps you can add a constructive post, or do you want to start with 
> some
> "yo momma" jokes?

He didn't call anyone names. He said that was a bike shed argument. Which is 
getting more accurate the longer this "discussion" goes on.

You've argued your point. If you like dash, then use it. There are valid 
reasons (which several others have made) not to force others to use it.

The decision isn't yours. Or mine. And the ones whose decision it is will 
switch or they won't.

Let's move on.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Wed, 1 Mar 2017 09:42:27, cyg Simple wrote:

Oh but it is.  We're discussing the bike shed named /bin/sh.
One of the color names is bash the other is dash; it's still the same
bike shed.


I see, when you realize your argument does not hold water, you resort to name
calling. Here are concrete arguments:

1. Debian uses Dash as /bin/sh
2. Ubuntu uses Dash as /bin/sh
3. The POSIX standard defines sh, and by definition Dash is closer to this than
  Bash
4. Dash is 3-5 times faster than Bash
5. While it might be upsetting to some users, _they_ have made an error in
  assuming Bash is /bin/sh, it would not be our error in switching to a shell
  that is closer to POSIX sh

Now, perhaps you can add a constructive post, or do you want to start with some
"yo momma" jokes?


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

RE: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Matt Seitz (matseitz)]

Personally, I would invoke a variation of Linus's attitude to breaking user 
space apps:  the distribution should not break existing user's scripts, even if 
they are not following "the rules".

If you want to speed up the scripts distributed by Cygwin, then I would suggest 
modifying those scripts to use #!/bin/dash.  Or give the user an option to 
change "/bin/sh" to point to dash, but don't make that the default setting.

But breaking who-knows-how-many user scripts without the user's informed 
consent seems like an undesirable solution.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[cyg Simple]

On 2/28/2017 4:43 PM, Steven Penny wrote:
> On Tue, 28 Feb 2017 15:52:15, cyg Simple wrote:
>> Ironic that *you* should make the same argument for using #!/bin/bash as
>> I've made to you about using #!/bin/dash.
> 
> Its not the same argument:
> 

Oh but it is.  We're discussing the bike shed named /bin/sh.

> - You are talking about people assuming Dash is /bin/sh
> - I am talking about people assuming Bash is /bin/sh
> 

One of the color names is bash the other is dash; it's still the same
bike shed.

-- 
cyg Simple

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Tue, 28 Feb 2017 15:52:15, cyg Simple wrote:

Ironic that *you* should make the same argument for using #!/bin/bash as
I've made to you about using #!/bin/dash.


Its not the same argument:

- You are talking about people assuming Dash is /bin/sh
- I am talking about people assuming Bash is /bin/sh

Both are wrong, but assuming Dash is /bin/sh is leagues more innocuous:

- Dash doesnt have arrays
- Dash doesnt have herestrings
- Dash doesnt have process substitution
- etc

So even if someone does mess up and include a Dashism with /bin/sh, it should be
trivial to fix. If you include heavy Bashisms with /bin/sh, it can be very hard
to undo that. Dash is not interchangable with POSIX sh, but it is certainly
closer than Bash.


But that doesn't mean you should expect /bin/sh to be dash.


I certainly dont expect that. I expect /bin/sh to be POSIX sh, and try to write
my scripts accordingly. Dash helps with this because it forces you to lose
Bashisms when writing scripts.


The idea is only sound because it uses less resources and when we're talking
about Windows a little resource makes a big difference.


Exactly. Even with Dash as /bin/sh, we are still about 5x slower than Linux
/bin/sh. People who are using /bin/sh with the assumption that it is Bash can
just change their shebang, or remove the Bashisms from their /bin/sh scripts.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[cyg Simple]

On 2/27/2017 7:51 PM, Steven Penny wrote:
> On Tue, 28 Feb 2017 10:12:47, Duncan Roe wrote:
>> "we" being you and who else?
>> /bin/sh has been bash for a long time and I would prefer it stays that
>> way.
> 
> “That’s the way it’s always been done” is not a good reason to keep doing
> something. /bin/sh has been defined for 20 years, and it is not Bash:
> 
> http://pubs.opengroup.org/onlinepubs/7908799/xcu/sh.html
> 
> If you want to write a Bash script, you should be using #!/bin/bash, not
> #!/bin/sh. If we are to continue using Bash, it should be for a good
> reason,
> like the one Eric presented, not just because you want to save 2
> characters on
> your shebang line.
> 

Ironic that *you* should make the same argument for using #!/bin/bash as
I've made to you about using #!/bin/dash.  If you want to ensure that
#!/bin/sh is a certain resemblance of a shell then you must test for it
and take other actions if not.  You cannot portably rely on /bin/sh
having certain qualities but you know that already.

Your argument is that dash is "more" POSIX conforming than bash as sh
and faster because its footprint is smaller.  Should we change, maybe
give it a test before the next release.  It really doesn't matter other
than the work required to make it so and the work required to help those
who get surprised.  But that doesn't mean you should expect /bin/sh to
be dash.  The idea is only sound because it uses less resources and when
we're talking about Windows a little resource makes a big difference.

-- 
cyg Simple

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Tue, 28 Feb 2017 10:12:47, Duncan Roe wrote:

"we" being you and who else?
/bin/sh has been bash for a long time and I would prefer it stays that way.


“That’s the way it’s always been done” is not a good reason to keep doing
something. /bin/sh has been defined for 20 years, and it is not Bash:

http://pubs.opengroup.org/onlinepubs/7908799/xcu/sh.html

If you want to write a Bash script, you should be using #!/bin/bash, not
#!/bin/sh. If we are to continue using Bash, it should be for a good reason,
like the one Eric presented, not just because you want to save 2 characters on
your shebang line.

On Fri, 24 Feb 2017 08:43:30, Eric Blake wrote:

But we are severely limited in volunteer manpower compared to Debian,
and I suspect that 1) there won't be enough testers (we won't know the
real impact until it is no longer experimental, but that is too late),
and 2) even if testers are diligent, we will be unable to patch all the
fallout in any sort of timely manner.



--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Duncan Roe]

On Sat, Feb 25, 2017 at 09:08:55AM -0800, Steven Penny wrote:
> On Sat, 25 Feb 2017 11:46:08, cyg Simple wrote:
> > If *your* script has a dependency to run using *dash* instead of *sh*
> > then you _must_ use #!/bin/dash anyway.
>
> I think you have a fundamental misunderstanding of what Dash is. Dash is a
> minimal shell, similar to the "sh" defined by POSIX:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html
>
> So saying "a dependency to run using *dash*" makes about as much sense as 
> saying
> "Cygwin has a dependency to run using Cygwin".
>
> > And what I _*I*_ don't want dash as /bin/sh? You see not everyone in the
> > community will agree.
>
> With respect to the community, yes of course you get a say. If the community
> doesn???t agree on this we will continue to use Bash. However you do not get 
> a say
> with respect to the standard. The POSIX standard specified that /bin/sh is the
> shell I have linked above. Bash is a superset of that, so by definition those
> extra features are not defined or guaranteed to exist. When writing /bin/sh
> scripts, you should be assuming the users /bin/sh only has those features
> defined by POSIX. Example:
>
> #!/bin/bash -> script that follows might have Bashisms
> #!/bin/dash -> script that follows might have Dashisms (echo -n, local)
> #!/bin/sh -> script that follows should be a POSIX script
>
> > You can make the suggestion to those using your script to do so otherwise 
> > but
> > forcing dash as /bin/sh worldwide isn't something you can actually do.
>
> Yes we certainly can. Debian and Ubuntu have already done this.
>
"we" being you and who else?

/bin/sh has been bash for a long time and I would prefer it stays that way.

OTOH, I have started prefacing scripts with

   /bin/bash -p

to be on the safe side. "-p" stops bash from importing functions from the
environment BTW

Cheers ... Duncan.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Mon, 27 Feb 2017 11:20:54, Csaba Raduly wrote:

Nice strawman argument.
You seem to assume that Dash implements nothing beyond POSIX (and
hence anything that runs on dash must run on every other shell). This
is not the case.


Perhaps you should have read my full post. Here, let me quote myself and link
for you:

On Sat, 25 Feb 2017 09:08:55, Steven Penny wrote:

#!/bin/bash -> script that follows might have Bashisms
#!/bin/dash -> script that follows might have Dashisms (echo -n, local)
#!/bin/sh -> script that follows should be a POSIX script


http://cygwin.com/ml/cygwin/2017-02/msg00320.html

So, as I said before and now again, Dash is a minimal shell, similar to POSIX
shell. Yes, it is possible to be in a situation like this:

- script has #!/bin/sh
- script has Dashisms, which can cause problem on POSIX shell

However being that "echo -n" and "local" are probably the only Dashisms, that is
trivial to fix. Compare to a Bash problem:

- script has #!/bin/sh
- script has Bashisms, which can cause problem on POSIX shell

Even switching from a Bash script to Dash script can be a herculean task, if the
script heavily relies on Bashisms like arrays, herestrings, process
substitution, etc.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Csaba Raduly]

On Sat, Feb 25, 2017 at 6:08 PM, Steven Penny  wrote:
> On Sat, 25 Feb 2017 11:46:08, cyg Simple wrote:
>>
>> If *your* script has a dependency to run using *dash* instead of *sh*
>> then you _must_ use #!/bin/dash anyway.
>
>
> I think you have a fundamental misunderstanding of what Dash is. Dash is a
> minimal shell, similar to the "sh" defined by POSIX:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html
>
> So saying "a dependency to run using *dash*" makes about as much sense as
> saying
> "Cygwin has a dependency to run using Cygwin".

Nice strawman argument.
You seem to assume that Dash implements nothing beyond POSIX (and
hence anything that runs on dash must run on every other shell). This
is not the case.

>From the dash man page:

dash is the standard command interpreter for the Linux system. The
current version of dash is in the process of being changed to conform
with the POSIX 1003.2 and 1003.2a specifications for the shell. This
version has many features which make it appear similar in some
respects to the Korn shell, but it is not a Korn shell clone. Only
features designated by POSIX, plus a few Berkeley extensions, are
being incorporated into this shell.

Csaba
-- 
GCS a+ e++ d- C++ ULS$ L+$ !E- W++ P+++$ w++$ tv+ b++ DI D++ 5++
The Tao of math: The numbers you can count are not the real numbers.
Life is complex, with real and imaginary parts.
"Ok, it boots. Which means it must be bug-free and perfect. " -- Linus Torvalds
"People disagree with me. I just ignore them." -- Linus Torvalds

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Sat, 25 Feb 2017 11:46:08, cyg Simple wrote:

If *your* script has a dependency to run using *dash* instead of *sh*
then you _must_ use #!/bin/dash anyway.


I think you have a fundamental misunderstanding of what Dash is. Dash is a
minimal shell, similar to the "sh" defined by POSIX:

http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html

So saying "a dependency to run using *dash*" makes about as much sense as saying
"Cygwin has a dependency to run using Cygwin". 


And what I _*I*_ don't want dash as /bin/sh? You see not everyone in the
community will agree.


With respect to the community, yes of course you get a say. If the community
doesn’t agree on this we will continue to use Bash. However you do not get a say
with respect to the standard. The POSIX standard specified that /bin/sh is the
shell I have linked above. Bash is a superset of that, so by definition those
extra features are not defined or guaranteed to exist. When writing /bin/sh
scripts, you should be assuming the users /bin/sh only has those features
defined by POSIX. Example:

#!/bin/bash -> script that follows might have Bashisms
#!/bin/dash -> script that follows might have Dashisms (echo -n, local)
#!/bin/sh -> script that follows should be a POSIX script


You can make the suggestion to those using your script to do so otherwise but
forcing dash as /bin/sh worldwide isn't something you can actually do.


Yes we certainly can. Debian and Ubuntu have already done this.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Fri, 24 Feb 2017 08:43:30, Eric Blake wrote:

But we are severely limited in volunteer manpower compared to Debian,
and I suspect that 1) there won't be enough testers (we won't know the
real impact until it is no longer experimental, but that is too late),
and 2) even if testers are diligent, we will be unable to patch all the
fallout in any sort of timely manner.



Are you really prepared to force the Cygwin community through that much
growing pain?


This is a fantastic question, thank you for asking. For me to answer it, I need
to quantify how much pain is actually involved. To truly answer that I would
need to download an check every single Cygwin package. That is probably overkill
because surely some packages are rarely used. I used this methodology to come up
with a list of packages to test:

1. Base category and recursive dependencies
2. Search my own repositories for "sage install", and got recursive dependencies
  for those too
3. Add checkbashisms package and recursive dependencies

This yields the list at the bottom of this post. Probably some packages need to
be added to this list, but it is a start. Next, I put all files from those
packages through this Awk selector:

   FNR == 1 && $0 ~ "^#!.*/sh"

To get potential scripts that need to be fixed. Then I ran checkbashism on those
files. The results are here:

http://github.com/svnpenn/a/blob/0a85bf7/docs/cygwin-bash-to-dash.txt

I am willing to correct these, if you are willing to proceed.



_autorebase alternatives autoconf autoconf2.1 autoconf2.5 automake automake1.10
automake1.11 automake1.12 automake1.13 automake1.14 automake1.15 automake1.4
automake1.5 automake1.6 automake1.7 automake1.8 automake1.9 base-cygwin
base-files bash binutils bison bzip2 ca-certificates checkbashisms cmake
coreutils csih curl cygrunsrv cygutils cygwin cygwin-devel dash dejavu-fonts
desktop-file-utils dialog diffutils dri-drivers ed editrights file findutils
flex gamin gawk gcc-core getent ghostscript ghostscript-fonts-other
ghostscript-fonts-std git gnupg grep groff gsettings-desktop-schemas gzip
hostname info ipc-utils less libarchive13 libargp libatomic1 libattr1 libblkid1
libbz2_1 libcairo2 libcom_err2 libcrypt0 libcurl4 libdb5.3 libdialog13 libedit0
libEGL1 libexpat1 libfam0 libffi6 libfontconfig1 libfontconfig-common
libfreetype6 libgcc1 libgd3 libgdbm4 libGL1 libglapi0 libglib2.0_0 libgmp10
libgnutls28 libgomp1 libgraphite2_3 libgs9 libgssapi_krb5_2 libguile17
libharfbuzz0 libharfbuzz-icu0 libhogweed2 libICE6 libiconv libiconv2 libicu57
libicu58 libidn11 libidn2_0 libintl8 libisl13 libjbig2 libjpeg8 libk5crypto3
libkpathsea6 libkrb5_3 libkrb5support0 liblcms2_2 libllvm3.8 libllvm3.9 libltdl7
liblz4_1 liblzma5 liblzo2_2 libmetalink3 libmpc3 libmpfr4 libncurses-devel
libncursesw10 libnettle4 libnghttp2_14 libnspr4 libnss3 libopenjpeg1
libopenldap2_4_2 libopenssl100 libp11-kit0 libpaper1 libpaper-common libpcre1
libpipeline1 libpixman1_0 libpng16 libpoppler62 libpopt0 libpopt-common libpsl5
libptexenc1 libquadmath0 libreadline7 libsasl2_3 libsigsegv2 libSM6
libsmartcols1 libsqlite3_0 libssh2_1 libssp0 libstdc++6 libsynctex1 libtasn1_6
libtexlua52_5 libtexluajit2 libtiff6 libtool libtxc_dxtn libunistring2 libusb0
libuuid1 libuuid-devel libvtv0 libwebp5 libX11_6 libX11-xcb1 libXau6 libXaw7
libxcb1 libxcb-glx0 libxcb-render0 libxcb-shm0 libXdmcp6 libXext6 libXft2 libXi6
libxml2 libXmu6 libXpm4 libXrender1 libXt6 libzzip0.13 login m4 make man-db
mingw64-x86_64-binutils mingw64-x86_64-curl mingw64-x86_64-gcc-core
mingw64-x86_64-gcc-g++ mingw64-x86_64-headers mingw64-x86_64-libgnurx
mingw64-x86_64-libssh2 mingw64-x86_64-nghttp2 mingw64-x86_64-openssl
mingw64-x86_64-pkg-config mingw64-x86_64-runtime
mingw64-x86_64-windows-default-manifest mingw64-x86_64-win-iconv
mingw64-x86_64-winpthreads mingw64-x86_64-zlib mintty ncurses openssh openssl
p11-kit p11-kit-trust perl perl_autorebase perl_base perl-Carp perl-Digest-SHA
perl-Encode-Locale perl-Error perl-File-Listing perl-HTML-Parser
perl-HTML-Tagset perl-HTTP-Cookies perl-HTTP-Daemon perl-HTTP-Date
perl-HTTP-Message perl-HTTP-Negotiate perl-IO-HTML perl-IO-String
perl-libwww-perl perl-LWP-MediaTypes perl-Net-HTTP perl-Pod-Simple perl-Socket
perl-TermReadKey perl-Tk perl-Tk-Pod perl-Try-Tiny perl-Unicode-Normalize
perl-URI perl-WWW-RobotRules perl-XML-Parser pkg-config poppler-data
publicsuffix-list-dafsa python rebase rsync run sed shared-mime-info tar
terminfo texinfo texlive texlive-collection-basic
texlive-collection-fontsrecommended texlive-collection-latex tzcode tzdata unzip
util-linux vim-minimal w32api-headers w32api-runtime wget which
windows-default-manifest words xz yasm zip zlib0


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[cyg Simple]

On 2/23/2017 2:46 PM, Steven Penny wrote:
> On Thu, 23 Feb 2017 20:38:08, Andrey Repin wrote:
>> $ cat /etc/postinstall/zp_dash-as-bin-sh.dash
>> #!/bin/dash
> 
>> test ! -h /bin/sh.exe && {
>>   _f="$(mktemp /bin/sh.XX)"
>>   {
>> mv /bin/sh.exe "$_f"
>> ln -s /bin/dash.exe /bin/sh.exe
>>   } || mv "$_f" /bin/sh.exe
>>   rm "$_f"
>> }
>> exit 0
> 
> This is a bad idea.
> 
> If script succeeds, then yes sh is now a symlink to Dash, but only for my
> system. I still have to use #!/bin/dash with my scripts, because otherwise
> people with Bash -> sh will not get the speed boost. The whole point of
> this
> thread is that it should be changed for everyone.

If *your* script has a dependency to run using *dash* instead of *sh*
then you _must_ use #!/bin/dash anyway.  And what I _*I*_ don't want
dash as /bin/sh?  You see not everyone in the community will agree.  You
can make the suggestion to those using your script to do so otherwise
but forcing dash as /bin/sh worldwide isn't something you can actually do.

-- 
cyg Simple

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Brian Inglis]

On 2017-02-24 07:32, Eric Blake wrote:
> On 02/23/2017 05:44 PM, Steven Penny wrote:
>> On Thu, 23 Feb 2017 12:59:27, Brian Inglis wrote:
>>> Some people may continue to need or prefer using bash in POSIX
>>> mode.
>> You might know this already, but "Bash in POSIX mode" is not quite 
>> POSIX. For example, arrays are not defined by POSIX:
> You may not know this, but POSIX doesn't forbid extensions. Bash in 
> POSIX mode does NOT cripple all extensions, merely those that are 
> incompatible with POSIX. That's why so many people end up writing 
> #!/bin/sh scripts with bashisms, then get surprised when they break
> on a different shell.
> If you want a STRICT shell, Debian's posh is even better than dash
> at ferreting out use of non-portable extensions.

For those interested, Cygwin packages are available for posh, a 
"stripped down version of pdksh", and checkbashisms, a Perl script for 
flagging any "shell feature that is not required to be supported by 
POSIX".
Neither dash nor posh have dependencies other than cygwin1.dll, but 
posh is not exactly lightweight at 598KB compared to dash at 99KB.
While posh may be good for checking scripts when installed as /bin/sh, 
dash is a better choice for running scripts as a production /bin/sh.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Lee Dilkie]

On 2/24/2017 9:43 AM, Eric Blake wrote:

On 02/23/2017 10:57 PM, Steven Penny wrote:

Or more likely, many people likely have pre-existing scripts wrongly
written as #!/bin/sh but which use bash-ism rather than portable
POSIX-specified shell


However, I think it is worth the trouble.


If you'd like, I can post experimental versions of both bash and dash,
which MUST be upgraded (or downgraded) in lockstep, where I move /bin/sh
over to the dash package (do it wrong, and you could be left with no
/bin/sh at all, which is not a good idea - although maybe I can use some
postinstall scripts so that at least the upgrade side tries to play nice
even when someone only does a partial upgrade).  If people will then
test with those experimental versions installed, and report breakage, we
could get a feel for how many scripts installed by default are broken.

But we are severely limited in volunteer manpower compared to Debian,
and I suspect that 1) there won't be enough testers (we won't know the
real impact until it is no longer experimental, but that is too late),
and 2) even if testers are diligent, we will be unable to patch all the
fallout in any sort of timely manner.

Are you really prepared to force the Cygwin community through that much
growing pain?

I agree that /bin/sh as dash is much faster at executing configure
scripts. But configure scripts aren't the only scripts in the wild. We
do have checkbashisms ported to Cygwin, and that can help, but it is not
a panacea.



If dash doesn't support all the features of bash (which is incorrectly assumed
in some cases for sh), aren't you just asking for trouble by breaking things?

-lee

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Eric Blake]

On 02/23/2017 10:57 PM, Steven Penny wrote:
>> Or more likely, many people likely have pre-existing scripts wrongly
>> written as #!/bin/sh but which use bash-ism rather than portable
>> POSIX-specified shell
> 
> However, I think it is worth the trouble.

If you'd like, I can post experimental versions of both bash and dash,
which MUST be upgraded (or downgraded) in lockstep, where I move /bin/sh
over to the dash package (do it wrong, and you could be left with no
/bin/sh at all, which is not a good idea - although maybe I can use some
postinstall scripts so that at least the upgrade side tries to play nice
even when someone only does a partial upgrade).  If people will then
test with those experimental versions installed, and report breakage, we
could get a feel for how many scripts installed by default are broken.

But we are severely limited in volunteer manpower compared to Debian,
and I suspect that 1) there won't be enough testers (we won't know the
real impact until it is no longer experimental, but that is too late),
and 2) even if testers are diligent, we will be unable to patch all the
fallout in any sort of timely manner.

Are you really prepared to force the Cygwin community through that much
growing pain?

I agree that /bin/sh as dash is much faster at executing configure
scripts. But configure scripts aren't the only scripts in the wild. We
do have checkbashisms ported to Cygwin, and that can help, but it is not
a panacea.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Eric Blake]

On 02/23/2017 05:44 PM, Steven Penny wrote:
> On Thu, 23 Feb 2017 12:59:27, Brian Inglis wrote:
>> Some people may continue to need or prefer using bash in POSIX mode.
> 
> You might know this already, but "Bash in POSIX mode" is not quite
> POSIX. For
> example, arrays are not defined by POSIX:

You may not know this, but POSIX doesn't forbid extensions. Bash in
POSIX mode does NOT cripple all extensions, merely those that are
incompatible with POSIX.  That's why so many people end up writing
#!/bin/sh scripts with bashisms, then get surprised when they break on a
different shell.

If you want a STRICT shell, Debian's posh is even better than dash at
ferreting out use of non-portable extensions.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Yaakov Selkowitz]

On 2017-02-23 14:41, Eric Blake wrote:

Or more likely, many people likely have pre-existing scripts wrongly
written as #!/bin/sh but which use bash-ism rather than portable
POSIX-specified shell (if it is portable, then bash vs. dash executing
the script shouldn't matter - but as Debian found out when they switched
to dash, there are a lot of non-portable scripts).  On the other hand,
Debian has already fixed much of the pre-packaged badness out there and
pushed fixes upstream where feasible, so these days it's not as hairy to
switch as when Debian first did it.

But part of me is still reluctant to make the switch as long as Fedora
still favors bash over dash.


I concur; we simply don't have the manpower to handle this ourselves.

--
Yaakov

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Thu, 23 Feb 2017 22:15:37, "Larry Hall (Cygwin)" wrote:

Since making a change of this magnitude is going to be an undertaking, we
should be sure we're going to see the intended benefits before enduring the
pain such a change would bring, though hopefully that pain would be
short-lived and/or minor. :-)


I am open to others debunking my previous test with other/better tests, but from
where I sit the benefit can be clearly seen. Not just with script times, but
startup time will see a 2-3 fold increase as well. This can be hugely helpful
with situations like using Autotools (configure), where "sh" is spawned hundreds
of times.

I have done heavy testing "living" with Dash. Between my repos I have over 80
Dash scripts, with usage going back to 2015:

http://github.com/svnpenn/a/commit/b4908b7

I think Eric said it well:

On Thu, 23 Feb 2017 14:41:52, Eric Blake wrote:

Or more likely, many people likely have pre-existing scripts wrongly
written as #!/bin/sh but which use bash-ism rather than portable
POSIX-specified shell


However, I think it is worth the trouble.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Larry Hall (Cygwin)]

On 02/23/2017 06:01 PM, Tony Kelman wrote:

The big question remains, where this speed boost coming from?
Is this a startup time? Or some internal slowness?
Because in latter case, given your STC, this is a bash issue and should be
reported upstream.


Dunno what you meant by STC, but upstream is well aware:

$ man bash | tail -n 21 | head -n 2
BUGS
It's too big and too slow.


Forgive me but this entire thread has been giving me a serious case of
deja-vu.  We've been down a similar path to this before, the last time
around with "ash" rather than "dash" but the arguments sound very familiar.
Brian Dessent has a nice summary a paragraph into his response below:

https://sourceware.org/ml/cygwin/2008-03/msg00369.html

Now this was "ash" and not "dash" and the actual transition from "bash" to
"ash" and back again occurred well over 10 years ago, so allot of things
have changed for sure.  But I bring this up because I think the parallels
are there.  Since making a change of this magnitude is going to be an
undertaking, we should be sure we're going to see the intended benefits
before enduring the pain such a change would bring, though hopefully that
pain would be short-lived and/or minor. :-)

--
Larry

_

A: Yes.
> Q: Are you sure?
>> A: Because it reverses the logical flow of conversation.
>>> Q: Why is top posting annoying in email?

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Larry Hall (Cygwin)]

On 02/23/2017 06:01 PM, Tony Kelman wrote:

The big question remains, where this speed boost coming from?
Is this a startup time? Or some internal slowness?
Because in latter case, given your STC, this is a bash issue and should be
reported upstream.


Dunno what you meant by STC, but upstream is well aware:

$ man bash | tail -n 21 | head -n 2
BUGS
It's too big and too slow.


Forgive me but this entire thread has been giving me a serious case of
deja-vu.  We've been down a similar path to this before, the last time
around with "ash" rather than "dash" but the arguments sound very familiar.
Brian Dessent has a nice summary a paragraph into his response below:

https://sourceware.org/ml/cygwin/2008-03/msg00369.html

Now this was "ash" and not "dash" and the actual transition from "bash" to
"ash" and back again occurred well over 10 years ago, so allot of things
have changed for sure.  But I bring this up because I think the parallels
are there.  Since making a change of this magnitude is going to be an
undertaking, we should be sure we're going to see the intended benefits
before enduring the pain such a change would bring, though hopefully that
pain would be short-lived and/or minor. :-)

--
Larry

_

A: Yes.
> Q: Are you sure?
>> A: Because it reverses the logical flow of conversation.
>>> Q: Why is top posting annoying in email?

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Thu, 23 Feb 2017 12:59:27, Brian Inglis wrote:

Some people may continue to need or prefer using bash in POSIX mode.


You might know this already, but "Bash in POSIX mode" is not quite POSIX. For
example, arrays are not defined by POSIX:

http://mywiki.wooledge.org/Bashism#Arrays

Yes Bash even in POSIX mode handles them:

   $ bash --posix -c 'z=(2 3 4); echo ${z[2]}'
   4

Dash rightfully does not handle them:

   $ dash -c 'z=(2 3 4); echo ${z[2]}'
   dash: 1: Syntax error: "(" unexpected


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Andrey Repin]

Greetings, Tony Kelman!

>> The big question remains, where this speed boost coming from?
>> Is this a startup time? Or some internal slowness?
>> Because in latter case, given your STC, this is a bash issue and should be
>> reported upstream.

> Dunno what you meant by STC,

https://cygwin.com/acronyms/#STC

> but upstream is well aware:

> $ man bash | tail -n 21 | head -n 2
> BUGS
>It's too big and too slow.

That's not answering my question.


-- 
With best regards,
Andrey Repin
Friday, February 24, 2017 02:18:52

Sorry for my terrible english...


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Fri, 24 Feb 2017 00:50:08, Andrey Repin wrote:

In fact, you should use specific shell name, if you want specific behavior.


That is not at issue here. Both before and after this proposed change, I can use
#!/bin/dash to explicitly call Dash and #!/bin/bash to explicitly call Bash.
What _is_ at issue here is what is called with #!/bin/sh for _all_ users, not
just myself or those whom have run your homebrewed script.

In that regard, since we are giving deference to Debian by using their shell, I
say we continue that pattern and use Dash as /bin/sh.


The big question remains, where this speed boost coming from?
Is this a startup time? Or some internal slowness?
Because in latter case, given your STC, this is a bash issue and should be
reported upstream.


Dash is faster than Bash. This is not a question, it is a fact. It is not a
_bug_ any more than a Ducati being faster than an 18-wheeler is a bug. Dash by
design has less features than Bash _so_ that it can be faster than Dash. It
would suit you well to do some research of your own on this topic before posting
further.

Cheers


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Vince Rice]

> On Feb 23, 2017, at 5:04 PM, Eliot Moss  wrote:
> 
> On 2/23/2017 6:01 PM, Tony Kelman wrote:
>>> The big question remains, where this speed boost coming from?
>>> Is this a startup time? Or some internal slowness?
>>> Because in latter case, given your STC, this is a bash issue and should be
>>> reported upstream.
>> 
>> Dunno what you meant by STC,
> 
> Indeed, this gives around 20 possibilities, none of which seem to fit:
> 
> http://www.acronymfinder.com/Slang/STC.html

If it's used here, then the first place to look is on the Cygwin acronym page.

https://cygwin.com/acronyms/#STC
--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Kenneth Wolcott]

On Thu, Feb 23, 2017 at 3:04 PM, Eliot Moss  wrote:
> On 2/23/2017 6:01 PM, Tony Kelman wrote:
>>>
>>> The big question remains, where this speed boost coming from?
>>> Is this a startup time? Or some internal slowness?
>>> Because in latter case, given your STC, this is a bash issue and should
>>> be
>>> reported upstream.
>>
>>
>> Dunno what you meant by STC,
>
>
> Indeed, this gives around 20 possibilities, none of which seem to fit:
>
> http://www.acronymfinder.com/Slang/STC.html
>
> Cheers!  Eliot Moss

Small Test Case.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Eliot Moss]

On 2/23/2017 6:01 PM, Tony Kelman wrote:

The big question remains, where this speed boost coming from?
Is this a startup time? Or some internal slowness?
Because in latter case, given your STC, this is a bash issue and should be
reported upstream.


Dunno what you meant by STC,


Indeed, this gives around 20 possibilities, none of which seem to fit:

http://www.acronymfinder.com/Slang/STC.html

Cheers!  Eliot Moss

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Tony Kelman]

> The big question remains, where this speed boost coming from?
> Is this a startup time? Or some internal slowness?
> Because in latter case, given your STC, this is a bash issue and should be
> reported upstream.

Dunno what you meant by STC, but upstream is well aware:

$ man bash | tail -n 21 | head -n 2
BUGS
   It's too big and too slow.

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Andrey Repin]

Greetings, Steven Penny!

> On Thu, 23 Feb 2017 20:38:08, Andrey Repin wrote:
>> $ cat /etc/postinstall/zp_dash-as-bin-sh.dash
>> #!/bin/dash

>> test ! -h /bin/sh.exe && {
>>   _f="$(mktemp /bin/sh.XX)"
>>   {
>> mv /bin/sh.exe "$_f"
>> ln -s /bin/dash.exe /bin/sh.exe
>>   } || mv "$_f" /bin/sh.exe
>>   rm "$_f"
>> }
>> exit 0

> This is a bad idea.

This is only bad idea for people assuming too much.

> If script succeeds, then yes sh is now a symlink to Dash, but only for my
> system. I still have to use #!/bin/dash with my scripts,

In fact, you should use specific shell name, if you want specific behavior.

> because otherwise
> people with Bash -> sh will not get the speed boost. The whole point of this
> thread is that it should be changed for everyone.

The big question remains, where this speed boost coming from?
Is this a startup time? Or some internal slowness?
Because in latter case, given your STC, this is a bash issue and should be
reported upstream.


-- 
With best regards,
Andrey Repin
Friday, February 24, 2017 00:47:31

Sorry for my terrible english...


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Eric Blake]

On 02/23/2017 01:59 PM, Brian Inglis wrote:
>> If script succeeds, then yes sh is now a symlink to Dash, but only
>> for my system. I still have to use #!/bin/dash with my scripts,
>> because otherwise people with Bash -> sh will not get the speed
>> boost. The whole point of this thread is that it should be changed
>> for everyone.
> 
> And should be done using update_alternatives so it and associated 
> man pages can easily be changed if anyone suffers breakage.
> Some people may continue to need or prefer using bash in POSIX mode.

Or more likely, many people likely have pre-existing scripts wrongly
written as #!/bin/sh but which use bash-ism rather than portable
POSIX-specified shell (if it is portable, then bash vs. dash executing
the script shouldn't matter - but as Debian found out when they switched
to dash, there are a lot of non-portable scripts).  On the other hand,
Debian has already fixed much of the pre-packaged badness out there and
pushed fixes upstream where feasible, so these days it's not as hairy to
switch as when Debian first did it.

But part of me is still reluctant to make the switch as long as Fedora
still favors bash over dash.

One problem with using update_alternatives: the alternatives system
relies on symlinks, but Windows is not so smart about executing through
symlinks; are there people that will have a broken setup if /bin/sh is a
symlink rather than an actual executable (hard links are fine, but
symlinks is where I worry).

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Brian Inglis]

On 2017-02-23 12:46, Steven Penny wrote:
> On Thu, 23 Feb 2017 20:38:08, Andrey Repin wrote:
>> $ cat /etc/postinstall/zp_dash-as-bin-sh.dash
>> #!/bin/dash
> 
>> test ! -h /bin/sh.exe && {
>>   _f="$(mktemp /bin/sh.XX)"
>>   {
>> mv /bin/sh.exe "$_f"
>> ln -s /bin/dash.exe /bin/sh.exe
>>   } || mv "$_f" /bin/sh.exe
>>   rm "$_f"
>> }
>> exit 0
> 
> This is a bad idea.
> 
> If script succeeds, then yes sh is now a symlink to Dash, but only
> for my system. I still have to use #!/bin/dash with my scripts,
> because otherwise people with Bash -> sh will not get the speed
> boost. The whole point of this thread is that it should be changed
> for everyone.

And should be done using update_alternatives so it and associated 
man pages can easily be changed if anyone suffers breakage.
Some people may continue to need or prefer using bash in POSIX mode.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada

--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Thu, 23 Feb 2017 20:38:08, Andrey Repin wrote:

$ cat /etc/postinstall/zp_dash-as-bin-sh.dash
#!/bin/dash



test ! -h /bin/sh.exe && {
  _f="$(mktemp /bin/sh.XX)"
  {
mv /bin/sh.exe "$_f"
ln -s /bin/dash.exe /bin/sh.exe
  } || mv "$_f" /bin/sh.exe
  rm "$_f"
}
exit 0


This is a bad idea.

If script succeeds, then yes sh is now a symlink to Dash, but only for my
system. I still have to use #!/bin/dash with my scripts, because otherwise
people with Bash -> sh will not get the speed boost. The whole point of this
thread is that it should be changed for everyone.


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Andrey Repin]

Greetings, Steven Penny!

> On Fri, 17 Feb 2017 07:50:34, "Eric Blake (cygwin)" wrote:
>> For now, there are no immediate plans of replacing /bin/sh
>> with dash, but the possibility remains for the future.

> I am a fan of this.

$ cat /etc/postinstall/zp_dash-as-bin-sh.dash
#!/bin/dash

test ! -h /bin/sh.exe && {
  _f="$(mktemp /bin/sh.XX)"
  {
mv /bin/sh.exe "$_f"
ln -s /bin/dash.exe /bin/sh.exe
  } || mv "$_f" /bin/sh.exe
  rm "$_f"
}
exit 0


> Debian does this already, and it would dramatically speed up
> shell scripts:

> $ cat alfa.sh
> x=40
> until [ "$((x -= 1))" = 0 ]
> do :
> done

> $ time bash alfa.sh
> real0m5.735s

> $ time dash alfa.sh
> real0m1.112s

> Dash can handle large scripts; I am already using it with one (545 SLOC):

> http://github.com/svnpenn/sage/blob/master/sage


-- 
With best regards,
Andrey Repin
Thursday, February 23, 2017 20:36:35

Sorry for my terrible english...


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple

Re: [ANNOUNCEMENT] Updated: dash-0.5.9.1-1

[Steven Penny]

On Fri, 17 Feb 2017 07:50:34, "Eric Blake (cygwin)" wrote:

For now, there are no immediate plans of replacing /bin/sh
with dash, but the possibility remains for the future.


I am a fan of this. Debian does this already, and it would dramatically speed up
shell scripts:

   $ cat alfa.sh
   x=40
   until [ "$((x -= 1))" = 0 ]
   do :
   done

   $ time bash alfa.sh
   real0m5.735s

   $ time dash alfa.sh
   real0m1.112s

Dash can handle large scripts; I am already using it with one (545 SLOC):

http://github.com/svnpenn/sage/blob/master/sage


--
Problem reports:   http://cygwin.com/problems.html
FAQ:   http://cygwin.com/faq/
Documentation: http://cygwin.com/docs.html
Unsubscribe info:  http://cygwin.com/ml/#unsubscribe-simple