Re: Substring search in dash [ _NOT_ bash ] shell

[Thomas Schmitt]

Hi,

Greg Wooledge wrote:
> https://lists.gnu.org/archive/html/bug-bash/2015-07/msg00125.html
which states

  $ test -n '<' -a true
  -bash: test: too many arguments

This looks rather like the effect of having an operator named '<':

  $ test 1 '<' 2 && echo yes
  yes
  $ test 3 '<' 2 && echo yes
  $ 

One gets the same problem with traditional operator '='

  $ test -n '=' -a 1 = 1 && echo yes
  bash: test: too many arguments

because the parser of "test" wants to be smart with this

  $ test -n '=' -a || echo no
  no
  $ test -n '=' && echo yes
  yes

The problem gets covered if only three arguments are given to "test"
or if the reserved operator name is not the second argument.

This is well an argument against attempts to be smart with "test",
i have to confess.


Have a nice day :)

Thomas

Re: Substring search in dash [ _NOT_ bash ] shell

[Greg Wooledge]

On Tue, Sep 27, 2016 at 05:05:08PM +0200, Thomas Schmitt wrote:
> Greg Wooledge wrote:
> > Do not use -o and -a in a test command.
> > http://mywiki.wooledge.org/BashPitfalls#pf6
> 
> As if anybody would dare to touch the old "test" command. :))
> 
> Rather i avoid "[" brackets, not to allow any implementor an excuse.
> But in plaintext mode i wanted to stay near to the given original example.
> It worked when i tested in dash and bash.

The test and [ commands have the same features and limitations.  The only
difference is the final argument of [ has to be ].

People do run into the problem occasionally in real life:

https://lists.gnu.org/archive/html/bug-bash/2015-07/msg00125.html

But, I will admit it's rare.  (And hard to Google, though this is mostly
due to Google's extremely inconvenient refusal to let you use punctuation
in searches.)

Re: Substring search in dash [ _NOT_ bash ] shell

[Thomas Schmitt]

Hi,

i wrote:
> > if you cannot find a dash tutorial then get a tutorial for bash or sh and 
> > test in dash whether the proposals apply properly.

Greg Wooledge wrote:
> First, there are more bad tutorials out there in the wild than good
> tutorials, by at least one order of magnitude.

But the conclusion can hardly be "Use no tutorial and test in dash whether
the proposals apply properly".


> Second, trying to work backwards from a bash tutorial to a POSIX sh/dash
> programming style is going to be maddening.

It will keep the reader from acquiring bashisms.
(Of course a shell tutorial should always state what is portable to
 other shell variations and what is a peculiarity of the particular shell.
 But, well, orders of magnitude ...)


> Third, even dash has a few extensions to the POSIX shell syntax.

And it will hardly ever drop support for them.
The original request was explicitely for "dash" and not for POSIX shell.


> Kernighan & Pike's "The Unix Programming Environment".
> 
> for i in $@

Ouch. Indeed, Bourne shows similar reluctance to use quotation marks:

  for i in *
  do  if test -d $d/$i
  ...

Nevertheless the examples still work in descendants of Bourne's shell.
Not better than in the past, but not worse either.


> It is terrifying!  It's every single BAD shell practice all thrown together
> into a single example!

I feel with you.

Especially the "set" stunt is perverted and invites for due punishment
when the failure will hurt most.


> 40 years of this.

At least we proved that one can survive it.
For me it's only 31 years now. In the beginning it was a bit frightening.


> You are redirecting stdout, not stderr.

Yeah. As said. My fingers were in plaintext mode. No parser active.

I redirected stdout so that grep does not print its findings.
stderr is not redirected so that one has a chance to notice own errors.

Main reason for my post was to explain that "case"-"esac" does not
have the same one-liner syntax as "if"-"fi".
It seems Richard needs to state it as one-liner.


> Do not use -o and -a in a test command.
> http://mywiki.wooledge.org/BashPitfalls#pf6

As if anybody would dare to touch the old "test" command. :))

Rather i avoid "[" brackets, not to allow any implementor an excuse.
But in plaintext mode i wanted to stay near to the given original example.
It worked when i tested in dash and bash.


> The main problem with writing for sh is that nothing works the way
> you expect.

Whenever i am not sure how to state it in shell or whether shell does
it fast enough, i break out the C compiler. (Which still understands
the C examples in Bourne and Kernighan-Ritchie.)


> That is sh.

Awful indeed ... as long as you do not compare it to GUI-only systems.

Shell is good glue code. Not more. Not less.
Dialog leads to scripting smoothly.


Have a nice day :)

Thomas

Re: Substring search in dash [ _NOT_ bash ] shell

[Greg Wooledge]

On Tue, Sep 27, 2016 at 10:07:14AM +0200, Thomas Schmitt wrote:
> if you cannot find a dash tutorial then get a tutorial for bash or sh and 
> test in dash whether the proposals apply properly.

This is potentially bad advice, for several reasons.

First, there are more bad tutorials out there in the wild than good
tutorials, by at least one order of magnitude.  The chances of randomly
stumbling upon one of the good ones are low.

Second, trying to work backwards from a bash tutorial to a POSIX sh/dash
programming style is going to be maddening.  If you're writing for sh,
definitely read documents aimed at sh, rather than some other shell.

Third, even dash has a few extensions to the POSIX shell syntax.
Which, if you're writing specifically for *dash* instead of POSIX sh
in general, is OK... but then you're tying yourself to one specific
shell which isn't in widespread use outside of Debian/Ubuntu.

> Bash and dash both stem from S.R.Bourne's sh. The shell chapters of
> his book "The Unix System" from 1983 still apply.

I have not read this book, but I have a copy of Kernighan & Pike's "The
Unix Programming Environment".  While it's a pretty good book for learning
the concepts, the shell script examples in it are atrociously BAD.
Adopting them as a style guide would be a horrible idea.  They're full
of all of the old shell scripting bugs and assumptions that make programs
fail in a real-life environment, where filenames can contain spaces, etc.

Essentially, every shell script or tutorial written before the year 2000
(and >= 95% of the ones written *after* that) is pure rubbish.

Want to know how bad it is?  Install the Debian package "manpages-posix"
and then read "man 1p sh" -- the SH(P) manual page for the POSIX shell,
written by the *people who made POSIX* (an older version) in 2003.

Search for the word EXAMPLE (all caps) and then scroll up to the example
right before it.  Here is it, reproduced as plain text:

  #
  # Installation time script to install correct POSIX shell pathname
  #
  # Get list of paths to check
  #
  Sifs=$IFS
  IFS=:
  set $(getconf PATH)
  IFS=$Sifs
  #
  # Check each path for 'sh'
  #
  for i in $@
  do
  if [ -f ${i}/sh ];
  then
  Pshell=${i}/sh
  fi
  done
  #
  # This is the list of scripts to update. They should be of the
  # form '${name}.source' and will be transformed to '${name}'.
  # Each script should begin:
  #
  # !INSTALLSHELLPATH -p
  #
  scripts="a b c"
  #
  # Transform each script
  #
  for i in ${scripts}
  do
  sed -e "s|INSTALLSHELLPATH|${Pshell}|" < ${i}.source > ${i}
  done

It is terrifying!  It's every single BAD shell practice all thrown together
into a single example!

1) Attempting to save IFS in a variable and restore it.  This fails if IFS
   is originally unset.

2) Unquoted $(command) substitution, relying on IFS word splitting but
   forgetting to disable globbing.

3) for i in $@
   instead of: for i in "$@"
   or simply: for i

   There is NO excuse for this sloppiness.  None.  Let alone from people
   writing an official POSIX manual!

4) Unquoted ${i} expansions all over the place.  Putting in useless curly
   braces is NOT a substitute for quoting.

5) Inconsistent use of useless curly braces.  If you're going to use them
   around i and Pshell, why omit them around IFS, Sifs and @?  (Of course,
   I would just omit *all* of them.)

6) Storing a space-delimited list in a string variable (scripts="...")
   and then using it as a pseudo-array, again without disabling globbing.
   What makes this one *especially* bad is that the example already used
   the @ pseudo-array earlier.  The author *knows* how to use @ instead
   of a string variable, but didn't do it!

THIS is what we're fighting against.  40 years of this.

So, in short, avoid the older books, manuals, tutorials and examples.
And be cautious even of the newer ones.

> ---

> The "test" expression used is "A = B". There are operators like "-o" for
> logical "or". "A -o B" is true if a is true, or if be is true, or both are
> true.

Do not use -o and -a in a test command.  It's not portable.  Instead,
use two separate test commands with || or && between them:

if [ A ] || [ B ]; then
 ...
fi

See http://mywiki.wooledge.org/BashPitfalls#pf6

> For substring search you would have to employ a program like "grep".
> It returns 0 if something was found, else it returns 1.
> 
>   if hostname | grep 'bob' >/dev/null ; then echo bob.cfg ; fi

grep -q 'bob' is slightly more efficient than grep 'bob' 

Re: Substring search in dash [ _NOT_ bash ] shell

[Thomas Schmitt]

Hi,

for a moment, my fingers were faster than my brain.

I wrote:
> The "test" expression used is "A = B". There are operators like "-o" for
> logical "or". "A -o B" is true if a is true, or if be is true, or both are
> true.

Translation via brain:

The "test" expression used is
  "`hostname`" = bob

There are operators like "-o" for logical "or":

  "`hostname`" = bob -o "`hostname`" = mary -o "`hostname`" = sam

I.e. "test" will return 0 if one of the "=" expressions evaluates
to true.


Have a nice day :)

Thomas

Re: Substring search in dash [ _NOT_ bash ] shell

[Thomas Schmitt]

Hi,

if you cannot find a dash tutorial then get a tutorial for bash or sh and 
test in dash whether the proposals apply properly.
(You can reach dash in dialog by typing "dash" into a bash window.)

Bash and dash both stem from S.R.Bourne's sh. The shell chapters of
his book "The Unix System" from 1983 still apply.

---

As others already stated, the most easy way to look for a substring
is to use the "case" flow-control construct and its capability to
apply search patterns.

One should note the peculiarity of "case ... esac" when it shall be condensed
to a single line. In nearly all shell situations one replaces line break
by a ';' character.
The actually documented dash form

  if [ "`hostname`" = bob ]
  then echo bob.cfg
  fi

may be written as

  if [ "`hostname`" = bob ]; then echo bob.cfg; fi


But with "case" it is different. man dash descibes it by:

  case word in
  [(]pattern) list ;;
  ...
  esac

Now if you do the condensation by the usual ';' characters

  case "`hostname`" in ; bob) echo bob.cfg ;; ; esac

then you earn protest from both shells:

  dash: 1: Syntax error: word unexpected (expecting ")")

  bash: syntax error near unexpected token `;'

You rather have to use in both shells the form from man bash without
extra semicolons.

  case word in [ [(] pattern [ | pattern ] ... ) list ;; ] ... esac

I.e. for strings which contain "bob":

  case "`hostname`" in *bob*) echo bob.cfg ;; esac

Note that a more modern form of `hostname` is $(hostname).
Both execute program hostname and the shell use their output text instead
the `` or $() constructs themselves.

---

To get back to "if": It is less specialized.

Your initial line uses "[" which is an alias of command "test".
"test" returns 0 if it evaluates its arguments as "true", and 1 if "false".
(IT people had a strange humor back in the 80s.)

The "test" expression used is "A = B". There are operators like "-o" for
logical "or". "A -o B" is true if a is true, or if be is true, or both are
true.

For substring search you would have to employ a program like "grep".
It returns 0 if something was found, else it returns 1.

  if hostname | grep 'bob' >/dev/null ; then echo bob.cfg ; fi

grep has a very powerful expression language which is not the same as
the shell patterns. It puts out to stderr what it finds. Here we dump
those messages to /dev/null.
Of course grep has its own fat man page.

---

Have a nice day :)

Thomas

Re: Substring search in dash [ _NOT_ bash ] shell

[David]

On 27 September 2016 at 13:49, Richard Owlett  wrote:
> On 9/26/2016 10:42 AM, Greg Wooledge wrote:
>> On Mon, Sep 26, 2016 at 10:39:59AM -0500, Richard Owlett wrote:
>>>
>>> #  string if [ "`hostname`" = bob ]; then echo bob.cfg; fi
>>>
>>> I need for the case when evaluates to marybobsam .
>>> I can find examples in bash shell, but I need dash.
>>
>> Are you asking how to check for the substring "bob" in your input,
>> in POSIX sh?  Use case, and a glob:
>>
>> case $input in
>>*bob*) echo "Bob found." ;;
>> esac
>>
>> case $(hostname) in
>>*bob*) echo "I must be a Bob." ;;
>> esac
>
> Thank you. I was expecting an if...fi construct.

Shell programming is an arcane topic full of corner cases and traps for
new players. Thus very often the answers to simple questions are far more
complex than we might expect. Consequently you are unlikely to find any
document that spoon feeds you what you need. Diligence and
apprenticeship is the only way here.

And I have to say too, because you are probably unaware ... Greg is a guru.
Consider yourself blessed to be receiving advice from an expert, without
having made much effort :)

> Is dash documented anywhere?

man dash

> All I've found are pages saying that some "bashisms" do not work in dash :<

And the most useful of those is probably this:
http://mywiki.wooledge.org/Bashism

Also you can read this:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html

Asking #bash on freenode.net is another source of excellent guidance.

Re: Substring search in dash [ _NOT_ bash ] shell

[The Wanderer]

On 2016-09-26 at 23:49, Richard Owlett wrote:

> On 9/26/2016 10:42 AM, Greg Wooledge wrote:
>
>> On Mon, Sep 26, 2016 at 10:39:59AM -0500, Richard Owlett wrote:
>>
>>> #  string if [ "`hostname`" = bob ]; then echo bob.cfg; fi
>>>
>>> I need for the case when evaluates to marybobsam .
>>> I can find examples in bash shell, but I need dash.
>>
>> Are you asking how to check for the substring "bob" in your input,
>> in POSIX sh?  Use case, and a glob:
>>
>> case $input in
>>*bob*) echo "Bob found." ;;
>> esac
>>
>> case $(hostname) in
>>*bob*) echo "I must be a Bob." ;;
>> esac
> 
> Thank you. I was expecting an if...fi construct.

Some such thing could be used when checking for an exact string, but for
substring matching, the example given is better.

> Is dash documented anywhere?
> All I've found are pages saying that some "bashisms" do not work 
> in dash :<

The dash manual page ('man dash') seems fairly comprehensive, on brief
observation. It doesn't seem as lengthy or as detailed as the bash man
page (which I _have_ actually read full-length, believe it or not), but
it does seem to cover most things.

-- 
   The Wanderer

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man. -- George Bernard Shaw



signature.asc
Description: OpenPGP digital signature

Re: Substring search in dash [ _NOT_ bash ] shell

[Richard Owlett]

On 9/26/2016 10:42 AM, Greg Wooledge wrote:

On Mon, Sep 26, 2016 at 10:39:59AM -0500, Richard Owlett wrote:

#  string if [ "`hostname`" = bob ]; then echo bob.cfg; fi

I need for the case when evaluates to marybobsam .
I can find examples in bash shell, but I need dash.


Are you asking how to check for the substring "bob" in your input,
in POSIX sh?  Use case, and a glob:

case $input in
   *bob*) echo "Bob found." ;;
esac

case $(hostname) in
   *bob*) echo "I must be a Bob." ;;
esac




Thank you. I was expecting an if...fi construct.
Is dash documented anywhere?
All I've found are pages saying that some "bashisms" do not work 
in dash :<


I've no experience with bash (or any shell) and in any case I'm 
constrained to using dash as I'm working with preseed.cfg .

Re: Substring search in dash [ _NOT_ bash ] shell

[Greg Wooledge]

On Mon, Sep 26, 2016 at 10:39:59AM -0500, Richard Owlett wrote:
> #  string if [ "`hostname`" = bob ]; then echo bob.cfg; fi
> 
> I need for the case when evaluates to marybobsam .
> I can find examples in bash shell, but I need dash.

Are you asking how to check for the substring "bob" in your input,
in POSIX sh?  Use case, and a glob:

case $input in
  *bob*) echo "Bob found." ;;
esac

case $(hostname) in
  *bob*) echo "I must be a Bob." ;;
esac

Substring search in dash [ _NOT_ bash ] shell

[Richard Owlett]

The Installation Manual in _B.5.3. Chainloading preconfiguration 
files_ says in part:
# More flexibly, this runs a shell command and if it outputs the 
names of

# preconfiguration files, includes those files.
#d-i preseed/include_command \
#  string if [ "`hostname`" = bob ]; then echo bob.cfg; fi

I need for the case when evaluates to marybobsam .
I can find examples in bash shell, but I need dash.

Are there equivalents of
  https://www.gnu.org/software/bash/manual/bash.html
or

http://www.tldp.org/LDP/Bash-Beginners-Guide/html/Bash-Beginners-Guide.html
BUT for dash *NOT* bash?

TIA