Re: clarification needed: shell 'exec' + function (builtin, ???)

[Steffen Nurpmeso]

[I bring back austin-group-l, ok?

Thorsten Glaser wrote in
 :
 |Steffen Nurpmeso dixit:
 |
 |>  #include 
 |>  #include 
 |>  #include 
 |>  #include 
 |>  int main(void){
 |> char inb[16], oub[16], *inbp, *oubp;
 |> iconv_t id;
 |> size_t inl, oul;
 |>
 |> memcpy(inbp = inb, "a\303\244c", sizeof("a\303\244c"));
 |> inl = sizeof("a\303\244c") -1;
 |
 |Not -1 otherwise oub will not be NUL-terminated and end with junk:
 |
 |$ ./a.out
 |Converting 4 
 |GOT 

Sure thing.  Just like below.  Normally stack pages are cow forked
from zero if i understand that right.  But maybe i do not.

 |Without the trailing NUL, stateful conversation may also be
 |incomplete…
 |
 |> oul = sizeof oub;
 |> oubp = oub;
 |>
 |> if((id = iconv_open("ascii", "utf8")) == (iconv_t)-1)
 |>   return 1;
 |
 |Throws 1 because you need "utf-8", but with it, see above.

Well names and iconv are a thing.  Especially regarding Unicode
(and nl_langinfo(CODESET), if i remember UnixWare right).

 |> fprintf(stderr, "Converting %lu <%s>\n",(unsigned long)inl, inbp);
 |> if(iconv(id, , , , ) == (size_t)-1){
 |>fprintf(stderr, "Fail <%s>\n", strerror(errno));
 |>return 2;
 |>}
 |> fprintf(stderr, "GOT <%s>\n", oub);
 |> iconv_close(id);
 |> return 0;
 |>}
 |>
 |>you should get replacement characters out of the box?
 |
 |Citrus iconv agrees. Its manpage says:
 |
 | If the string pointed to by *src contains a character which is valid
 | under the source codeset but can not be converted to the destination
 | codeset, the character is replaced by an "invalid character" which
 | depends on the destination codeset, e.g., '?', and the conversion \
 | is con-
 | tinued. iconv() returns the number of such "invalid conversions".

That was my thinking.
Thanks for confirming this.

 --End of 

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: mail encoding not-fun (was Re: clarification needed: shell 'exec' + function (builtin, ???))

[Steffen Nurpmeso]

Thorsten Glaser via austin-group-l at The Open Group wrote in
 :
 |Steffen Nurpmeso via austin-group-l at The Open Group dixit:
 |
 |>|This is because m4.opengroup.org runs qmail, the arsehole under the MTAs,
 |>|which auto-converted the mail from quoted-printable to 8bit, sending it
 |>|as 8bit even to MTAs that don't offer 8BITMIME (I configured my sendmail
 |>|not to do that as well, so I got the same truncated mail back :( other
 |>|than qmail, exim is known to break the MIME and SMTP standards like \
 |>|that).
 |>
 |>Naaah, not true Thorsten.  At least this time.
 |
 |This one *is* correct, as I got the broken message back as well.
 |It contains an embedded NUL.
 |
 |But apparently, this was not the cause of J�rg’s problem ☻

Evil, you.  Hey, i live also on IRC since ~1.5 years for the first
time ever, and on this Linux-Distro (just released 3.6 two days
ago) here is someone active from Düsseldorf, and i now sometimes
listen to La Düsseldorf from La Düsseldorf.  (Of course Neu! and
Lilo Engel are less populist.)  Lots of longing for the 70s here.
Tja.

All he would need to do would be to upgrade to a newer version,
i think i innocently prod him two times on that.

 |>Related to my MUA.
 |[…]
 |>I have been able to save the mail as file and to run iconv(1) on the \
 |>content.
 |
 |oic
 |
 |>Maybe a problem is that the first missing line is a line with a character \
 |>that
 |>is not part of ISO-8859-1
 |
 |Yes, of course, I have been writing in UTF-8 for a while.

Not so here, even though i have ~/.kent.xmodmaprc keycode
adjustments for some German and French quotation marks, i find it
hard to go "random Unicode".  I usually use altgr/g-a in vim and
the look in UnicodeData.txt to find the codepoint ;)

 |[00:02]  gecko: benutzt du emacs ?
 |[00:03]  nö  [00:03]  nur n normalen mac

Graphical selection may be a winner here, indeed.

 |[00:04]  argl   [00:04]  ne den editor
 | -- Vutral und gecko2 in #deutsch (NB: Editor? Betriebssystem.)
 --End of 

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: clarification needed: shell 'exec' + function (builtin, ???)

[Steffen Nurpmeso]

Hallo Jörg, all,

Joerg Schilling wrote in
 <20201210004945.i3n8e%sch...@schily.net>:
 |Steffen Nurpmeso  wrote:
 |> this is an iconv(3)-related error that was fixed in later version
 |> of the mailer you use.  The very error came up on the ML this
 |> year[1], basically you use LATIN1 on your box, as could be
 |> expected, but Thorsten is known to be a Unicode character
 |> "junkie", so to say.
 |
 |You are correct,

Yep -- unfortunately.

 |I have been able to save the mail as file and to run iconv(1) on the \
 |content.

Yes, we temporarily did not restart for ILSEQ, if your prompt
would include "set prompt='\${^ERRNAME}', for example, you would
have seen that an error happened.
But of course we are tolerant for weird base64, so we should be
tolerant for weird iconv, thus i "restored the original
behavirour", so to say.

That reminds me of iconv weirdness regarding hard-to-test
replacement characters, which makes testing really hard.  Wasn't
there an issue on that going on, being able to specify it
explicitly, and whether it stands for an entire character or for
by-byte sequences would be a great improvement.

While talking about iconv, i got closed glibc bug[1] as "resolved
invalid", but wouldn't you all agree that in the following

  #include  
  #include 
  #include 
  #include 
  int main(void){
 char inb[16], oub[16], *inbp, *oubp;
 iconv_t id;
 size_t inl, oul;

 memcpy(inbp = inb, "a\303\244c", sizeof("a\303\244c"));
 inl = sizeof("a\303\244c") -1;
 oul = sizeof oub;
 oubp = oub;

 if((id = iconv_open("ascii", "utf8")) == (iconv_t)-1)
   return 1;
 fprintf(stderr, "Converting %lu <%s>\n",(unsigned long)inl, inbp);
 if(iconv(id, , , , ) == (size_t)-1){
fprintf(stderr, "Fail <%s>\n", strerror(errno));
return 2;
 }  
 fprintf(stderr, "GOT <%s>\n", oub);
 iconv_close(id);
 return 0;
  }

you should get replacement characters out of the box?
I said by then

   $ /tmp/zt
   Converting 4 
   Fail 

  whereas musl gives

   $ ./zt
   Converting 4 
   GOT 

and i still think musl is totally right (also by giving only one
replacement character.

  [1] https://sourceware.org/bugzilla/show_bug.cgi?id=22908

 |Maybe a problem is that the first missing line is a line with a character \
 |that
 |is not part of ISO-8859-1

Yes, transliteration should possibly be possible.
On the other hand, if i change the above to

   if((id = iconv_open("ascii//TRANSLIT", "utf8")) == (iconv_t)-1)

i get

  Converting 4 
  GOT 

and with

   if((id = iconv_open("ascii//TRANSLIT", "utf8")) == (iconv_t)-1)

we are back at the error.

Ciao,

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: clarification needed: shell 'exec' + function (builtin, ???)

[Steffen Nurpmeso]

Hallo Jörg,

Joerg Schilling wrote in
 <2020120933.yyo5w%sch...@schily.net>:
 |"shwaresyst via austin-group-l at The Open Group"  wrote:
 ...
 |> Hi *,
 |
 |Hi,
 |
 |here is where the original mail ended for me. Interesting that you did get
 |more content. Is there any idea, why I received only the first line \
 |from the
 |original mail?

this is an iconv(3)-related error that was fixed in later version
of the mailer you use.  The very error came up on the ML this
year[1], basically you use LATIN1 on your box, as could be
expected, but Thorsten is known to be a Unicode character
"junkie", so to say.

commit a9ec20d6
    Author: Steffen Nurpmeso 
AuthorDate: 2020-04-23 17:29:57 +0200

Fix: "revert" [ab0cd3b8] from 2017-10-20.. (Claus Assmann)..

(FIX iconv for main body part (since EVER!) (Doug McIlroy,
Random832)..) was nonsense in sofar as we now generated ILSEQ
errors, but without giving any error message, so that users
(without nice *prompt*) normally had no indication of what
happened, but looked at a partial message.  This is inacceptible,
so instead simply use replacement until we possibly have a better
way out at some later time.

  This changeset is in v14.9.19.

  [1] https://www.mail-archive.com/s-mailx@lists.sdaoden.eu/msg01013.html

P.S.: i will release v14.9.20 before Christian Christmas, and that
should just work neatly on SchilliX, too, including readily
prepared catman manual (to be downloaded as an extra).

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: clarification needed: shell 'exec' + function (builtin, ???)

[Steffen Nurpmeso]

austin-grou...@opengroup.org wrote in
 :
 |Joerg Schilling via austin-group-l at The Open Group dixit:
 |
 |>here is where the original mail ended for me. Interesting that you did get
 |
 |This is because m4.opengroup.org runs qmail, the arsehole under the MTAs,
 |which auto-converted the mail from quoted-printable to 8bit, sending it
 |as 8bit even to MTAs that don't offer 8BITMIME (I configured my sendmail
 |not to do that as well, so I got the same truncated mail back :( other
 |than qmail, exim is known to break the MIME and SMTP standards like that).

Naaah, not true Thorsten.  At least this time.
Related to my MUA.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: An additional quote test for check.t

[Steffen Nurpmeso]

Thorsten Glaser  wrote:
 |>i think, the \r is expanded, the output is
 |
 |a bit unsure what exactly you’re talking about, but you cannot
 |ever use echo to display strings portably.
 |
 |Use 'print -r -- "$foo"' to display the contents of the foo parameter.
 |
 |Is there anything you think mksh fails to do correctly? If so,
 |please indicate _exactly_ where and what and how.

Oh yes, cool, ehm, that is right.  With printf and arguments all
is fine indeed.
Great.
Enough shame for the weekend..
Sorry.
Ciao.

--steffen
|
|Ralph says i must not use signatures which spread the light!

An additional quote test for check.t

[Steffen Nurpmeso]

Hallo Thorsten,

I do not know whether you have followed the thread on the standard
list, but Robert Elz has written a shell function which allows
round-tripping quoted strings from and to the positional parameter
stack, Stephane did so in awk(1), effectively we are talking about

  stephane_quote() {
LC_ALL=C awk -v q="'" -v b='\\' '
  function quote(s){
gsub(q, q b q q, s)
return q s q
  }
  BEGIN{
sep = ""
for(i = 1; i < ARGC; i++){
  printf "%s", sep quote(ARGV[i])
  sep = " "
}
if(sep)
  print ""
  }' "$@"
  }

mksh fails to roundtrip the string

  set -- x 'a \ b' "foo'" "\\'b\\a\\r\\" A
  echo "$#: <${1}><${2}><${3}><$4><$5>"
  saved_parameters=$(robert_quote "$@")
  eval "set -- $saved_parameters"
  echo "$#: <${1}><${2}><${3}><$4><$5>"

i think, the \r is expanded, the output is

  \><\'b

instead of

  5: <\'b\a\r\>

Interested in the attached addition to check.t?

--steffen
|
|Ralph says i must not use signatures which spread the light!
diff --git a/check.t b/check.t
index e0c8d5e..7cc08d6 100644
--- a/check.t
+++ b/check.t
@@ -6835,6 +6835,61 @@ expected-stdout:
 	3 10 .
 	4 -2147483646 .
 ---
+name: quote-roundtrip
+description:
+	Check roundtrip quoting behaviour via Robert Elz quote()
+stdin:
+	robert__quote() {
+	case "$1" in
+	*\'*)   ;;
+	*)  printf "'%s'" "$1"; return 0;;
+	esac
+	__A__="$1" __S__= __E__=
+	while case "$__A__" in
+	\'*)__A__=${__A__#?}; __S__="${__S__}'";;
+	*\')__A__=${__A__%?}; __E__="${__E__}'";;
+	'') printf "${__S__}${__E__}"; return 0;;
+	*)  false;;
+	esac
+	do
+	continue
+	done
+	_save_IFS="${IFS}"; ${IFS+":"} unset _save_IFS
+	_save_OPTS="$(set +o)"
+	IFS=\'
+	set -f
+	set -- $__A__
+	_result_="${1}"
+	shift
+	for __A__
+	do
+	_result_="${_result_}'\\''${__A__}"
+	done
+	printf "${__S__}'%s'${__E__}" "${_result_}"
+	# now clean up
+	IFS="${_save_IFS}"; ${_save_IFS+":"} unset IFS
+	eval "${_save_OPTS}"
+	unset _result_ _save_IFS _save_OPTS __A__ __E__ __S__
+	return 0;
+	}
+	robert_quote() {
+	j=
+	for i
+	do
+	[ -n "$j" ] && printf ' '
+	j=' '
+	robert__quote "$i"
+	done
+	}
+	set -- x 'a \ b' "foo'" "\\'b\\a\\r\\" A
+	echo "$#: <${1}><${2}><${3}><$4><$5>"
+	saved_parameters=$(robert_quote "$@")
+	eval "set -- $saved_parameters"
+	echo "$#: <${1}><${2}><${3}><$4><$5>"
+expected-stdout:
+	5: <\'b\a\r\>
+	5: <\'b\a\r\>
+---
 name: readonly-0
 description:
 	Ensure readonly is honoured for assignments and unset

Re: alias expansion bug with matching { ... }

[Steffen Nurpmeso]

Hallo!

Thorsten Glaser  wrote:
 |Dixi quod…
 |>Commit ID:   10058E5A1067AFA1BBA
 ...
 |Oh well. For the sake of consistency. But fixing this DID uncover
 |a re-entrancy bug in 'typeset -f' ouput (which IS also used in some
 |other places internally):
 ...
 |When deferring alias interpretation until the end, we do get:
 ...
 |This is problematic (in the first two) and inconsistent (in the last) case.
 ...
 |To make this work but not get “\./pfn” stupidly, I had to limit the
 |allowed characters in “alias”, but ksh93 does so too, as does POSIX.

Would you mind re-allowing hyphen-minus again?  That really has
become very common in $PATH, and the new ksh_isalias() seems to
omit it.  For example,

  ?0[steffen@wales]$ alias|wc -l
  86
  ?0[steffen@wales]$ alias|grep -F -- -|wc -l
  78
  ?0[steffen@wales]$ ls /usr/bin|wc -l
  1565
  ?0[steffen@wales]$ ls /usr/bin|grep -F -- -|wc -l
  372

I seem to remember that i have opened an Austin Group issue to
include hyphen-minus in the set of allowed characters, that is
[1], … i seem to be too lazy to look into the standard to resolve
the aberrations on notational aspects of the desired change that
have evolved on the tracker, though.  Still, bash etc. support it.
In the other window i updated the desired action.

  [1] http://austingroupbugs.net/view.php?id=1050

Ciao!

--steffen

Re: trap recursion until SIGSEGV (non-interactive)

[Steffen Nurpmeso]

Thorsten Glaser t...@mirbsd.de wrote:
 |Steffen Nurpmeso dixit:
 |An interactive shell or '/usr/bin/mksh -c exec ./t.sh' limits
 |the storm in the same way that bash(1) does

 |Running 'bash t.sh' shows…
 |
 |au
 |au
 |au
 |after zap
 |au
 |au
 |au
 |au
 |au
[ au]
 |exit
 |au
 |
 |… weirdly enough, but this is hardly consistent with

..consistency was what i had in mind.

 |the documentation of the trap builtin. And you cannot
 |portably catch stack underruns anyway (which is what
 |happens here, just like if you have a function that
 |recurses infinitely).

I would prefer seeing a stack recursion limit exceeded or
similar before SIGSEGV happens because of recursive functions,
but it is possibly no good if mksh would be different.

--steffen

Re: trap recursion until SIGSEGV (non-interactive)

[Steffen Nurpmeso]

Thorsten Glaser t...@mirbsd.de wrote:
 |Steffen Nurpmeso dixit:
 |I would prefer seeing a stack recursion limit exceeded or
 |similar before SIGSEGV happens because of recursive functions,
 |
 |Yeah, but I have no idea how to do that. If you find one… tell me.

Anyway i'm thankful i was worth an answer for that -- current bash
(i usually only have a stale 3.2) shows the same behaviour and
after searching the web a person that ran into the same problem
with bash didn't get one!
And i would have said the same as someone else i saw in
a different, even older message: use a recursion counter, how
wacky that may be.
But i guess it is not worth any effort, especially given that
No. 1 changed _to_ this behaviour (even completely, exec and even
interactive [but via subshell]!) and that i saw it the first time
because of a bug.
It's just that i am with the one that got no answer in that it
didn't feel right.  Just forget about it.

--steffen

trap recursion until SIGSEGV (non-interactive)

[Steffen Nurpmeso]

Hello Thorsten,

i wrote a buggy shell script similar to the crap below, but it
crashed mksh(1):

  cat  t.sh _EOT
  trap echo au; kill $$ TERM
  trap echo exit EXIT
  zap=`kill -TERM $$`
  echo after zap
  exit
  _EOT

An interactive shell or '/usr/bin/mksh -c exec ./t.sh' limits
the storm in the same way that bash(1) does, but running the
script directly will loop until crash.  (dash(1)s HEAD behaves the
very same btw.)

--steffen

Bum behaviour of hash(1)

[Steffen Nurpmeso]

Hello,

just stumbled over a makefile which does

  RANLIB=(hash ranlib) /dev/null 21 || exit 0; ranlib

and had a problem:

  ?0[]$ for i in ba da k mk z; do
 echo --${i}sh:;
 eval ${i}sh -c '(hash nosuch)||echo ok; (hash vim)||echo bad';
   done 
  --bash:
  bash: line 0: hash: nosuch: not found
  ok
  --dash:
  dash: 1: hash: nosuch: not found
  ok
  --ksh:
  --mksh:
  --zsh:
  zsh:hash:1: no such command: nosuch
  ok

And also (packed in shell archives, grmpf)

  ?1[]$ bourne.sh + -c '(hash vim)||echo au'
  ?0[]$ bourne.sh + -c '(hash vims)||echo au'
  /var/folders/Da/DaZX3-k5G8a57zw6MSmjJTM/-Tmp-//.bournesh/sh: vims: not 
found
  au
  ?0[]$ schily.sh + -c '(hash vim)||echo au'
  ?0[]$ schily.sh + -c '(hash vims)||echo au'
  /var/folders/Da/DaZX3-k5G8a57zw6MSmjJTM/-Tmp-//.schily.sh/sh: vims: not 
found
  au

Ciao,

--steffen

Re: Bum behaviour of hash(1)

[Steffen Nurpmeso]

Thorsten Glaser t...@mirbsd.de wrote:
 |Steffen Nurpmeso dixit:
 |
 |just stumbled over a makefile which does
 |
 |Fix that Makefile.

Your advice comes too late.
Sorry for that.
Ciao,

--steffen