Re: print -R compatibility

[Thorsten Glaser]

Hi Jean,

>Sorry for the late reply, I have been away from home due to unexpected
>family issues.

no problem.

>for -e must be fairly limited. As I understand it, the only purpose of
>the option is to cancel a previous -r, which is kind of a corner case.

The ksh93 manpage documents it as such, yes.

>Fair enough. I hope this minor incompatibility won't be a problem for
>out customer.

OK. Handling _that_ would be maximally tricky (escape from the getopts
framework, etc).

>> This will be in the next mksh release.
>
>Thank you very much. I have backported your work to mksh R50 as this is
>what our product includes, and will send to our customer shortly for
>testing.

OK, wonderful, thanks!

>Now that the print command supports option -e and it is documented,
>it should also appear in the command synopsis.

Good catch! I merged it, and while there, fixed another small bug.

bye,
//mirabilos
-- 
I believe no one can invent an algorithm. One just happens to hit upon it
when God enlightens him. Or only God invents algorithms, we merely copy them.
If you don't believe in God, just consider God as Nature if you won't deny
existence.  -- Coywolf Qi Hunt

Re: print -R compatibility

[Jean Delvare]

Hi Thorsten,

Sorry for the late reply, I have been away from home due to unexpected
family issues.

On ven., 2017-02-17 at 22:37 +, Thorsten Glaser wrote:
> Jean Delvare dixit:
> >Our previous products included ksh93.
> 
> OK. (Found out as well that ksh93 has print -e, which we had code
> for but no member in the getopt call…)

Good catch. As ‘\’ sequences are processed by default, the use cases
for -e must be fairly limited. As I understand it, the only purpose of
the option is to cancel a previous -r, which is kind of a corner case.
That must be why nobody noticed so far. But for compatibility with
other ksh implementations, I agree it is better to support it, so
thanks for having fixed it.

> (...)
> I looked at ksh93 and investigated several approaches and ended
> up codifying “as soon as -R is encountered, we jump into the BSD
> echo (formerly called Debian echo) codepath”.
> 
> This matches ksh93 in that things like a previous -n, -uN, -s, …
> are *not* reset:
> 
> tg@blau:/usr/src/bin/mksh $ obj/mksh -c 'print -u2 -R -42' >/dev/null
> -42
> 
> This allows your use cases:
> 
> tg@blau:/usr/src/bin/mksh $ obj/mksh -c 'print -R -n -4; print -R 2'
> -42

Looks good.

> Parsing after -R like ksh93 does is not supported:
> 
> tg@blau:/usr/src/bin/mksh $ ksh93 -c 'print -Rn foo; print bar'
> foobar
> tg@blau:/usr/src/bin/mksh $ obj/mksh -c 'print -Rn foo; print bar'
> foo
> bar
> 
> But:
> 
> tg@blau:/usr/src/bin/mksh $ obj/mksh -c 'print -R -n foo; print bar'
> foobar

Fair enough. I hope this minor incompatibility won't be a problem for
out customer.

> This will be in the next mksh release.

Thank you very much. I have backported your work to mksh R50 as this is
what our product includes, and will send to our customer shortly for
testing.

-- 
Jean Delvare
SUSE L3 Support

Re: print -R compatibility

[Thorsten Glaser]

Jean Delvare dixit:

>So, what is the next step?

Me getting any spare time…

Sorry,
//mirabilos, currently under high pressure for two customer projects
-- 
Stéphane, I actually don’t block Googlemail, they’re just too utterly
stupid to successfully deliver to me (or anyone else using Greylisting
and not whitelisting their ranges). Same for a few other providers such
as Hotmail. Some spammers (Yahoo) I do block.

Re: print -R compatibility

[Jean Delvare]

On Fri, 10 Feb 2017 13:10:20 + (UTC), Thorsten Glaser wrote:
> That’s ksh88 where you(r customer) come(s) from?

Our previous products included ksh93.

> >> We have the echo you mean in POSIX mode though…
> >
> >Not sure what you mean here. I see that the echo command behavior is
> >changed if Flag(FPOSIX), but I can't seem to be able to set that flag
> >(I tried "export POSIX=1" but that doesn't seem to change anything?)
> 
> “set -o posix” ;-)

Got it, thanks.

> >(...)
> >For reference, here is the patch I came up with. The idea is to jump to
> >the "echo" command handling code a soon as we see -R. The rest of the
> >changes is to remove po.pminusminus as it is no longer needed. Known
> 
> That’s about what I’d have done as well (unless the BSD echo
> behaves different from classical echo in which 'print -R -ex'
> would be the same as 'print -r -- -ex'; the alternative would
> be to parse the -e from -ex and only then output it, turning
> it into 'print -- -ex'). This is why some investigation is
> likely still needed.

I don't think it makes sense to consider one half of a parameter as an
option and the other half as a non-option. And definitely not to handle
a given letter as an option and then still printing it... "-e" is not
necessarily relevant as ksh93's print command doesn't support it in the
first place, but I tested "-n":

$ print -R -nx
-nx
$

So it only treats -n as an option if not mixed with unsupported option
letters. That is exactly what mksh does for the echo command, and I
think we should stick to that for consistency.

> If I take your patch, do you wish to have your name added to
> the list of copyright holders at the top of the file?

No need, this is only a minor contribution.

> >caveat: -E would be handled as a valid option, while it was not
> 
> True, but that can be circumvented.

Sure it can. My patch was more of a proof-of-concept (one of many, I
tried different approaches before) and details can be discussed.

Thanks for your time,
-- 
Jean Delvare
SUSE L3 Support

Re: print -R compatibility

[Thorsten Glaser]

Jean Delvare dixit:

>I din't think -R currently emulates either properly (but I suppose I
>can't complain, as "to some degree" could be read as implying just
>that.) BSD echo will print arguments starting with a dash, other than a

Hmm, okay. In that case (I’ll recheck with legacy BSD sources though)
I’ll likely change the behaviour.

>leading -n. So if "print -R -42" is supposed to be like BSD "echo -42",
>it should print "-42", as is the case with legacy ksh.

That’s ksh88 where you(r customer) come(s) from?

>> We have the echo you mean in POSIX mode though…
>
>Not sure what you mean here. I see that the echo command behavior is
>changed if Flag(FPOSIX), but I can't seem to be able to set that flag
>(I tried "export POSIX=1" but that doesn't seem to change anything?)

“set -o posix” ;-)

>Also that would only change the behavior of echo anyway, not print.

Yes, of course.

>Actually I already suggested to the customer that they could replace
>all the instances of "print -R" by "print -r --" in their scripts. They
>did not answer yet.

If it’s that easy…

>I believe "print -r --" is a better choice in general regardless of the

(yes, it is)

>outcome of our discussion, as -R will treat -n and -e (and combinations
>thereof) as options, and the user may not expect that.

… depends on what the user wishes, but if they really only want to
echo out arbitrary strings, “print -r -- "something"” is the way to go.

>I agree, and I would go that way if it was my code (straight to step 2,
>even.) But it's not, and my experience is that customers are often
>reluctant to change scripts that have been running for a long time and
>are still deployed on many, sometimes heterogeneous, systems. I can
>make suggestions but they get to make decisions.

I *completely* understand, having a dayjob in IT as well.

>I did suggest to them to just use "echo" at one point and they turned
>it down exactly because of portability concerns. Their script is
>running on different OS flavors where echo behaves differently. They

True.

>> Hrm. Funnily enough:
>>
>> tglase@tglase-nb:~ $ print -R -- -42
>> -- -42
>> tglase@tglase-nb:~ $ print -R -x -42
>> /bin/mksh: print: -x: unknown option
>
>I did notice this inconsistency too as part of my investigation, and
>that's one of the reasons why I'm not sure the current behavior is
>really by design.

I noticed that mostly after writing the rest of the mail, which is why…

>> Perhaps I’ll have to investigate “the BSD echo(1) command”
>> further (no problem thanks to TUHS) and maybe the implementation
>> is indeed wrong… but the result will likely end up in a formal
>> mksh release too late for your customer anyway. I’ll make a note.

I wrote that.

>That would not be a problem. If I know for sure that such a change will
>make it into a future release of mksh, I can backport the change
>immediately to whatever version our customers are running.

OK.

>For reference, here is the patch I came up with. The idea is to jump to
>the "echo" command handling code a soon as we see -R. The rest of the
>changes is to remove po.pminusminus as it is no longer needed. Known

That’s about what I’d have done as well (unless the BSD echo
behaves different from classical echo in which 'print -R -ex'
would be the same as 'print -r -- -ex'; the alternative would
be to parse the -e from -ex and only then output it, turning
it into 'print -- -ex'). This is why some investigation is
likely still needed.

If I take your patch, do you wish to have your name added to
the list of copyright holders at the top of the file?

>caveat: -E would be handled as a valid option, while it was not

True, but that can be circumvented.

>supported by print -R before. Actually legacy ksh only supports -n
>after -R (as BSD echo does), not -n nor -E.

Which one are you coming from, again? ;)

bye,
//mirabilos
-- 
13:22⎜«neurodamage» mira, what's up man? I have a CVS question for you in #cvs
13:22⎜«neurodamage» since you're so good w. it │ «neurodamage:#cvs» i love you
13:28⎜«neurodamage:#cvs» you're a handy guy to have around for systems stuff ☺
16:06⎜ Thank god I found you =)   20:03│«bioe007:#cvs» mira2k: ty
17:14⎜ Thanks big help you are :-)mira|nwt: ty again

Re: print -R compatibility

[Jean Delvare]

Hi Thorsten,

Thanks for your quick answer.

On jeu., 2017-02-09 at 18:46 +, Thorsten Glaser wrote:
> Jean Delvare dixit:
> >In mksh, "print -R" is not as close to "echo" as it was in original
> >ksh. Specifically, after "-R", mksh's print command keeps parsing
> >the command line in search of options, and stops as soon as it
> >finds an
> 
> Indeed, this is as documented:
> 
>     The -R option is used to emulate, to some degree, the BSD echo(1)
>     command which does not process ‘\’ sequences unless the -e option
>     is given.  As above, the -n option suppresses the trailing new‐
>     line.
> 
> The -R option does _not_ emulate the echo you mean but another.

I din't think -R currently emulates either properly (but I suppose I
can't complain, as "to some degree" could be read as implying just
that.) BSD echo will print arguments starting with a dash, other than a
leading -n. So if "print -R -42" is supposed to be like BSD "echo -42",
it should print "-42", as is the case with legacy ksh.

> We have the echo you mean in POSIX mode though…

Not sure what you mean here. I see that the echo command behavior is
changed if Flag(FPOSIX), but I can't seem to be able to set that flag
(I tried "export POSIX=1" but that doesn't seem to change anything?)

Also that would only change the behavior of echo anyway, not print.

> >Example with the original ksh:
> >
> >$ print -R -42
> >-42
> 
> Quick workaround (this will lose you support for -n though):
> 
> function print {
> if [[ $1 = -R ]]; then
> shift
> builtin print -r -- "$@"
> else
> builtin print "$@"
> fi
> }

Actually I already suggested to the customer that they could replace
all the instances of "print -R" by "print -r --" in their scripts. They
did not answer yet.

I believe "print -r --" is a better choice in general regardless of the
outcome of our discussion, as -R will treat -n and -e (and combinations
thereof) as options, and the user may not expect that.

> >I looked at the mksh code and was able to modify the print_c function
> >to get mksh's print command to behave the same with -R as echo and the
> >original ksh print with -R. I have a patch ready. Are you interested in
> >it, or is the different behavior on purpose?
> 
> While I think you’re the first user of “print -R”, this looks as if it
> was intended — I’d address this on the customer side, maybe in a
> two-step process:
> 
> 1. Check all uses of 'print.*-[^ ]R' for which syntax is used, e.g.
>    if we need to handle “print -R -n”, “print -Rn”, “print -nR” or
>    somesuch; adjust the abovementioned print function, use it
> 
> 2. Convert all uses of print’s -R option to 'print -r --' or
>    'print -nr --' in the scripts (yes, more effort, but worth it).

I agree, and I would go that way if it was my code (straight to step 2,
even.) But it's not, and my experience is that customers are often
reluctant to change scripts that have been running for a long time and
are still deployed on many, sometimes heterogeneous, systems. I can
make suggestions but they get to make decisions.

> I’d advocate against trying to do something with echo that even
> pretends portability; we have no less than three different echo
> implementations in mksh alone (and which is chosen depends on
> several factors).

I did suggest to them to just use "echo" at one point and they turned
it down exactly because of portability concerns. Their script is
running on different OS flavors where echo behaves differently. They
are using "print -R" instead precisely because that appeared to be
portable across all their systems. Until they started migrating to
mksh, that is.

> Nevertheless, thank you for mailing about this!
> 
> 
> Hrm. Funnily enough:
> 
> tglase@tglase-nb:~ $ print -R -- -42
> -- -42
> tglase@tglase-nb:~ $ print -R -x -42
> /bin/mksh: print: -x: unknown option

I did notice this inconsistency too as part of my investigation, and
that's one of the reasons why I'm not sure the current behavior is
really by design.

> Perhaps I’ll have to investigate “the BSD echo(1) command”
> further (no problem thanks to TUHS) and maybe the implementation
> is indeed wrong… but the result will likely end up in a formal
> mksh release too late for your customer anyway. I’ll make a note.

That would not be a problem. If I know for sure that such a change will
make it into a future release of mksh, I can backport the change
immediately to whatever version our customers are running.

For reference, here is the patch I came up with. The idea is to jump to
the "echo" command handling code a soon as we see -R. The rest of the
changes is to remove po.pminusminus as it is no longer needed. Known
caveat: -E would be handled as a valid option, while it was not
supported by print -R before. Actually legacy ksh only supports -n
after -R (as BSD echo does), not -n nor -E.

Subject: print_c: Make option -R behave more like echo

In original ksh, print -R 

Re: print -R compatibility

[Thorsten Glaser]

Jean Delvare dixit:

>Hello mksh developer,

Hi!

>In mksh, "print -R" is not as close to "echo" as it was in original
>ksh. Specifically, after "-R", mksh's print command keeps parsing the
>command line in search of options, and stops as soon as it finds an

Indeed, this is as documented:

The -R option is used to emulate, to some degree, the BSD echo(1)
command which does not process ‘\’ sequences unless the -e option
is given.  As above, the -n option suppresses the trailing new‐
line.

The -R option does _not_ emulate the echo you mean but another.
We have the echo you mean in POSIX mode though…

>Example with the original ksh:
>
>$ print -R -42
>-42

Quick workaround (this will lose you support for -n though):

function print {
if [[ $1 = -R ]]; then
shift
builtin print -r -- "$@"
else
builtin print "$@"
fi
}

>I looked at the mksh code and was able to modify the print_c function
>to get mksh's print command to behave the same with -R as echo and the
>original ksh print with -R. I have a patch ready. Are you interested in
>it, or is the different behavior on purpose?

While I think you’re the first user of “print -R”, this looks as if it
was intended — I’d address this on the customer side, maybe in a
two-step process:

1. Check all uses of 'print.*-[^ ]R' for which syntax is used, e.g.
   if we need to handle “print -R -n”, “print -Rn”, “print -nR” or
   somesuch; adjust the abovementioned print function, use it

2. Convert all uses of print’s -R option to 'print -r --' or
   'print -nr --' in the scripts (yes, more effort, but worth it).

I’d advocate against trying to do something with echo that even
pretends portability; we have no less than three different echo
implementations in mksh alone (and which is chosen depends on
several factors).

Nevertheless, thank you for mailing about this!


Hrm. Funnily enough:

tglase@tglase-nb:~ $ print -R -- -42
-- -42
tglase@tglase-nb:~ $ print -R -x -42
/bin/mksh: print: -x: unknown option

Perhaps I’ll have to investigate “the BSD echo(1) command”
further (no problem thanks to TUHS) and maybe the implementation
is indeed wrong… but the result will likely end up in a formal
mksh release too late for your customer anyway. I’ll make a note.

bye,
//mirabilos
-- 
“It is inappropriate to require that a time represented as
 seconds since the Epoch precisely represent the number of
 seconds between the referenced time and the Epoch.”
-- IEEE Std 1003.1b-1993 (POSIX) Section B.2.2.2