Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Geoff Clare via austin-group-l at The Open Group]

Austin Group Bug Tracker wrote, on 07 Oct 2022:
>
> -- 
>  (0005990) geoffclare (manager) - 2022-10-07 15:56
>  https://austingroupbugs.net/view.php?id=267#c5990 
> -- 
> Suggested changes ...

[...]

> [...] to:When
> time is used in any of the following circumstances, via a simple
> command for which the word time is the command name (see [xref to
> 2.9.1.1]), and none of the characters in the word time is quoted,
> the results (including parsing of later words) are unspecified: 
> The simple command for which the word time is the command name
> includes one or more redirections (see [xref to 2.7]) or is (directly) part
> of a pipeline (see [xref to 2.9.2]).
> The next word that follows time would, if the word time
> were not present, be recognized as a reserved word (see [xref to 2.4]) or a
> control operator (see [xref to XBD 3.83]).
> The utility operand names a special built-in utility (see [xref
> to 2.14]), an intrinsic utility (see [xref to 1.7]), or a function (see
> [xref to 2.9.5]). 

After posting this I realised that this is the wrong place for the
last bullet item, as it should always apply, not just under the
conditions stated here. So I have edited the note to put this as
replacement text for the current similar text in OPERANDS (instead
of deleting that text).

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Fri, 6 Mar 2020 07:21:29 +
From:Austin Group Bug Tracker 
Message-ID:  

  | A NOTE has been added to this issue. 

  | A few explanations related to ...

Ugh.   I hate the mantis web interface!That note escaped from me
(twice) before I was finished with it, hence it was later edited (twice).

Ignore the e-mail about the (this) note, what's there is incomplete nonsense
instead look at what is on mantis now, alternatively, this is what that
note ended up saying, after editing (just to finish content, there might
still be typos).

kre

A few explanations related to Note: 0004794

A few of the changes are just intended to clarify what is stated now in
the way I feel it should be stated to avoid potential problems, eg: where
XCU 2.4 says:

  Words that are the concatenation of a name and a  (':')
  are reserved; their use produces unspecified results.

it doesn't say that this applies only where the word appears in a
place where a reserverd word might be recognised, so for example,
taken literally this would mean

  echo Usage: whatever

(ignoring whatever effect my use of echo might have here) would have
unspecified results, because Usage: is a word that is the concatenation
of a name and a colon. That kind of thing should be fixed whatever else
happens, and as making time back being an unspecified reserved word means
changes all around that area (and ideally we need a method to introduce new
reserved words, without stepping on the planned label namespace) this seemed
like a good opportunity to fix this kind of wording.

Apart from that the intent is to make unspecified all of the cases where
time being a reserved word, and time being a utility are different, while
leaving the simple case
  time cmd [arg...]
as meaning the same as it does where the time utility is invoked, so that
usage remains safe to use (but only in this very simplest of cases).
That will make things like attempting to define time as a function 
non-conformant as reserved words cannot be redefined that way, while
leaving it open for any implementation that treats time as a reserved word
to largely do as it wants (within the confines of what reserved words can do).

We can work on an entirely new method for timing shell operations (including
utility invocations) later - for issue 9 or something...

RE: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Schwarz, Konrad]

> From: Nick Stoughton  
> Sent: Tuesday, February 25, 2020 00:27
> To: Chris F.A. Johnson  Cc: austin-group-l@opengroup.org
> Subject: Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)
> 
> If we are bicycle-shedding around a name for a new utility,
> I like the idea of using the currently reserved namespace of appending a 
> colon ... "time: "
> 
> Since this is already reserved it shouldn't break either existing shells or 
> utilities.
> I know of no shells that implement a goto (which is why it was reserved in 
> the first place).
> 
> XRAT states "The restriction on ending a name with a  is to allow 
> future implementations that support
> named labels for flow control; see the RATIONALE for the break built-in 
> utility.", and the mentioned break rationale provides an
> example, but I have not heard of any shell that uses it.

I would prefer to retain the option of labeled loops and breaks.

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Wed, 26 Feb 2020 03:10:31 + (UTC)
From:shwaresyst 
Message-ID:  <1721701622.609847.1582686631...@mail.yahoo.com>

  | No keyword, on its own, presently in the standard is required to produce ou=
  | tput that may be redirected.

That is certainly correct.   That doesn't mean that a new keyword couldn't
produce output, which is what time does, and if it did, its output would
logically get redirected along with that of the commands it runs (which would
be then subjected to their own more specific redirections, as the fantasy
examples I gave showed).   An alternative to that would be to have an
"output" word in such a design (since we only need to redirect output from
time, it has no input) or to allow its redirections to occur in the words
section - but this version then precludes use of its output into a pipe,
unless we also added a "pipeout" word - and that's going to absurd lengths.

The problem the current "time" reserved word has in this area is that its
syntax is (approximately)

time command

and command can have redirections anywhere, and contain just about anything,
so there is nowhere except possibly before the "time" word where redirections
could be placed which would apply to the output from time itself - and before
a reserved word is the one place that redirections cannot go, so that is out
as well.   This means the only way left is to embed the time command in a
group, and redirect the group instead, which is simply different (and more
limited) syntax than the fantasy examples I gave.

  | What redirections are permitted apply to utilities used with those
  | keywords, not the keywords themselves, and this is where the conflict
  | appears - does a stderr redirect with time as a keyword apply only to
  | the output of time, which is desired but not implemented;

I don't know who would desire that, but it wouldn't be me.  That would be
a truly horrible result.

  | only whatever command(s) are being processed, as is currently implemented;
  | or both as with the utility.

One of those is what it has to be, and the former is more flexible, if ugly.

But once again, I believe you're wasting time even considering any of this.

 [Incidentally, in my previous message, where I wrote ...

   More likely the real problem here is how to define the grammar for time as
   a reserved word to make it match the actual implementations correctly.
   That I haven't considered, as I don't think it should even be invented.

 the final word was meant to be "considered" - I guess my brain was already
 planning the next paragraph, which was using "invent" and my fingers got
 confused...]

When this was originally discussued, I assume the standards writers of the
time (no pun...) looked at "time", the reserved word, saw how badly designed
it was, along with the conflicts with "time". the utility, and simply could
not bring themselves to inflict that nonsense on anyone.   They should be
applauded for that.  We should not be going anywhere near changing that,
all we need to do is to make the existing implementations of the time
reserved word no longer be non-conformant with the standard (which is
what they clearly desired to achieve ... we should simply finish that job).

That's it.   The only hard part of this (aside from writing the correct
standards speak and formatting it properly, which I find mind blowing)
is to make sure we identify all the places where there are currently
problems, so we can fix them - which means - make it unspecified.

So, please, everyone, cease all efforts on trying to find ways to specify
the time reserved word, and instead use your considerable talents in various
areas to look for any other obscure cases where using time as a reserved word
currently violates the standard in any way.

  | What I proposed could be grafted onto existing utility implementations,
  | perhaps, but making it robust is much harder then, that I see, as you
  | have to guard against date being used to modify the clock somehow,

I have absolutely no idea what you're thinking of here, but the shell is
also just a utility, and has no special privs wrt clock changes than any
other utility - anything the shell can do anything else can do as well.

But once again, it is simply *impossible* to get anything like the time
reserved word implemented as a utility, even a "must be built in" kind of
utility.   Just stop dreaming about that, it cannot be achieved, or at
least not unless we totally redefine what it means to be a utility in the
shell.

The closest it is possible to come would be to make time like eval,
(as mentioned in a much earlier message of mine) which is given a string
to parse and execute (and in the case of time, also provide timing output).
That would be a must-be-builtin utility (as eval is) and could work ...
but then adds all of the quoting issues that arise with eval to uses of time.
eval's purpose is largely to deal with various different kinds of quoting

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[shwaresyst]

No keyword, on its own, presently in the standard is required to produce output 
that may be redirected. What redirections are permitted apply to utilities used 
with those keywords, not the keywords themselves, and this is where the 
conflict appears - does a stderr redirect with time as a keyword apply only to 
the output of time, which is desired but not implemented; only whatever 
command(s) are being processed, as is currently implemented; or both as with 
the utility.

What I proposed could be grafted onto existing utility implementations, 
perhaps, but making it robust is much harder then, that I see, as you have to 
guard against date being used to modify the clock somehow, for one type of 
implementation. It would be less efficient also, as the shell could not apply 
various optimizations it may have to bypass portions of setting up a utility 
environment for special built-ins. The differences are minor in theory between 
regular and special, I agree, but they do affect how utility implementations 
are designed.

As to getting it invented, roughly 80% of the bash implementation for posix 
mode can be reused, I estimate, and changing from keyword to built-in as a new 
wrapper for that is straightforward, it appears. The backend for the new 
switches, as new code, is not as straightforward but shouldn't have any 
insurmountable difficulties; I'd need to spelunk the codebase more to verify 
that. The point is dumb can be fixed, without sacrificing backwards 
compatibility or needing any new keywords. 
On Tuesday, February 25, 2020 Robert Elz  wrote:
    Date:        Tue, 25 Feb 2020 17:47:23 + (UTC)
    From:        shwaresyst 
    Message-ID:  <1701159045.321204.1582652843...@mail.yahoo.com>

  | The thing is, various shells have implemented time as keyword, so this bug
  | is trying to get the standard to reflect actual practice that ignored that
  | Rationale.

The first part of that I understand, and what's more, that was predicted, even
known, at the time, as the implementations (at least one) already existed.

Further it was intended to be permitted - that much is clear, they just
didn't realise all they needed to make unspecified in order for that to work.

  | The debate is more some things allowed to utilities aren't allowed
  | for keywords, and vice versa, so there are advantages to both.

Then that's a pretty silly debate, as absolutely anything can be allowed
for a keyword - part of defining a keyword (which is a grammar token) is
defining the grammar rules that use it, and those can be just about anything
that's desired, in any form that's desired.  It just needs to be consistent
(ie: not cause a conflict, I don't mean be similar to) with the rest of the
grammar.

More likely the real problem here is how to define the grammar for time as
a reserved word to make it match the actual implementations correctly.
That I haven't considered, as I don't think it should even be invented.

We should not be inventing new reserved words that are not in the standard.
An unspecified reserved word (one which might be reserved, or might not be)
is as far as is reasonable to go,  and even that is a huge push at the limits.

  | As another option, if an approach is to be invented, the utility could
  | change from being a regular utility to a special built-in,

That doesn't help - the only differences between them are some execution
time weirdnesses (error consequences, effects of var assigns, etc).

Further, that must be the case, as the relevant part of the script will have
been parsed before we even know what kind of command it is (whether built in
or not, and if built in, whether special or not).  It is way too late to
make syntax decisions, of any kind, by then.

  | This method adds capability not available to keywords
  | such as redirections

keywords can have redirections, almost all of them do, or at least the
structures they define do (all of for/while/until/case/()/{} can be redirected
(subshells aren't implemented with keywords, but with operators, but the
effect is the same for that particular operator pair in this regard).

  | and multiple utilities,

Sorry, don't understand what that means.

  | pipelined or not

Where a program structure introduced by a keyword fits in the grammar
is entirely up to the grammar to determine - and provided it isn't made
ambiguous by so doing, it can have different structures in different
circumstances if desired.

But a utility can only ever be an element in a pipeline, we have no idea
what utility is to be invoked until after the pipeline structure has been
parsed.

  | and requires no grammar changes.

That is certainly correct.

I suspect a large part of the problem might be because of the limits of
the current implemented time reserved word.  That's horrid, and most likely
only in the form it is because it was copying csh, and for some reason,
apparently a need was felt to make the syntax compatible with csh's syntax,
which was a patently absurd 

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 25 Feb 2020 17:47:23 + (UTC)
From:shwaresyst 
Message-ID:  <1701159045.321204.1582652843...@mail.yahoo.com>

  | The thing is, various shells have implemented time as keyword, so this bug
  | is trying to get the standard to reflect actual practice that ignored that
  | Rationale.

The first part of that I understand, and what's more, that was predicted, even
known, at the time, as the implementations (at least one) already existed.

Further it was intended to be permitted - that much is clear, they just
didn't realise all they needed to make unspecified in order for that to work.

  | The debate is more some things allowed to utilities aren't allowed
  | for keywords, and vice versa, so there are advantages to both.

Then that's a pretty silly debate, as absolutely anything can be allowed
for a keyword - part of defining a keyword (which is a grammar token) is
defining the grammar rules that use it, and those can be just about anything
that's desired, in any form that's desired.   It just needs to be consistent
(ie: not cause a conflict, I don't mean be similar to) with the rest of the
grammar.

More likely the real problem here is how to define the grammar for time as
a reserved word to make it match the actual implementations correctly.
That I haven't considered, as I don't think it should even be invented.

We should not be inventing new reserved words that are not in the standard.
An unspecified reserved word (one which might be reserved, or might not be)
is as far as is reasonable to go,  and even that is a huge push at the limits.

  | As another option, if an approach is to be invented, the utility could
  | change from being a regular utility to a special built-in,

That doesn't help - the only differences between them are some execution
time weirdnesses (error consequences, effects of var assigns, etc).

Further, that must be the case, as the relevant part of the script will have
been parsed before we even know what kind of command it is (whether built in
or not, and if built in, whether special or not).   It is way too late to
make syntax decisions, of any kind, by then.

  | This method adds capability not available to keywords
  | such as redirections

keywords can have redirections, almost all of them do, or at least the
structures they define do (all of for/while/until/case/()/{} can be redirected
(subshells aren't implemented with keywords, but with operators, but the
effect is the same for that particular operator pair in this regard).

  | and multiple utilities,

Sorry, don't understand what that means.

  | pipelined or not

Where a program structure introduced by a keyword fits in the grammar
is entirely up to the grammar to determine - and provided it isn't made
ambiguous by so doing, it can have different structures in different
circumstances if desired.

But a utility can only ever be an element in a pipeline, we have no idea
what utility is to be invoked until after the pipeline structure has been
parsed.

  | and requires no grammar changes.

That is certainly correct.

I suspect a large part of the problem might be because of the limits of
the current implemented time reserved word.  That's horrid, and most likely
only in the form it is because it was copying csh, and for some reason,
apparently a need was felt to make the syntax compatible with csh's syntax,
which was a patently absurd decision.   Had time been something like

time list
with word ...
emit

which would have been sh type syntax (that can be rearranged as desired,
the word... specify the options, format, ...  "with" can be "using") or
perhape
time word ...
do
list
emit   (or done)

to avoid adding too many new reserved words. and use would be
something like

time format '%E elapsed %.3U/%.3S user/sys\n' summary; do
for f in *.c; do grep time_me "$f"; done >&3 3>&-
emit 3>&1 >/tmp/time_log

but this is all just fantasy.


  | A time format could be an argument to the -b switch, getting rid
  | of any dependence on a shell variable being set or not too.

Yes, for any new time like thing that is added, this would clearly be
a good idea, but the csh way didn't do that, so ksh's time didn't either.
Dumb.

  | It has the advantages of both other approaches, in other words, and more.

No form of utility has any advantages over the time utility we have now.

We can easily add more options to that if they're implemented somewhere,
and any shell can make it a built-in (being special changes nothing that
matters) - but that is not what is needed as an additional feature, and
not what the time reserved word provides.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[shwaresyst]

The thing is, various shells have implemented time as keyword, so this bug is 
trying to get the standard to reflect actual practice that ignored that 
Rationale. The debate is more some things allowed to utilities aren't allowed 
for keywords, and vice versa, so there are advantages to both.

As another option, if an approach is to be invented, the utility could change 
from being a regular utility to a special built-in, because then it is fairly 
simple to implement -b(egin) and -e(nd) switches so use of the utility can 
bracket arbitrary amounts of script, with the shell being tasked to maintain an 
internal variable for the start or elapsed time, like the read utility can set 
visible variables. This method adds capability not available to keywords, such 
as redirections and multiple utilities, pipelined or not, and requires no 
grammar changes. A time format could be an argument to the -b switch, getting 
rid of any dependence on a shell variable being set or not too. It has the 
advantages of both other approaches, in other words, and more.
On Tuesday, February 25, 2020 Robert Elz  wrote:
    Date:        Tue, 25 Feb 2020 10:08:53 +
    From:        Geoff Clare 
    Message-ID:  <20200225100853.GA7003@lt2.masqnet>

  | In particular, ripping out most of the pipeline changes

Yes, that part is obvious, and I am not sure you need "most", all of
it should simply go.  No grammar changes are required at all.

But:
  | and adding the time reserved
  | word on top of the pipefail changes instead.

That is what cannot be done.  We cannot at this late stage reverse the
decision of the original standard not to make time a required reserved
word - making it even an unspecified reserved word will be a substantial
reversal of the earlier decision, and if there was any way that we could
implement the original intent (or most of it) of the original decision
without doing that, I wouldn't be in favour of that either.  But right
now I don't see one.

All that is needed to resolve the issues is to make time unspecified
reserved (what I have been calling reserved reserved - a word without
defined meaning that scripts should not use it), which means that it
cannot be safely used as an alias or function name (which is the right
thing to do as those don't work in shells where time is a reserved
word).

That would be sufficient, though the time utility should probably also
be dropped if it was done that simply, as there would be no defined way
to use it anyway.  Which of course doesn't mean anyone would delete it
from their distributions.

Better would be to spefify just enough of the time reserved word, with
language like

    If the unspecified reserved word time is implemented as a
    reserved word, then when used with a simple command, and without
    any redirections (including pipelines) it must operate as specified
    for the time utility.

    When redirections are present, or when used as an element of
    a pipeline, the results are unspecified.

There can also be added app notes, etc, showing how to use the time
utility in ways that seem to be unspecified, by quoting, by embedding it
in { }, ... and making clear some ways that are not speficied to give
any particular result.    And there should be plenty about this mess
added to the rationale.

We absolutely do not, in fact, must not, specify anything at all about
how a time reserved word should work.  That means there is no need to
waste time discussing nonsense like which format characters in TIMEFORMAT
mean what, as TIMEFORMAT won't even be mentioned.

None of this depends upon there being a proposal for some new way to
do what the time reserved word does in shells that have it.  They will
still have it after all (just be conformant in this area after the changes,
if we get it all correct), and scripts will still not be able to rely upon
it existing, as other shells don't have a reserved "time" - all that is the
correct result, as this is the current state of the world.

We are not required to provide such a solution - it would be nice if
one can be worked out, and agreed, and then implemented enough to be
standardised, but all of that will take a long time - no way it is going
to be ready for issue 8.

  | It's a work in progress but you can see what we have so far in the
  | etherpad at https://posix.rhansen.org/p/bug267

Yes, I know, I have been watching (sporadically).  That is what I have
been commenting on.  Sorry if that was not clear.  I haven't looked again
since you said (or the minutes said, or something) that this issue was
being delayed for a month, as I am assuming that it isn't currently being
worked upon (in the conferences).

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 25 Feb 2020 10:08:53 +
From:Geoff Clare 
Message-ID:  <20200225100853.GA7003@lt2.masqnet>

  | In particular, ripping out most of the pipeline changes

Yes, that part is obvious, and I am not sure you need "most", all of
it should simply go.  No grammar changes are required at all.

But:
  | and adding the time reserved
  | word on top of the pipefail changes instead.

That is what cannot be done.   We cannot at this late stage reverse the
decision of the original standard not to make time a required reserved
word - making it even an unspecified reserved word will be a substantial
reversal of the earlier decision, and if there was any way that we could
implement the original intent (or most of it) of the original decision
without doing that, I wouldn't be in favour of that either.   But right
now I don't see one.

All that is needed to resolve the issues is to make time unspecified
reserved (what I have been calling reserved reserved - a word without
defined meaning that scripts should not use it), which means that it
cannot be safely used as an alias or function name (which is the right
thing to do as those don't work in shells where time is a reserved
word).

That would be sufficient, though the time utility should probably also
be dropped if it was done that simply, as there would be no defined way
to use it anyway.   Which of course doesn't mean anyone would delete it
from their distributions.

Better would be to spefify just enough of the time reserved word, with
language like

If the unspecified reserved word time is implemented as a
reserved word, then when used with a simple command, and without
any redirections (including pipelines) it must operate as specified
for the time utility.

When redirections are present, or when used as an element of
a pipeline, the results are unspecified.

There can also be added app notes, etc, showing how to use the time
utility in ways that seem to be unspecified, by quoting, by embedding it
in { }, ... and making clear some ways that are not speficied to give
any particular result.And there should be plenty about this mess
added to the rationale.

We absolutely do not, in fact, must not, specify anything at all about
how a time reserved word should work.   That means there is no need to
waste time discussing nonsense like which format characters in TIMEFORMAT
mean what, as TIMEFORMAT won't even be mentioned.

None of this depends upon there being a proposal for some new way to
do what the time reserved word does in shells that have it.  They will
still have it after all (just be conformant in this area after the changes,
if we get it all correct), and scripts will still not be able to rely upon
it existing, as other shells don't have a reserved "time" - all that is the
correct result, as this is the current state of the world.

We are not required to provide such a solution - it would be nice if
one can be worked out, and agreed, and then implemented enough to be
standardised, but all of that will take a long time - no way it is going
to be ready for issue 8.

  | It's a work in progress but you can see what we have so far in the
  | etherpad at https://posix.rhansen.org/p/bug267

Yes, I know, I have been watching (sporadically).   That is what I have
been commenting on.  Sorry if that was not clear.   I haven't looked again
since you said (or the minutes said, or something) that this issue was
being delayed for a month, as I am assuming that it isn't currently being
worked upon (in the conferences).

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Geoff Clare]

Robert Elz  wrote, on 25 Feb 2020:
> 
> However contuing along the lines of the current 267 resolution proposal
> is totally the wrong thing to do,

When I said "the current direction" I was referring to the work we
have been doing in recent teleconferences to reduce the previous
bug 267 resolution to something more acceptable. In particular,
ripping out most of the pipeline changes and adding the time reserved
word on top of the pipefail changes instead.

It's a work in progress but you can see what we have so far in the
etherpad at https://posix.rhansen.org/p/bug267

(username and password at the end of the minutes).

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 25 Feb 2020 03:37:59 + (UTC)
From:shwaresyst 
Message-ID:  <968694195.52968.1582601879...@mail.yahoo.com>

  | I disagree with the notion of using names with colons,
  | especially for new utilities,

For utilities, I agree, it would be a terrible idea, but that is
not the proposal, it is to use them for new reserved words, and
for that:

  | mainly because colon is the character used as path separator in PATH

is part of the appeal, as it (close to) guarantees that no untility or
script will ever have a ':' in its name (anywhere), so a new reserved word
does not need to worry about that, in the (current) standard at least,
functions need to be NAMES, so cannot have colons (and even if/when we
extend the rules for function names, which we should, this won't be an issue)
- we are just about home free on being able to pick any name we want (and
it helps that names ending in ':' have been reserved forever, which is why
even with relaxed function names, names ending in ':' would not be
conformant there anyway).

':' (at the end - which is already reserved) is the right choice for new
reserved words.   No suggested NAME type word I can see is safe enough to
risk (though some are most likely not going to be a real problem).   Other
in-word non alpha-num characters that we could use aren't currently reserved
(nor is mid-word ':') and someone might be using them ... you might be
surprised at some of the command names I use, just about every single
character as a word has some defined meaning (a function, or script)
(though '^', because historically it wasn't a word-char, happens not to be
one of them, and nor is ':' because of the PATH issues) and some odd pairs
of chars as well.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[shwaresyst]

I disagree with the notion of using names with colons, especially for new 
utilities, mainly because colon is the character used as path separator in PATH 
and there may be usage ambiguities resulting from this overload. I'd be more in 
favor of implementing labels with another character entirely as distinguishing 
definer, such as caret, and reserving that instead.
On Monday, February 24, 2020 Robert Elz  wrote:
    Date:        Mon, 24 Feb 2020 15:27:22 -0800
    From:        Nick Stoughton 
    Message-ID:  


  | XRAT states "The restriction on ending a name with a  is to allow
  | future implementations that support
  | named labels for flow control; see the RATIONALE for the break built-in
  | utility.",

Ah.  Thanks for that.  I missed that part, and certainly never thought to
look at the pages for break ...

But we can even keep that possibility (as unimplemented as it is) alive
(I guess it sounded like a good idea at the time ... it still does, just
not so good as to be worth spending any effort implementing), by only allowing
such labels to be a NAME with an appended ':'.  Since that is all new
(fictional) usage, we can restrict it however we like...

Then we have the entire remaining name space (anything that is not a NAME)
with a solon appended to use to invent new reserved words as needed, by
making them something that is not a NAME (anything really) with a ':' appended
(it just needs to not require quoting).

One obvious example would be to end with two colons eg: statz::  or we
could do away with the s/s/z/ stuff as no longer being needed, and just
use  stats::  (I still prefer not to imply only timing related statistics
should be available by using anyything suggestive of that).

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Mon, 24 Feb 2020 15:27:22 -0800
From:Nick Stoughton 
Message-ID:  


  | XRAT states "The restriction on ending a name with a  is to allow
  | future implementations that support
  | named labels for flow control; see the RATIONALE for the break built-in
  | utility.",

Ah.  Thanks for that.   I missed that part, and certainly never thought to
look at the pages for break ...

But we can even keep that possibility (as unimplemented as it is) alive
(I guess it sounded like a good idea at the time ... it still does, just
not so good as to be worth spending any effort implementing), by only allowing
such labels to be a NAME with an appended ':'.   Since that is all new
(fictional) usage, we can restrict it however we like...

Then we have the entire remaining name space (anything that is not a NAME)
with a solon appended to use to invent new reserved words as needed, by
making them something that is not a NAME (anything really) with a ':' appended
(it just needs to not require quoting).

One obvious example would be to end with two colons eg: statz::   or we
could do away with the s/s/z/ stuff as no longer being needed, and just
use   stats::   (I still prefer not to imply only timing related statistics
should be available by using anyything suggestive of that).

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Mon, 24 Feb 2020 15:09:54 +
From:Geoff Clare 
Message-ID:  <20200224150954.GA3455@lt2.masqnet>

  | You are looking further ahead that we need to at this point.

I knew that, as you would have seen lower down, I only added that point
as you seemed as if you were waiting a month for a proposal from me (and
perhape Chet) for a replacement ... the implication being that perhape if
nothing reasonable was forthcoming by then, then something along the lines
of current proposed wording on 267 (cleaned up as needed perhaps) might/would
go ahead.But a month wouldn't be long enough to wait for that.

  | For now, the teleconference participants just need enough info to make
  | a decision(probably on March 23) of which direction to take for bug 267.

I would have thought the directions is already clear (and you summarise
it below) - what is needed now for that is the details.   A month seems
like a reasonable time to clean that up.

  | If your and Chet's new proposal looks promising, we will likely
  | abandon the current direction

Now I am back to thinking that you are expecting something new within
a month, and that's unreasonable.If it is to be good it needs to be
tested, and that means getting all kinds of different people with
different needs to use it, and give feedback, and then to react to that
by changing the p[roposal if needed.And all that is after design and
implementation.

But if you just want a rough idea, it will (I suspect) look rather like
the reserved word time does in shells that support it now, just with a
different reserved word (not yet selected) and most likely cleaned up
some (no longer needing to try and coexist with the time utility).

  | and start thinking about what changes
  | would be needed to the existing text in order to lay the groundwork
  | for adding the new feature later on.

The only think I can think of there would be to reserve the name asao
(if needed).   If we want to go that route, and of the names suggested
recently, Mark's suggested "statz" is the one that appeals to me most.

But we can also just agree to use a name ending in ':' in which case it
is already reserved, and nothing really needs to be done (you could add
words saying "names ending in : are reserved, unless defined by this
standard.  Currently there are none, but new reserved words might be
added in future editions" somwehere - in some rationale, or future
directions, I guess).   That might be worth doing anyway, just to allow
a mechanism to add unrelated new reserved words if it ever seems
advisable (I cannot think of anything right now, but who knows).

That is unless there is some other good reason that names ending in ':'
must be kept reserved and never used for anything.Anyone?

  | You missed the issue that was the reason bug 267 was raised - redirection
  | of the timing information.

You're right, I did.   So we need to make that unspecified as well.
Making stuff unspecified is always something of a cop-out ... but it does
tend to reflect reality.

  | Another issue I discovered recently is that shells with time as a
  | reserved word behave strangely when the command being timed is stopped
  | by a signal.

Ugh...   Hadn't even considered that possibility.  There is a lot of
unliness around process that stop, and very little consistency in what
happens in many of them (processes that stop in loops, ..., are another
source of continuous problems ... not relevant to this issue.)

  | If we end up deciding to add your and Chet's new feature,

I think that regardless of that, this ...

  | then I think what we should do for "time" is add it as an unspecified
  | reserved word (alongside select and [[) and advise that conforming
  | applications should avoid using the reserved word and instead make
  | sure they use the time utility, e.g. by quoting the word "time",
  | using "command time ...", or having a word expansion result in the
  | word "time".

is roughly what we must do.   There really is no need to wait a month
to make that decision.   I'd prefer it, for the sake of existing 
conforming scripts (or some/most of them anyway) if we can find a way to
make some uses of
time simple-command
(no pipelines or redirectios involved) still acceptable (defined) when
it is a simple enough case that it works everywhere, just to avoid some
future shell deciding to go in some other direction completely, and relying
upon the "applications are not allowed to use unquoted 'time'" wording).

But if we cannot find suitable wording to make that happen, then so be it.
But that is what I would prefer to spend time over the next month
woirking on, rather than trying to come up with a half baked (which is
all it could be) proposal for a replacement within that period.

However contuing along the lines of the current 267 resolution proposal
is totally the wrong thing to do, we cannot simply reverse the considered
decision of the earlier 

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Nick Stoughton]

If we are bicycle-shedding around a name for a new utility, I like the idea
of using the currently reserved namespace of appending a colon ... "time:
"

Since this is already reserved it shouldn't break either existing shells or
utilities. I know of no shells that implement a goto (which is why it was
reserved in the first place).

XRAT states "The restriction on ending a name with a  is to allow
future implementations that support
named labels for flow control; see the RATIONALE for the break built-in
utility.", and the mentioned break rationale provides an example, but I
have not heard of any shell that uses it.
-- 
Nick

On Mon, Feb 24, 2020 at 3:16 PM Chris F.A. Johnson 
wrote:

> On Mon, 24 Feb 2020, Geoff Clare wrote:
>
> > Chet Ramey  wrote, on 16 Feb 2020:
> >>
> >> On 2/15/20 9:42 PM, Robert Elz wrote:
> >>
> >>> Let's design & implement something better, and then have that
> (eventually)
> >>> standardised, rather than compromising the standard with this horror.
> There
> >>> is of course, no reason that the shells that have chosen to copy ksh's
> >>> extension cannot continue to do so, no matter how ugly and difficult it
> >>> is to do.
> >>
> >> OK. Let's start with a proposal. The existing syntax will have to stick
> >> around for backwards compatibility, of course.
> >
> > In case you two don't read the teleconference minutes, in Thursday's
> > meeting we decided to postpone further work on bug 267 for a month to
> > allow you time to work on this new proposal.
> >
> >> (The simplest proposal is to replace the ksh/bash/mksh/zsh(?) `time'
> with
> >> `ptime' and go from there.)
> >
> > The new name needs to be something that is unlikely to be in use by
> > applications or users already.  I think ptime is not a good candidate,
> > as it is a likely choice for an existing script, function or alias name.
>
>Almost anything you can think of is probably in use somewhere already.
>
>My recommendation is 'tm', analogous to 'cp', 'ls', 'mv' etc. (the
>first two consonants of the full word). And, yes, I do have a
>script with that name, but I haven't used it in years, nor am I
>likely to.
>
>
> --
> Chris F.A. Johnson 
> === Author: ===
> Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
> Pro Bash Programming: Scripting the GNU/Linux shell (2009, Apress)
>
>

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Chris F.A. Johnson]

On Mon, 24 Feb 2020, Geoff Clare wrote:


Chet Ramey  wrote, on 16 Feb 2020:


On 2/15/20 9:42 PM, Robert Elz wrote:


Let's design & implement something better, and then have that (eventually)
standardised, rather than compromising the standard with this horror.  There
is of course, no reason that the shells that have chosen to copy ksh's
extension cannot continue to do so, no matter how ugly and difficult it
is to do.


OK. Let's start with a proposal. The existing syntax will have to stick
around for backwards compatibility, of course.


In case you two don't read the teleconference minutes, in Thursday's
meeting we decided to postpone further work on bug 267 for a month to
allow you time to work on this new proposal.


(The simplest proposal is to replace the ksh/bash/mksh/zsh(?) `time' with
`ptime' and go from there.)


The new name needs to be something that is unlikely to be in use by
applications or users already.  I think ptime is not a good candidate,
as it is a likely choice for an existing script, function or alias name.


  Almost anything you can think of is probably in use somewhere already.

  My recommendation is 'tm', analogous to 'cp', 'ls', 'mv' etc. (the
  first two consonants of the full word). And, yes, I do have a
  script with that name, but I haven't used it in years, nor am I
  likely to.


--
   Chris F.A. Johnson 
   === Author: ===
   Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
   Pro Bash Programming: Scripting the GNU/Linux shell (2009, Apress)

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Joerg Schilling]

Geoff Clare  wrote:

> > (The simplest proposal is to replace the ksh/bash/mksh/zsh(?) `time' with
> > `ptime' and go from there.)
>
> The new name needs to be something that is unlikely to be in use by
> applications or users already.  I think ptime is not a good candidate,
> as it is a likely choice for an existing script, function or alias name.

Correct!

And ptime is in use since 1994, (written by Roger Faulkner) as part of the 
ptools for procfs.

# ptime
usage: ptime [-mh] [-p pid | command [ args ... ]]
  (time a command using microstate accounting)

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Geoff Clare]

Robert Elz  wrote, on 24 Feb 2020:
>
> But we really do not need to defer bug 267 until all that is worked out,
> this will take a long time, as not only do we need a proposal, we need
> implementation experience, and then application use to verify it is
> suitable, and shouldn't be improved before it is finalised, and then more
> widespread implementation -- if we happen, just by chance of course, to make
> the new reserved word, at least in its basic form - some extensions might
> be nice - mean just the same as what "time" does now in those shells which
> implement it as a reserved word (modulo what stuff in TIMEFORMAT means, which
> looks like another giant mess, so we should not usurp that one) then I suspect
> getting implementations in those shells might not be all that difficult.

You are looking further ahead that we need to at this point.  For now,
the teleconference participants just need enough info to make a decision
(probably on March 23) of which direction to take for bug 267.

If your and Chet's new proposal looks promising, we will likely
abandon the current direction and start thinking about what changes
would be needed to the existing text in order to lay the groundwork
for adding the new feature later on.

[...]
> We can make sure that implementations that have implemented time as a
> reserved word aren't contrary to the standard though, since it seems that
> was the intent.
> 
> At the minute I see just one real issue, are there others?
> 
> 
> What that leaves, to me, is the issue with functions and aliases.  When
> time is a reserved word, it cannot be defined as an alias (who cares) or
> as a function (a slightly bigger issue).   Since there is no reasonable
> way out of this, I believe the only thing to do is to prohibit its use
> for those purposes (an automatic side effect of making it a reserved
> reserved word), as scripts that attempt it simply won't work with
> many shells (and consequently I doubt there are very many such usages).

You missed the issue that was the reason bug 267 was raised - redirection
of the timing information.

Another issue I discovered recently is that shells with time as a
reserved word behave strangely when the command being timed is stopped
by a signal.  Bash gives timing output when the command is stopped (and
none when it completes).  Ksh88 on HP-UX and ksh93 on Linux do the same.
With ksh88 on Solaris, timing a command prevents it from being stopped
with Ctrl-Z.  Ksh93 on Solaris produces no timing information at all.
(All tested with the relevant shell as the login shell and using
"time /bin/sleep 10" then hitting Ctrl-Z and - if the command was
stopped - using fg to continue it.)

If we end up deciding to add your and Chet's new feature, then I think
what we should do for "time" is add it as an unspecified reserved word
(alongside select and [[) and advise that conforming applications should
avoid using the reserved word and instead make sure they use the time
utility, e.g. by quoting the word "time", using "command time ...", or
having a word expansion result in the word "time".

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[shwaresyst]

If you're going to go with 'z' suffix, "statz" is an option too; for 
session/process statistics or "state of the machine".
On Monday, February 24, 2020 Scott Lurndal  wrote:
On Mon, Feb 24, 2020 at 09:23:16AM +, Geoff Clare wrote:
> Chet Ramey  wrote, on 16 Feb 2020:
> >
> > On 2/15/20 9:42 PM, Robert Elz wrote:
> > 
> > > Let's design & implement something better, and then have that (eventually)
> > > standardised, rather than compromising the standard with this horror.  
> > > There
> > > is of course, no reason that the shells that have chosen to copy ksh's
> > > extension cannot continue to do so, no matter how ugly and difficult it
> > > is to do.
> > 
> > OK. Let's start with a proposal. The existing syntax will have to stick
> > around for backwards compatibility, of course.
> 
> In case you two don't read the teleconference minutes, in Thursday's
> meeting we decided to postpone further work on bug 267 for a month to
> allow you time to work on this new proposal.
> 
> > (The simplest proposal is to replace the ksh/bash/mksh/zsh(?) `time' with
> > `ptime' and go from there.)
> 
> The new name needs to be something that is unlikely to be in use by
> applications or users already.  I think ptime is not a good candidate,
> as it is a likely choice for an existing script, function or alias name.
> 

System V had 'timex';  perhaps 'timez' would be a good candidate name?

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Scott Lurndal]

On Mon, Feb 24, 2020 at 09:23:16AM +, Geoff Clare wrote:
> Chet Ramey  wrote, on 16 Feb 2020:
> >
> > On 2/15/20 9:42 PM, Robert Elz wrote:
> > 
> > > Let's design & implement something better, and then have that (eventually)
> > > standardised, rather than compromising the standard with this horror.  
> > > There
> > > is of course, no reason that the shells that have chosen to copy ksh's
> > > extension cannot continue to do so, no matter how ugly and difficult it
> > > is to do.
> > 
> > OK. Let's start with a proposal. The existing syntax will have to stick
> > around for backwards compatibility, of course.
> 
> In case you two don't read the teleconference minutes, in Thursday's
> meeting we decided to postpone further work on bug 267 for a month to
> allow you time to work on this new proposal.
> 
> > (The simplest proposal is to replace the ksh/bash/mksh/zsh(?) `time' with
> > `ptime' and go from there.)
> 
> The new name needs to be something that is unlikely to be in use by
> applications or users already.  I think ptime is not a good candidate,
> as it is a likely choice for an existing script, function or alias name.
> 

System V had 'timex';   perhaps 'timez' would be a good candidate name?

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Mon, 24 Feb 2020 09:23:16 +
From:Geoff Clare 
Message-ID:  <20200224092316.GA32284@lt2.masqnet>

  | The new name needs to be something that is unlikely to be in use by
  | applications or users already.  I think ptime is not a good candidate,
  | as it is a likely choice for an existing script, function or alias name.

I don't think anything with time in its name would be right anyway, as
I said earlier (much more information is typically available than that,
and even if it was not to be required, a name that better suited the
complete common use would be better).

It is going to be hard to find something even half way reasonable that
we can hope that scripts are not using though (or that add on packages aren't
using as a command name - it isn't just scripts, functions and aliases that
matter) - but maybe we could usurp that "words ending in : are reserved"
part (why is that there?   The only thing I can think of is for labels, but
did any Bourne shell derivative (as distinct from Thompson shell & csh and
descendants) ever have labels?) and pick a nice word, and simply append a ':'
and define that (and weasel word the text that says "any word ending in :"
or whatever the words are, into "any word not mentioned above ending in : ..."

But we really do not need to defer bug 267 until all that is worked out,
this will take a long time, as not only do we need a proposal, we need
implementation experience, and then application use to verify it is
suitable, and shouldn't be improved before it is finalised, and then more
widespread implementation -- if we happen, just by chance of course, to make
the new reserved word, at least in its basic form - some extensions might
be nice - mean just the same as what "time" does now in those shells which
implement it as a reserved word (modulo what stuff in TIMEFORMAT means, which
looks like another giant mess, so we should not usurp that one) then I suspect
getting implementations in those shells might not be all that difficult.

What is needed for 267 is to settle the time mess - and for that we
simply cannot require shells to make time a reserved word - especially
when the original standards developers quite deliberately decided that
was not the correct course of action.

We can make sure that implementations that have implemented time as a
reserved word aren't contrary to the standard though, since it seems that
was the intent.

At the minute I see just one real issue, are there others?

It isn't that one can do (with reserved words)

time ( subshell-commands )
or
time { grouped commands;  }
or
time for x in ...
(etc)

those aren't issues, as they would all normally generate syntax errors in
a shell that doesn't implement time as a reserved word, and shells have
always been permitted to give some meaning to what is a syntax error
(as no valid script can ever reasonably be expecting such a thing to work).

Similarly whether or not one can time (rationally) a builtin or function
depends more upon whether time is builtin to the shell or not (and chooses
to instrument itself enough to make the data available) than whether it is
a reserved word or not.


What that leaves, to me, is the issue with functions and aliases.  When
time is a reserved word, it cannot be defined as an alias (who cares) or
as a function (a slightly bigger issue).   Since there is no reasonable
way out of this, I believe the only thing to do is to prohibit its use
for those purposes (an automatic side effect of making it a reserved
reserved word), as scripts that attempt it simply won't work with
many shells (and consequently I doubt there are very many such usages).

Beyond that, and adding time in that reserved reserved word list,
we need to create wording to make it clear that

time simple-command

(not used as part of a pipeline) must continue to work, whether it is
implemented as a reserved word, an external time utility, or a built-in
utility doesn't matter - and certainly under no circumstances should anything
depend upon what options might happen to be used.

time a | b already gives unspecified results, we don't need to require
time -p a | b  to magically become specified - it never has been before.

To work out proposed wording for this resoluton of 267 the month's
delay is a reasonable action, and I appreciate that, for 100% computing
unrelated reasons I am very busy right now, and barely even reading
mail, let alone keeping up with actions like this one.   But within a
month I should be able to come up with something (which like everything
from me, I wouldn't be surprised if it needs wordsmithing into something
suitable for the standard) - so if you (all) haven't heard from me on
this in 2 (maybe 3) weeks, please prompt me.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Geoff Clare]

Chet Ramey  wrote, on 16 Feb 2020:
>
> On 2/15/20 9:42 PM, Robert Elz wrote:
> 
> > Let's design & implement something better, and then have that (eventually)
> > standardised, rather than compromising the standard with this horror.  There
> > is of course, no reason that the shells that have chosen to copy ksh's
> > extension cannot continue to do so, no matter how ugly and difficult it
> > is to do.
> 
> OK. Let's start with a proposal. The existing syntax will have to stick
> around for backwards compatibility, of course.

In case you two don't read the teleconference minutes, in Thursday's
meeting we decided to postpone further work on bug 267 for a month to
allow you time to work on this new proposal.

> (The simplest proposal is to replace the ksh/bash/mksh/zsh(?) `time' with
> `ptime' and go from there.)

The new name needs to be something that is unlikely to be in use by
applications or users already.  I think ptime is not a good candidate,
as it is a likely choice for an existing script, function or alias name.

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Geoff Clare]

Robert Elz  wrote, on 18 Feb 2020:
>
> Date:Mon, 17 Feb 2020 15:22:08 +
> From:Geoff Clare 
> 
>   | Not at all.  If the discrepancy between ksh88 and POSIX.2-1992 had
>   | come to light before those systems were certified, the standard would
>   | have been amended to allow the ksh88 behaviour at that time.  The end
>   | result would be the same, just the timing would be different.
> 
> That might or might not be correct, it is impossible to say now - after
> all "select" was known at the time, and not standardised, my guess is
> that the reserved word "time" would, at best, have been treated the same,
> but we can never know for sure now.

You misunderstood my point.  The ksh time reserved word was well known at
the time, and the POSIX.2-1992 developers believed they had written the
description of the time utility in a way that allowed it.  This is clear
from the RATIONALE on the time utility page.

What has come to light since is that there are other ways in which the
behaviour of the time reserved word in ksh is not allowed by POSIX,
besides those that the standard developers explicitly allowed.  This
is what I meant by "the discrepancy between ksh88 and POSIX.2-1992".

> However I would note XRAT C.2.4 ...
> 
>   The list of unspecified reserved words is from the KornShell,
>   so conforming applications cannot use them in places a reserved
>   word would be recognized. This list contained time in early proposals,
>   but it was removed when the time utility was selected for the Shell
>   and Utilities volume of POSIX.1-2017.
> 
> which to me suggests the exact opposite of what you believe would
> have happened (someone with access to earlier copies of the standard
> would need to check and see how long that text has been there, certainly
> that it mentions "POSIX.1-2017" means nothing, that's just "current version").

It is clear from the RATIONALE on the time utility page that the above
means that they considered making time an unspecified reserved word like
select, but then decided instead to accommodate its behaviour in the
description of the time utility.

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Stephane CHAZELAS]

2020-02-19 11:52:21 +0100, Joerg Schilling:
[...]
> > $ zsh -c 'time -p uname | cat'
> > zsh:1: command not found: -p
> 
> BTW: I remember that you frequently write that zsh intends to emulate the csh 
> behavior, but csh _is_ able to get timing for builtin commands:
> 
> csh
> % time alias
> 0.0u 0.0s 0:00 0% 0+0k 0+0io 0pf+0w
> % 
[...]

Like I said, csh forks there to be able to report the usage:

tcsh> alias a b
tcsh> alias
a   b
tcsh> time alias c d
0.000u 0.000s 0:00.00 0.0%  0+0k 0+0io 0pf+0w
tcsh> alias
a   b

(missing "c  d" in that output as alias c d was run in a
subshell).

zsh does have a csh emulation mode (emulate csh), which makes it
more csh-like, but there's no way it could be twisted into being
a csh interpreter. That's more of a helper mode to make it more
friendly to users who come from tcsh.

Like ksh or bash, zsh got many features from csh. Of the three
shells, it's probably the one which got the most (like it's the
one which got the most features from rc, or more generally the
most features).

Here, when it comes to time, it got the full getrusage() from
csh/BSD, it extended it by timing each pipeline component
separately (very useful to spot bottlenecks).

Like csh, it report *process* resource usage.

Contrary to csh, it supports timing pipelines, compounds,
functions (csh has no functions anyway).

And like I said, contrary to csh, it doesn't implicitly add an
extra fork. That would change the functionality and would not be
practical given the point above.

That does mean it can't report resource usages of things that
are not run in a separate process. That's a sometimes annoying
drawback (though some easy workaround), but that's a tradeoff.

-- 
Stephane

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Joerg Schilling]

Stephane Chazelas  wrote:

> Note that AT ksh and bosh revert to calling the standalone
> time utility in that case so they can't be used to time
> builtin/functions.

Well, if you enforce to implement the old POSIX behavior, you get what the old 
POSIX definition says: timing is only possible for external commands.

If you use the extension, you get more

> Which seem to be calling the standalone "time" utility in all
> implementations in my tests.
>
> $ mksh -c 'time -p uname | cat'
> mksh: -p: inaccessible or not found

Given  that mksh claims to emulate ksh88 and sometimes ksh93, it could be seen 
as a bug.

> $ zsh -c 'time -p uname | cat'
> zsh:1: command not found: -p

BTW: I remember that you frequently write that zsh intends to emulate the csh 
behavior, but csh _is_ able to get timing for builtin commands:

csh
% time alias
0.0u 0.0s 0:00 0% 0+0k 0+0io 0pf+0w
% 

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Chet Ramey]

On 2/18/20 12:00 PM, shwaresyst wrote:
> 
> Don't see why, lparen, like "=", is not a char that stops collection of a 
> token body

I'm not exactly sure what a `token body' is supposed to be, but the left
paren does delimit a token. A `=' does not.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 18 Feb 2020 17:00:08 + (UTC)
From:shwaresyst 
Message-ID:  <944836912.4626676.1582045208...@mail.yahoo.com>

  | Don't see why, lparen, like "=", is not a char that stops collection
  | of a token body;

What standard are you reading?

XCU 2.3 (Token Recognition), rule 6:

6. If the current character is not quoted and can be used as the first
   character of a new operator, the current token (if any) shall be delimited.
   The current character shall be used as the beginning of the next
   (operator) token.

Together with XBD 3.260

Operator
In the shell command language, either a control operator
or a redirection operator.

and XBD 3.113

Control Operator

In the shell command language, a token that performs a
control function. It is one of the following symbols:

& && ( ) ; ;; newline | ||

The end-of-input indicator used internally by the shell
is also considered a control operator.

Note:  Token Recognition is defined in detail in XCU
Section 2.3 (on page 2347).

Do you see that '(' in that list?   I do.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Stephane CHAZELAS]

2020-02-18 14:50:50 +0100, Joerg Schilling:
[...]
> There however is a shell that has massive problems: "zsh".

> zsh cannot time functins or builtins at all

Zsh's "time" keyword reports *process* resource usage (for each
component of the timed pipeline).

If there's no process, there's nothing to report.

If you want to time a process running a function, builtin or
compound command, you need a subshell:

time (...)

Note that  that resource usage reporting is not only about CPU
and elapse time as zsh supports reporting the same things as csh
or GNU time does. For things like "The average amount in
(shared) text space used in kilobytes" (%X), approaches that
substract usage after and before are not really usable.

See how tcsh does fork by itself when you try to time builtins.

>   time -p sleep 1
>   zsh: command not found: -p

"time" is a keyword in zsh. keywords don't take options.

I suppose zsh could add the same ugly kludge as in ksh93 for
POSIX compliance in sh mode, but nobody every bothered asking
for it it would seem.

"time" is a debugging/interactive tool mostly. It's hardly
relevant in portable scripts.

Which is why I'd tend to agree with kre that it's probably not
worth the bother POSIX specifying it.

> 
> as you see, the command line documentd by POSIX does not work.
[...]

If you did need zsh to interpret a sh script that used "time
-p" (not that I've ever come across such a script), you could
always do:

emulate sh 'disable -r time; . /path/to/that-script'

-- 
Stephane

Re: Re: Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[shwaresyst]

Strictly from the syntax, yes, if what is before the lparen is a valid NAME. 
Practically, I'd expect an 'keyword redefinition attempt' message rather than 
'syntax error', because "time()" does meet the requirements of the syntax, but 
the standard doesn't require this so silently storing the definition, and 
confusing the user, are more the conforming behavior.


-Original Message-
From: Joerg Schilling 
To: shwaresyst ; kre 
Cc: gwc ; austin-group-l 
Sent: Tue, Feb 18, 2020 12:08 PM
Subject: Re: Re: [1003.1(2004)/Issue 6 267]: time (keyword)


shwaresyst  wrote:>> Don't see why, lparen, like "=", is 
not a char that stops collection of a token body; the function name is 
extracted, supposedly, when the parser finds the lparen rparen pair; it is used 
to start a background job only when it is the first character of the token 
body.Do you expect any function name to be accepted when a function is 
defined?I am not sure whether this is a good idea, as a function that uses the 
name of a reserved word will never be executed and this may cause more 
amazement to the user than an error message that flags the function 
definition.Jörg--  EMail:jo...@schily.net(home) Jörg 
Schilling D-13353 Berlinjoerg.schill...@fokus.fraunhofer.de (work) Blog: 
http://schily.blogspot.com/ URL: http://cdrecord.org/private/ 
http://sf.net/projects/schilytools/files/'

Re: Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Joerg Schilling]

shwaresyst  wrote:

>
> Don't see why, lparen, like "=", is not a char that stops collection of a 
> token body; the function name is extracted, supposedly, when the parser finds 
> the lparen rparen pair; it is used to start a background job only when it is 
> the first character of the token body.

Do you expect any function name to be accepted when a function is defined?

I am not sure whether this is a good idea, as a function that uses the name of 
a reserved word will never be executed and this may cause more amazement to the 
user than an error message that flags the function definition.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[shwaresyst]

Don't see why, lparen, like "=", is not a char that stops collection of a token 
body; the function name is extracted, supposedly, when the parser finds the 
lparen rparen pair; it is used to start a background job only when it is the 
first character of the token body.

-Original Message-
From: Robert Elz 
To: shwaresyst 
Cc: Joerg.Schilling ; gwc 
; austin-group-l 
Sent: Tue, Feb 18, 2020 11:34 AM
Subject: Re: [1003.1(2004)/Issue 6 267]: time (keyword)


Date:Tue, 18 Feb 2020 15:26:43 + (UTC)From:
shwaresyst Message-ID:  
<2085395192.4579856.1582039603...@mail.yahoo.com>  | It shouldn't matter 
whether a token prefix matches a reserved word or not;  | because the lparen is 
not separated by a  function definition should  | be the "best, longest" 
match productionI'm not quite sure what to say to that, other than that that 
interpretationof how things work must be from some other shell entirely, as it 
is noteven close to how POSIX shells work.kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 18 Feb 2020 15:26:43 + (UTC)
From:shwaresyst 
Message-ID:  <2085395192.4579856.1582039603...@mail.yahoo.com>

  | It shouldn't matter whether a token prefix matches a reserved word or not;
  | because the lparen is not separated by a  function definition should
  | be the "best, longest" match production

I'm not quite sure what to say to that, other than that that interpretation
of how things work must be from some other shell entirely, as it is not
even close to how POSIX shells work.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Chet Ramey]

On 2/18/20 10:36 AM, Robert Elz wrote:
> Date:Tue, 18 Feb 2020 09:58:32 -0500
> From:Chet Ramey 
> Message-ID:  <22f16ef4-41cf-60b0-5968-f608dc988...@case.edu>
> 
>   | The 1992 version of the standard knew about time, standardized it as part
>   | of the UPE, and acknowledged that it worded the definition to allow the
>   | ksh88 reserved word implementation.
> 
> Well, kind of - they made it unspecified what
> 
>   time a | b 
> 
> timed ("a" or "a|b")

Yes. "The times reported are unspecified."

 but ksh (all versions I believe) bash and bosh
> all allow
>   time(sleep 1)
> which nothing in the standard explains (or allows).  

"If the current character is not quoted and can be used as the first
character of a new operator, the current token (if any) shall be
delimited."

 Similarly
>   time { sleep 1; }
> and
>   time if true; then sleep 1; else sleep 2; fi

They're both pipelines according to the shell grammar.

> And in a subsequent message, chet.ra...@case.edu said:
>   | There are people on the bash mailing lists who would like a word. 
> 
> There are some very strange people on those lists!

A couple of representative examples.

https://lists.gnu.org/archive/html/help-bash/2018-12/msg00110.html
https://lists.gnu.org/archive/html/help-bash/2018-12/msg00092.html


-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 18 Feb 2020 09:58:32 -0500
From:Chet Ramey 
Message-ID:  <22f16ef4-41cf-60b0-5968-f608dc988...@case.edu>

  | The 1992 version of the standard knew about time, standardized it as part
  | of the UPE, and acknowledged that it worded the definition to allow the
  | ksh88 reserved word implementation.

Well, kind of - they made it unspecified what

time a | b 

timed ("a" or "a|b") but ksh (all versions I believe) bash and bosh
all allow
time(sleep 1)
which nothing in the standard explains (or allows).   Similarly
time { sleep 1; }
and
time if true; then sleep 1; else sleep 2; fi

except that if Jörg is right, then the "sleep" and "true" in those
would need to be quoted just in case they've also been turned into
reserved words, accepting some random syntax we don't know about.

And in a subsequent message, chet.ra...@case.edu said:
  | There are people on the bash mailing lists who would like a word. 

There are some very strange people on those lists!

kre

ps: I am not saying that any of this functionality (that provided by
ksh, etc) is not useful, it's just hard to reconcile in any reasonable
way with what the standard allows, and other shells implement.

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[shwaresyst]

It shouldn't matter whether a token prefix matches a reserved word or not; 
because the lparen is not separated by a  function definition should be 
the "best, longest" match production that it is interpreted as, and so wouldn't 
be a syntax error. It's on use that keyword recognition would hide whatever 
definition was done, and which might confuse end users, but it appears those 
shells are buggy in not allowing it.
On Tuesday, February 18, 2020 Robert Elz  wrote:
    Date:        Tue, 18 Feb 2020 11:31:54 +0100
    From:        Joerg Schilling 
    Message-ID:  <5e4bbd1a.ixp7vvhraxdcl4i4%joerg.schill...@fokus.fraunhofer.de>


  | There is an official and documented way already, call:
  |
  |     time -p command1 | command2

That's nice, but in a POSIX compatible shell (the FreeBSD shell in
this case)

fbsh $ time() { command time -p "$@"; }
fbsh $ time sleep 1
real        1.00
user        0.00
sys          0.00
fbsh $ 

Also works in dash, yash, nbsh, and zsh

But in ksh93, mksh, bosh, pdksh, and bash, we get (variations of)

bosh $ time() { command time -p "$@"; }
syntax error: `)' unexpected
bosh $ 

That's because time there is a reserved word, and as written that
looks to be an attempt to time the empty sub-shell (which is a
syntax error, correctly).

  | The time utility in POSIX requires to use the -p option to be specified.
  | Otherwise, the results are unspecified.

As written above, the time utility was called with the -p option.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Chet Ramey]

On 2/18/20 9:59 AM, Robert Elz wrote:

> Why someone would want to time a builtin I'm not sure
> (with the possible exception of elapsed time of wait)

There are people on the bash mailing lists who would like a word.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Chet Ramey]

On 2/18/20 5:31 AM, Joerg Schilling wrote:

> All shells I am aware detect the -p option in the parser already and switch 
> to 
> the time utility instead of the time keyword.

Bash doesn't do that; it's not useful and difficult to do using a bison-
based parser.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 18 Feb 2020 14:50:50 +0100
From:Joerg Schilling 
Message-ID:  <5e4bebba.XCR/ahv7oqvibm+7%joerg.schill...@fokus.fraunhofer.de>

  | If you like to get the previous POSIX behavior, call:
  |
  | time -p some-command
  |
  | This is what POSIX did require in the past...

No, it didn't, the -p is needed only to get the POSIX specific
output format.

  | When you do that, you of course cannot time functions or builtins.

Not directly, one can time a script that runs the function/builtin
and subtract the time for running an empty script, but that is kind
of complex.Why someone would want to time a builtin I'm not sure
(with the possible exception of elapsed time of wait)

  | There however is a shell that has massive problems: "zsh".
  |
  | zsh cannot time functins or builtins at all and even more:
  |
  | time -p sleep 1
  | zsh: command not found: -p
  |
  | as you see, the command line documentd by POSIX does not work.

But
time sleep 1
works in it just fine...

zsh $ time sleep 1
sleep 1  0.00s user 0.00s system 0% cpu 1.004 total

The output format isn't POSIX standard, but the data is there.

Further:

zsh $ \time -p sleep 1
real 1.00
user 0.00
sys  0.00
zsh $ command time -p sleep 1
real 1.00
user 0.00
sys  0.00

also both work, so it isn't quite as hopeless as it seems.  Both of
those suppress the reserved word processing, and I guess I really
should have used

zsh $ \command time -p sleep 1
real 1.00
user 0.00
sys  0.00

just in case "command" has been turned into a reserved word as well
(since apparently that's possible, and entirely POSIX compatible).

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Chet Ramey]

On 2/17/20 9:17 PM, Robert Elz wrote:
> Date:Mon, 17 Feb 2020 15:22:08 +
> From:Geoff Clare 
> Message-ID:  <20200217152208.GA9739@lt2.masqnet>
> 
> 
>   | Not at all.  If the discrepancy between ksh88 and POSIX.2-1992 had
>   | come to light before those systems were certified, the standard would
>   | have been amended to allow the ksh88 behaviour at that time.  The end
>   | result would be the same, just the timing would be different.
> 
> That might or might not be correct, it is impossible to say now - after
> all "select" was known at the time, and not standardised, my guess is
> that the reserved word "time" would, at best, have been treated the same,
> but we can never know for sure now.

The 1992 version of the standard knew about time, standardized it as part
of the UPE, and acknowledged that it worded the definition to allow the
ksh88 reserved word implementation. They considered invalidating the ksh88
implementation, but chose not to, since the S5 `time' utility was
essentially useless.

The language from the rationale was present in the 1992 edition.

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 18 Feb 2020 14:27:38 +0100
From:Joerg Schilling 
Message-ID:  <5e4be64a.b7iehtvuvp4jwlze%joerg.schill...@fokus.fraunhofer.de>

  | The problem here is that it is forbidden to create a function
  | with a name that is equal to a keyword.

Yes, I knew that.

  | Since POSIX does not forbid to have keywords that are beyond the list from 
  | POSIX,

Hmm - POSIX allows lots of extenstions, but if that were true, then
the only way to ever write a portable script would be to quote every
command name, and there would be no way to ever define a function.

XCU 2.4 goes to great lengths to list the reserved words, some other
words that have to be treated as if they were reserved (just in case)
and all words ending with a colon ... the latter two sets would not
be needed if scripts were required to treat every word as if it might
be reserved.

I kind os suspect that the vast majority of sh script writers would be
kind of amazed by this revelation.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 18 Feb 2020 11:31:54 +0100
From:Joerg Schilling 
Message-ID:  <5e4bbd1a.ixp7vvhraxdcl4i4%joerg.schill...@fokus.fraunhofer.de>

  | There is an official and documented way already, call:
  |
  | time -p command1 | command2

That's actually specifically unspecified, see lines 110983-8 (page 3297)
That's one thing this bug planned on changing, but that would be a
truly bad result.

  | The time utility in POSIX requires to use the -p option to be specified.
  | Otherwise, the results are unspecified.

The only thing unspecified in that case is what the actual output
format is (and perhaps just what data is output).   Wven without -p
time(1) is required to run the utility given by its first (non-option)
arg, and report on its execution times.   See lines 110971-3 (same page).

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Joerg Schilling]

Stephane Chazelas  wrote:

> 2020-02-18 11:31:54 +0100, Joerg Schilling:
> [...]
> > There is an official and documented way already, call:
> > 
> > time -p command1 | command2
> > 
> > The time utility in POSIX requires to use the -p option to be specified.
> > Otherwise, the results are unspecified.
> > 
> > All shells I am aware detect the -p option in the parser already and switch 
> > to 
> > the time utility instead of the time keyword.
> [...]
>
> Only bash, AT ksh and bosh (of those shells that have a "time"
> keyword) and only when the -p is literal (not "-p",
> ${POSIXLY_CORRECT+-p}, though ksh93/bosh seem to recognise -"p"
> and -pp) and is the next word after "time".
>
> Note that AT ksh and bosh revert to calling the standalone
> time utility in that case so they can't be used to time
> builtin/functions.

It seems that you did not understand what I like to say:

If you like to get the previous POSIX behavior, call:

time -p some-command

This is what POSIX did require in the past...
When you do that, you of course cannot time functions or builtins.

There however is a shell that has massive problems: "zsh".

zsh cannot time functins or builtins at all and even more:

time -p sleep 1
zsh: command not found: -p

as you see, the command line documentd by POSIX does not work.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Stephane Chazelas]

2020-02-18 11:31:54 +0100, Joerg Schilling:
[...]
> There is an official and documented way already, call:
> 
>   time -p command1 | command2
> 
> The time utility in POSIX requires to use the -p option to be specified.
> Otherwise, the results are unspecified.
> 
> All shells I am aware detect the -p option in the parser already and switch 
> to 
> the time utility instead of the time keyword.
[...]

Only bash, AT ksh and bosh (of those shells that have a "time"
keyword) and only when the -p is literal (not "-p",
${POSIXLY_CORRECT+-p}, though ksh93/bosh seem to recognise -"p"
and -pp) and is the next word after "time".

Note that AT ksh and bosh revert to calling the standalone
time utility in that case so they can't be used to time
builtin/functions.

Portably, you'd need

"time" -p command1 | command2

Or

command time -p command1 | command2

Which seem to be calling the standalone "time" utility in all
implementations in my tests.

$ mksh -c 'time -p uname | cat'
mksh: -p: inaccessible or not found
0m00.00s real 0m00.00s user 0m00.00s system
$ zsh -c 'time -p uname | cat'
zsh:1: command not found: -p
-p uname  0.00s user 0.00s system 86% cpu 0.001 total
cat  0.00s user 0.00s system 82% cpu 0.002 total
$ bosh -c 'time 2>&1 -p uname'
bosh: -p: not found

real   0.000254
user   0.00
sys0.000251
$ ksh -c 'time \-p uname'
ksh: -p: not found

real0m0.000s
user0m0.000s
sys 0m0.000s
$ bash -o posix -c 'time ${POSIXLY_CORRECT+-p} uname'
bash: -p: command not found

real0m0.002s
user0m0.001s
sys 0m0.000s

-- 
Stephane

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Joerg Schilling]

Robert Elz  wrote:

>   | There is an official and documented way already, call:
>   |
>   |   time -p command1 | command2
>
> That's nice, but in a POSIX compatible shell (the FreeBSD shell in
> this case)
>
> fbsh $ time() { command time -p "$@"; }

The problem here is that it is forbidden to create a function with a name that 
is equal to a keyword.

Since POSIX does not forbid to have keywords that are beyond the list from 
POSIX, I see no problem with the behavior of the other shells.

POSIX compatible does not forbid enhancements.


Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Tue, 18 Feb 2020 11:31:54 +0100
From:Joerg Schilling 
Message-ID:  <5e4bbd1a.ixp7vvhraxdcl4i4%joerg.schill...@fokus.fraunhofer.de>


  | There is an official and documented way already, call:
  |
  | time -p command1 | command2

That's nice, but in a POSIX compatible shell (the FreeBSD shell in
this case)

fbsh $ time() { command time -p "$@"; }
fbsh $ time sleep 1
real 1.00
user 0.00
sys  0.00
fbsh $ 

Also works in dash, yash, nbsh, and zsh

But in ksh93, mksh, bosh, pdksh, and bash, we get (variations of)

bosh $ time() { command time -p "$@"; }
syntax error: `)' unexpected
bosh $ 

That's because time there is a reserved word, and as written that
looks to be an attempt to time the empty sub-shell (which is a
syntax error, correctly).

  | The time utility in POSIX requires to use the -p option to be specified.
  | Otherwise, the results are unspecified.

As written above, the time utility was called with the -p option.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Joerg Schilling]

Geoff Clare  wrote:

> Treating time something like select and [[ would be an alternative way
> around the problem.  What you suggest for "use of the time utility" is
> not quite right, though.  Use of the time _reserved_word_ should be as
> you say - unspecified unless it prefixes a simple command.  Use of the
> time _utility_ can stay specified the same as now.  For example:
>
> "time" command1 2>&1 | command2
>
> is a use of the time utility to time the execution of command1, with
> its stdout and stderr, and the timing information, all being fed into
> command2. Quoting "time" means it is not recognized as a reserved word.

There is an official and documented way already, call:

time -p command1 | command2

The time utility in POSIX requires to use the -p option to be specified.
Otherwise, the results are unspecified.

All shells I am aware detect the -p option in the parser already and switch to 
the time utility instead of the time keyword.

Jörg

-- 
 EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
 URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Mon, 17 Feb 2020 15:22:08 +
From:Geoff Clare 
Message-ID:  <20200217152208.GA9739@lt2.masqnet>


  | Not at all.  If the discrepancy between ksh88 and POSIX.2-1992 had
  | come to light before those systems were certified, the standard would
  | have been amended to allow the ksh88 behaviour at that time.  The end
  | result would be the same, just the timing would be different.

That might or might not be correct, it is impossible to say now - after
all "select" was known at the time, and not standardised, my guess is
that the reserved word "time" would, at best, have been treated the same,
but we can never know for sure now.

However I would note XRAT C.2.4 ...

The list of unspecified reserved words is from the KornShell,
so conforming applications cannot use them in places a reserved
word would be recognized. This list contained time in early proposals,
but it was removed when the time utility was selected for the Shell
and Utilities volume of POSIX.1-2017.

which to me suggests the exact opposite of what you believe would
have happened (someone with access to earlier copies of the standard
would need to check and see how long that text has been there, certainly
that it mentions "POSIX.1-2017" means nothing, that's just "current version").

But regardless of that, decades have passed, implementations and
applications have been built based upon the spec as it was written.
Making substantial changes without a good reason (a very good reason,
and retrospectively correcting missing conformance tests is not one)
is unreasonable.

[me previously...]
  | > specify that use of the time utility (specified as unquoted "time")

  | Treating time something like select and [[ would be an alternative way
  | around the problem.  What you suggest for "use of the time utility" is
  | not quite right, though.  Use of the time _reserved_word_ should be as
  | you say - unspecified unless it prefixes a simple command.

That's what I meant by the '(specified as unquoted "time")' - that is,
where it could be a reserved word.  Note that "time" is not currently a
reserved word, so
time command1 } command2
pipes the output of "time command1" to command2.   Anything to allow
time to be a reserved word breaks that usage.   That's unfortunate, but
as we know, is already broken in many shells, and updating the standard
so it matches current reality is a good reason.   Mandating copying of
the way ksh does it is not justified though.

  | Use of the time _utility_ can stay specified the same as now.

Yes, sure.   Except when "time" (unquoted) might be treated as a
reserved word, and is used in a place where more than a simple
command follows.   Note that the "reserved reserved words" are a
constraint upon conforming applications (which cannot use them in
places where they might be treated as reserved words), not any
kind of requirement on implementations.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Geoff Clare]

Robert Elz  wrote, on 17 Feb 2020:
>
> Date:Mon, 17 Feb 2020 09:52:52 +
> From:Geoff Clare 
> 
>   | We can't just reject issue 267.  There are certified systems that
>   | do not behave as the standard requires (probably all of them).
>   | We need a formal interpretation saying that there is a defect in
>   | the standard in order for those systems not to have to remove the
>   | time keyword from their "sh" shells.  Once we say that there is a
>   | defect in the standard, we have to fix it.
> 
> Let me interpret that, and see if I understand correctly.
> 
> You plan on changing the rules for shells that implemented the
> standard (in this area) as it is written, in order to allow systems
> that were incorrectly certified to become correctly certified without
> needing to change anything.

Not at all.  If the discrepancy between ksh88 and POSIX.2-1992 had
come to light before those systems were certified, the standard would
have been amended to allow the ksh88 behaviour at that time.  The end
result would be the same, just the timing would be different.

>   | So what I would suggest is that for bug 267 we continue with the
>   | addition of a minimal set of changes to add the time keyword in
>   | a way that is compatible with posixified-ksh88 and bash.
> 
> If something absolutely has to be done with time, then make it
> a reserved reserved word, just like select and [[ (etc), and then
> specify that use of the time utility (specified as unquoted "time")
> in any way other than with a simple command produces unspecified results.

Treating time something like select and [[ would be an alternative way
around the problem.  What you suggest for "use of the time utility" is
not quite right, though.  Use of the time _reserved_word_ should be as
you say - unspecified unless it prefixes a simple command.  Use of the
time _utility_ can stay specified the same as now.  For example:

"time" command1 2>&1 | command2

is a use of the time utility to time the execution of command1, with
its stdout and stderr, and the timing information, all being fed into
command2. Quoting "time" means it is not recognized as a reserved word.

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Mon, 17 Feb 2020 09:52:52 +
From:Geoff Clare 
Message-ID:  <20200217095252.GA6718@lt2.masqnet>

  | We can't just reject issue 267.  There are certified systems that
  | do not behave as the standard requires (probably all of them).
  | We need a formal interpretation saying that there is a defect in
  | the standard in order for those systems not to have to remove the
  | time keyword from their "sh" shells.  Once we say that there is a
  | defect in the standard, we have to fix it.

Let me interpret that, and see if I understand correctly.

You plan on changing the rules for shells that implemented the
standard (in this area) as it is written, in order to allow systems
that were incorrectly certified to become correctly certified without
needing to change anything.

In other areas that would be called a "cover up".

  | So what I would suggest is that for bug 267 we continue with the
  | addition of a minimal set of changes to add the time keyword in
  | a way that is compatible with posixified-ksh88 and bash.

If something absolutely has to be done with time, then make it
a reserved reserved word, just like select and [[ (etc), and then
specify that use of the time utility (specified as unquoted "time")
in any way other than with a simple command produces unspecified results.

When used as it works with time(1) whether the time is built in, and/or a
reserved word, really makes no (significant) difference.   When used in some
other way (as in a pipeline) it becomes simply an unspecified permitted
extension, and the results vary (which is exactly what happens now,
depending upon what shell is used).

I believe that way we get to have our cake and ...

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Geoff Clare]

Robert Elz  wrote, on 17 Feb 2020:
>
> Agree with that, we need something that works, sanely, and is implemented
> first, which is why I support simply rejecting issue 267 (this one).

We can't just reject issue 267.  There are certified systems that
do not behave as the standard requires (probably all of them).
We need a formal interpretation saying that there is a defect in
the standard in order for those systems not to have to remove the
time keyword from their "sh" shells.  Once we say that there is a
defect in the standard, we have to fix it.

So what I would suggest is that for bug 267 we continue with the
addition of a minimal set of changes to add the time keyword in
a way that is compatible with posixified-ksh88 and bash.  Then,
once there is existing practice for a new way to time pipelines,
shell builtins and functions, a new bug should be submitted which
adds the new way and marks the time keyword and the time utility as
obsolescent.

-- 
Geoff Clare 
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Sun, 16 Feb 2020 17:03:11 -0500
From:Chet Ramey 
Message-ID:  <7dc21b31-75d4-5f1a-b4b5-ed47b4061...@case.edu>


  | but I could get behind this, as long as the standard didn't invent it.

Agree with that, we need something that works, sanely, and is implemented
first, which is why I support simply rejecting issue 267 (this one).

  | OK. Let's start with a proposal. The existing syntax will have to stick
  | around for backwards compatibility, of course.

In the shells that implement it, naturally.

  | (The simplest proposal is to replace the ksh/bash/mksh/zsh(?) `time' with
  | `ptime' and go from there.)

Yes, though I would get rid of "time" in the name, I have been trying to
think of a better label (something I hate doing, and am certainly not good
at) and all I have really decided so far is that this is generally not
simply reporting time info, but all kinds of resource usage.   I'd suggest
"usage" as a command name, but that would break far too many scripts, so
instead something relate to resources in general, not just time.

But the name is a side issue, and we can get to that later, the most
important first decision is whether we want it to be a new "only works
when built in" type utility, or a reserved word, and new syntax, as the
whole design of how it functions will depend upon that.   Either way would
work for me, the utility version is easier to implement, but makes it
harder to use (have to deal with all the quoting issues that plague eval).
Not impossible for scripts, but harder.  A new reserved word is slightly
harder to implement (more changes internally) but easier for scripts to use.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Chet Ramey]

On 2/15/20 9:42 PM, Robert Elz wrote:
> Date:Sat, 15 Feb 2020 17:56:36 -0500
> From:Chet Ramey 
> Message-ID:  <17c5318a-a4f7-8bd8-8792-c1ad4d82b...@case.edu>
> 
>   | The only reason to specify it as a reserved word is to time pipelines.
> 
> Yes, that much I understood, at least the intent.
> 
>   | You can time builtins or shell functions with a builtin (special
>   | builtins should not be a thing), but you have to change the parser to
>   | time pipelines.
> 
> eval is a special builtin (or just a builtin if you would prefer it
> that way) and can eval pipelines.   There is no reason that time could
> not have been made an analog of eval, except a desire to make the thing
> look like csh syntax (ignoring that csh was based upon an entirely different
> underlying syntax structure).

Sure.

Korn could easily have required that non-simple commands be quoted, and
that would have been upwards compatible, but he chose not to.

> 
>   | It's genuinely new functionality. The historical time command cannot time
>   | pipelines, builtins, or shell functions.
> 
> Yes, understood, but new functionality should have had a new name, not
> steal the name of an existing command.

Maybe. I'm not generally interested in relitigating 30-year-old decisions,
but I could get behind this, as long as the standard didn't invent it.

> Let's design & implement something better, and then have that (eventually)
> standardised, rather than compromising the standard with this horror.  There
> is of course, no reason that the shells that have chosen to copy ksh's
> extension cannot continue to do so, no matter how ugly and difficult it
> is to do.

OK. Let's start with a proposal. The existing syntax will have to stick
around for backwards compatibility, of course.

(The simplest proposal is to replace the ksh/bash/mksh/zsh(?) `time' with
`ptime' and go from there.)

Chet

-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

Date:Sat, 15 Feb 2020 17:56:36 -0500
From:Chet Ramey 
Message-ID:  <17c5318a-a4f7-8bd8-8792-c1ad4d82b...@case.edu>

  | The only reason to specify it as a reserved word is to time pipelines.

Yes, that much I understood, at least the intent.

  | You can time builtins or shell functions with a builtin (special
  | builtins should not be a thing), but you have to change the parser to
  | time pipelines.

eval is a special builtin (or just a builtin if you would prefer it
that way) and can eval pipelines.   There is no reason that time could
not have been made an analog of eval, except a desire to make the thing
look like csh syntax (ignoring that csh was based upon an entirely different
underlying syntax structure).

  | It's genuinely new functionality. The historical time command cannot time
  | pipelines, builtins, or shell functions.

Yes, understood, but new functionality should have had a new name, not
steal the name of an existing command.

Or a solution could have been to replace the existing command entirely
(as you indicate you have done with bash), requiring \time if the old command
was to be wanted - which sould normally be rare if the new syntax is a
genuine replacement - reserved words with "options" might be new, but
there is nothing new about reserved words that have (even optional) further
syntax (eg: case, for) rather than just being a single word.   There could
even be an "option" to indicate how much the reserved word should apply
to (next command, next pipeline, or even next and-or list).

  | Bash just times the null command, doesn't mess around with the ksh93-like
  | magic conversion to `times'.

Either way, what the command does isn't what matters, it is that the
word "time" is syntax in one usage, and a command name in a different
one, that is the issue here.If time (or some replacement token to
avoid conflicting with the command) were genuinely a reserved word, and
worked the same way as all the other reserved words, then if I had a
script that seemed to be over resource hungry, and it wasn't obvious
why, then I could simply insert

time # Z

lines ahead of every possible command (including groups, sub-shells, ...)
and then once the output was available and the issue identified, and
perhaps corrected:

sed -e /Z/d < debug-script > real-script

(other debugging printf, etc, commands with Z in comments, may have
been added as well) to clean it all up.   This kind of automated undo
is much harder when the timed command is required to start on the same line
as the reserved word (as sh has no in-line comment syntax) and that is
required to allow time to also be a command.

That is, since "time" by itself would be meaningless, the grammar could
legitimately allow a linebreak after it (as it does after "if", "while"
etc) - perhaps that would come after any "options", using whatever syntax
were chosen for them, to control how it operates) as there would always be
required to be some command (or pipeline, or and-or list) to operate upon.

All in all, my general opinion is that all of this is just so ugly, and
so badly conceived, that it ought not be added to the standard in anything
like its current form.

Let's design & implement something better, and then have that (eventually)
standardised, rather than compromising the standard with this horror.  There
is of course, no reason that the shells that have chosen to copy ksh's
extension cannot continue to do so, no matter how ugly and difficult it
is to do.

kre

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Chet Ramey]

On 2/14/20 5:19 PM, Robert Elz wrote:

> It is not that I have anything against a time reserved word, though
> being a reserved word rather than a builtin, or special builtin does
> make it impossible to redirect (as has been noted) and also raises
> questions as to what exactly happens on error (like an invalid format
> string in TIMEFORMAT, or a command not found, etc) - does it act like
> a special builtin with an error, or a regular command.   Those kinds
> of things could (and should) be fixed .. but that's more difficult
> for a reserved word which doesn't have the utility page with its
> standard format in which all of this has a natural place.

The only reason to specify it as a reserved word is to time pipelines.
You can time builtins or shell functions with a builtin (special
builtins should not be a thing), but you have to change the parser to
time pipelines.

> 
> But the proposed grammar (and also one of the proposed replacements)
> is broken, the "linebreak" non-terminal cannot be used the way it was
> (and apparently still is) being proposed - doing so would make the
> grammar ambiguous. 

True.

> Now to the actual subject matter ... first, and comparatively minor,
> is it really desirable to make a reserved word ("time") (and I mean
> when it is a reserved word, but I'll come to that below) into also
> being a command?   This would be a first for the shell I believe, until
> this, reserved words are only ever syntax, never commands. 

It's genuinely new functionality. The historical time command cannot time
pipelines, builtins, or shell functions.

  Even some
> of what is often considered syntax is actually implemented as commands.
> Have all of the ramifications of this been considered?   Note that this
> relates to the use of "time" as a command, not to its use as a prefix for
> a pipeline (where it is just syntax).   I'd suggest just avoiding the
> isssue, and drop the command form (a pipeline that is exactly "time"
> with nothing else) -- all that it generates is a poor man's "times" command
> with the only extra being the ability for users to control the output
> format, better to add that ability to "times" than pervert the structure
> of the shell.

Bash just times the null command, doesn't mess around with the ksh93-like
magic conversion to `times'.


> But what is worst about all of this is...
> 
>"however, the word \fBtime\fP shall not be
>  recognized as a reserved word if the first
>  character of the next token is '-' ()."
> 
> Huh?

Yeah, that's pretty much impossible to implement in a straightforward way
using yacc/bison. You really have to abuse the tokenizer. The best you can
do without that is to try and clean it up when executing it as some kind of
compound command, but why bother at that point?

That's why bash tries to support `time -p' and `time -p --' (though,
unfortunately, not `time --' -- I'll have to fix that).

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, UTech, CWRUc...@case.eduhttp://tiswww.cwru.edu/~chet/

Re: [1003.1(2004)/Issue 6 0000267]: time (keyword)

[Robert Elz]

It is certainly a good thing that this has been reopened, as what
was proposed to be inserted is, frankly, nonsense.

It is not that I have anything against a time reserved word, though
being a reserved word rather than a builtin, or special builtin does
make it impossible to redirect (as has been noted) and also raises
questions as to what exactly happens on error (like an invalid format
string in TIMEFORMAT, or a command not found, etc) - does it act like
a special builtin with an error, or a regular command.   Those kinds
of things could (and should) be fixed .. but that's more difficult
for a reserved word which doesn't have the utility page with its
standard format in which all of this has a natural place.

But the proposed grammar (and also one of the proposed replacements)
is broken, the "linebreak" non-terminal cannot be used the way it was
(and apparently still is) being proposed - doing so would make the
grammar ambiguous.   That might be OK if you're feeding it to yacc,
where the shift/reduce conflict resolution rules will settle on one
interpretation, but is definitely inappropriate to use in a standard.

"linebreak" can only ever be used in a rule when there is at least
one more required syntactic element to follow - that isn't to say
that it cannot be the last element of a rule - but only when that
rule is embedded in another (or which is itself embedded in another)
in which something else (not a command termination) is required.

Or in other words, it is only ever used in places where the current
command cannot possibly end at the current place, and it is desired
to allow a newline to be inserted, such as after most reserved words,
after && || | (and similar, including '(' itself).

So
  pipeline : pipe_sequence
   | Bang linebreak
   |  (other stuff)

cannot be correct, as

linebreak : newline_list
  | /* empty */

  and_or  :  pipeline
  |   (other stuff)

list  :  and_or
  |(other stuff)

 complete_command : list separator_op
  | list

 complete_commands: complete_commands newline_list complete_command
  | complete_command

 program  : linebreak complete_commands linebreak
  | linebreak

If we were to have simply

 !\n

as input (where the \n means a single newline character) then it is
ambiguous whether the newline matches the linebreak that is in "pipeline"
or the one in "program" and while that might seem insignificant, it isn't
for a grammar (and in more complex cases may make a real difference).

This problem is still in Mark's proposed replacement grammar, but isn't
in Stephane's (which is much better).

The original was much worse than this, I don't even want to contemplate
what

pipeline:  pipe_sequence
| Bang linebreak
| Bang pipeline
|   (even more)

is useful for, or why anyone could conceivably want that.

But since there are proposed replacements without that nonsense, I assume
that you're aware of just how absurd this all is, and so I can stop this
part here.

But the above raises another issue, this bug report is supposed to be
about the "time" reserved word (keyword) - why would any solution to
that be making any changes at all to the way that the ! reserved word
works?   Surely that would need to be done in a bug about either ! itself,
or a more general pipeline syntax bug report.   Dealing with "time" as a
reserved word is complex enough without trying to slip in modifications
to "!" at the same time (and doing a half baked job of it), for now can
I assume that the proposed changes to the way ! is defined to work will
be dropped from this issue ... if there is some real desire for this
change make it be a new issue (and then I will say why I think it is
a very poor idea there, rather than here).

(The same is true of the changes that seem to make minor unrelated
changes to the grammar, to fix minor issues that were noticed - strictly
they should be separate issues (or at least one) ... it is hard enough
to search for stuff in mantis to see what is in the queue to be changed
already, but it becomes impossible when changes are buried within other
issues that are unrelated ... but as these look to be trivial, and not
controversial at all, unless I missed something, never mind).


Now to the actual subject matter ... first, and comparatively minor,
is it really desirable to make a reserved word ("time") (and I mean
when it is a reserved word, but I'll come to that below) into also
being a command?   This would be a first for the shell I believe, until
this, reserved words are only ever syntax, never commands.   Even some
of what is often considered syntax is actually implemented as commands.
Have all of the ramifications of this been considered?   Note that this
relates to the use of "time" as a command,