Re: [Issue 8 drafts 0001776]: find -newer any symlinks

[Don Cragun via austin-group-l at The Open Group]

Hi Stephane,
We missed the point you raised about the example when we resolved this issue 
during our 2023-11-20 conference call.  Since those changes have already been 
applied to Issue 8 draft 4 I will file a new bug to address this point in a 
later draft.

Thank you for pointing that we missed this.

Sincerely,
Don

> On Dec 14, 2023, at 10:54 AM, Stephane Chazelas via austin-group-l at The 
> Open Group  wrote:
> 
> 2023-11-20 17:30:32 +, Austin Group Bug Tracker via austin-group-l at The 
> Open Group:
> [...]
>> On page 2920 line 97596 section find (-newer), change:The
>> primary shall evaluate as true if the modification time of the current file
>> is more recent than the modification time of the file named by the pathname
>> file.to:The primary shall evaluate as true
>> if the modification time of the current file is more recent than the
>> modification time of the file named by the pathname file. If
>> file names a symbolic link, the modification time used shall be that
>> of the file referenced by the symbolic link if either the -H or
>> -L is specified; if neither -H nor -L is specified, it
>> is unspecified whether the modification time is that of the symbolic link
>> itself or of the file referenced by the symbolic link. In either case,  if
>> the referenced file does not exist, the modification time used shall be
>> that of the link itself. If file is a relative pathname, it shall be
>> resolved relative to the current working directory that was inherited by
>> find when it was invoked.
> [...]
> 
> Thanks, looks good.
> 
> What about the example:
> 
>> - fix the example at line 97747 to use -H, and maybe add a
>> "provided both file1 and file2 are accessible".
> 
> ?
> 
> -- 
> Stephane
> 

Re: pathnames with two leading slashes

[Don Cragun via austin-group-l at The Open Group]

Hi Cris,
Please find comments in-line below...

> On Jul 31, 2023, at 4:05 PM, Christoph Anton Mitterer via austin-group-l at 
> The Open Group  wrote:
> 
> Hey.
> 
> POSIX 4.13 Pathname Resolution says:
>> If a pathname begins with two successive  characters, the
>> first component following the leading  characters may be
>> interpreted in an implementation-defined manner, although more than
>> two leading  characters shall be treated as a single 
>> character.
> 
> I wondered what exactly this means.
> If the *first* component following the leading // is allowed to be
> interpreted in an implementation-defined manner, what would that mean
> for any remaining components? That they are not?
> 
> It doesn't seem to make too much sense to have the remaining components
> not in an implementation-defined manner if the first one already is, or
> does it?

On some systems when the initial POSIX standards were being created, a pathname 
that started with exactly two  characters was interpreted to be a 
specific directory on the server on the local network named by the first 
component.  On other systems, the leading // was treated exactly like / and 
just referred to the directory named named by the first component on the root 
directory on the local system.  After that directory (local or on the network) 
has been identified pathname resolution continues from that directory in the 
standard manner.  It is only the evaluation of the first component that is 
implementation-defined.

> 
> 
> Also, in POSIX a pathname component / filename cannot contain ,
> but would the above text mean that - since component may be interpreted
> in an implementation-defined manner - it could contain  and
> there was only one component at all?

No.  Each group of  characters separates components no matter what 
directory is identified by the first component.

> 
> And POSIX 3.271 Pathname says:
>> Multiple successive  characters are considered to be the same
>> as one , except for the case of exactly two leading 
>> characters. 
> 
> Would that only mean that the two leading es aren't collapsed
> into one, but any others are as in:
> ///foo///bar
> would be the same as:
> ///foo/bares.

Yes.

> 
> or would it mean that *no* collapsing takes place at all, that is the
> two pathnames from above would not be the same?

No.

> 
> Thanks
> Chris.
> 

Hope this helps,
Don

Re: [1003.1(2016/18)/Issue7+TC2 0001614]: XSH 3/mktime does not specify EINVAL and should

[Don Cragun via austin-group-l at The Open Group]

Robert & Geoff have been arguing about whether or not giving a struct tm to 
mktime() that specifies a time in the gap between standard time and daylight 
time is allowed to be treated as an error (assuming that the seconds since the 
Epoch would otherwise fit in a time_t).

Robert is arguing that if (after adjusting other fields to
bring them into the ranges specified in ) mktime()
should return an error if tm_hour has a value that does not
occur on days when the timezone shifts.  Geoff is arguing
that when tm_hour has a value that does not occur on those
days, mktime() is required to adjust tm_hour (and any other
fields that might need to be adjusted for cases where the
shift isn't 1 hour) to a value that does yield a struct tm
containing values that time() could return.

I agree with Geoff.

If we accept Robert's argument, then it isn't just gaps in time caused by a 
timezone shift that would be affected. As an example, if I call time() on 
January 30, 2023 at noon, it will return a struct tm with tm_mon set to 11 (it 
has a normal range of 0-11) and tm_mday set to 29 (it has a normal range of 
1-31).  If I then add 1 to tm_mon and call mktime() with the resulting struct 
tm, I'm asking mktime to give me back a struct tm for noon on February 29, 
2023. This date does not exist (since 2023 is not a leap year). However, both 
tm_mon and tm_mday are in the ranges specified by .  So, if Robert is 
being consistent, he is insisting that mktime() should return an error because 
there is no February 29 in 2023.  Fortunately all existing implementations of 
mktime() adjust tm_mon and tm_mday to noon on March 1, 2023 instead of 
returning an error.  I sincerely hope that none of us would want an error in 
this case.

The standard (200x draft 2.1 P1311, L43850-43870 says:
The mktime() function shall convert the broken-down
time, expressed as local time, in the structure
pointed to by timeptr, into a time since the Epoch
value with the same encoding as that of the values
returned by time(). The original values of the
tm_wday and tm_yday components of the structure shall
be ignored, and the original values of the other
components shall not be restricted to the ranges
described in .

A positive or 0 value for tm_isdst shall cause
mktime() to presume initially that Daylight Saving
Time, respectively, is or is not in effect for the
specified time. A negative value for tm_isdst shall
cause mktime() to attempt to determine whether
Daylight Saving Time is in effect for the specified
time.

Local timezone information shall be set as though
mktime() called tzset().

The relationship between the tm structure (defined in
the  header) and the time in seconds since
the Epoch is that the result shall be as specified in
the expression given in the definition of seconds
since the Epoch (see XBD Section 4.17, on page 95)
corrected for timezone and any seasonal time
adjustments, where the names other than tm_yday in
the structure and in the expression correspond, and
the tm_yday value used in the expression is the day
of the year from 0 to 365 inclusive, calculated from
the other tm structure members specified in 
(excluding tm_wday).

Upon successful completion, the values of the tm_wday
and tm_yday components of the structure shall be set
appropriately, and the other components shall be set
to represent the specified time since the Epoch, but
with their values forced to the ranges indicated in
the  entry; the final value of tm_mday shall
not be set until tm_mon and tm_year are determined.

I see nothing above that says that mktime() should just
adjust tm_mday to a valid day within the range 1-31 and give
an error if that day doesn't exist in the calculated month.
The mktime() function is required to return a struct tm
containing adjustments corresponding to a time that time()
could return.  I see no reason to believe that just because
tm_hour was initially set to 1 means that it is in any way
exempt from being adjusted to yield a tm_hour value (and a
tm_isdst value) that are valid whether or not there was a
timezone shift.

 - Don

Re: stty default output/-a _POSIX_VDISABLE character "undef", contrasts with "" on all known implementations? Formatting loss->overinterpretation since Issue 6.

[Don Cragun via austin-group-l at The Open Group]

Hi наб,
Thank you for your work on this issue.  It has been filed as Austin Group 
Defect Tracker issue #1604.  See https://austingroupbugs.net/view.php?id=1604

Cheers,
Don

> On Sep 10, 2022, at 5:09 AM, наб via austin-group-l at The Open Group 
>  wrote:
> 
> Hi!
> 
> Issue 7 and Issue 8 Draft 2.1 say this (XCU, stty, STDOUT, last para.):
>  If control characters are written as part of the default output, or if
>  the −a option is specified, control characters shall be written as:
>"%s = %s;", , 
>  where  is either the character, or some visual representation of
>  the character if it is non-printable, or the string undef if the
>  character is disabled.
> 
> undef is italicised, and matches the argument to character-valued
> attributes like kill. On first glance, this makes sense.
> 
> However, every BSD release on the CSRG CDs since 3BSD,
> SysIII, and SysVr{1,2,3,4} ‒ that is, every historical system /with/ a
> disabling functionality ‒ as well as modern BSD, Illumos,
> and the GNU system, render this as "".
> 
> This is very curious! /I/ was very curious, at least.
> 
> XPG3 stty (same as XPG2's) does not define the output format
> (stty's primary purpose isn't to generate text,
> so that fits with the XPG's "don't rely on this format" clause).
> 
> The XPG3-XPG4 migration guide lists some changes,
> but none of them relevant here.
> 
> POSIX.2 Draft 11.2 (the only one extant on the internet :/) says:
>  If control characters are written as part of the default output,
>  or if the −a option is
>  specified, control characters shall be written as:
>"%s = %s;", , 
>  where value is either the character, or some visual representation
>  of the character if it is nonprintable,
>  or the string  if the character is disabled.
> 
> And  is monospace here!
> This is "correct": it matches every implementation.
> 
> XPG4.2/SUSv1 in its change history cites
>  Aligned with the ISO/IEC 9945-2: 1993 standard.
> for Issue 4 amd says:
>  If control characters are written as part of the default output,
>  or if the −a option is specified,
>  control characters will be written as:
>"%s = %s;", <-character name>, 
>  where value is either the character,
>  or some visual representation of the character if it is non-printable,
>  or the string  if the character is disabled.
> 
> With  in normal font
> (this matches the formatting of its own control-character-argument table,
> which has ^- and undef in normal font as well).
> 
> SUSv2/Issue 5 matches SUSv2 (except for the ^- in the table).
> Its FUTURE DIRECTIONS says something about interpretations of P1003.2b,
> but in draft 12 of that I didn't see anything relevant to this.
> 
> SUSv3/Issue 6 is the first one that formats this in the problematic way
> described: both undef in the control-character-argument table and in the
> STDOUT section are no-<> and italic:
>  If control characters are written as part of the default output,
>  or if the -a option is specified,
>  control characters shall be written as:
>"%s = %s;", , 
>  where  is either the character, or some visual representation
>  of the character if it is non-printable, or the string undef if the
>  character is disabled.
> 
> The only diff items for Issue 6 seem to be removing legacy items and
> fixing the description of nl and -nl in XCU/TC1/D6/37.
> 
> I only checked the HTML version, but the formatting and wording are the
> same for Issue 7 (HTML) and Issue 8 Draft 2.1 (PDF).
> 
> My naive interpretation of this is that, after loss of monospacing from
> POSIX.2 to SUSv1, at some point in Issue 6's creation, "" was
> taken to mean literal undef, i.e. italic undef, which is wrong,
> but makes sense since use of <>s is very common to mean
> "enclosed literal" or "literal symbol".
> 
> The fix would be to simply change italic undef on line 108235 (D2.1)
> to monospace  or bold .
> 
> Best,
> наб
> 
> Note: idk how well this mail will render on the Open Group webmail
>  interface due to its heavy use of <>;
>  I've been pestering the contact box to fix "injecting the literal
>  mail text as HTML" but this appears to be impossible.

Re: [1003.1(2016/18)/Issue7+TC2 0001546]: BREs: reserve \? \+ and \|

[Don Cragun via austin-group-l at The Open Group]

Hey Chris,
Note that the definition of "escape sequence" is being added in section 9.1 
(titled" Regular Expression Definitions") in the Base Definitions Volume of the 
standard; not in Section 3 where definitions go that pertain to the entire 
standard.

Cheers,
Don

> On Apr 7, 2022, at 4:05 PM, Christoph Anton Mitterer via austin-group-l at 
> The Open Group  wrote:
> 
> Hey.
> 
> btw: It should not be forgotten, that this definition of escape
> sequence/character, escaped character is in principle only valid for
> the RE chapter. Except of course, something explicitly refers to that
> or - like with sed - extends the RE language.
> 
> Just wanted to point that out again, to be kept in mind,... when the
> same terms are used in other places of the standard.
> 
> 
> Apart from that,... thanks Goff and Robert for your efforts in this
> issue.
> 
> 
> Cheers,
> Chris.
> 
> 

Re: wait and stopped jobs

[Don Cragun via austin-group-l at The Open Group]

> On Feb 28, 2022, at 3:20 PM, Robert Elz via austin-group-l at The Open Group 
>  wrote:
> 
>Date:Mon, 28 Feb 2022 16:12:03 -0500
>From:"Garrett Wollman via austin-group-l at The Open Group" 
> 
>Message-ID:  <25117.15011.846137.211...@khavrinen.csail.mit.edu>
> 
>  ... ... ...

> It is worth pointing out though that testing this in interactive shells
> is somewhat of a weird case - it should probably be done in a script
> instead.   That's because of the effect of the (implicit) jobs command
> before each PS1 output, that should have reported the background job as
> stopped when the Ctl-Z was sent, before the wait command is issued.  Normally
> each status change gets just one result from wait() - and the one from
> jobs counts (that's why in interactive shells there's no need, and no
> point since it doesn't work, issuing wait commands for known already completed
> background jobs ... they already no longer exist).   How all this interacts
> with stopped jobs and wait(1) (as distinct from waitpid(2) (or (3)) seems
> to be wholly unspecified - hence running these tests in a script, where
> there's no PS1 and hence no implicit jobs command is a much more reliable
> test methodology.

The requirement for the shell to keep the exit status for a known terminated 
process until wait is called is in section 2.9.3.1 of the shell command 
language (202x D2.1 P2338, L75542-75560):

2.9.3.1 Asynchronous Lists
If a command is terminated by the control operator  ('&'), the shell 
shall execute
the command asynchronously in a subshell. This means that the shell shall not 
wait for the
command to finish before executing the next command.

The format for running a command in the background is:
command1 & [command2 & ... ]

... ... ...

When an element of an asynchronous list (the portion of the list ended by an 
,
such as command1, above) is started by the shell, the process ID of the last 
command in the
asynchronous list element shall become known in the current shell execution 
environment; see
Section 2.12. This process ID shall remain known until:
1. The command terminates and the application waits for the process ID.
2. Another asynchronous list is invoked before "$!" (corresponding to the 
previous
asynchronous list) is expanded in the current execution environment.

The implementation need not retain more than the {CHILD_MAX} most recent 
entries in its list
of known process IDs in the current shell execution environment.

> 
> kre
> 

Re: [1003.1(2016/18)/Issue7+TC2 0001558]: require [^...] in addition to [!...] for bracket expression negation

[Don Cragun via austin-group-l at The Open Group]

Chris,
No, I hadn't.  Thanks for pointing it out.  The enhancement request has been 
reopened.

Cheers,
Don

> On Feb 17, 2022, at 8:28 PM, Christoph Anton Mitterer via austin-group-l at 
> The Open Group  wrote:
> 
> Don,
> 
> On Fri, 2022-02-18 at 04:04 +, Austin Group Bug Tracker via austin-
> group-l at The Open Group wrote:
>>  (0005678) Don Cragun (manager) - 2022-02-18 04:04
>>  https://www.austingroupbugs.net/view.php?id=1558#c5678 
>> -
>> - 
>> Withdrawn at the request of the submitter in
>> https://www.austingroupbugs.net/view.php?id=1558#c5677. 
> 
> You had seen, that mirabilos who commented in 0005677 was *not* the
> submitter?
> 
> 
> Cheers,
> Chris.
> 
> 

Re: [Issue 8 drafts 0001532]: "stty -g" output should not have to be split

[Don Cragun via austin-group-l at The Open Group]

Hi Wayne,
I don't see this being a real problem.  The current output of stty -g on the 
system I'm using is about 225 bytes.  The minimum acceptable value for LINE_MAX 
in the current standard is 2048 (including the terminating  
character).  I'm not expecting terminal settings to need an additional 1800 
bytes to be represented in text form in the foreseeable future.

- Don

> On Feb 11, 2022, at 7:43 PM, Wayne Pollock via austin-group-l at The Open 
> Group  wrote:
> 
> This is probably a silly question, but how long can the output of stty -g be?
> Because as worded ("line of text") the output would be limited to less than
> LINE_MAX.  If that might be an issue the wording could be changed to something
> like "string of characters terminated by a newline".
> 
> -- 
> Wayne Pollock
> 
> On Thu, Feb 3, 2022, 2:06 PM Austin Group Bug Tracker via austin-group-l at 
> The Open Group  > wrote:
> 
> >
> 

Re: sed and delimiters that are also special characters to REs

[Don Cragun via austin-group-l at The Open Group]

> On Jan 13, 2022, at 10:07 PM, Oğuz via austin-group-l at The Open Group 
>  wrote:
> 
> On Thu, Jan 13, 2022 at 6:36 PM Christoph Anton Mitterer
>  wrote:
>> And where does it say that? I mean in the standard.
>> I.e. where does it say, that parsing is only allowed to happen in one
>> stage from left to right, especially not only with respect to an RE
>> itself, but also when an RE is embedded in a command with delimiters.
> 
> It is what makes sense.
> 
>> 
>> Where does it say, whether:
>> s.[.].X.
>> is:
>> a) s/[/]/
>>   followed by X.
>> 
>> or:
>> 
>> b) s/[.]/X/
> 
> It is clearly `a'. The second period is not preceded by a backslash.

It isn't that simple.  The string between the first and second delimiters in a 
substitute command has to be an RE (and "[" can't be the end of an RE; it is 
the start of a bracket expression).  Note also that  is not special 
inside a bracket expression.  When you find the first delimiter, you can't look 
for the second delimiter in the middle of a bracket expression and you can't 
look for the second delimiter in the middle of a subexpression ("\("..."\)").

> 
>> Then, with a delimiter that is also a special character, the special
>> character would no longer be usable as such.
> 
> Again, that's why any character other than backslash and newline can
> be used as the delimiter. The standard makes an exception for when the
> special sequence `\n' is used with the `y' command, but that's all.
> The point of custom delimiters is to save users from the escaping
> hell. I don't think anyone put further thought into it.
> 
>> btw: busybox sed, behaves as you say:
>> $ printf '%s\n' '.' | busybox sed 's.\..X.'
>> X
>> $ printf '%s\n' 'a' | busybox sed 's.\..X.'
>> a
> 
> So do Solaris sed and Unixware sed. On the other hand, all seds I tested do 
> this
> 
>$ echo a | sed 's1(.)1\11'
>a
> 
> except GNU sed; it prints 1 instead. The behavior varies widely with
> other special characters and sequences, so the best standard
> developers could do is to state that the results are unspecified when
> they are used as delimiters; OR, leave the text as is, which already
> implies that it is unspecified.
> 
I agree that the wording:
"If the character designated by c appears
 following a , then it shall be
 considered to be that literal character,
 which shall not terminate the RE."
on 202x_d2.1 P3134, L106088-106090 is ambiguous and needs to be clarified.  I 
believe that it means that the escaped delimiter character is treated as the 
literal character (not as a delimiter) and that the listeral character is still 
treated as a special character if that character is special at that point in an 
RE.  But, I agree that it could be interpreted to mean that it is not only not 
a delimiter, but also not special.

I don't think I have ever worried about this wording because I just never use a 
special character as a delimiter if I'm going to have to escape that character 
in an RE.

Re: sed and delimiters that are also special characters to REs

[Don Cragun via austin-group-l at The Open Group]

> On Jan 10, 2022, at 2:15 PM, Christoph Anton Mitterer via austin-group-l at 
> The Open Group  wrote:
> 
> Hey.
> 
> Maybe someone can help me with this (or point out that it's undefined -
> and perhaps should be):
> 
> I was looking into when BREs/EREs are used with delimiters (i.e. sed's
> s- and y-commands) that are also special characters with respect to the
> respective RE syntax.
> 
> 
> There were some things which I found a bit ambiguous:
> 
> 
> 
> 
> 1) In "Addresses in sed" it says vaguely:
> "preceded and followed by a delimiter, usually a )"
> (line 106071, et seq., in the draft)
> 
> ... merely indicating that other characters may be used, but e.g. not
> clearly excluding (or including) '\' or newline - which are explicitly
> excluded as delimiters for the s- and y-commands.
> 
Hi Chris,
About 16 lines later in 202x_d2.1, in the subsection
titled "Regular Expressions in sed" on lines
106087-106091 it says:

  * In a context address, the construction "\cREc",
where c is any character other than 
or , shall be identical to "/RE/". If
the character designated by c appears following
a , then it shall be considered to be
that literal character, which shall not terminate
the RE. For example, in the context address
"\xabc\xdefx", the second x stands for itself, so
that the RE is "abcxdef".

Doesn't this answer your questions?

Cheers,
Don

> 2) Now with respect to delimiters that are special characters in the RE
> (using BREs here), either by themselves, (like '.') or when escaped via
> a preceding '\', like '\('.
> 
> The documentation says merely:
> "Within the RE and the replacement, the RE delimiter itself can be used
> as a literal character if it is preceded by a ."
> (line 106204, et seq., in the draft)
> 
> Maybe I'm just blind, but I couldn't find answers to the following:
> 
> 
> 
> a) Is it defined, how the unescaping of delimiters vs. special
> characters happens?
> Consider the following example:
> s(\\((X(
> There are at least two ways to parse that:
> - splitting up into RE an replacement parts first by going through the
>  string and looking for any (non-escaped) delimiter (here '('),
>  removing the escapes for any others, so one would end up with:
>  RE='\('
>  replacement='X'
> - going from left to right unescaping both at once, which would lead
>  here to:
>  RE='\\'
>  followed by two delimiters ((, followed by an unknown flag and thus
>  an error
>  (which is what implementations seem to do)
> 
> 
> b) There doesn't seem to be any way (or mentioning of that) to fully
> use a special characters as delimiter
> Consider e.g.:
> s.\..X.
> which is used as:
> s/./X/
> but it seems impossible to get:
> s/\./X/
> when using '.' as delimiter.
> 
> Same for a character that gets only special when escaped with '\':
> s(\((X(
> which is used as:
> s/(/X/
> but it seems impossible to get:
> s/\(/X/
> when using '(' as delimiter.
> (I know the above lacks a closing '\)' and something within the
> subexpression ... but that could be easily added.)
> 
> I found that e.g GNU an BusyBox' sed allow the following workaround:
> s.[.].X.
> which seems used as:
> s/[.]/X/
> 
> But again, at least to me the standard seems to be ambiguous with
> respect to how this should be parsed.
> While the bracket expression itself is defined to take the '.'
> literally, it nowhere seems to say that this is even to be seen as a
> '.' for the RE and not as the next delimiter.
> 
> 
> c) The documentation uses the phrase "literal character" for the
>   delimiter when being escaped with a '\' at least in these places:
> "If the character designated by c appears following a , then
> it shall be considered to be that literal character, which shall not
> terminate the RE."
> (line 106088, et seq., in the draft)
> 
> and
> 
> "Within the RE and the replacement, the RE delimiter itself can be used
> as a literal character if it is preceded by a ."
> (line 106204, et seq., in the draft)
> 
> Maybe it's worth to clarify this, that it it means literal with respect
> to the command and not being a delimiter for that, but that the
> character may not be literal with respect to the RE.
> 
> E.g. AFAIU:
> s.\..X.
> would be the same as:
> s/./X/
> and '.' would be used as special character.
> 
> 
> 
> 
> Any insights with respect to whether and what POSIX mandates in these
> points would be appreciated.
> 
> 
> Thanks,
> Chris.
> 

Re: [1003.1(2016/18)/Issue7+TC2 0001440]: Calling `system("-some-tool")` fails (although it is a valid `sh` command)

[Don Cragun via austin-group-l at The Open Group]

The standard does specify exceptions to the Utility Syntax Guidelines in the 
OPTIONS section of each utility that needs an exception.  For example, if you 
look at the start of the OPTIONS section in the description of the c99 utility, 
you will find:
The c99 utility shall conform to XBD Section
12.2 (on page 199), except that:
 • Options can be interspersed with operands.
 • The order of specifying the −L, −l, and −R
   options, and the order of specifying −l
   options with respect to pathname operands is
   significant.
 • Conforming applications shall specify each
   option separately; that is, grouping option
   letters (for example, −cO) need not be
   recognized by all implementations.

 - Don

> On Nov 1, 2021, at 2:12 PM, Wayne Pollock via austin-group-l at The Open 
> Group  wrote:
> 
>> ...
>> For context, the example was . nohup should be, and as 
>> far as I know, is required to support invocations as
>> , treating that first <--> as the end of the options. The GNU 
>> extension that options and the end-of-options indicator can also follow 
>> operands is as far as I know not supported by POSIX in the first place 
>> (Utility Syntax Guideline 9: "All options should precede operands on the 
>> command line.") 
> 
> The POSIX utility "c99" (hopefully renamed in issue 8) does have options that
> follow operands.  So Guideline 9 you quoted needs to be fixed to say "except 
> as noted".
> There are other such utilities too, although I don't think any in POSIX/SUS.  
> (For example
> Gnu ps and tar.)
> 
>> and accordingly disabled in GNU utilities when invoked in POSIX mode, so in 
>> any POSIX utility, if the arguments start with sh -c --, I think the fact 
>> that sh can only be interpreted as an operand already means that -c and -- 
>> must also be interpreted as operands.
> 
> I agree that's how it should be, but I am not certain the standard is clear on
> this.  As a non-expert I will leave matters to you and the other experts.  I 
> just
> appreciate the opportunity to raise my concerns occasionally on this list.
> 
> -- 
> Wayne Pollock
> 

Re: Adding %n$ conversions to the printf utility (printf(1))

[Don Cragun via austin-group-l at The Open Group]

> On Sep 10, 2021, at 3:29 PM, Joerg Schilling via austin-group-l at The Open 
> Group  wrote:
> 
> "Stephane Chazelas via austin-group-l at The Open Group" 
>  wrote:
> 
>> 2021-09-10 22:46:26 +0100, Stephane Chazelas via austin-group-l at The Open 
>> Group:
>> [...]
>>> I've personally used the feature to reorder items in sets, so
>>> would object to precluding reusing arguments.
>> [...]
> 
> I agree...
> 
>> As mentioned on that mailing list and it's still undocumented,
>> -r can be used with print -f to disable reuse:
> 
> The problem, POSIX defines printf(1) to not handle options. So I don't think, 
> we could add that.
> 
> Jörg

I think you're confusing the requirements for printf and echo.  The standard 
echo is not allowed to accept options. The standard printf does not define any 
options, but allows them as implementation extensions.

- Don

Re: utilities and write errors

[Don Cragun via austin-group-l at The Open Group]

> On Jun 29, 2021, at 3:20 PM, tg...@mirbsd.org via austin-group-l at The Open 
> Group  wrote:
> 
>> ... ... ...
> 
> Ah, so you meant…
> 
>> "/tmp cd ~ >& /dev/zero"
> 
> … which was not clear from the punctuation salad you posted.
> 
>> "Violates POSIX on the parse level?!"  Eek! Next
> 
> It does: >& is parsed as > & by POSIX rules, TTBOMK.

No.  According to XCU Section 2.7.6 (Duplicating an Output File Descriptor) 
redirection operators of the form:
[n]>&word
closes file descriptor n if word is "-" and opens file descriptor n as a 
duplicate of the file descriptor specified by word if word is a string of 
digits.

If n is not specified, standard output is closed or made to be a duplicate of 
the file descriptor specified by word.

If word (in this case an empty string) is not a string of digits and word is 
not "-", the results are explicitly unspecified.

Cheers,
Don

> 
> bye,
> //mirabilos
>  ... ... ...

Re: utilities and write errors

[Don Cragun via austin-group-l at The Open Group]

> On Jun 27, 2021, at 2:33 PM, tg...@mirbsd.org wrote:
> 
> Don Cragun via austin-group-l at The Open Group dixit:
> 
>>  • When an unrecoverable error condition is
>>encountered, the utility shall exit with a
>>non-zero exit status.

Please do not cherry pick what you quote from my message!
The lead in to the above quote was:
   The following shall apply to each utility, unless
   otherwise stated:
... ... ...

When the CONSEQUENCES OF ERRORS section of the specification of the pwd utility 
says:
If an error is detected, output shall not be
written to standard output, a diagnostic message
shall be written to standard error, and the exit
status is not zero.
the standard clearly states OTHERWISE for the behavior of that utility.  When 
the standard states OTHERWISE, the requirements following "The following shall 
apply to each utility, UNLESS OTHERWISE STATED" do not apply.
> 
> Is “pwd >/dev/full” an “unrecoverable error condition” as regards
> the pwd utility?
> 
> I think not. It validated the current working directory and attempted
> to hand the information over; what the user does with it is the user’s
> fault.

Note also that without options, the pwd utility does not validate the current 
working directory unless the PWD environment variable is not present, does not 
contain an absolute pathname, or contains a component of the pathname that is 
"." or "..".  Under normal circumstances, the pwd utility is just supposed to 
copy the contents of the PWD environment variable to stdout.

And, if /dev/full is not an existing file that is writable or the name of a 
file that can be successfully created by the user invoking pwd, then the shell 
will report an error and not even try to invoke pwd.

> 
> More context: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990265
> and https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=990264 (though
> they are pretty much the same information).

The pwd utility CONSEQUENCES OF ERRORS quote from the standard is also noted in 
both of the above references.  And, I agree with Vincent in those references 
when he says that when using the pwd built-in utility "one would expect a 
diagnostic message (and I think that one should also get one for the other 
builtins)".

Sincerely,
Don

> 
> Thanks in advance,
> //mirabilos
> -- 
> Gestern Nacht ist mein IRC-Netzwerk explodiert. Ich hatte nicht damit
> gerechnet, darum bin ich blutverschmiert… wer konnte ahnen, daß SIE so
> reagier’n… gestern Nacht ist mein IRC-Netzwerk explodiert~~~
>   (as of 2021-06-15 The MirOS Project temporarily reconvenes on OFTC)
> 

Re: utilities and write errors

[Don Cragun via austin-group-l at The Open Group]

Section 1.4 of the Commands and Utilities Volume of the standard (Utility 
Description Defaults) already does this in the CONSEQUENCES OF ERRORS section 
on P2303-2304, L74122-74151 in P1003.1-202x Draft 2 and on P2341-2342, 
L74540-74569 in P1003.1-2017.  Note especially:
The following shall apply to each utility, unless
otherwise stated:
... ... ...
• When an unrecoverable error condition is
  encountered, the utility shall exit with a
  non-zero exit status.
• A diagnostic message shall be written to
  standard error whenever an error condition
  occurs.

Cheers,
Don Cragun, IEEE PASC OR

> On Jun 24, 2021, at 12:18 PM, Vincent Lefevre via austin-group-l at The Open 
> Group  wrote:
> 
> Shouldn't POSIX require that when a utility cannot write its output
> (e.g. due to ENOSPC or EPIPE), it shall be regarded as an error, i.e.
> the exit status be non zero, with a diagnostic message if possible?
> 
> -- 
> Vincent Lefèvre  - Web: <https://www.vinc17.net/>
> 100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
> Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
> 

Re: Did I ever say how much I detest mantis...

[Don Cragun via austin-group-l at The Open Group]

Hi again,
The Section and Page Number sections have also been fixed.

 - Don

> On Aug 13, 2020, at 1:08 PM, Robert Elz via austin-group-l at The Open Group 
>  wrote:
> 
> When I first submitted issue 1389, I had forgotten to fill in
> the "Category" field (select "shell and Utilities" from the
> drop down menu.
> 
> So after getting the error because of that (that was fine) I
> used my browser's "back" button to return to the form, filled
> in the missing field, and submitted again.
> 
> I didn't notice that (for some unknowable reason) half the other
> fields had returned to their default values.Why?
> 
> In any case 1389 is supposed to be an Severity:"Objection" and
> Type:"Error".
> 
> Clearly my name is not "Your Name Here"!   (Seeing that value in that
> field should result in rejection of the submission - if that had happened
> I would almost certainly have noticed everything else that changed as well).
> 
> The section numbers were intended to be:
> 2.9.1.1 1.c (2.9.1.4 1.c in Issue8 Draft 1), 2.9.1.1 1.d, 2.9.1.1 1.e.i.a
> 
> And the page numbers
> 2367 (2335 in Issue8 draft 1)
> 
> For some unknown reason it retained the line numbers.
> 
> Can someone correct it please?
> 
> Thanks,
> 
> kre
> 

Re: Did I ever say how much I detest mantis...

[Don Cragun via austin-group-l at The Open Group]

Hi Robert,
The Severity, Type, and Name fields have all been corrected on bugid:1389 and 
all three bugs have been related to each other.

 - Don

> On Aug 13, 2020, at 1:08 PM, Robert Elz via austin-group-l at The Open Group 
>  wrote:
> 
> When I first submitted issue 1389, I had forgotten to fill in
> the "Category" field (select "shell and Utilities" from the
> drop down menu.
> 
> So after getting the error because of that (that was fine) I
> used my browser's "back" button to return to the form, filled
> in the missing field, and submitted again.
> 
> I didn't notice that (for some unknowable reason) half the other
> fields had returned to their default values.Why?
> 
> In any case 1389 is supposed to be an Severity:"Objection" and
> Type:"Error".
> 
> Clearly my name is not "Your Name Here"!   (Seeing that value in that
> field should result in rejection of the submission - if that had happened
> I would almost certainly have noticed everything else that changed as well).
> 
> The section numbers were intended to be:
> 2.9.1.1 1.c (2.9.1.4 1.c in Issue8 Draft 1), 2.9.1.1 1.d, 2.9.1.1 1.e.i.a
> 
> And the page numbers
> 2367 (2335 in Issue8 draft 1)
> 
> For some unknown reason it retained the line numbers.
> 
> Can someone correct it please?
> 
> Thanks,
> 
> kre
> 

Re: [1003.1(2016)/Issue7+TC2 0001345]: date(1) default format

[Don Cragun]

As the IEEE PASC OR, I also agree with Geoff's statements.

Sincerely,
Don Cragun

> On Jul 13, 2020, at 4:02 AM, Eric Blake  wrote:
> 
> On 7/13/20 4:07 AM, Geoff Clare wrote:
>> J William Piggott  wrote, on 12 Jul 2020:
>>> 
>>> On Mon, 6 Jul 2020, Geoff Clare wrote:
>>> 
>>>> There is no way we are going to change the required d_t_fmt value for
>>>> the POSIX locale.
>>> 
>>> Why?
>> Because every implementation would have to change, and all applications
>> that rely on the current value would potentially break (depending on
>> which fields they use).
>> We would need a very good reason to make such a change.
>>> Has it been discussed with 'we'? Would any of them like to comment on
>>> this please?
>> I've been using "we" to refer to the whole Austin Group, i.e. everyone
>> on this mailing list, so yes it has been discussed (in this thread).
>> The three people who's opinions matter are the organisational
>> representatives who would vote on it if it came to that.  I'm sure if
>> any of them disagree with what I've said they will comment.
> 
> To make matters clearer, as one of the organisational reps (Open Group), I'm 
> of the mind that mandating a change to existing practice is undesirable.  
> Standardizing a new format value (especially if there is existing practice to 
> copy from) or adding better documentation to make it clear about the 
> intentional differences between strftime vs. date are both less invasive than 
> a mandatory change to the contents of an existing format value.  So I'm 
> concurring with Geoff's handling of the responses so far.
> 
> -- 
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.   +1-919-301-3226
> Virtualization:  qemu.org | libvirt.org

Re: sh: aliases in command substitutions

[Don Cragun]

The base shell for the POSIX.2-1992 standard was ksh88 with a few cases where 
David Korn said that ksh88 didn't work properly and he had fixed it in the test 
shell update he had in the lab (which eventually became ksh93).

In a few cases we also allowed either Bourne shell or Korn shell behavior (on 
David's advice where he thought Bourne sh compatibility  had been 
unintentionally broken by ksh88).  I don't remember what those cases were at 
this point.  (After all this was in a meeting a little over 30 years ago.)

We didn't have David for nearly as much interaction as we would have liked.  We 
just had him for about 2 days in a huge conference room in the Waikiki Hilton 
during one of our working group sessions.  So there are several parts of the 
shell details that I'm sure he never reviewed.  It would have been nice if he 
had done that so we could have avoided casting mistakes into stone in the form 
of the standard.

-- Don

> On Apr 21, 2020, at 5:42 AM, Robert Elz  wrote:
> 
>Date:Tue, 21 Apr 2020 13:59:47 +0200
>From:Joerg Schilling 
>Message-ID:  
> <5e9ee033.AKUL4ZOBGnR/szdz%joerg.schill...@fokus.fraunhofer.de>
> 
>  | If I am right, then this text is from a time before ksh93 exists.
> 
> Probably not, it was an interpretation of the 1992 standard, and so
> was probably not earlier than 1993 (I see some 1996 copyrights, but wjether
> that's for the specific interpretation or just the collection I don't know).
> 
> And in any case:
> 
>  | So ksh88 was the current shell and the Bourne Shell may have
>  | been considered "historic".
> 
> Since that didn't support aliases at all, changing the rules for processing
> aliases in double quoted $() command substitutions (which the historic Bourne
> shell also didn't support) based upon what those shells didn't do would have
> been kind of pointless.
> 
> Much more likely it was based upon old versions of ksh, which did support
> aliases, and which did (try to) detect the end of a $() cmd sub by parenthesis
> counting.
> 
> kre

Re: Mantis project for Issue8 bug reports?

[Don Cragun]

Hi Robert,
Please find comments mixed in below...

> On Mar 29, 2020, at 1:14 PM, Robert Elz  wrote:
> 
>Date:Sun, 29 Mar 2020 12:33:04 -0700
>From:    Don Cragun 
>Message-ID:  
> 
>  | Note that I'm making all of this up as I write it here.
> 
> Yes, understood.
> 
>  | I think I answered your question indirectly...  To get the Category values
>  | to match I think you need to use the same Project,
> 
> I think we have something of a breakdown in terminology, I was treating
> Categroy as what one would put in the field labelled that way in the
> "report an issue" form, ie: one of "Base Definition and Headers", "Front
> Matter", "Rationale", ...(which is an easy choice to make, it is the
> same as the original bug).   The "Project" here all has the same set of
> Categories, it is a matter of which version of the Project to select
> (as in Issue7-TC1 vs Issue7-TC2) not "Aadrvark Bugs" etc.
> 
>  | but you may be right in assuming that two or more projects have the
>  | same Category values.
> 
> Don't all of the ones that are about the (at least recent) POSIX standard
> (as opposed to other stuff) have the exact same categories?

Yes.  I'm sorry.  I was looking at the contents of the Category field when it 
is in the Viewing Issues display when Project is set to "All Projects".  In 
that case the display field labeled "Category" contains both the Project and 
the Category that were filled in when a bug is created.

> 
>  | The idea I was shooting for was to have the original bug and the new bug
>  | you are creating in the same oroject and category.
> 
> OK, in that case, the new bug should reference TC1 - TC2 didn't exist when
> the bug in question was filed.
> 
> So it is still acceptable to file bug reports against TC1 ?   Even though
> it is obsolete and replaced by TC2?

You can file bugs against any version of the standards documents that is listed 
in the menu of Projects.  Uf a bug against an old version of the standard still 
needs to be fixed in the current version of the standard, we can deal with the 
old page and line numbers.  If the text in the old standard has already been 
changed in the current standard, we'll probably either close it as a duplicate 
or ask for a new bug addressing the desired changes relative to the wording in 
the current standard.

> 
>  | (You might also consider adding a New Relationship making the new bug
>  | a child of the bug you are asking us to reconsider.)
> 
> Sorry, I have absolutely no idea how one would do that, there's nothing
> in the form that suggests of anything like that that I can see.
> 
> Perhaps that's an option open only to admin (or manager, or whatever)
> category users?
> 
> But yes, this new one should certainly be listed as linked to the
> original.   I assumed that would happen by someone able to do it, using
> whatever magic is usually used to link related bugs.

When you look at the Mantis bug report (at least for managers) there is a 
section between the Desired Action and the Notes that lists Relationships.  At 
least on the view managers see, the first line of that section allows one to 
add a new relationship.

If you look at bugid:1318 you should find two relationships in that section of 
the Mantis bug report.  Similarly, I added a reationship in the Mantis 
bugid:1329 that was filed today by Olaf 'Rhialto' Seibert.  I assume that this 
discussion started to get suggestons on how to file 1328.

> 
>  | There are cases where the bugnote resolving an old bug says that
>  | the line numbers given in that resolution are using line numbers
>  | from a different version than the line numbers used in the original bug.
> 
> That's not an issue here, this was a very little discussed bug, there are
> no (relevant) notes, the text added comes directly from the Desired Action.
> There is no bugnote that specifies anything, or even discusses anything
> (there is one administrative type note).
> 
> It seems to have been non-controversial (it is old) - at least one of the
> problems will be even less controversial (it is much like as if the text
> said that 1+2=4 when it should say 1+2=3 ... while more subtle than that,
> which is why it wasn't detected - I didn't notice it when I skimmed the
> report when summarising it for the NetBSD list - it is that obvious once
> pointed out.)The other issue, the bigger problem, looks to be an issue
> that no-one ever thought of, it is simply assumed what the answer will be,
> but unfortunately the example given either has the same problem as the
> previous one (which is likely) or is assuming the less likely answer.
> That makes things 

Re: Mantis project for Issue8 bug reports?

[Don Cragun]

Note that I'm making all of this up as I write it here.  There aren't any 
documented rules/guidelines for bugs filed against Applied bugs that have not 
yet been Approved by the three member organizations of the Austin Group.  What 
I have suggested in my last post and expanding upon here is what I would like 
to see in a bug report so that I can look at the old and new bug reports and 
figure out which part of the resolution is broken and where the new suggestion 
is to be placed.

I think I answered your question indirectly...  To get the Category values to 
match I think you need to use the same Project, but you may be right in 
assuming that two or more projects have the same Category values.  The idea I 
was shooting for was to have the original bug and the new bug you are creating 
in the same oroject and category.  That will make it easier for people looking 
at a project to find related bugs.  (You might also consider adding a New 
Relationship making the new bug a child of the bug you are asking us to 
reconsider.)

There are cases where the bugnote resolving an old bug says that the line 
numbers given in that resolution are using line numbers from a different 
version than the line numbers used in the original bug.  The important thing is 
that when referencing page and line numbers in a resolution to a bugid, you use 
the same version of the standard for page an line numbers in your new 
Descriptions and Desired Actions.  We want the page and line numbers in your 
new bug to match the page and line numbers used in the resolution of the old 
bug.

The only time you want to use different page and line numbers than what are 
mentioned in the original bug or in its resolutiohn is if some wording that was 
in the resolution has been modified by a more recent version of the standard.  
(This can happen if text changes in a TC that modifies something that can't be 
applied until the next Revision of the standard.)

Cheers,
Don

> On Mar 29, 2020, at 11:38 AM, Robert Elz  wrote:
> 
>Date:Sun, 29 Mar 2020 10:19:58 -0700
>From:Don Cragun 
>Message-ID:  <0c9446a8-1e1e-4c15-ab8b-68d7939ce...@sonic.net>
> 
>  | You can't file a bug agains 1003.1(20xx)/Issue 8 because there
>  | is no such thing until [...]
> 
> Yes, that's what I suspecxted.
> 
>  | If you believe that there is a problem in the resolution of a bug
> 
> There definitely is.When you see it, no-one will disagree (of course
> this assumes that the text edited in is what the instrustions in the bug
> report say should be done ... this was not one of the cases where the
> "applied" message says something like "except I changed...")
> 
>  | that you can no longer add a comment to, I would suggest creating a
>  | new bug as follows:
> 
> Thanks, I will forward this, but you omitted the most important
> of the fields, the Project.   The original bug was submitted against
> 1003.1(2013)/Issue7+TC1 (but tagged for Issue8, so it wasn't applied
> in TC2).   However, I believe it is no longer appropriate to submit
> bug reports against TC1 is it - it is also not really appropriate to
> submit a bug report against TC2, as the relevant text does not exist
> there.
> 
> Maybe we need a project for bug reports against closed bug reports
> (perhaps just ones not yet present in a published standard) ?
> 
>  | 4.  In the Desired Action tell us exactly what text in that bug note needs 
> to be changed and give us new text that you believe will solve the problem.
> 
> Part of the problem with this one, is that it isn't clear to me (it might
> be to the person who found the problem, this one isn't really my area, ie:
> it isn't a shell issue...) what the solution should be, so if I were to
> submit the bug report, I wouldn't know what to do there, this is more a
> "there is a problem here that hasn't been considered, what do we do?" type
> issue.
> 
> Note that in the implementations that exist that might be answered (they
> might have selected one of the possibilities, perhaps not even realising
> that there is a conflict) I have no idea.
> 
>  | Hope this helps,
> 
> If does, and if you could clear up the Project question, that would be
> enough for a bug report to get filed (after being advised of the issue,
> it is 100% clear to me that there are problems that have to be corrected).
> 
> kre
> 

Re: Mantis project for Issue8 bug reports?

[Don Cragun]

You can't file a bug agains 1003.1(20xx)/Issue 8 because there is no such thing 
until it is approve by the balloting group at IEEE, ISO, and The Open Group.  
Trying to file bugs against a draft of the standard would lead to hundreds of 
different Category values (and a hundreds of large PDF files we would all have 
to keep in case a bug was filed against it).  Note that each note that you see 
in the mailing list that has Status Applied could indicate that a new draft has 
been created.

Despite Mark's belief that we should all be able to get a new draft whenever we 
want one, trying to keep track of what draft we're talking about in a situation 
like that would be horrendous.  (Nonetheless, it would be nice to be able to 
look at the current status of the draft to make it easier to see how all of the 
changes for related bugs fit together.)

If you believe that there is a problem in the resolution of a bug that you can 
no longer add a comment to, I would suggest creating a new bug as follows:
1.  Use the same Category as was in the bug that has been resolved.
2.  In the Summary say something like "Problem in resolution of bugid:" 
(where "bugid:" is literal text and "" is replaced by the bug number that 
you believe was incorrectly resolved).
3.  In the Description say what you believe the problem is in the resolution to 
bugid: as specified in bugnote: (where ) is the number of the note 
in that bug that you believe should change.
4.  In the Desired Action tell us exactly what text in that bug note needs to 
be changed and give us new text that you believe will solve the problem.
5.  If the changes needed are about new text that was added in the resolution 
you can leave the Page Number and Line Number fields blank and in the 
Description say something like paragraph added after Paaa, Lbbb-ccc (where aaa, 
bbb, and ccc specify where the text was added relative to the version of the 
standard given in the Category.  If the changes needed are about replacement 
text for existing text in the old standard, specify the page and line numbers 
in that standard that you believe were incorrectly resolved.

The above suggestion is a mild abuse of Mantis, but it should allow you to do 
what you need to do and shoula allow us to figure out exactly what changes you 
think were incorrectly resolved.

Hope this helps,
Don

> On Mar 29, 2020, at 8:47 AM, Robert Elz  wrote:
> 
> Is it time already for a new Project to be added to Mantis in
> which to file bugs on the new text which is not in Issue7, but
> which has been added (bug report closed and marked "Applied") ?
> 
> I have been sending breif summaries of what is being changed for
> Issue8 to the NetBSD mailing list (for changes I don't already
> know are irrelevant to NetBSD - eg: I didn't bothed with the 'i'
> (ignore case) flag added to sed 's' commands as I know we already
> have that) so people can be aware of things that might need to
> change (for some the answer is "nothing needed" in that I just
> didn't know that NetBSD already did whatever).
> 
> One of our developers/users found some (real) errors in one of the
> new applied sections of text (and no, not one I have complained
> about here...)   One of the errors is fairly simple, and could probably
> be fixed editorially by just sending a message to this list, but the
> other is a real issue that is going to need extra definition, and
> a decision (there are two ways it could be resolved, and they are
> quite different, and I personally have no idea which is right).
> 
> I know I'm being obscure - that's because I'm leaving it for the
> person who noticed the problem to report it.
> 
> So the actual issue isn't relevant to this message - but the correct
> form to use in Mantis to report a bug in text that has been edited
> into what will be Issue8 is unclear to me.   Is it still Issue7-TC2 ?
> The relevant text with the problem doesn't appear in it however (so
> there are no page/line numbers, etc).
> 
> kre
> 

Re: XCU: 'return' from subshell

[Don Cragun]

Would this issue be resolved if we change the last sentence of the description 
section of the return Special Built-In Utility from:
If the shell is not currently executing a function
or dot script, the results are unspecified.
to:
If the shell is not currently executing a function
or dot script running in the same shell execution
environment as the command that invoked the function
or dot script, the results are unspecified.
?

Dirk, notice that I said "in the same shell execution environment as the 
command that invoked ..."; not "in the same shell execution environment as the 
function's defining command".  If a function is invoked in a subshell of the 
environment that defined the function that should not affect the behavior of 
return as long as the function doesn't doesn't invoke return in a new shell 
execution environment that it created.)

Since current shells do not all treat return in a subshell as exit, I think we 
should leave that behavior unspecified.  (I see no reason why a conforming 
shell should not be able to report that return was invoked in a shell execution 
environment that is not current execution environment as the command that 
invoked the function or dot script.)

Cheers,
Don

> On Mar 11, 2020, at 10:07 AM, Dirk Fieldhouse  wrote:
> 
> On 11/03/20 15:23, Chet Ramey wrote:
>> ...>
>> What does a `return from the execution environment' mean, exactly? ...
> 
> To clarify, what I wrote was shorthand for "return from the function if
> the 'return' is executed in the same execution environment as" the
> function's defining command, or otherwise (ii) exit or (iii) unspecified
> behaviour.
> 
> So I think your agreement with case (i) means that you would be looking
> for some text like that in my originally (10/03/20 15:22) proposed
> Resolution (a), and therefore (with the clarification above) a
> definition of when a 'return' is "in" a function for case (i) that would
> make it equivalent to case (iii).
> 
> Then the remaining question is whether (case (ii)) there is benefit in
> codifying, normatively or not, the 'exit'-like behaviour when a 'return'
> that is textually contained in a function definition is run in a
> 'separate execution environment' from the function's defining command,
> rather than leaving all other cases unspecified.
> 
> The case (naively) of 'return' used textually outside a function
> definition would be covered by existing 2.14 text "If the shell is not
> currently executing a function or dot script", except that it is widely
> held that "not currently executing a function" includes the situation of
> the previous paragraph. Whatever the wording, it appears controversial
> whether the 'exit'-like behaviour should apply here.
> 
> Without wishing to extend the scope of discussion, the model against
> which readers are likely to interpret the standard's text on functions
> is that of a programming language like C, where a function foo() is said
> to be executing while any statements textually contained in the function
> are executing, and possibly more than once if it performed a successful
> fork(). Any revised wording needs to invalidate such a mental model, if
> that is what the standard intends.
> 
> /df
> 
> --
> London SW6
> UK
> 

Re: awk: FS matching 0 or more characters

[Don Cragun]

Hi Martijn,
In the description of REs in the standard, "match" is described (on
P181-182, L5969-5993 in the 2017 edition of the standar) as:
"A sequence of zero or more characters shall be said to
be matched by a BRE or ERE when the characters in the
sequence correspond to a sequence of characters defined by
the pattern.

"Matching shall be based on the bit pattern used for encoding
the character, not on the graphic representation of the
character. This means that if a character set contains two
or more encodings for a graphic symbol, or if the strings
searched contain text encoded in more than one codeset, no
attempt is made to search for any other representation of
the encoded symbol. If that is required, the user can
specify equivalence classes containing all variations of
the desired graphic symbol.

"The search for a matching sequence starts at the beginning
of a string and stops when the first sequence matching the
*   ``begins earliest in the string’’. If the pattern permits
*   a variable number of matching characters and thus there is
*   more than one such sequence starting at that point, the
*   longest such sequence is matched. For example, the BRE
"bb*" matches the second to fourth characters of the
string "abbbc", and the ERE "(wee|week)(knights|night)"
matches all ten characters of the string "weeknights".

*   "Consistent with the whole match being the longest of the
*   leftmost matches, each subpattern, from left to right,
*   shall match the longest possible string. For this purpose,
*   a null string shall be considered to be longer than no
*   match at all. For example, matching the BRE "\(.*\).*"
*   against "abcdef", the subexpression "(\1)" is "abcdef",
*   and matching the BRE "\(a*\)*" against "bc", the
*   subexpression "(\1)" is the null string.

"When a multi-character collating element in a bracket
expression (see Section 9.3.5, on page 184) is involved,
the longest sequence shall be measured in characters
consumed from the string to be matched; that is, the
collating element counts not as one element, but as the
number of characters it matches."

Noting the part of this definition that is on lines shown above
with a leading asterisk, I believe the standard is clear and
that Busybox awk does not conform.

Hope this helps,
Don

> On Feb 3, 2020, at 1:50 PM, Martijn Dekker  wrote:
> 
> Consider:
> 
> echo 'one!two!!three!!!end' | awk -v 'FS=!*' \
>   '{ for (i=NF; i>0; i--) print $i; }'
> 
> Onetrueawk, mawk, GNU awk, and Solaris awk all print:
> 
>> end
>> three
>> two
>> one
> 
> However, Busybox awk prints:
> 
>> d
>> n
>> e
>> e
>> e
>> r
>> h
>> t
>> o
>> w
>> t
>> e
>> n
>> o
> 
> In a way, the Busybox awk behaviour makes more sense. The "!*" ERE means: 
> match zero or more "!", and that's exactly what it did.
> 
> Changing the ERE to '!+' makes all awks behave consistently, so that's the 
> obvious fix.
> 
> But what, if anything, does POSIX have to say about an FS ERE matching zero 
> or more characters?
> 
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html#tag_20_06_13_04
> 
> I can only find:
> 
>> 1. If FS is a null string, the behavior is unspecified.
> 
> That doesn't really apply; FS is a non-null ERE, though one that may match 
> the null string.
> 
>> 3. [...] Each occurrence of a sequence matching the extended regular
>> expression shall delimit fields.
> 
> Is a null string matching the ERE a "sequence" that matches it?
> 
> So at this point I'm not sure whether to report a bug in Busybox awk, or an 
> area in the standard that needs further specification or clarification, or 
> neither...
> 
> - Martijn
> 
> -- 
> modernish -- harness the shell
> https://github.com/modernish/modernish
> 

Re: A question on file flags after fork

[Don Cragun]

Hi Danny and Ronald,
The austin-group-l e-mail list is a perfectly fine place to discuss the 
contents and interpretation of the current standard as well as the development 
of future revisions and amendments to the standard.  Places like 
unix.stackexchange.com and unix.com are fine places to go when you have 
questions about how to do something on BSD, Linux, and UNIX systems especially 
when discussing features that have not been standardized (jand frequently vary 
considerably in various versions of those operating systems, and, to a lesser 
degree even in various releases of a given version operating systems).

Any time you want an official interpretation of the standard, this is the place 
to come.

Hi Ronald,
The terms "file descriptor" and "file description" are both taken directly fro 
historic UNIX system documentation; neither was invented by developers of the 
standard.  Historic UNIX system man pages from 40 years ago seldom spelled out 
the relationship between those two terms as well of the current standard does.  
For example, the first two paragraphs of the DESCRIPTION section of the open() 
function man page (from page 1408, lines 46754-46762 in the 2017 edition of the 
standard):

The open() function shall establish the connection between
a file and a file descriptor. It shall create an open file
description that refers to a file and a file descriptor that
refers to that open file description. The file descriptor is
used by other I/O functions to refer to that file. The path
argument points to a pathname naming the file.

The open() function shall return a file descriptor for the
named file, allocated as described in Section 2.14 (on page
549). The open file description is new, and therefore the
file descriptor shall not share it with any other process in
the system. The FD_CLOEXEC file descriptor flag associated
with the new file descriptor shall be cleared unless the
O_CLOEXEC flag is set in oflag.

tries to clearly specify the relationship between the file descriptor and the 
file description that are created by a successful call to open().  Note also 
that the magic phrase "as described in" is almost always followed by a 
hyperlink in the PDF and in the HTML versions of the standard that you can 
click on to see that description.

We all realize that there are a lot of defined terms that you need to 
understand while reading the standard and that until you are used to the level 
of detail specified, it is easy to get confused.  When you use the standard 
frequently you get used to the terms used and learn where to find the 
definitions of those terms.  (Look especially at chapter 2 of the System 
Interfaces and Headers volume of the standard and at chapter 3 of the Base 
Definitions volume of the standard.  If you understand the concepts and 
definitions in those two places, many of your questions will have been 
answered.)

I hope this helps and look forward to seeing more of your questions in the 
future.

Sincerely,
Don Cragun
IEEE Portable Applications Standards Committee organizational representative to 
The Austin Group

> On Jan 13, 2020, at 11:25 PM, Ronald F. Guilmette  
> wrote:
> 
> In message , 
> Danny Niu wrote:
> 
>> Anyway, this mailing list should focus on **standard development**, 
>> questions like this of yours should go to places like unix.stackexchange.com.
> 
> I can only thank you for your kindness, grace, generosity, and understanding.
> I'm sure that these will all inspire others more worthy than myself to wish
> to contribute meaningfully to the ongoing standardization effort.
> 
> Regards,
> rfg
> 

Re: What do letters for symlink options (-P -L -H) stand for?

[Don Cragun]

Hi Danny,
I apologize for any confusion I have caused about XPG
versus SUS numbering.  I should know better than to try
to respond to technical e-mails just before going to bed
after midnight.

Hi Nick & Geoff,
Thank you for correcting and amplifying what I said in my
first response on this topic.

Hi Jörg,
You might have noticed that Solaris systems don't have a
/usr/xpg5/bin even though Solaris 10 and later releases
have conformed to XPG4, SUSv1, and SUSv2.  This was
because there weren't any changes in Issue 4V2 or in Issue
5 that could not be satisfied by updating utilities in
/usr/xpg4/bin in ways that would not break any applications
that were using the standard's description of those
utilities in Issue 4.  See the Oracle Solaris release 11.4
standards(7) man page for details on which Solaris releases
first conformed to the various X/Open Portability Guides
and POSIX standards.  That man page also tells you which
directories have to be included in $PATH to get utilities
conforming to the various standards.  The Solaris standards(7)
man page can be found here:
https://docs.oracle.com/cd/E88353_01/html/E37853/standards-7.html#REFMAN7standards-7

> On Nov 22, 2019, at 6:21 AM, Nick Stoughton  wrote:
> 
> BTW Danny, in  case you don't have a copy of the PDF downloaded, you can find 
> the information Don highlighted on the HTML version at 
> https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap03.html
> 
> And for completeness, that page states:
> 
> -L => Logical
> -H => Half Logical
> -P => Physical
> 
> On Fri, Nov 22, 2019 at 5:08 AM Joerg Schilling 
>  wrote:
> Geoff Clare  wrote:
> 
> > > > Issue 7 = SUSv4
> > > > Issue 6 = SUSv3
> > > > Issue 5 = SUSv2
> > > > Issue 4V2 = SUSv1
> > > > Issue 4 = XPG4
> > > > Issue 3 = XPG3
> > > > Issue 2 = XPG2
> > > > Issue 1 = XPG1
> > > 
> > > Interesting, could you please help with xpg6? /usr/xpg6/bin is used for 
> > > binaries on Solaris that are P.2001 compatible IIRC...
> >
> > That was Sun's choice to use the same naming convention as /usr/xpg4/bin
> > with the 4 and 6 being the Issue numbers.  The last X/Open Portability
> > Guide was XPG4.
> 
> OK, then the current standard would result in /usr/xpg7/* and I understand 
> the 
> background.
> 
> Jörg
> 
> -- 
>  EMail:jo...@schily.net(home) Jörg Schilling D-13353 
> Berlin
> joerg.schill...@fokus.fraunhofer.de (work) Blog: 
> http://schily.blogspot.com/
>  URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'
> 

Re: What do letters for symlink options (-P -L -H) stand for?

[Don Cragun]

Hi Danny,
Look at the rationale for the definition of symbolic
links in section A.3 Definitions in the Rationale
volume of the standard.  In the 2018 edition it starts
on page 3503 line 118500 and continues through the end
of page 3507 at line 118685.  Table A-1 and the end of
the last page answers your questions about which
utilities (at least those required by the standard)
have options specifying how symbolic links are to be
handled).  I think the earlier discussion in that
section answers the rest of your questions.

Note that the 2018 edition of the standard is SUSv7;
not SUSv4.  But I don't think that matters for this
discussion as long as you really are talking about the
2018 edition of the standards.

If after reading that rationale section you still have
questions, feel free to post them here and we'll try
to help.

Hope this helps,
Don

> On Nov 21, 2019, at 11:47 PM, Danny Niu  wrote:
> 
> I posted a question to unix.stackexchange.com, 
> but answers I got so far aren't quite satisfactory, as
> they are hardly backed by references. 
> 
> I hope expertized folks on this mailing list may have a 
> better answer for this question << EOF
> 
> I've summarized a list of commands that accepts 
> symbolic link options according to SUSv4-2018ed:
> 
> cd chgrp chown chmod cp find ln ls pax rm
> 
> The full list also includes their defaults and 
> other related options supported (such as -h and -d), 
> and I stored it on my HDD for reference.
> 
> I've previously seen (GNU documents if I was correct) 
> referring to -P -L options as "physical" and "logical" respectively, 
> and I think that's probably where the option letters come from, 
> but the latest docs as of Nov 2019 refer to them 
> as "--no-dereference" and "--dereference" now.
> 
> My question is: where do -P -L -H come from? 
> Is it SUS, XPG, POSIX, SVID, or vendor documentation? 
> And what do they initially stand for?
> 
> EOF

Re: The case against ctermid()

[Don Cragun]

Hello Marc,
Please find comments interspersed below...

> On Nov 9, 2019, at 9:51 PM, Marc Aurèle La France  wrote:
> 
> Greetings.
> 
> The specification for ctermid() doesn't define any error conditions.  Yes, the
> function is allowed to return an empty string (instead of NULL) if the
> controlling terminal name cannot be determined.  But it cannot set errno to
> indicate a more precise error condition.  Thus, for portability reasons,
> implementations are functionally encouraged to avoid this situation by always
> successfully returning a pathname that is guaranteed to exist (but not
> necessarily be openable).  The standard currently requires this pathname to be
> /dev/tty, but the actual value doesn't really matter to the arguement below.

The standard does not require the pathname /dev/tty to be the name of the 
controlling terminal for every process.  Not all processes have a controlling 
terminal.  Background jobs do not have a controlling terminal.  A process that 
has just called setsid() and not yet attempted to assign a controlling terminal 
for the new session does not have a controlling terminal.

> 
> Aside from the dubious usefulness of, in effect, requiring the return of a
> string constant, there is no guarantee any buffer provided by the caller is
> large enough to hold this value, leading to buffer overruns.

The standard requires that the header  define the symbolic constant 
L_ctermid with the length of a buffer (including the terminating NUL) that is 
long enough to hold the name of any controlling terminal on that 
implementation.  If an application chooses to ignore this defined value and 
create buffers that are shorter than that defined value, then there can be 
overruns; but that is not because the standard does not define a safe way to 
allocate a buffer for ctermid().

> 
> And so, via ctermid(), the standard mandates the implementation of a security
> hole.

No.  It does not.

> 
> In my view, ctermid() is completely useless and unfixable, as functional
> changes to its specification would break existing code.
> 
> Desired action:
> Either remove ctermid() (as was previously done to cuserid()) and any
> references to it, or mark it as obsolete and scheduled for future removal.
> 
> Here, coming up with a better replacement for ctermid() is considered a
> separate endeavour.

We don't generally consider this to be a separate endeavor.  When removing an 
interface that applications are using, we almost always require a replacement 
to be made available to application writers before removing an interface or 
marking it deprecated.

> 
> Marc.
> 

Cheers,
Don

Re: [1003.1(2013)/Issue7+TC1 0000981]: Remove oudated set -o nolog

[Don Cragun]

I don't remember whether the type utility was invented by POSIX
or adopted from ksh (88 or an early version of 93), but I believe
the DESCRIPTION section for type:

The type utility shall indicate how each argument
would be interpreted if used as a command name.

is clear.  The command "type command_name" should search the hash
table for command_name and if it is found there, report what it
found.  If and only if command_name is not in the hash table,
type should search $PATH for command_name (and, if it is found,
add it to the hash table unless hashing is disabled).  I.e., type
should behave the same way the shell would behave when the shell
encounters a command name that is to be executed.

If you want to look for command_name by searching $PATH no matter
what is in the hash table, use "hash command_name" instead of
"type command_name".

Cheers,
Don

> On Oct 28, 2019, at 8:51 AM, Joerg Schilling 
>  wrote:
> 
> Robert Elz  wrote:
> 
>>  | But why don't you like the "type" behavior of the Bourne Shell?
>> 
>> As I recall it, the Bourne shell (which is the shell written and
>> maintained by Steve Bourne, and distributed with the pdp-11 v7, and
>> vax 32V research unix systems, anything else is a modified derivative)
>> had no type command, so there was nothing in it to like or not like.
> 
> This is an interesting interpretation.
> 
> bosh is still a direct descendent of the Bourne Shell source code written by
> Stephen Bourne and the "type" command has been added by AT&T people around
> 1982. ksh93 is something similar, as both shells started with the source from
> Stephen Bourne and modified it.
> 
> With your interpretation, however the beast at 
> 
>   https://github.com/att/ast/tree/master/src/cmd/ksh93
> 
> is no more than a modified ksh93 derivate, it did even cause more harm to the 
> ksh93 original than bosh did on the Bourne Shell original.
> 
>> If you mean what bosh does, then that's OK, though having the output
>> vary from "ls is /bin/ls" to "ls is hashed (/bin/ls)" is not what I'd
>> prefer I think (despite being more or less exactly what the NetBSD shell
>> also does).   But as the output from the type command is more or less
>> anything goes, there is no problem with it.   Having bosh not add the
>> result of "type ls" to the has table just seems like an unnecessary
>> ineffeciency (neither correct nor incorrect.)
> 
> Well, "type ls" does a pure lookup.
> 
> If you try to use ls, it adds an "enter" in case ls is not in the cache.
> 
> This seems to be the result os an AT&T decision from around 1982.
> 
> Jörg
> 
> -- 
> EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin
>joerg.schill...@fokus.fraunhofer.de (work) Blog: 
> http://schily.blogspot.com/
> URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'
> 

Re: x[ as first word in sh

[Don Cragun]

> On Jul 29, 2019, at 3:30 PM, Stephane Chazelas  
> wrote:
> 
> 2019-07-29 22:25:29 +0100, Stephane Chazelas:
>> 2019-07-29 21:33:50 +0100, Stephane Chazelas:
>> [...]
>>> BTW, what's the point of 2.10.2 7a? It seems 7b already says
>>> what 7a says (if the TOKEN doesn't contain a =, then rule 1
>>> applies). The distinction seems to be caused by the distinction
>>> between cmd_name vs cmd_word grammar rules which seems to be
>>> about keywords not being recognised after redirections or
>>> assignments, but I can't see how 7a/7b help there.
>> [...]
>> 
>> It seems it was broken in the 2016 edition by the resolution of
>> http://austingroupbugs.net/view.php?id=839
> [...]
> 
> 
> I see it (and several other issues) was already raised in
> http://austingroupbugs.net/view.php?id=1100
> which was rejected.
> 
> The grammar in the suggested resolution there makes a lot more
> sense to me than the current one but that may be down to me
> missing some background on yacc parsing and misunderstanding how
> it works.
> 
> Since 1100 was rejected, shouldn't the other issues it was meant
> to factorize (which doesn't include this one though) and were
> closed as duplicate reopened?
> 
> -- 
> Stephane

Hi Stephane,
No.  The reasons why bugid:1100 was rejected are clearly detailed in 
bugnote:4038.  None of those issues have been addressed in the more than 14 
months since that bug report was rejected.  There is no reason to believe that 
if we reopened all of the bugs that Mark said were replaced by bugid:1100 that 
we would not just be wasting our time and rejecting each one of them with a 
duplicate of bugnote:4038 as the reason for rejection.

Please do not tell us to go back and redo a lot of work we did over a year ago.

If you think you understand the problem(s) better than Mark did, please file a 
new bug (or a few new bugs) that clearly states the problem that each is 
addressing, gives examples of shell productions that are incorrectly handled by 
the current grammar, explains how your changes fix the problem (without 
creating new problems), and defines all new terms used by your changes.  All of 
this is detailed in bugnote:4038.

Sincerely,
Don

Re: [1003.1(2016)/Issue7+TC2 0001212]: Enhance trap command

[Don Cragun]

> On Apr 9, 2019, at 7:35 AM, Geoff Clare  wrote:
> 
>> -- 
>> (0004360) nick (manager) - 2019-04-09 14:27
>> http://austingroupbugs.net/view.php?id=1212#c4360 
>> -- 
>> ... ... ...
>> 
>> I believe it is incorrect to say "shall silently ignore" here. Many shells
>> permit a trap of these signals, even though they cannot be caught or
>> ignored.
> 
> But the existing text says that if you try to set a trap for SIGKILL
> or SIGSTOP the behavior is undefined.
> 
> So at the point where you execute "trap -p" the output either indicates
> that they are set to default or you are already in the realms of
> undefined behaviour.
> 
> -- 
> Geoff Clare 
> The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England

Hi Geoff,
If the shell's man page says what happens in those undefined cases and a 
script's documentation says that the environment in which it runs has to 
support that extension, the application could still be a Conforming POSIX 
Application Using Extensions.  And, if the shell is going to report (for 
instance) that a trap has been set for the SIGKILL signal that means that it 
previously allowed the application to set a trap for the SIGKILL signal.  
Requiring that it accept an attempt to reset it to a value that it previously 
accepted doesn't seem to me to be a requirement that any shell should reject.

If a shell doesn't want to reset a signal that won't be effective, it shouldn't 
report the setting of a trap for that signal in the output it produces for the 
"trap" or "trap -p" commands.

However, concerning a question that was raised earlier about which shells are 
required to process the output produced by the "trap" an "trap -p" commands... 
I think that the requirement only applies to the instance of the shell that 
produced the output.  A non-interactive shell is not allowed to trap or reset 
the behavior of a signal that was ignored when the shell was started.  So 
passing the output of "trap" from one instance of a shell to another instance 
of the same shell cannot require a non-interactive shell to change the state of 
a signal that was ignored when it was invoked.  Nonetheless, a parent shell 
gathering settings from a subshell, such as with
traps=$(trap -p)
has to be processed correctly by the parent when given the command:
eval "$traps"

Cheers,
Don

Re: [1003.1(2016)/Issue7+TC2 0001234]: in most shells, backslash doesn't have two meaning wrt pattern matching

[Don Cragun]

Hello kre et al,
The magic incantation:
bugnote:
(where  is replaced by the digits in a bug note number) in the text of a 
bug note, in the text of the Description, or in the text of the Desired Action 
produces a link to the bug note that is labeled:
Note: 
where  is the bug note number extended to 7 digits by adding leading zeroes.

The magic incantation:
bugid:
(where  is replaced by the digits in a bug number) in the text of a bug 
note, in the text of the Description, or in the text of the Desired Action 
produces a link to the bug that is labeled:

where  is the bug number extended to 7 digits by adding leading zeroes.

So, the text of bug note #4307 below could be changed to:
Re bugnote:4303 again...

I suspect that you added that note to the wrong bug report.  Bug bugid:1235
falls squarely into what is/was being discussed in bug bugid:985 whereas
this one is not.

This is more related to bug bugid:1190, in which, in bugnote:4290, you 
explicitly
asked for new issues to be opened to discuss these side issues of the
original report.

to produce the formatting I think you wanted.  (And I have edited the text of 
your bug note to be the above text.  So, if you go to that bug note now, you 
can see your note with all of the links.)

You can find these and other useful formatting tips if you click on the "Docs" 
tab close to the top of every web page produced by The Austin Group Defect 
Tracker and then click on the "Mantis Reporting Help Documentation" link.

Cheers,
Don

> On Mar 11, 2019, at 8:37 PM, Austin Group Bug Tracker  
> wrote:
> 
> 
> A NOTE has been added to this issue. 
> == 
> http://austingroupbugs.net/view.php?id=1234 
> == 
> Reported By:stephane
> Assigned To:
> == 
> Project:1003.1(2016)/Issue7+TC2
> Issue ID:   1234
> Category:   Shell and Utilities
> Type:   Enhancement Request
> Severity:   Editorial
> Priority:   normal
> Status: New
> Name:   Stephane Chazelas 
> Organization:
> User Reference:  
> Section:2.13.1 
> Page Number:2382 
> Line Number:76212-76215 
> Interp Status:  --- 
> Final Accepted Text: 
> == 
> Date Submitted: 2019-03-08 23:58 UTC
> Last Modified:  2019-03-12 03:37 UTC
> == 
> Summary:in most shells, backslash doesn't have two meaning
> wrt pattern matching
> == 
> 
> -- 
> (0004307) kre (reporter) - 2019-03-12 03:37
> http://austingroupbugs.net/view.php?id=1234#c4307 
> -- 
> Re note 4303 again...
> 
> I suspect that you added that note to the wrong bug report.  Bug 1235
> falls squarely into what is/was being discussed in bug 985 (sorry, no idea
> what I should type to make those be links to the bug reports) whereas
> this one is not.
> 
> This is more related to bug 1190, in which, in note 4290, you explicitly
> asked for new issues to be opened to discuss these side issues of the
> original report. 
> 
> Issue History 
> Date ModifiedUsername   FieldChange   
> == 
> 2019-03-08 23:58 stephane   New Issue
> 2019-03-08 23:58 stephane   Name  => Stephane Chazelas
> 2019-03-08 23:58 stephane   Section   => 2.13.1  
> 2019-03-08 23:58 stephane   Page Number   => 2382
> 2019-03-08 23:58 stephane   Line Number   => 76212-76215 
> 2019-03-11 02:42 kreNote Added: 0004296  
> 2019-03-11 02:58 kreNote Edited: 0004296 
> 2019-03-11 03:07 kreNote Edited: 0004296 
> 2019-03-11 03:10 kreNote Added: 0004297  
> 2019-03-11 03:16 kreNote Edited: 0004296 
> 2019-03-11 03:23 kreNote Added: 0004298  
> 2019-03-11 07:24 stephane   Note Added: 0004300  
> 2019-03-11 07:31 stephane   Note Added: 0004301  
> 2019-03-11 09:41 geoffclare Note Added: 0004303  

Re: [1003.1(2016)/Issue7+TC2 0001187]: sigaltstack ss->ss_flags cannot contain extension flags

[Don Cragun]

> On Mar 4, 2019, at 12:31 PM, Robert Elz  wrote:
> 
>Date:Mon, 4 Mar 2019 16:26:28 +
>From:Austin Group Bug Tracker 
>Message-ID:  <3d03a7389560b5a70dc4195ba4fc9...@austingroupbugs.net>
> 
>  | The following issue has been RESOLVED. 
>  | == 
>  | http://austingroupbugs.net/view.php?id=1187 
> 
> A minor question about the resolution of that one ... was the removal
> of the   markup around ss_flags and the 2nd occurrence of ss
> in the text (the first use retains it) deliberate, or an oversight ?
> 
> kre
> 

It was my mistake.  I copied the text from the etherpad too soon.

It has now been corrected.

Thanks for pointing out my mistake.

 - Don

Re: Typos in strtod documentation

[Don Cragun]

Hi Oleksii,
The bug report has been updated to also address that problem.

Sorry,
Don

> On Sep 29, 2018, at 6:44 AM, Oleksii Vilchanskyi 
>  wrote:
> 
>> printf("It's an integer with value

Re: Typos in strtod documentation

[Don Cragun]

Hi Oliksii,
Mantis bug report #1213 has been filed to report your issue.

Cheers,
Don

> On Sep 29, 2018, at 5:09 AM, Oleksii Vilchanskyi 
>  wrote:
> 
> 
> Hello,
> 
> the documentation of strtod()
> 
> contains a snippet in 'application usage' section, which has multiple
> typos in it:
> 
> if (s != endp && *endp == `\0')
> should instead be
> if (s != endp && *endp == '\0')
> 
> if (s != endp && *endp == `\0')
> should instead be
> if (s != endp && *endp == '\0')
> 
> printf("It's an integer with value %ld\n", 1);
> should instead be
> printf("It's an integer with value %ld\n", l);
> 
> You might wonder why I am posting this on the list instead of using
> Mantis. Despite all of my attempts at logging into
>  under the username I have registered via
> , and being subscribed to this list,
> and further wait (>12 hours), the result was unsuccessful, so I give
> up on the bug tracker and post the information directly on the list.
> Sorry if I have missed something.
> -- 
> < Regards, | S pozdravem >
> Oleksii Vilchanskyi
> PGP:0x8D3A0E046BDE941F2A53867CE3FD952D48C0B338
> 

Re: What exit status for an executable that is found but cannot be invoked?

[Don Cragun]

> On Apr 19, 2018, at 4:04 PM, Martijn Dekker  wrote:
> 
> Op 20-04-18 om 00:34 schreef Shware Systems:
>> Whatever the reason for the load failure, at that point in time it is
>> "not an executable utility", in other words.
> 
> That is an interesting interpretation of the words "executable utility". I do 
> not believe that most people would read it this way.

Interpret them as specified in section 2.9.1.1 of the standard (titled Command 
Search and Execution in the Shell Command Language part of the Shell and 
Utilities volume of the standard).  It goes into explicit detail about when 126 
and 127 are used as exit code when the shell tries to run a utility.  If you 
have the PDF version of the 2016 or 2017 version of the standard, this section 
can be found on pages 2367-2368 lines 75546-75627.

Note, however, that it is possible for the shell to successfully start a 
utility and have it return any exit status from 0 through 255.  When the exit 
status from that utility is a value like 126 or 127, the shell will not modify 
that exit status to avoid confusion that might be caused by the utility's 
choice of exit status.

> 
> Also consider error conditions such as E2BIG. If the utility is found and the 
> argument list is too long, that does not make the utility non-executable even 
> by your definition. There is no load failure; loading the file is never 
> attempted to begin with, because there is not enough space to pass it the 
> list of arguments.

It is strange that you believe that.  If an attempt to execute the utility 
fails, that seems to me to be a clear indication that it is not executable (at 
least with the environment and arguments that were passed to it on this 
invocation).  Why would you believe that the exit status should be 127 
indicating that the file not found if the file was found, but the exec failed?

> 
> POSIX does not define the term "executable utility" exactly but it does 
> define "executable file" under 3.154:
> | A regular file acceptable as a new process image file by the
> | equivalent of the exec family of functions, and thus usable as one
> | form of a utility.
> 
> This defines "executable" in terms of the file's acceptability to 'exec' and 
> friends, which I take to mean properties such as its format and permissions. 
> It says nothing about errors invoking the file that have nothing to do with 
> the nature of the file, e.g. I/O error, argument list too long, etc. -- so it 
> seems to me that those error conditions do not influence the meaning of 
> "executable".

One meaning of executable is that the file is in a format that can be executed 
by the underlying system.  Another meaning is that the file and the needed 
resources to run it are available at the time an attempt was made to execute 
it.  According to the standard, both of these meanings are supposed to result 
in exit code 126.

> 
> - M.

Cheers,
Don

Re: Should shell quoting within glob bracket patterns be effective?

[Don Cragun]

> On Apr 12, 2018, at 8:07 AM, Robert Elz  wrote:
> 
>Date:Thu, 12 Apr 2018 14:25:32 +0100
>From:Geoff Clare 
>Message-ID:  <20180412132532.GA9483@lt2.masqnet>
> 
>  | It treats them as literal characters, just as 2.2.3 says.
> 
> I thought that might have been the response, in that case in
> 
>   "${xxx}"
> 
> The '{' has to be treated as a literal character, as inside double
> quotes, and not being one of the magic few, that's what the text
> you quoted says, and apparently, everywhere else in the shell
> is supposed to follow that same interpretation.
> 
> That is, the '{' above cannot be treated the same as the one in
> 
>   ${xxx}
> 
> (unquoted) where it is a part of the syntax of the variable expansion,
> because then it would not be being treated literally.
> 
> Which way do you want it?
> 
> kre

The fact that the $ is special is what is the key.  Since $ is
special and parameter expansion and command substitution are
performed inside double-quotes, sections 2.6.2 and 2.6.3 come
into play... and that is where {, #, ##, %, %%, and } in
parameter expansions may become special and where ( and ) may
become special in command substitutions, respectively.

Cheers,
Don

Re: complex.h functions and errno

[Don Cragun]

Hi Hal,
I agree with what others have already said in this thread, but I think there is 
a more
basic issue at play here.  In the example you provided:
cabs(1.7e308+I*1.7e308);
the multiplication is not being performed by the function.  It is performed by 
the compiler evaluating arguments to be passed to the functipn.  So the 
floating point overflow will occur before cabs() has a chance to perform any 
calculations.

The C99 standard (section 6.5 "Expressions") says:
"If an exceptional condition occurs during the evaluation of an 
expression
 (that is, if the result is not mathematically defined or not in the 
range
 of representable values for its type), the behavior is undefined."
which seems to be in conflict with the math error handling requirements.  (I 
haven't
tried to determine if this has been fixed in C11 or later drafts.)

 - Don

> On Nov 7, 2017, at 3:22 PM, Hal Finkel  wrote:
> 
> Hi, everyone,
> 
> My reading of the combined C and POSIX specifications leads me to believe 
> that functions in complex.h don't set errno on POSIX systems. This came up 
> recently during a code review for the Clang compiler [1]. It has been pointed 
> out that, under glibc, cabs() can set errno (e.g., cabs(1.7e308+I*1.7e308)) 
> although POSIX says "No errors are defined" for cabs.
> 
> To elaborate, in the C specification, 7.12 specifies the requirements for 
> functions in math.h. For those functions, 7.12.1 (Treatment of error 
> conditions) says that overflows do set ERANGE, and that it's implementation 
> defined if the same is true for underflows. That's true for functions in 
> math.h in general. 7.3 specifies the requirements for functions in complex.h. 
> Here, 7.3.2 says, "An implementation may set errno but is not required to." 
> That, as I understand it, applies to all functions in complex.h (unless 
> otherwise noted, I presume). However, because setting errno is not required 
> by C for functions in complex.h, when POSIX says "No errors are defined." 
> that constrains the choice (POSIX is constraining the implementation choice 
> allowed by C as it does in many other cases). As a result, under POSIX, the 
> functions in complex.h don't set errno.
> 
> Is my reading correct? If it is correct, is it intentional? I'd prefer that 
> these functions don't set errno, because that makes compiler optimization 
> easier, but my overriding concern is that the compiler interprets the 
> requirements correctly.
> 
> Thanks in advance,
> 
> Hal
> 
> [1] https://reviews.llvm.org/D39611
> 
> -- 
> Hal Finkel
> Lead, Compiler Technology and Programming Languages
> Leadership Computing Facility
> Argonne National Laboratory

Re: fscanf white space does not include carraige return

[Don Cragun]

Sorry Mark,
I disagree strongly.  The definition of white space (XBD section 3.442, P104
[using the 2016 edition page and line numbers], L2873-2877) and the definition
of the space character class in the description of Locales (XBD section 7.3.1,
P140, L4163-4170):

3.442 White Space
A sequence of one or more characters that belong to the space character 
class as
defined via the LC_CTYPE category in the current locale.

In the POSIX locale, white space consists of one or more  
( and
 characters), , , , and 

characters.

space   Define characters to be classified as white-space characters.

In the POSIX locale, exactly , , , 
,
, and  shall be included.

In a locale definition file, no character specified for the keywords 
upper,
lower, alpha, digit, graph, or xdigit shall be specified. The , 
,
, , , and  of the portable
character set, and any characters included in the class blank are 
automatically
included in this class.

make it abundantly clear that  is a white-space character in 
the POSIX
locale.  The description of white-space characters in fscanf()'s DESCRIPTION 
section on
P949, L32229 is wrong and needs to be fixed.  The current text:

The format is a character string, beginning and ending in its initial 
shift
state, if any, composed of zero or more directives. Each directive is 
composed
of one of the following: one or more white-space characters (, 
,
, , or ); an ordinary character 
(neither '%'
nor a white-space character); or a conversion specification.

is wrong both because it omits  from the list and because it 
fails to
mention that the list is only true in the POSIX (or C) locale.  In other 
locales, the
list of white-space characters is the set of characters contained in that 
locale's
space character class.

The first parenthetical element in the last quote above should just be removed 
(which
does  not appear in the C Standard) or should be changed to match the 
parenthetical
element in the later reference on P950, L32253-32254:

Input white-space characters (as specified by isspace()) shall be 
skipped,
unless the conversion specification includes a [, c, C, or n conversion 
...

(which does appear in the C Standard).

Note that the C standard has two pseudo-definitions of white space.  In 
pre-processor
statements, white-space characters are just  and  (but  may 
have
replaced comments and other white-space characters in translation phase 3).  In 
the
library section of the C Standard, isspace() returns true for "standard" 
white-space
characters (the same characters that appear in the POSIX space character class) 
and
any locale-specific characters chosen by the implementation (but no character 
can
yield a true response from both isspace() and isalnum()).  In the C Locale, 
only the
standard white-space characters yield a true response from isspace().

> On Sep 7, 2017, at 4:50 PM, Shware Systems  wrote:
> 
> I suspect  in that list is meant to include both  or , 
> possibly according to a 
> platform's or terminal's line ending convention, rather than list the codes 
> separately.
> I could
>  see an addition to Application Usage that applications may have to take 
> conventions into account when crafting control strings, but this is more a 
> bug in C not being specific enough, to me.
> 
> On Thursday, September 7, 2017 Schwarz, Konrad  
> wrote:
> 
> Hi,
> 
> fscanf states that white space characters are skipped, and lists the white 
> space characters.  This list does not include carriage return.  The POSIX 
> definition of white space (link anker seems to be wrong) includes carriage 
> return.
> 
>  
> C0x.pdf speaks generically of white space characters.
> 
>  
> Is the omission of carriage return in fscanf deliberate?
> 
>  
> Regards
> 
>  
> Konrad Schwarz
> 

Re: [1003.1(2016)/Issue7+TC2 0001154]: Add 'rm -v'

[Don Cragun]

> On Jun 28, 2017, at 1:53 AM, Job Snijders  wrote:
> 
> On Tue, Jun 27, 2017 at 04:15:50PM -0700, Scott Lurndal wrote:
>> On Tue, Jun 27, 2017 at 11:57:14PM +0200, Job Snijders wrote:
>>> On Mon, Jun 26, 2017 at 11:06:14AM -0700, Scott Lurndal wrote:
 On Sun, Jun 25, 2017 at 06:52:16PM +, Austin Group Bug Tracker wrote:
> You are right that "Print names of files as they are processed" is a bit
> ambiguous. All "-v" implementations I am aware of specifically print
> _after_ a successful deletion (through a system call to unlink() or
> rmdir()). Perhaps "Print names of files as they are successfully deleted"?
 
 Although deleted is imprecise here - the directory entry is removed
 but the file may still exist via a different pathname on the same
 device (i.e. a hard link).
>>> 
>>> Do you have a suggestion how to improve the phrasing?
>> 
>> As the rest of the description of 'rm' refers to 'directory entries',
>> that seems the appropriate term in this instance instead of 'files'.
>> 
>> E.g. "Print directory entries as they are successfully removed".
> 
> Thank you, that is better yes.
> 
> As you may have noticed, I am somewhat now to this process and working
> group. Where do we go from here? What is the proper way to update the
> bugreport in the mantis system to reflect improvements?
> 
> Kind regards,
> 
> Job

Hi Job,
Add a note to the bug report saying that the wording has been discussed on the 
mailing list and that all of part of the Desired Action section of the bug 
report should be replaced with your new proposed wording.

Note that to avoid ambiguity, you would probably be better with something like:
 -vPrint file after it has been unlinked.
Saying "unlinked" is better than "removed" because the contents of the file 
will not be removed until the last link to a file has been unlinked AND all 
open file descriptors associated with that file are closed (see the 2nd 
paragraph of the description of the unlink() function on P2197, L70220-70223 in 
the 2016 edition of the standard).

Re: [1003.1(2013)/Issue7+TC1 0001044]: Calling setsid as part of posix_spawn

[Don Cragun]

Hi Geoff,
I like the idea of updating the example implementation to add code
to match the current specification, but I don't like the idea of
changing the value assigned to POSIX_SPAWN_RESETIDS.  The list of
#defines on P3695, L126575-126580 already contains several
definitions that are not in alphabetic order, so, instead of
changing reassigning the value of POSIX_SPAWN_RESETIDS in the
current standard to POSIX_SPAWN_SETSID.  I would prefer to have it
keep its current value and just add:
   #define POSIX_SPAWN_SETSID   0x40
after L126580.

 - Don

> On Apr 5, 2017, at 1:29 AM, Geoff Clare  wrote:
> 
>> -- 
>> (0003658) daurnimator (reporter) - 2017-04-05 02:50
>> http://austingroupbugs.net/view.php?id=1044#c3658 
>> -- 
>> I just noticed the example implementation of posix_spawn in "B.3.3 Examples
>> for Spawn". Should we also update that? 
> 
> Yes, I think we should.  If nobody objects, I will edit my bugnote 3650
> to add the following:
> 
> On page 3695 line 126580 section B.3.3 Examples for Spawn, change:
> 
>#define POSIX_SPAWN_RESETIDS 0x20
> 
> to:
> 
>#define POSIX_SPAWN_SETSID   0x20
>#define POSIX_SPAWN_RESETIDS 0x40
> 
> 
> On page 3696 line 126646 section B.3.3 Examples for Spawn, change:
> 
>/* Worry about process group */
> 
> to:
> 
>/* Worry about creating a new session */
>if (attrp->posix_attr_flags & POSIX_SPAWN_SETSID)
>{
>   /* Create a new session */
>   if (setsid() == -1)
>   {
>   /* Failed */
>   _exit(127);
>   }
>}
> 
>/* Worry about process group */
> 
> 
> Between page 3696 line 126653 and page 3699 line 126791, change all
> occurrences of:
> 
>exit(127);
> 
> to:
> 
>_exit(127);
> 
> 
> The editors may also wish to consider changing all the "Worry about ..."
> comments in this code example to "Handle ...".
> 
> -- 
> Geoff Clare 
> The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
> 

Re: I/O semantics of pipe and FIFO.

[Don Cragun]

While it may be true that one or more systems have this "feature",
the script you have provided does not demonstrate that behavior.
(And, I am not aware of any system that does behave the way you
have described.)

Since a writer is blocked from writing into a FIFO if the FIFO is
not open for reading, both of your asynchronous echo commands will
block until dd opens the FIFO for reading.  When the dd starts,
one of the echo commands will be unblocked and write 4 bytes into
the FIFO.  There is then a race as to whether the 2nd echo will
write another 4 bytes into the FIFO or dd will read 4 bytes from
the FIFO.  In the example you have shown, dd reads the contents
of the FIFO before the 2nd echo wakes up and writes the other 4
bytes into the FIFO and then dd reads those 4 bytes as a second
partial read.

With a more complicated test that opens the FIFO for reading
(without actually reading any data from the FIFO) before
executing the echo commands, then runs the echo commands and
then runs the dd command (before the other reader closes the
FIFO), we can see that the output from the two echo commands is
seen by dd in 1 read from the FIFO.  Unfortunately, at this
point we have dd hung waiting for another writer, to allow its
blocked read to continue, but adding another echo takes care of
that.  If a read from a FIFO returned at most the data written by
a single write system call, the dd in the following script would
show 3 partial reads instead of two.

opener.c:
=
#include 
#include 
#include 
#include 
#include 
#include 
#include 

int
main(int argc, char *argv[]) {
int fd;

fd = open(argv[1], O_RDONLY);
if(fd == -1) {
fprintf(stderr, "Can't open '%s' (%s)\n", argv[1], 
strerror(errno));
exit(1);
}
sleep(30);
printf("opener done.\n");
}

shell script:
#!/bin/ksh
[ ! -p myfifo ] && mkfifo myfifo
ls -l myfifo
date
./opener myfifo&
sleep 1
date
/bin/echo abc > myfifo
date
/bin/echo 123 > myfifo
date
dd if=myfifo&
date
sleep 40
date
/bin/echo XYZ > myfifo
date

Output produced by running the above shell script on macOS Sierra version 
10.12.3:
==
prw-r--r--  1 dwc  staff  0 Mar  4 22:16 myfifo
Sat Mar  4 22:16:38 PST 2017
Sat Mar  4 22:16:39 PST 2017
Sat Mar  4 22:16:39 PST 2017
Sat Mar  4 22:16:39 PST 2017
Sat Mar  4 22:16:39 PST 2017
opener done.
Sat Mar  4 22:17:19 PST 2017
abc
123
XYZ
0+2 records in
0+1 records out
12 bytes transferred in 0.002135 secs (5621 bytes/sec)
Sat Mar  4 22:17:19 PST 2017

 - Don


> On Mar 4, 2017, at 6:25 PM, Danny Niu  wrote:
> 
> Found it: http://man.cat-v.org/unix_8th/2/write
> 
> Should I file an editorial bug on the rationale section of the write() 
> interface, to note that some systems have this feature? Since we've already 
> mentioned the special case of making a zero-length write request. 
> 
> From: Danny Niu 
> Sent: Sunday, March 5, 2017 10:05
> To: Stephane Chazelas
> Cc: Austin Group Mailing List
> Subject: Re: I/O semantics of pipe and FIFO.
> 
> Just so you guys can reproduce what I was talking about, here's essentially 
> what I did:
> 
> #!/bin/sh
> 
> cd ~
> mkfifo myfifo
> /bin/echo 123 >myfifo & /bin/echo abc >myfifo &
> sleep 1
> dd if=myfifo #dd defaults to block size of 512 in this case.
> 
> output from dd (stdout+stderr):
> 123
> abc
> 0+2 records in
> 0+1 records out
> 8 bytes transferred in 0.004737 secs
> 
> Not that I use /bin/echo instead of just echo to ensure two separate write 
> system calls are made.
> 
> From: Stephane Chazelas 
> Sent: Sunday, March 5, 2017 05:48
> To: Danny Niu
> Cc: Austin Group Mailing List
> Subject: Re: I/O semantics of pipe and FIFO.
> 
> 2017-03-04 13:14:08 +, Danny Niu:
>> Hi all.
>> 
>> I couldn't remember where I saw it saying, that when reading
>> from a pipe or a FIFO, the read syscall returns the content of
>> at most one write call. It's a bit similar to the
>> message-nondiscard semantics of dear old STREAM.
>> 
>> Currently, I'm reading through the text to find out a bit
>> more, and I appreciate a bit of pointer on this.
> [...]
> 
> (echo x; echo y) | (sleep 1; dd count=1 2> /dev/null)
> 
> outputs both x and y in all of Linux, FreeBSD and Solaris in my
> tests.
> 
> That a read wouldn't read what's currently in the pipe would be
> quite surprising.
> 
> I also wouldn't expect pipes to store the writes as individual
> separate message but use one buffer.
> 
> In:
> 
> (
>  dd bs=4 count=1 if=/dev/zero 2> /dev/null
>  echo first through >&2
>  dd bs=4 count=1 if=/dev/zero 2> /dev/null
>  echo second through >&2
> ) | (sleep 1; dd bs=10 count=1 2> /dev/null) | wc -c
> 
> That is where the second write blocks because the pipe is full,
> the reading dd still reads both writes in Linux and Solaris in
> my tests (on Solaris (10 on amd64 at least), red

Re: Siginfo_t question:

[Don Cragun]

> On Nov 14, 2016, at 4:33 PM, Philip Guenther  wrote:
> 
> On Mon, Nov 14, 2016 at 3:38 PM, Robert Elz  wrote:
>>From:Philip Guenther 
>>  | I don't think we want to start reading the spec as giving
>>  | implementations leeway to overlay arbitrary struct members.
>> 
>> You're interpreting the world backwards.   This group doesn't define new
>> interfaces that the world then goes out and implements (or shouldn't).
> 
> I don't understand how my message, taken as a whole, was interpreted that way.
> What's your opinion on the suggestion that I was replying to and which
> you trimmed?
> 
> 
>> For better or worse, the implementations extended siginfo_t using a
>> union, with different values for different purposes.   Then the standard
>> needs to find a way to describe that.
> 
> Since I was apparently unclear in my previous message:
> 
> * I think the current wording in the standard is wrong and should be fixed.
> 
> * I think this should be done by explicitly stating when and where
> members should be overlaid, like the sigaction case I quoted.
> 
> * I understood Scott Lurndal's message as suggesting that we solve
> this by (re)interpreting the standard as permitting implementations to
> -- in general -- overlay struct members without the standard
> explicitly stating which and where.  I think that would be too much
> leeway for implementations and too limiting on applications.
> 
> 
> Philip Guenther
> 
All of this is covered in detail in XSH section 2.4.3 (Signal Actions) under 
the heading "Pointer to a Function" where it explicitly states the conditions 
under which certain of those structure members are required to be defined and 
which certain of those structure members are undefined depending on what signal 
is being caught and what generated the signal.  For those of you with the PDF 
for the 2016 Edition of the standard (including TC1 and TC2), it is on 
P492-493, L17053-17096.  Or, look at the description of the use of the 
SA_SIGINFO flag in the DESCRIPTION of the sigaction() function on XSH P1951, 
L62839-62862 which also contains a back reference to section 2.4.3 above.

You might also want to read the APPLICATION USAGE section on the XBD  
page on P339, L11487-11493.

While I realize that it would be handy if you could look at any page in the 
standard and understand all of the requirements in the standard without having 
to read the rest of the standard or look at anything else (like the general 
concepts describing how signals work and the functions that generate and catch 
signals), doing that would make the standard immensely larger and significantly 
increase the chance that an update in one place in the standard will introduce 
an inconsistency with other text that repeats the requirements in other 
sections of the standard.  Clearly you would also like us to throw away the SEE 
ALSO sections in the standard since they are just wasting space; but if you had 
bothered to read the pages referenced in the SEE ALSO sections, all of the 
information you say is missing from the standard would have been found.

 - Don

Re: "sh" utility and "-" and "--"

[Don Cragun]

Hi Stephane,
Does the 5th paragraph in the RATIONALE section on the page you pointed to:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html
help?

 - Don

> On Aug 8, 2016, at 12:35 PM, Stephane Chazelas  
> wrote:
> 
> Can anybody please explain this text at
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html#tag_20_117_05
> 
>> OPERANDS
>> 
>>The following operands shall be supported:
>> 
>>-
>>A single  shall be treated as the first
>>operand and then ignored. If both '-' and "--" are
>>given as arguments, or if other operands precede the
>>single , the results are undefined.
> 
> Unless I'm missing something, that implies that the behaviour is
> undefined for things like:
> 
> sh myscript foo -
> 
> Or:
> 
> sh - myscript -- foo
> 
> Or:
> 
> sh -c 'inline' sh - ...
> 
> Or:
> 
> sh -c - 'inline' --
> 
> Or
> 
> sh -s - -- x y
> 
> Which doesn't make any sense, especially for the first 2 ones
> which we generally depend on. Think of
> 
> #! /bin/sh -
> getopts ...
> 
> For instance.
> 
> Thanks,
> Stephane
> 

Re: Are the results of echo $(FOO=bar foobar) really unspecified in POSIX shell?

[Don Cragun]

> On Aug 5, 2016, at 7:52 PM, Mark Galeck  wrote:
> 
> Hello,
> 
> I am trying to understand your POSIX shell standard, found at 
> 
> Shell Command Language
>  
... ... ...  Blank lines deleted.
>  
> Shell Command Language
> <<< Previous Home Next >>> The Open Group Base Specifications Issue 7 IEEE 
> Std 1003.1, 2013 Edition Copyright © 2001-2013 The IEEE and The Open Group 2. 
> Shel...
> View on pubs.opengroup.org
> Preview by Yahoo
>  
> 
> In the following command:
> 
> echo $(FOO=bar foobar)
> 
> there are two tokens for the top level shell:
> echo and $(FOO=bar foobar)

Yes.  This comes from the grammar for a simple_command expanding to:
cmd_name cmd_suffix
and rule 7a is applied to cmd_name and the WORD  is recognized in the 
grammar as the name of a utility to be invoked (since  is not a shell 
keyword).  The grammar for cmd_suffix in this case chooses the token WORD as a 
match for <$(FOO=bar foobar)>.  This WORD is subject to the Word Expansions 
specified in section 2.6 of the standard and this particular expansion is a 
command substitution that is described in section 2.6.3.  Section 2.6.3 says 
that the command substitution results in the simple command  
being executed and the standard output produced as a result of running that 
command will then be parsed as subject to field splitting and pathname 
expansions (since the command substitution was not enclosed in double-quotes).

In the command substitution we again have two tokens  and  
that will match the grammar of a simple_command this time expanding to the:
cmd_prefix cmd_word
grammar item and the cmd_prefix  will match ASSIGNMENT_WORD using rule 
7b and cmd_word  will match WORD and rule 7b will be applied again.  
But, since there is no equal sign in , it will not be treated as an 
ASSIGNMENT_WORD and rule 1 will be applied thereby treating  as a 
utility name (since  is not the 1st word of this simple command).

> 
> This is a useful type of command and not "contrived", so it should be defined 
> what its effects are.  But when you try to parse it using the grammar given, 
> the second token is unspecified by Rule 7b, because it contains = and before 
> that is not a valid name.

Nobody here is saying this is contrived (other than the foobar utility).
 
> 
> Are the results really unspecified?  Or am I not understanding something?

This is fully specified by the standard as explained above.  Did my explanation 
help you see what you missed?

> 
> I tried to ask this question on sites such as StackOverflow, but it seems 
> that, even the "gurus" there, fail to understand my question.
> 
> Thank you,
> 
> Mark

 - Don

Re: [1003.1(2013)/Issue7+TC1 0001008]: 1. clarify iconv(3) reset usage; 2. truly support Unicode character input

[Don Cragun]

Hi Steffen,
You should still be able to add notes to 
http://austingroupbugs.net/view.php?id=1008 .

But, the point is that an application using iconv() to convert text doesn't 
need to know whether or not the target codeset has locking shift states.  It 
should always make a final call similar to:
ret = size_t iconv(cd, NULL, 0, outbuf, outbytesleft);
with outbytesleft large enough to handle any string that might be required to 
terminate the output buffer with whatever string is needed to get back to the 
initial shift state for the target codeset.  (This will be a no-op for many 
target codesets, but it won't hurt an application to make this final call 
before calling:
ret = iconv_close(cd);
to terminate the conversion even if the target codeset does not need to add any 
text to get back to an initial shift state.)

 - Don


> On Aug 3, 2016, at 6:26 PM, Steffen Nurpmeso  wrote:
> 
> Good evening.
> 
> |http://austingroupbugs.net/view.php?id=1008 
> |== 
> |Project:1003.1(2013)/Issue7+TC1
> |Issue ID:   1008
> 
> |Section:Vol.2, System Interfaces, iconv 
> |Page Number:1109 
> |Line Number:37302 ff. 
> 
> |Date Submitted: 2015-11-16 22:24 UTC
> |Last Modified:  2016-08-04 16:35 UTC
> 
> |Summary:1. clarify iconv(3) reset usage; \
> |2. truly support Unicode character input
> 
> | (0003326) geoffclare (manager) - 2016-08-04 16:35
> | http://austingroupbugs.net/view.php?id=1008#c3326 
> |-- 
> |Add to application usage as a new paragraph after P1110, L37364:
> |
> |It is the responsibility of the application to ensure that, if the output
> |codeset has a locking-shift encoding, the output buffer is returned to its
> |initial shift state when conversion is completed.  This can be accomplished
> |by calling iconv() with inbuf as a null pointer, or with
> |inbuf pointing to a null pointer, before calling
> |iconv_close(). 
> 
> As of today an application cannot know whether a character set has
> a stateful encoding or not.  I have always read the current
> standard text, regarding state-dependency, as a guideline and as
> an informational context, due to that.  Adding a clause that puts
> a conditional "responsibility" upon an application is something
> that it cannot conform to, because it cannot detect the condition
> as such.  Because of that i would remove the "if the output
> codeset has a locking-shift encoding" condition from the accepted
> text.
> 
> P.S.: i've tried to place that on Mantis but the issue is now
> read-only.
> 
> --steffen
>