Re: C macro for maximum path length?

2020-12-04 Thread David Crayford

On 3/12/2020 11:12 pm, Charles Mills wrote:

I believe you, but why then is the macro undefined? Why is the definition now 
commented out?

 From  (actually CEE.SCEEH.H(LIMITS)) on z/OS V2R4:


For the usual POSIX compliance reasons !

FWIW, in IBM z/OS C++ code that's used by the JVM they set PATH_MAX to 
1024 
*https://github.com/eclipse/omr/blob/c3c479f9d5958df679349d0e02ed204329fda2da/port/unix/omrsysinfo.c#L181*





/*
  *  POSIX.1 1990 Section 2.8.5 Statement 1065 -
  *  these macros "shall be omitted on specific
  *  implementations where the corresonding value is
  *  >= the stated minimum, but where the value
  *  can vary depending on the file to which it is
  *  applied."
  *
  *
  */
  
  /*

   * #define LINK_MAX
   * #define MAX_CANON
   * #define MAX_INPUT
   * #define NAME_MAX
   * #define PATH_MAX
   * #define PIPE_BUF
   */

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of David Crayford
Sent: Thursday, December 3, 2020 5:32 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: C macro for maximum path length?

PATH_MAX on z/OS is 1023. This is documented in lots of error messages and BPX 
assembler services.


On 3 Dec 2020, at 7:16 am, Charles Mills  wrote:

I have some code that compiles both under Windows Visual Studio and z/OS
XLC.

In Windows the maximum length of a file path is defined by _MAX_PATH and
__MAX_PATH (I guess MS thinks two macros are better than one).

What is the equivalent macro for XLC? Failing that, what *is* the maximum
path length so I can define my own macro?

Posix defines PATH_MAX and I see references to it in the z/OS doc but it
does not seem to be defined for my compile. How do I pick that up? And yes,
I have

#define _XOPEN_SOURCE_EXTENDED 1
#include 

z/OS V2R4

Thanks,

Charles

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-04 Thread Paul Gilmartin
On Fri, 4 Dec 2020 09:25:07 -0800, Charles Mills  wrote:

>@Gil, weren't you going to try a mkdir wombat/cd wombat endless loop and see 
>where it failed?
> 
My latest attempt:
# ###
#! /bin/sh
# Doc: Test limits of path length (PATH_MAX)
#  and directory nesting depth.

Build() { F="$1"
( while mkdir "$F" & cd "$F"; do pwd; done ) | tee "$F.log"; }

set x "Ridiculously long file name"

for I; do
Build "$I"; done

echo; uname -sr
for I; do
wc "$I.log"
tail -1 "$I.log" | wc; done
exit
# ###
I've long feared to try such things on z/OS because it's too easy to
create an object on which "rm -r" fails because of either excessive
recursion depth or excessive geenerated path length.

I'm not afflicted with a DOS system to test.

On Linux, it "fails cleanly":
Linux 4.19.0-12-amd64
   20412041 4196296 x.log
  1   14096
   145  31900 298410 Ridiculously long file name.log
  1 4364074

The limitation seems to be path length (~4096) rather than
nesting depth (at least 2041).

On MacOS it generates a stream of error messages until I interrupt it:
chdir: error retrieving current directory: getcwd: cannot access parent 
directories: No such file or directory
pwd: error retrieving current directory: getcwd: cannot access parent 
directories: No such file or directory

But never sets error status.  I suspect that setting the environment
variable PWD may mask an underlying problem.

>What happens if you make ~the maximum length path (~1024?) and then mount it 
>on some mount point?
>
>It seems like any "opt-in" for a longer path name would want to be program by 
>program (again, analogous to long parms) 
>
YA program object attribute?  Or JCL JOB option

I feel the same way about REFRPROT.  A site might need to keep
REFRPROT(NO) for backward compatibility while conscientious
programmers should enable REFRPROT(YES) for their own programs.
(REUS(REFR,REALLY)?)

>because I as a vendor would have no control over whether a customer did a 
>"global" opt-in in PARMLIB. (I would just get the "it blew up" ticket.)

>-Original Message-
>...
>On Fri, 4 Dec 2020 08:19:34 -0500, Peter Relson wrote:
>>
>>And that's why z/OS will never change the maximum path length by default 
>>(I actually thought it was 1024, but my knowledge is only from what 
>>CSVQUERY implemented and documents for returning the path name). There 
>>would have to be some sort of "opt-in" for a longer name.
>>
z/OS seems to be inviting the pitfall by  not providing the POSIX-required and
safe "allocating" form of realpath()?
>
A programmer with FOSS code that compiles successfully on a POSIX-conformant
system and fails on z/os because of the lack of PATH_MAX and who is most
familiar with:  Authorized Assembler Services Guide
DALPATH specifies the pathname of the z/OS UNIX file to be allocated.
... The maximum length of the pathname is 255 characters
and: z/OS: MVS JCL Reference; DD: PATH
pathname ...
• Has a length of 1 through 254 characters, not including the slash.

... is apt to take 255 as gospel and  code in desperation:
#ifndef PATH_MAX
#define PATH_MAX 255
#endif

(Why is the DALPATH parameter limited to 255 bytes, given that the
length is passed in a halfword?)

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-04 Thread Charles Mills
@Gil, weren't you going to try a mkdir wombat/cd wombat endless loop and see 
where it failed?

What happens if you make ~the maximum length path (~1024?) and then mount it on 
some mount point?

It seems like any "opt-in" for a longer path name would want to be program by 
program (again, analogous to long parms) because I as a vendor would have no 
control over whether a customer did a "global" opt-in in PARMLIB. (I would just 
get the "it blew up" ticket.)

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Paul Gilmartin
Sent: Friday, December 4, 2020 7:02 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: C macro for maximum path length?

It would be a courtesy to leave a citation when appending to a thread.
A reader might wish to refer to the ped text.

On Fri, 4 Dec 2020 08:19:34 -0500, Peter Relson wrote:

>
>The real question is not "how long can a path be [today]?" but rather "how 
>long might a path be at any future point when this compilation is 
>running?" 
>
>
>And that's why z/OS will never change the maximum path length by default 
>(I actually thought it was 1024, but my knowledge is only from what 
>CSVQUERY implemented and documents for returning the path name). There 
>would have to be some sort of "opt-in" for a longer name.
>
Why, then, does z/OS not provide the POSIX-required and safe "allocating" 
form of realpath()?

Are any other utilities/functions affected?

Is it IBM's uniform policy never to relax any limit because a suitably
mischievous (or stupid) programmer might leverage it into an exposure?

How does this cover:
o NFS, where IBM can't control the content of the guest filesystem?
o The possibility that administrators might mount a tower of
  filesystems to a height beyond the putative hard-coded PATH_MAX
  ("putative" in that the value is mentioned in (obsolete?) M but
  no macro is defined)?

Study: https://eklitzke.org/path-max-is-tricky

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-04 Thread Paul Gilmartin
It would be a courtesy to leave a citation when appending to a thread.
A reader might wish to refer to the ped text.

On Fri, 4 Dec 2020 08:19:34 -0500, Peter Relson wrote:

>
>The real question is not "how long can a path be [today]?" but rather "how 
>long might a path be at any future point when this compilation is 
>running?" 
>
>
>And that's why z/OS will never change the maximum path length by default 
>(I actually thought it was 1024, but my knowledge is only from what 
>CSVQUERY implemented and documents for returning the path name). There 
>would have to be some sort of "opt-in" for a longer name.
>
Why, then, does z/OS not provide the POSIX-required and safe "allocating" 
form of realpath()?

Are any other utilities/functions affected?

Is it IBM's uniform policy never to relax any limit because a suitably
mischievous (or stupid) programmer might leverage it into an exposure?

How does this cover:
o NFS, where IBM can't control the content of the guest filesystem?
o The possibility that administrators might mount a tower of
  filesystems to a height beyond the putative hard-coded PATH_MAX
  ("putative" in that the value is mentioned in (obsolete?) M but
  no macro is defined)?

Study: https://eklitzke.org/path-max-is-tricky

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-04 Thread Peter Relson

The real question is not "how long can a path be [today]?" but rather "how 
long might a path be at any future point when this compilation is 
running?" 


And that's why z/OS will never change the maximum path length by default 
(I actually thought it was 1024, but my knowledge is only from what 
CSVQUERY implemented and documents for returning the path name). There 
would have to be some sort of "opt-in" for a longer name.

Peter Relson
z/OS Core Technology Design


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-03 Thread Mike Schwab
Linux allows 4K UTF-8 characters, so 16K bytes is possible.

https://listserv.ua.edu/cgi-bin/wa?A2=ind1905=IBM-MAIN=R19142=0C1F3202D837335EDF-
May 2019, Request For Enhancement to z/OS.
16 bit length so 64K possible, current limit for dynamic allocation at
255, defined limit of 1023.

https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.10/client/c_cmd_filespecsyntax.html

The length of a file specification is limited.

On AIX, Solaris, and Mac: The maximum number of characters for a file
name is 255. The maximum combined length of the file name and path
name is 1024 characters. The Unicode representation of a character can
occupy several bytes, so the maximum number of characters that a file
name might contain can vary. [ UTF-8 goes to 4 bytes, so 4096 possible
- Mike Schwab]

On Linux: The maximum length for a file name is 255 bytes. The maximum
combined length of both the file name and path name is 4096 bytes.
This length matches the PATH_MAX that is supported by the operating
system. The Unicode representation of a character can occupy several
bytes, so the maximum number of characters that comprises a path and
file name can vary. The actual limitation is the number of bytes in
the path and file components, which might correspond to an equal
number of characters. [ UTF-8 goes to 4 bytes, so 16K possible - Mike
Schwab]

On Linux: For archive or retrieve operations, the maximum length that
you can specify for a path and file name (combined) remains at 1024
bytes. [ UTF-8 goes to 4 bytes, so 4096 possible - Mike Schwab]

The maximum number of bytes for a file name and file path when
combined is 6255. However, the file name itself cannot exceed 255
bytes. Furthermore, directory names (including the directory
delimiter) within a path are limited to 255 bytes. The Unicode
representation of a character can occupy several bytes, so the maximum
number of characters that a file name might contain can vary.  [ UTF-8
goes to 4 bytes, so 1563-6255 possible - Mike Schwab]

When using the open file support feature with VSS, the backup-archive
client adds the snapshot volume name to the path of the objects being
processed. The resulting path (snapshot volume name plus object path)
must adhere to the limits shown. The snapshot volume name can be up to
1024 bytes.


On Thu, Dec 3, 2020 at 6:16 PM Charles Mills  wrote:
>
> Is that a z/OS limit or an FTP limit? Reading the doc leads me to think it is 
> an FTP limit, not a z/OS limit.
>
> Charles
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Mike Schwab
> Sent: Thursday, December 3, 2020 3:26 PM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: C macro for maximum path length?
>
> Sorry, here is the error message.
> https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.cs3cod0/ftp550113.htm
> z/OS Unix System Services specifies the maximum file name of 255 and
> path of 1023.
>
> https://www.ibm.com/support/pages/apar/PI70999
> Rhapsody on Windows before 10 had a limit of 260 characters in a path.
>
> On Thu, Dec 3, 2020 at 4:56 PM Charles Mills  wrote:
> >
> > I see
> >
> > #define FILENAME_MAX  1024
> >
> > in stdio.h V2R4
> >
> > Where does that fit into this conundrum?
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN



-- 
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-03 Thread Charles Mills
Is that a z/OS limit or an FTP limit? Reading the doc leads me to think it is 
an FTP limit, not a z/OS limit.

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Mike Schwab
Sent: Thursday, December 3, 2020 3:26 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: C macro for maximum path length?

Sorry, here is the error message.
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.cs3cod0/ftp550113.htm
z/OS Unix System Services specifies the maximum file name of 255 and
path of 1023.

https://www.ibm.com/support/pages/apar/PI70999
Rhapsody on Windows before 10 had a limit of 260 characters in a path.

On Thu, Dec 3, 2020 at 4:56 PM Charles Mills  wrote:
>
> I see
>
> #define FILENAME_MAX  1024
>
> in stdio.h V2R4
>
> Where does that fit into this conundrum?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-03 Thread Mike Schwab
Sorry, here is the error message.
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.1.0/com.ibm.zos.v2r1.cs3cod0/ftp550113.htm
z/OS Unix System Services specifies the maximum file name of 255 and
path of 1023.

https://www.ibm.com/support/pages/apar/PI70999
Rhapsody on Windows before 10 had a limit of 260 characters in a path.

On Thu, Dec 3, 2020 at 4:56 PM Charles Mills  wrote:
>
> I see
>
> #define FILENAME_MAX  1024
>
> in stdio.h V2R4
>
> Where does that fit into this conundrum?
>
> Charles
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Charles Mills
> Sent: Thursday, December 3, 2020 9:41 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: C macro for maximum path length?
>
> Thanks @Gord, yes, I saw pathconf().
>
> I am starting to "get" the problem.
>
> > I suspect there's a buffer overrun hazard associated with a statically 
> > compiled
> > PATH_MAX.
>
> Never mind the exact lack of a macro and never mind z/OS. Let's say one were 
> to build a hypothetical program with a buffer of hard-coded length 'n', the 
> maximum path length for the environment. What is to stop a future release of 
> the environment from allowing path names with a length greater than 'n'? Now 
> all of a sudden the program is in danger of buffer overrun.
>
> The real question is not "how long can a path be [today]?" but rather "how 
> long might a path be at any future point when this compilation is running?" 
> and that is kind of an unanswerable question, barring some sort of vendor SOD 
> like "it is our intention never to increase maximum path length beyond 4095." 
> It is a problem somewhat analogous to the problem of old programs and an EXEC 
> PARM longer than 100 bytes.
>
> Yes, it is a question that pathconf() answers, although it seems to me that 
> doing it filename by filename is overly complicated: why not just a single, 
> static for a given environment, "maximum path length of any possible file on 
> this system"? (Answer: because they didn't ask me. @Gil's link below provides 
> a better answer.)
>
> I am simply going to recode my routine to finesse the problem. It currently 
> has a "work buffer" of length PATH_MAX. It is kind of a lazy programmer's 
> approach. I am going to re-code it to malloc (actually, to "new" -- it's C++) 
> a buffer of the necessary length for every path that it processes. I need a 
> dedicated area for each name eventually anyway (and yes, that I delete 
> eventually). That's why I say the "work buffer" was kind of a lazy programmer 
> approach.
>
> Thanks all for your patience and explanations.
>
> Charles
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Paul Gilmartin
> Sent: Thursday, December 3, 2020 9:08 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: C macro for maximum path length?
>
> On Thu, 3 Dec 2020 11:49:05 -0500, Gord Tomlin wrote:
>
> >On 2020-12-03 10:12 AM, Charles Mills wrote:
> >> I believe you, but why then is the macro undefined? Why is the definition 
> >> now commented out?
> >>
> I suspect there's a buffer overrun hazard associated with a statically 
> compiled
> PATH_MAX.  Interesting, apparently well-researched link:
> https://eklitzke.org/path-max-is-tricky
>
> >> >From  (actually CEE.SCEEH.H(LIMITS)) on z/OS V2R4:
> >> /*
> >>   *  POSIX.1 1990 Section 2.8.5 Statement 1065 -
> >>   *  these macros "shall be omitted on specific
> >>   *  implementations where the corresonding value is
> >>   *  >= the stated minimum, but where the value
> >>   *  can vary depending on the file to which it is
> >>   *  applied."
> >>   *...
> >>* #define LINK_MAX
> >>* #define MAX_CANON
> >>* #define MAX_INPUT
> >>* #define NAME_MAX
> >>* #define PATH_MAX
> >>* #define PIPE_BUF
> >>*/
> >
> >"an application may use the/fpathconf/()  
> ><https://pubs.opengroup.org/onlinepubs/009696699/functions/fpathconf.html>,/pathconf/()
> >  <https://pubs.opengroup.org/onlinepubs/009696699/functions/pathconf.html>, 
> >and/sysconf/()  
> ><https://pubs.opengroup.org/onlinepubs/009696699/functions/sysconf.html>  
> >functions to
> >determine the actual value of a limit at runtime."
> >
> >(<https://pubs.opengroup.org/onlinepubs/009696699/basedefs/limits.h.html>)
> >
> OpenGroup/Single UNIX requires an &

Re: C macro for maximum path length?

2020-12-03 Thread Mike Schwab
That is the maximum length of a directory or file name.  Add 1 and
multiply by maximum directory depth (if any).

On Thu, Dec 3, 2020 at 4:56 PM Charles Mills  wrote:
>
> I see
>
> #define FILENAME_MAX  1024
>
> in stdio.h V2R4
>
> Where does that fit into this conundrum?
>
> Charles
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Charles Mills
> Sent: Thursday, December 3, 2020 9:41 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: C macro for maximum path length?
>
> Thanks @Gord, yes, I saw pathconf().
>
> I am starting to "get" the problem.
>
> > I suspect there's a buffer overrun hazard associated with a statically 
> > compiled
> > PATH_MAX.
>
> Never mind the exact lack of a macro and never mind z/OS. Let's say one were 
> to build a hypothetical program with a buffer of hard-coded length 'n', the 
> maximum path length for the environment. What is to stop a future release of 
> the environment from allowing path names with a length greater than 'n'? Now 
> all of a sudden the program is in danger of buffer overrun.
>
> The real question is not "how long can a path be [today]?" but rather "how 
> long might a path be at any future point when this compilation is running?" 
> and that is kind of an unanswerable question, barring some sort of vendor SOD 
> like "it is our intention never to increase maximum path length beyond 4095." 
> It is a problem somewhat analogous to the problem of old programs and an EXEC 
> PARM longer than 100 bytes.
>
> Yes, it is a question that pathconf() answers, although it seems to me that 
> doing it filename by filename is overly complicated: why not just a single, 
> static for a given environment, "maximum path length of any possible file on 
> this system"? (Answer: because they didn't ask me. @Gil's link below provides 
> a better answer.)
>
> I am simply going to recode my routine to finesse the problem. It currently 
> has a "work buffer" of length PATH_MAX. It is kind of a lazy programmer's 
> approach. I am going to re-code it to malloc (actually, to "new" -- it's C++) 
> a buffer of the necessary length for every path that it processes. I need a 
> dedicated area for each name eventually anyway (and yes, that I delete 
> eventually). That's why I say the "work buffer" was kind of a lazy programmer 
> approach.
>
> Thanks all for your patience and explanations.
>
> Charles
>
>
> -Original Message-
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On 
> Behalf Of Paul Gilmartin
> Sent: Thursday, December 3, 2020 9:08 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: C macro for maximum path length?
>
> On Thu, 3 Dec 2020 11:49:05 -0500, Gord Tomlin wrote:
>
> >On 2020-12-03 10:12 AM, Charles Mills wrote:
> >> I believe you, but why then is the macro undefined? Why is the definition 
> >> now commented out?
> >>
> I suspect there's a buffer overrun hazard associated with a statically 
> compiled
> PATH_MAX.  Interesting, apparently well-researched link:
> https://eklitzke.org/path-max-is-tricky
>
> >> >From  (actually CEE.SCEEH.H(LIMITS)) on z/OS V2R4:
> >> /*
> >>   *  POSIX.1 1990 Section 2.8.5 Statement 1065 -
> >>   *  these macros "shall be omitted on specific
> >>   *  implementations where the corresonding value is
> >>   *  >= the stated minimum, but where the value
> >>   *  can vary depending on the file to which it is
> >>   *  applied."
> >>   *...
> >>* #define LINK_MAX
> >>* #define MAX_CANON
> >>* #define MAX_INPUT
> >>* #define NAME_MAX
> >>* #define PATH_MAX
> >>* #define PIPE_BUF
> >>*/
> >
> >"an application may use the/fpathconf/()  
> ><https://pubs.opengroup.org/onlinepubs/009696699/functions/fpathconf.html>,/pathconf/()
> >  <https://pubs.opengroup.org/onlinepubs/009696699/functions/pathconf.html>, 
> >and/sysconf/()  
> ><https://pubs.opengroup.org/onlinepubs/009696699/functions/sysconf.html>  
> >functions to
> >determine the actual value of a limit at runtime."
> >
> >(<https://pubs.opengroup.org/onlinepubs/009696699/basedefs/limits.h.html>)
> >
> OpenGroup/Single UNIX requires an "allocating" form of realpath();
> z/OS XL C/C++ fails to provide one.
>
> More:
> What is the OP doing with this?
>
>

Re: C macro for maximum path length?

2020-12-03 Thread Charles Mills
I see

#define FILENAME_MAX  1024

in stdio.h V2R4

Where does that fit into this conundrum?

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Charles Mills
Sent: Thursday, December 3, 2020 9:41 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: C macro for maximum path length?

Thanks @Gord, yes, I saw pathconf().

I am starting to "get" the problem. 

> I suspect there's a buffer overrun hazard associated with a statically 
> compiled
> PATH_MAX.

Never mind the exact lack of a macro and never mind z/OS. Let's say one were to 
build a hypothetical program with a buffer of hard-coded length 'n', the 
maximum path length for the environment. What is to stop a future release of 
the environment from allowing path names with a length greater than 'n'? Now 
all of a sudden the program is in danger of buffer overrun.

The real question is not "how long can a path be [today]?" but rather "how long 
might a path be at any future point when this compilation is running?" and that 
is kind of an unanswerable question, barring some sort of vendor SOD like "it 
is our intention never to increase maximum path length beyond 4095." It is a 
problem somewhat analogous to the problem of old programs and an EXEC PARM 
longer than 100 bytes.

Yes, it is a question that pathconf() answers, although it seems to me that 
doing it filename by filename is overly complicated: why not just a single, 
static for a given environment, "maximum path length of any possible file on 
this system"? (Answer: because they didn't ask me. @Gil's link below provides a 
better answer.)

I am simply going to recode my routine to finesse the problem. It currently has 
a "work buffer" of length PATH_MAX. It is kind of a lazy programmer's approach. 
I am going to re-code it to malloc (actually, to "new" -- it's C++) a buffer of 
the necessary length for every path that it processes. I need a dedicated area 
for each name eventually anyway (and yes, that I delete eventually). That's why 
I say the "work buffer" was kind of a lazy programmer approach.

Thanks all for your patience and explanations.

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Paul Gilmartin
Sent: Thursday, December 3, 2020 9:08 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: C macro for maximum path length?

On Thu, 3 Dec 2020 11:49:05 -0500, Gord Tomlin wrote:

>On 2020-12-03 10:12 AM, Charles Mills wrote:
>> I believe you, but why then is the macro undefined? Why is the definition 
>> now commented out?
>> 
I suspect there's a buffer overrun hazard associated with a statically compiled
PATH_MAX.  Interesting, apparently well-researched link:
https://eklitzke.org/path-max-is-tricky

>> >From  (actually CEE.SCEEH.H(LIMITS)) on z/OS V2R4:
>> /*
>>   *  POSIX.1 1990 Section 2.8.5 Statement 1065 -
>>   *  these macros "shall be omitted on specific
>>   *  implementations where the corresonding value is
>>   *  >= the stated minimum, but where the value
>>   *  can vary depending on the file to which it is
>>   *  applied."
>>   *...
>>* #define LINK_MAX
>>* #define MAX_CANON
>>* #define MAX_INPUT
>>* #define NAME_MAX
>>* #define PATH_MAX
>>* #define PIPE_BUF
>>*/
>
>"an application may use the/fpathconf/()  
><https://pubs.opengroup.org/onlinepubs/009696699/functions/fpathconf.html>,/pathconf/()
>  <https://pubs.opengroup.org/onlinepubs/009696699/functions/pathconf.html>, 
>and/sysconf/()  
><https://pubs.opengroup.org/onlinepubs/009696699/functions/sysconf.html>  
>functions to
>determine the actual value of a limit at runtime."
>
>(<https://pubs.opengroup.org/onlinepubs/009696699/basedefs/limits.h.html>)
> 
OpenGroup/Single UNIX requires an "allocating" form of realpath();
z/OS XL C/C++ fails to provide one.

More:
What is the OP doing with this?

Even more:
   https://eklitzke.org/path-max-is-tricky

Rexx ADDRESS SYSCALL realpath
returns strings which appear correct but seem to exceed the
PATH_MAX computed by pathconf().  When I expressed astonishment
about that on MVS-OE, WJS replied:
   http://vm.marist.edu/htbin/wlvtype?MVS-OE.61764
   I suspect C passes a buffer of PATH_MAX+1.  The REXX support tends to use a
   local 4K scratch buffer for system call output areas when it can, and does
   in this case.
I.e. just define it as 4K and hope for the best.

I'll try this on Linux:
   ( set -x; while true; do mkdir wombat; cd wombat; pwd; done )

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-03 Thread Charles Mills
Thanks @Gord, yes, I saw pathconf().

I am starting to "get" the problem. 

> I suspect there's a buffer overrun hazard associated with a statically 
> compiled
> PATH_MAX.

Never mind the exact lack of a macro and never mind z/OS. Let's say one were to 
build a hypothetical program with a buffer of hard-coded length 'n', the 
maximum path length for the environment. What is to stop a future release of 
the environment from allowing path names with a length greater than 'n'? Now 
all of a sudden the program is in danger of buffer overrun.

The real question is not "how long can a path be [today]?" but rather "how long 
might a path be at any future point when this compilation is running?" and that 
is kind of an unanswerable question, barring some sort of vendor SOD like "it 
is our intention never to increase maximum path length beyond 4095." It is a 
problem somewhat analogous to the problem of old programs and an EXEC PARM 
longer than 100 bytes.

Yes, it is a question that pathconf() answers, although it seems to me that 
doing it filename by filename is overly complicated: why not just a single, 
static for a given environment, "maximum path length of any possible file on 
this system"? (Answer: because they didn't ask me. @Gil's link below provides a 
better answer.)

I am simply going to recode my routine to finesse the problem. It currently has 
a "work buffer" of length PATH_MAX. It is kind of a lazy programmer's approach. 
I am going to re-code it to malloc (actually, to "new" -- it's C++) a buffer of 
the necessary length for every path that it processes. I need a dedicated area 
for each name eventually anyway (and yes, that I delete eventually). That's why 
I say the "work buffer" was kind of a lazy programmer approach.

Thanks all for your patience and explanations.

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Paul Gilmartin
Sent: Thursday, December 3, 2020 9:08 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: C macro for maximum path length?

On Thu, 3 Dec 2020 11:49:05 -0500, Gord Tomlin wrote:

>On 2020-12-03 10:12 AM, Charles Mills wrote:
>> I believe you, but why then is the macro undefined? Why is the definition 
>> now commented out?
>> 
I suspect there's a buffer overrun hazard associated with a statically compiled
PATH_MAX.  Interesting, apparently well-researched link:
https://eklitzke.org/path-max-is-tricky

>> >From  (actually CEE.SCEEH.H(LIMITS)) on z/OS V2R4:
>> /*
>>   *  POSIX.1 1990 Section 2.8.5 Statement 1065 -
>>   *  these macros "shall be omitted on specific
>>   *  implementations where the corresonding value is
>>   *  >= the stated minimum, but where the value
>>   *  can vary depending on the file to which it is
>>   *  applied."
>>   *...
>>* #define LINK_MAX
>>* #define MAX_CANON
>>* #define MAX_INPUT
>>* #define NAME_MAX
>>* #define PATH_MAX
>>* #define PIPE_BUF
>>*/
>
>"an application may use the/fpathconf/()  
><https://pubs.opengroup.org/onlinepubs/009696699/functions/fpathconf.html>,/pathconf/()
>  <https://pubs.opengroup.org/onlinepubs/009696699/functions/pathconf.html>, 
>and/sysconf/()  
><https://pubs.opengroup.org/onlinepubs/009696699/functions/sysconf.html>  
>functions to
>determine the actual value of a limit at runtime."
>
>(<https://pubs.opengroup.org/onlinepubs/009696699/basedefs/limits.h.html>)
> 
OpenGroup/Single UNIX requires an "allocating" form of realpath();
z/OS XL C/C++ fails to provide one.

More:
What is the OP doing with this?

Even more:
   https://eklitzke.org/path-max-is-tricky

Rexx ADDRESS SYSCALL realpath
returns strings which appear correct but seem to exceed the
PATH_MAX computed by pathconf().  When I expressed astonishment
about that on MVS-OE, WJS replied:
   http://vm.marist.edu/htbin/wlvtype?MVS-OE.61764
   I suspect C passes a buffer of PATH_MAX+1.  The REXX support tends to use a
   local 4K scratch buffer for system call output areas when it can, and does
   in this case.
I.e. just define it as 4K and hope for the best.

I'll try this on Linux:
   ( set -x; while true; do mkdir wombat; cd wombat; pwd; done )

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-03 Thread Paul Gilmartin
On Thu, 3 Dec 2020 11:49:05 -0500, Gord Tomlin wrote:

>On 2020-12-03 10:12 AM, Charles Mills wrote:
>> I believe you, but why then is the macro undefined? Why is the definition 
>> now commented out?
>> 
I suspect there's a buffer overrun hazard associated with a statically compiled
PATH_MAX.  Interesting, apparently well-researched link:
https://eklitzke.org/path-max-is-tricky

>> >From  (actually CEE.SCEEH.H(LIMITS)) on z/OS V2R4:
>> /*
>>   *  POSIX.1 1990 Section 2.8.5 Statement 1065 -
>>   *  these macros "shall be omitted on specific
>>   *  implementations where the corresonding value is
>>   *  >= the stated minimum, but where the value
>>   *  can vary depending on the file to which it is
>>   *  applied."
>>   *...
>>* #define LINK_MAX
>>* #define MAX_CANON
>>* #define MAX_INPUT
>>* #define NAME_MAX
>>* #define PATH_MAX
>>* #define PIPE_BUF
>>*/
>
>"an application may use the/fpathconf/()  
>,/pathconf/()
>  , 
>and/sysconf/()  
>  
>functions to
>determine the actual value of a limit at runtime."
>
>()
> 
OpenGroup/Single UNIX requires an "allocating" form of realpath();
z/OS XL C/C++ fails to provide one.

More:
What is the OP doing with this?

Even more:
   https://eklitzke.org/path-max-is-tricky

Rexx ADDRESS SYSCALL realpath
returns strings which appear correct but seem to exceed the
PATH_MAX computed by pathconf().  When I expressed astonishment
about that on MVS-OE, WJS replied:
   http://vm.marist.edu/htbin/wlvtype?MVS-OE.61764
   I suspect C passes a buffer of PATH_MAX+1.  The REXX support tends to use a
   local 4K scratch buffer for system call output areas when it can, and does
   in this case.
I.e. just define it as 4K and hope for the best.

I'll try this on Linux:
   ( set -x; while true; do mkdir wombat; cd wombat; pwd; done )

-Original Message-
From: Paul Gilmartin 
Sent: Wednesday, December 2, 2020 3:33 PM

read: https://www.zsh.org/mla/workers/2000/msg03393.html
etc. and weep.

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-03 Thread Gord Tomlin

On 2020-12-03 10:12 AM, Charles Mills wrote:

I believe you, but why then is the macro undefined? Why is the definition now 
commented out?

>From  (actually CEE.SCEEH.H(LIMITS)) on z/OS V2R4:

/*
  *  POSIX.1 1990 Section 2.8.5 Statement 1065 -
  *  these macros "shall be omitted on specific
  *  implementations where the corresonding value is
  *  >= the stated minimum, but where the value
  *  can vary depending on the file to which it is
  *  applied."
  *
  *
  */
  
  /*

   * #define LINK_MAX
   * #define MAX_CANON
   * #define MAX_INPUT
   * #define NAME_MAX
   * #define PATH_MAX
   * #define PIPE_BUF
   */

Charles


Tried this?

"an application may use the/fpathconf/()  
,/pathconf/()  
, and/sysconf/()  
  functions to
determine the actual value of a limit at runtime."

(https://pubs.opengroup.org/onlinepubs/009696699/basedefs/limits.h.html)

--

Regards, Gord Tomlin
Action Software International
(a division of Mazda Computer Corporation)
Tel: (905) 470-7113, Fax: (905) 470-6507
Support: https://actionsoftware.com/support/


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-03 Thread Charles Mills
I believe you, but why then is the macro undefined? Why is the definition now 
commented out?

>From  (actually CEE.SCEEH.H(LIMITS)) on z/OS V2R4:

/*   
 *  POSIX.1 1990 Section 2.8.5 Statement 1065 -  
 *  these macros "shall be omitted on specific   
 *  implementations where the corresonding value is  
 *  >= the stated minimum, but where the value   
 *  can vary depending on the file to which it is
 *  applied."
 *   
 *   
 */  
 
 /*  
  * #define LINK_MAX 
  * #define MAX_CANON
  * #define MAX_INPUT
  * #define NAME_MAX 
  * #define PATH_MAX 
  * #define PIPE_BUF 
  */ 

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of David Crayford
Sent: Thursday, December 3, 2020 5:32 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: C macro for maximum path length?

PATH_MAX on z/OS is 1023. This is documented in lots of error messages and BPX 
assembler services. 

> On 3 Dec 2020, at 7:16 am, Charles Mills  wrote:
> 
> I have some code that compiles both under Windows Visual Studio and z/OS
> XLC.
> 
> In Windows the maximum length of a file path is defined by _MAX_PATH and
> __MAX_PATH (I guess MS thinks two macros are better than one).
> 
> What is the equivalent macro for XLC? Failing that, what *is* the maximum
> path length so I can define my own macro?
> 
> Posix defines PATH_MAX and I see references to it in the z/OS doc but it
> does not seem to be defined for my compile. How do I pick that up? And yes,
> I have
> 
> #define _XOPEN_SOURCE_EXTENDED 1
> #include 
> 
> z/OS V2R4
> 
> Thanks,
> 
> Charles 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-03 Thread David Crayford
PATH_MAX on z/OS is 1023. This is documented in lots of error messages and BPX 
assembler services. 

> On 3 Dec 2020, at 7:16 am, Charles Mills  wrote:
> 
> I have some code that compiles both under Windows Visual Studio and z/OS
> XLC.
> 
> In Windows the maximum length of a file path is defined by _MAX_PATH and
> __MAX_PATH (I guess MS thinks two macros are better than one).
> 
> What is the equivalent macro for XLC? Failing that, what *is* the maximum
> path length so I can define my own macro?
> 
> Posix defines PATH_MAX and I see references to it in the z/OS doc but it
> does not seem to be defined for my compile. How do I pick that up? And yes,
> I have
> 
> #define _XOPEN_SOURCE_EXTENDED 1
> #include 
> 
> z/OS V2R4
> 
> Thanks,
> 
> Charles 
> 
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-02 Thread Paul Gilmartin
On Wed, 2 Dec 2020 16:43:11 -0800, Charles Mills  wrote:

>This code has been changed since I last compiled it on Z but I don't think I 
>changed that reference. It appears that PATH_MAX went away somewhere between 
>V2R2 and V2R4. I see some comments in limits.h that are consistent with that.
>
Longer, I think.  My utility realpath.c is dated 2012.
(I recompiled it on Mac OS and pedantic C issued 3 warnings;
one a genuine error in an unlikely path.)

>I picked up _POSIX_PATH_MAX from limits.h. I am running POSIX(ON). It that a 
>valid maximum path length or am I likely to get whacked with a buffer overflow?
>
>Re-stating the question, what is the maximum path length on z/OS UNIX? Can it 
>really be true that there is no hard maximum? I really have to ask pathconf() 
>every time? No maximum of possible maximums? The maximum is really whatever 
>z/OS wants it to be today?
>
Alas, I suspect yes. Single UNIX says:
https://pubs.opengroup.org/onlinepubs/9699919799/functions/realpath.html#tag_16_479_03
If resolved_name is a null pointer, the generated pathname shall be stored 
as a null-terminated
string in a buffer allocated as if by a call to malloc().
...
If the resolved_name argument is a null pointer, the pointer returned by 
realpath()
can be passed to free().
(Probably a good idea.)  But z/OS XL C C++:
 EINVAL
[the] resolved_name argument is a NULL pointer.

I suspect the concern may be that in the future the constraint may be relaxed
but you'll run a dusty program object allocating a static resolved_name and
incur a buffer overrun.

>What is FILENAME_MAX, referenced in 
>https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.4.0/com.ibm.zos.v2r4.bpxbd00/fldata.htm
> ?
>
Don't know.  Can that be misused to incur a buffer overflow?  I suppose if
you parse a pathname and copy a component into a static char[ FILENAME_MAX ].

-- gil

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-02 Thread Charles Mills
This code has been changed since I last compiled it on Z but I don't think I 
changed that reference. It appears that PATH_MAX went away somewhere between 
V2R2 and V2R4. I see some comments in limits.h that are consistent with that.

I picked up _POSIX_PATH_MAX from limits.h. I am running POSIX(ON). It that a 
valid maximum path length or am I likely to get whacked with a buffer overflow?

Re-stating the question, what is the maximum path length on z/OS UNIX? Can it 
really be true that there is no hard maximum? I really have to ask pathconf() 
every time? No maximum of possible maximums? The maximum is really whatever 
z/OS wants it to be today?

What is FILENAME_MAX, referenced in 
https://www.ibm.com/support/knowledgecenter/en/SSLTBW_2.4.0/com.ibm.zos.v2r4.bpxbd00/fldata.htm
 ?

Charles


-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Paul Gilmartin
Sent: Wednesday, December 2, 2020 3:25 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: C macro for maximum path length?

On Wed, 2 Dec 2020 15:16:10 -0800, Charles Mills wrote:

>I have some code that compiles both under Windows Visual Studio and z/OS
>XLC.
>
>In Windows the maximum length of a file path is defined by _MAX_PATH and
>__MAX_PATH (I guess MS thinks two macros are better than one).
>
>What is the equivalent macro for XLC? Failing that, what *is* the maximum
>path length so I can define my own macro?
>
>Posix defines PATH_MAX and I see references to it in the z/OS doc but it
>does not seem to be defined for my compile. How do I pick that up? And yes,
>I have
>
>#define _XOPEN_SOURCE_EXTENDED 1
>#include 
>
PATH_MAX is deprecated (I thought even by POSIX).  IBM says this
it is because the value differs among filesystems.  Here is some
code where I used the recommended alternative in a call to realpath().
(realpath() is dreadful: an invitation to buffer overflows.)

/* * */
/* Doc: Print the resolved pathname of each argument
*/
#include 
#include 
#include 
#include 
#undef PATH_MAX

int main( int argc, char ** argv ) {
char **file_name;
int status=0;

for ( file_name = argv; *++file_name; ) {
long PATH_MAX;

if ( ( PATH_MAX = pathconf( *file_name, _PC_PATH_MAX ) )>0 ) {
char *resolved_name;

if ( resolved_name = malloc( PATH_MAX + 1 ) ); {
if ( realpath( *file_name, resolved_name ) ) {
printf( "%s\n", resolved_name );
free( resolved_name );
continue; }
free( resolved_name ); } }

perror( *file_name );
status = 1; }
exit( status );  }

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


Re: C macro for maximum path length?

2020-12-02 Thread Paul Gilmartin
On Wed, 2 Dec 2020 15:16:10 -0800, Charles Mills wrote:

>I have some code that compiles both under Windows Visual Studio and z/OS
>XLC.
>
>In Windows the maximum length of a file path is defined by _MAX_PATH and
>__MAX_PATH (I guess MS thinks two macros are better than one).
>
>What is the equivalent macro for XLC? Failing that, what *is* the maximum
>path length so I can define my own macro?
>
>Posix defines PATH_MAX and I see references to it in the z/OS doc but it
>does not seem to be defined for my compile. How do I pick that up? And yes,
>I have
>
>#define _XOPEN_SOURCE_EXTENDED 1
>#include 
>
PATH_MAX is deprecated (I thought even by POSIX).  IBM says this
it is because the value differs among filesystems.  Here is some
code where I used the recommended alternative in a call to realpath().
(realpath() is dreadful: an invitation to buffer overflows.)

/* * */
/* Doc: Print the resolved pathname of each argument
*/
#include 
#include 
#include 
#include 
#undef PATH_MAX

int main( int argc, char ** argv ) {
char **file_name;
int status=0;

for ( file_name = argv; *++file_name; ) {
long PATH_MAX;

if ( ( PATH_MAX = pathconf( *file_name, _PC_PATH_MAX ) )>0 ) {
char *resolved_name;

if ( resolved_name = malloc( PATH_MAX + 1 ) ); {
if ( realpath( *file_name, resolved_name ) ) {
printf( "%s\n", resolved_name );
free( resolved_name );
continue; }
free( resolved_name ); } }

perror( *file_name );
status = 1; }
exit( status );  }

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


C macro for maximum path length?

2020-12-02 Thread Charles Mills
I have some code that compiles both under Windows Visual Studio and z/OS
XLC.

In Windows the maximum length of a file path is defined by _MAX_PATH and
__MAX_PATH (I guess MS thinks two macros are better than one).

What is the equivalent macro for XLC? Failing that, what *is* the maximum
path length so I can define my own macro?

Posix defines PATH_MAX and I see references to it in the z/OS doc but it
does not seem to be defined for my compile. How do I pick that up? And yes,
I have

#define _XOPEN_SOURCE_EXTENDED 1
#include 

z/OS V2R4

Thanks,

Charles 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN