Andrew, Your performance work in this area is very interesting to me. I would love to hear more if you ever write up your findings, even informally. We had a customer a few years ago who seem to have performance problems with fork/execs related to BPXAS address space startups. IBM worked on it for a long time and then somehow it magically went away and no one could say why.
I'm not sure exactly what you are doing in your benchmarks, but in COZBATCH we do an explicit local spawn of your login shell so as to get it to run in the same address space. Here's proof that pipeline shell commands can run in the job step address space: //SHELL EXEC PGM=COZBATCH //IN DD DSN=SYS1.MACLIB(ACB),DISP=SHR //OUT DD SYSOUT=* //STDIN DD * # run the user's login shell in the same address space fromdsn //DD:IN | grep IBM | todsn //DD:OUT // fromdsn(DD:IN)[N]: 79 records/6320 bytes read; 6399 bytes written in 0 milliseconds. todsn(DD:OUT)[N]: 243 bytes read; 3 records/240 bytes written in 0 milliseconds. CoZBatch[I]: returning rc=exitcode=0 there are all sorts of undocumented pitfalls surrounding where /bin/sh does a spawn vs fork. For example, a "builtin" is always forked when used in a pipeline, except for when it is last and you have -o pipecurrent set. You can hack this by using /bin/cp or /bin/cat (say, with a //DD:xxx). Gil will point out that this usage isn't explicitly documented by IBM (as with many things). Of course, this isn't that interesting to people who believe that things run "in z/OS Unix" rather than the reality that pretty much any job/task can *use* z/OS UNIX services, filesystems, programs, etc. :-) We find that using Unix shell scripts in z/OS batch works really nicely. On Wed, Oct 28, 2020 at 6:41 PM Andrew Rowley <and...@blackhillsoftware.com> wrote: > On 29/10/2020 8:40 am, Kirk Wolf wrote: > > /bin/sh (when it uses spawn() and _BPX_SHAREAS=YES/MUST) will try to > start > > the child process in the **same** address space as the parent. > > > > I just reran the same script with _BPX_SHAREAS=YES to see what it looked > like. > > It cut the number of SMF records from 20,000 to 4,000 with only 1 > process showing up per iteration - maybe required for the pipe? > > Elapsed time didn't change. CPU time was reduced by about 20%. > > With _BPX_SHAREAS=YES you could no longer see information on the > commands making up the shell script. From the runs without > _BPX_SHAREAS=YES the SMF data showed most of the elapsed time was due to > the setfacl command. > > -- > > Andrew Rowley > Black Hill Software > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
