Re: The Business Case for Pipes in the z/OS Base (was: Re: REXX - Interpret or Value - Which is better?)
On Mon, 20 Sep 2021 16:26:59 -0500, Hobart Spitz wrote: >... >For those who don't know, in the typical case, a record passes through all >possible stages before the next record begins the same trip. Each record >stays in the working page set, at least partially, during the entire time. >Parts that are referenced have a good chance of staying cache resident >between stages. > I'm guessing the atypical case is a stage such as FANOUT which necessarily copies the data. >... My best guess is that Pipes, in this analogy, would have only >5%-15% of the narrow openings as an equivalent UNIX piping command, meaning >that the data (sand) would flow 85-95% faster in the Pipes "hourglass". > ... I suspect the cost of moving data is overwhelmed by the cost of process switching. And z/OS UNIX is probably the worst of all UNIXem because its design hasn't been optimized for process switching. (I wonder whether nowadays z/OS creates more address spaces for job step initiation or for fork(). I'm confident that the design remains optimized for the former, regardless.) But remember that nowadays silicon is ofteen cheaper than carbon. With POSIX pipes I can: 407 $ ls | wc 2 2 17 (got it right the first time. A useless example, admittedly.) What would that look like using JCL an Batchpipes, replacing "ls" with LISTDS and "wc" with (utility of your chiice)? (I wouln't get it right the first time.) -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: The Business Case for Pipes in the z/OS Base (was: Re: REXX - Interpret or Value - Which is better?)
Yes, I was not aware of the subsystem idea and was just throwing out an idea. On Tue, Sep 21, 2021 at 10:14 PM Paul Gilmartin <000433f07816-dmarc-requ...@listserv.ua.edu> wrote: > > On Mon, 20 Sep 2021 22:30:32 -0500, Mike Schwab wrote: > > >How about > >//ddname DD DISP=(NEW,DELETE,DELETE), > >// DCB=(DSORG=PS[only],RECFM=F/FA/FM/V/VA/VM/U?/UA?/UM?/[no > >B]),BLKSIZE=n), > >// PIPES=(PROGNAME,'100/32k byte parm to select or modify records', > >// ddname1,ddname2,...,ddnameN), > >and the OPEN (ddname,OUTPUT) loads the PIPES program in (sub)task memory) > >and sets up the PUT/WRITE to DDNAME to call the program instead? > >The close unloads the PIPES program in (subtask memory). > >VSAM writes accepted like a DSORG=PS? > >Any other parameters possible? > > > What manual did you find that in? I don't see it in the JCL Ref. > > Are you makiing it up? How is it preferable to SUBSYS=PIPE? (BP01? > I see each among various IBM pages.) > > >A pipes DDNAME can use a PIPES parameter for a subsequent program. > > > "subsequent"? Not "concurrent"? (But perhaps you mean lexically subsequent > but temporally concurrent.) > > >Of course, ddnames can only be used by the EXEC PGM= or one of the > >pipes programs. > >And all pipes programs remain in memory until step end. > >In case of abend / restart the checkpoint is taken in EXEC PGM= and > >abandon updates / writes in any pipes program after the checkpoint. > > ... > > -- gil > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: The Business Case for Pipes in the z/OS Base (was: Re: REXX - Interpret or Value - Which is better?)
On Mon, 20 Sep 2021 22:30:32 -0500, Mike Schwab wrote: >How about >//ddname DD DISP=(NEW,DELETE,DELETE), >// DCB=(DSORG=PS[only],RECFM=F/FA/FM/V/VA/VM/U?/UA?/UM?/[no B]),BLKSIZE=n), >// PIPES=(PROGNAME,'100/32k byte parm to select or modify records', >// ddname1,ddname2,...,ddnameN), >and the OPEN (ddname,OUTPUT) loads the PIPES program in (sub)task memory) >and sets up the PUT/WRITE to DDNAME to call the program instead? >The close unloads the PIPES program in (subtask memory). >VSAM writes accepted like a DSORG=PS? >Any other parameters possible? > What manual did you find that in? I don't see it in the JCL Ref. Are you makiing it up? How is it preferable to SUBSYS=PIPE? (BP01? I see each among various IBM pages.) >A pipes DDNAME can use a PIPES parameter for a subsequent program. > "subsequent"? Not "concurrent"? (But perhaps you mean lexically subsequent but temporally concurrent.) >Of course, ddnames can only be used by the EXEC PGM= or one of the >pipes programs. >And all pipes programs remain in memory until step end. >In case of abend / restart the checkpoint is taken in EXEC PGM= and >abandon updates / writes in any pipes program after the checkpoint. > ... -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: The Business Case for Pipes in the z/OS Base (was: Re: REXX - Interpret or Value - Which is better?)
On Tue, 21 Sep 2021 21:03:14 -0500, Mike Schwab wrote: >If a SORT (or other similar temporary data store) program is one of >the pipe programs, when the EXEC PGM= program closes the output file >then the program holding the data needs to output the stored data to >output ddnames (pipe or output files). > Are you thinking of MS-DOS pseudo-"pipes" where the upstream program wrote a temporary file under-the-covers and the downstream program processed it? A pipe in syntax only. Even Windows is better nowadays. SORT is a bad conceptual example for Pipethink because SORT can't write its first output record until it has read its last input record. Better to envision a filter which re-formats log records from a long-running (or never-terminating) program, writing a file to be browsed with SDSF or tail -f in real time. -- gil -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: The Business Case for Pipes in the z/OS Base (was: Re: REXX - Interpret or Value - Which is better?)
If a SORT (or other similar temporary data store) program is one of the pipe programs, when the EXEC PGM= program closes the output file then the program holding the data needs to output the stored data to output ddnames (pipe or output files). On Tue, Sep 21, 2021 at 5:25 AM Mike Schwab wrote: > > Oh, and PIPES=(program,'parm',ddname1,ddname2,ddnaen) the ddname1 gets > as input the one record written to the //ddname DD PIPES ddname. > > On Mon, Sep 20, 2021 at 10:30 PM Mike Schwab wrote: > > > > How about > > //ddname DD DISP=(NEW,DELETE,DELETE), > > // DCB=(DSORG=PS[only],RECFM=F/FA/FM/V/VA/VM/U?/UA?/UM?/[no > > B]),BLKSIZE=n), > > // PIPES=(PROGNAME,'100/32k byte parm to select or modify records', > > // ddname1,ddname2,...,ddnameN), > > and the OPEN (ddname,OUTPUT) loads the PIPES program in (sub)task memory) > > and sets up the PUT/WRITE to DDNAME to call the program instead? > > The close unloads the PIPES program in (subtask memory). > > VSAM writes accepted like a DSORG=PS? > > Any other parameters possible? > > > > A pipes DDNAME can use a PIPES parameter for a subsequent program. > > Of course, ddnames can only be used by the EXEC PGM= or one of the > > pipes programs. > > And all pipes programs remain in memory until step end. > > In case of abend / restart the checkpoint is taken in EXEC PGM= and > > abandon updates / writes in any pipes program after the checkpoint. > > > > On Mon, Sep 20, 2021 at 9:12 PM Paul Gilmartin > > <000433f07816-dmarc-requ...@listserv.ua.edu> wrote: > > > > > > On Mon, 20 Sep 2021 19:10:16 -0500, Mike Schwab wrote: > > > > > > >So, in a sense, instead of pipes, the programs could be modified so > > > >that instead of outputting a record, call the next program passing the > > > >record as input. > > > > > > > No. > > > Rather than requiring every utility to be modified in order to use the > > > capability, it should be provided at the access method level or lower, > > > transparent, so the top programs are oblivious to it. > > > > > > And there should be provision for inserting filters between a producer > > > and a consumer to convert incompatible data formats. > > > > > > And ... what am I thinking? > > > > > > -- gil > > > > > > -- > > > For IBM-MAIN subscribe / signoff / archive access instructions, > > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > > > > > -- > > Mike A Schwab, Springfield IL USA > > Where do Forest Rangers go to get away from it all? > > > > -- > Mike A Schwab, Springfield IL USA > Where do Forest Rangers go to get away from it all? -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: DYNAM vs NODYNAM
If you use the integrated CICS translator you can use the newish >>CALLINTERFACE DYNAMIC statement in order to be able to use CALL literal and have it be a dynamic call. All calls to DFHEI1 are still statically called when you do this. I'd like to see the NODYNAM requirement for CICS lifted. From: IBM Mainframe Discussion List on behalf of Tony Thigpen Sent: Tuesday, September 21, 2021 8:47 AM To: IBM-MAIN@LISTSERV.UA.EDU Subject: Re: DYNAM vs NODYNAM Because the translator generates: Call 'DFHEI1' . And you don't want them dynamic. Tony Thigpen Erik Janssen wrote on 9/21/21 9:57 AM: > Hello All, > > In the COBOL documentation at > https://www.ibm.com/docs/en/cobol-zos/6.2?topic=environment-choosing-dynam-nodynam-compiler-option > it is said that for example CICS requires the NODYNAM compile option. My > idea was that the sole purpose of DYNAM vs NODYNAM was to control dynamic vs > static calls, where DYNAM will always lead to dynamic calls, but with NODYNAM > only a CALL literal will lead to a static call, a CALL identifier will still > lead to a dynamic call. So is it only possible under CICS to use CALL literal > in this case (must all calls be static), or can you still use dynamic calling > with a CALL identifier? And if the latter is the case, then why is there a > requirement to use NODYNAM? Is there some other effect of DYNAM vs NODYNAM > that I'm not seeing? > > Kind regards, > > Erik Janssen. > > -- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: DYNAM vs NODYNAM
Ah yes, That makes sense. Kind regards, Erik. On Tue, 21 Sep 2021 10:47:37 -0400, Tony Thigpen wrote: >Because the translator generates: > Call 'DFHEI1' . >And you don't want them dynamic. > >Tony Thigpen > >send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: DYNAM vs NODYNAM
Because the translator generates: Call 'DFHEI1' . And you don't want them dynamic. Tony Thigpen Erik Janssen wrote on 9/21/21 9:57 AM: Hello All, In the COBOL documentation at https://www.ibm.com/docs/en/cobol-zos/6.2?topic=environment-choosing-dynam-nodynam-compiler-option it is said that for example CICS requires the NODYNAM compile option. My idea was that the sole purpose of DYNAM vs NODYNAM was to control dynamic vs static calls, where DYNAM will always lead to dynamic calls, but with NODYNAM only a CALL literal will lead to a static call, a CALL identifier will still lead to a dynamic call. So is it only possible under CICS to use CALL literal in this case (must all calls be static), or can you still use dynamic calling with a CALL identifier? And if the latter is the case, then why is there a requirement to use NODYNAM? Is there some other effect of DYNAM vs NODYNAM that I'm not seeing? Kind regards, Erik Janssen. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
DYNAM vs NODYNAM
Hello All, In the COBOL documentation at https://www.ibm.com/docs/en/cobol-zos/6.2?topic=environment-choosing-dynam-nodynam-compiler-option it is said that for example CICS requires the NODYNAM compile option. My idea was that the sole purpose of DYNAM vs NODYNAM was to control dynamic vs static calls, where DYNAM will always lead to dynamic calls, but with NODYNAM only a CALL literal will lead to a static call, a CALL identifier will still lead to a dynamic call. So is it only possible under CICS to use CALL literal in this case (must all calls be static), or can you still use dynamic calling with a CALL identifier? And if the latter is the case, then why is there a requirement to use NODYNAM? Is there some other effect of DYNAM vs NODYNAM that I'm not seeing? Kind regards, Erik Janssen. -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
RMF SMF "broken" records
Now that we have established that a "broken" record is really a "broken-up" (i.e., "split") record, there are approaches for understanding the split and dealing with the multiple records that resulted from the split. Use of the timestamp in the SMF record is not the way. >From the RMF folks: Users who do not have 2.5 should use the fields in the Reassembly area of the RMF Product section mapped by ERBPRO to analyze the split records without reassembling them into a large SMF record. This can be done since the split records are self-containing, i.e. SMF header triplet (offset,length,number of data sections) are adjusted to reflect the right location and number of data sections. In other words, applications can analyze each single split SMF record and logically combine the extracted information in the application w/o reassembling multiple small records into one large record. The Reassembly area starts at offset '68'X of the RMF Product section and contains a 16-bytes header (mapped by RMFRHD) and multiple 4-bytes reassembly information blocks (mapped by RMFRBL). Header field RMFRBR can be read to understand how many split records have to be analyzed to get the "big picture". RMFRSQ tells the user the sequence number of the split SMF record. Use RMF Product section field RMFIST, RMFDAT, RMFINT and SMF header fields SMFRTY, SMFSTY, SMFSID (mapped by ERBHDR) to determine whether split records belong to the same original SMF record. z/OS 2.5 users can use the new GRBSMFR service as described on page 13ff in the z/OS Data Gatherer Programmer's Guide. https://www-01.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R5gc315701?OpenDocument The service is 2.5 only and is not available on earlier releases Peter Relson z/OS Core Technology Design Peter rel...@us.ibm.com 1-845-435-83908+295-8390 -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: The Business Case for Pipes in the z/OS Base (was: Re: REXX - Interpret or Value - Which is better?)
Oh, and PIPES=(program,'parm',ddname1,ddname2,ddnaen) the ddname1 gets as input the one record written to the //ddname DD PIPES ddname. On Mon, Sep 20, 2021 at 10:30 PM Mike Schwab wrote: > > How about > //ddname DD DISP=(NEW,DELETE,DELETE), > // DCB=(DSORG=PS[only],RECFM=F/FA/FM/V/VA/VM/U?/UA?/UM?/[no > B]),BLKSIZE=n), > // PIPES=(PROGNAME,'100/32k byte parm to select or modify records', > // ddname1,ddname2,...,ddnameN), > and the OPEN (ddname,OUTPUT) loads the PIPES program in (sub)task memory) > and sets up the PUT/WRITE to DDNAME to call the program instead? > The close unloads the PIPES program in (subtask memory). > VSAM writes accepted like a DSORG=PS? > Any other parameters possible? > > A pipes DDNAME can use a PIPES parameter for a subsequent program. > Of course, ddnames can only be used by the EXEC PGM= or one of the > pipes programs. > And all pipes programs remain in memory until step end. > In case of abend / restart the checkpoint is taken in EXEC PGM= and > abandon updates / writes in any pipes program after the checkpoint. > > On Mon, Sep 20, 2021 at 9:12 PM Paul Gilmartin > <000433f07816-dmarc-requ...@listserv.ua.edu> wrote: > > > > On Mon, 20 Sep 2021 19:10:16 -0500, Mike Schwab wrote: > > > > >So, in a sense, instead of pipes, the programs could be modified so > > >that instead of outputting a record, call the next program passing the > > >record as input. > > > > > No. > > Rather than requiring every utility to be modified in order to use the > > capability, it should be provided at the access method level or lower, > > transparent, so the top programs are oblivious to it. > > > > And there should be provision for inserting filters between a producer > > and a consumer to convert incompatible data formats. > > > > And ... what am I thinking? > > > > -- gil > > > > -- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN > > > > -- > Mike A Schwab, Springfield IL USA > Where do Forest Rangers go to get away from it all? -- Mike A Schwab, Springfield IL USA Where do Forest Rangers go to get away from it all? -- For IBM-MAIN subscribe / signoff / archive access instructions, send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
Re: How to use the FILTER command for OPERLOG in batch
My advice is to have a subroutine/function in your REXX that converts one or
more physical lines from the ISFLINE stem into a logical message.
This will mean processing multiple ISFLINE variables when you encounter a
multi-line WTO using the indicators in the first 2 bytes of ISFLINE and the
correlation number ('nnn') - for example :
MR000 SYS1 ddd hh:mm:ss.th jobname msgid text nnn
LR nnn
title_text
DR nnn
data_text
DR nnn
data_text
ER nnn
last_data_text
Rob Scott
Rocket Software
-Original Message-
From: IBM Mainframe Discussion List On Behalf Of
ibmmain
Sent: 20 September 2021 17:03
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: How to use the FILTER command for OPERLOG in batch
EXTERNAL EMAIL
Hi Rob
Thanks for your help!
I have another question.
When we use the SDSF “FILTER" command on the operlog, the output of
"FILTER" is the complete messages ( for multi-line messages)
If we use "POS" or "INDEX" on the OPERLOG rows, how to
handle multi-line messages?
Thanks a lot!
Jason Cai
发件人:
"IBM Mainframe Discussion List"
https://my.rocketsoftware.com/RocketCommunity/RCEmailSupport
Unsubscribe from Marketing Messages/Manage Your Subscription Preferences -
http://www.rocketsoftware.com/manage-your-email-preferences
Privacy Policy - http://www.rocketsoftware.com/company/legal/privacy-policy
This communication and any attachments may contain confidential information of
Rocket Software, Inc. All unauthorized use, disclosure or distribution is
prohibited. If you are not the intended recipient, please notify Rocket
Software immediately and destroy all copies of this communication. Thank you.
--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
