Sorry for the delay in responding. I'm guessing that Sri is thinking of BatcfhPipes between two otherwise independent batch JOBs. That is different animal entirely, even though BatchPipes and BatchPipesWorks (Pipelines/Pipes) can work together.
As for Jesse's question about how to do restart, the short answer is "it depends". In a typical scenario, each temporary JCL dataset becomes a pipe character ( | ), one or more JOB steps become a single stage, and the Pipe's run time is near that of the longest step. If that longest JCL step is heavily I/O bound on temporary or passed datasets, the run time could be much, much shorter. The point is implementing a multi-step job in as a single Pipe could slash the run time and eliminate any need for restart. You just rerun. If you still need to be able restart, you can put your PIPE command(s) in a loop via REXX, and batch (by record count), partition (by key), or otherwise track and record your progress. Code checks at program start-up, when the the process runs again, restart at the last point of progress (A.K.A checkpoint). Automatic restart, where you don't have to specify the restart point, may be compelling in some cases. That said, that you need to have a good understanding of your process, of JCL under the covers, and of Pipes and REXX. (Batch REXX is frequently be part of the solution.) And you have to be a bit creative. Yes, that was the short answer!!! I started to write the long answer out, but it just became way too long. There were just too many situations, conditions, and approaches to consider, and a moderate knowledge of what is going on under the covers when an initiator is processing JCL is needed. Despite giving talks on replacing JCL with REXX (plus a stub JOB), I realized that there was little benefit, to the mass conversion/implementation of batch REXX and/or PIpes. I further realized that the expertise does not exist in most sites to do such a conversion, and that it's easier to create new JCL by copying old JCL (mistakes and all), then to do it cleanly with a new, possibly risky, approach. The fact that z/OS is only major platform where the commonly used foreground scripting language differs from that used in the background. That said, there are situations where the efficiency of Pipes and/or flexibility of REXX can be crucial to solve otherwise intractable batch problems. Let me cite two actual examples from the last few years, where I've had to address JOB/process issues because critical batch windows were being missed almost daily. In both cases, there were processes that were failing, and which had to be rerun. (Only one used Pipes in the solution. The other was solved with a batch REXX exec.) The common thread was that by stepping back and looking at the big picture the issues were addressed, and the batch windows were met consistently Citing another example, I once replaced an over-maintained COBOL program of about 1000 lines with REXX exec of about a dozen lines. The only significant statement was a Pipe host command where all the processing was done. The point is that fresh eyes and fresh approaches can do wonders. The less JCL you write, versus Pipes and REXX, the more flexibility you have to do what your business needs require and not what the rules of JCL dictate. So to understand how to do restart, please suspend disbelief, at least briefly, and accept that most multi-step JOBs can be done in a single step, using REXX and/or Pipes. More generally, the best candidates for rewriting in Pipes and/or REXX are likely to be JOBs that meet one or more of these conditions; - Are many stepped jobs with significant I/O to/from temporary/passed datasets. - Have mostly utility steps versus application program steps. - Miss their batch windows and/or fail frequently. - Run more frequently than daily. - Have to be restarted frequently. - Have JCL that has been over-maintained and/or that are overly complex and would benefit from brevity. - Have many nearly identical instances that differ only in a few PROC parameters. - Run very long or are very costly. - Represent short running processes that you want to run in foreground. - Represent processes that you want to port to a different platform, e.g. z/VM, *nix, Windows, etc. - Have complex restart requirements or need automatic restart at resubmission. - Needs to check a condition and retry, rather than restart. - Are in need of application specific automation. What really closed down my idea of going into full detail, however, was when I was browsing tutorial videos on JCL, to get a feel for the current state of the what people were doing. (I've been on z/VM for about 90% of the time recently.) In one case, I found an IEFBR14 step with SYSPRINT, SYSOUT, and SYSUDUMP DDs. I was afraid I would go blind!!! ;-D (For those who may not know, IEFBR14 is a null program that does absolutely nothing; it allows you to use the side effects of DDs, typically. The entire program is SR 15,15; BR 14; hence the name. It does not use SYSPRINT, SYSOUT, or SYSUDUMP.) I plan to update and improve my Pipes and REXX talks and associated batch methods. They did not go far enough and much has changed in the intervening decades. Some of the routine initiator emulation funcitons can be done with common support routines, rather than code duplication. To do that will require access to a mainframe environment. I am considering offering, in exchange for access to either a TSO environment Pipes or z/VM, to review and, if warranted in my opinion, to rewrite one problematic batch JOB per corporation/organization, that might run better as REXX and/or Pipes, at no charge. (The target environment does not have to be the one that you let me use for updating my talks.) If you are interested, contact me at my personal email address with brief descriptions of up to 3 candidates and their problem(s): [email protected] . On Tue, Nov 21, 2017 at 8:24 PM, Paul Gilmartin < [email protected]> wrote: > On Tue, 21 Nov 2017 15:37:57 -0700, Sri h Kolusu wrote: > > > >If one end of the pipe fails for any reason, then both ends would have to > >be rerun. The redbook "Batch Modernization on z/OS" chapter 16 > >explains the use of Batch pipes. Martin Packer is one of the co-author. > > > >http://www.redbooks.ibm.com/abstracts/sg247779.html > > > SORT is a weak instructional tool for pipe concepts. As in: > CMS Pipelines User’s Guide and Reference: > sort—Order Records > ... > Record Delay: sort delays all records until end-of-file. > > I.e. SORT writes nothing until it has completely read its input. One > might as well write to a temp file and sort that. > > A better example might be to use ICETOOL to do a minor reformtting, > record-by-record, of its input. And generate that input with an EXEC > that emits a record every few seconds. And use SDSF to track its > output in real time. But beware: a Master Plumber is apt to jump in > with SPECS stages that do the same thing, faster, better, cheaper. > > QSAM buffering will introduce some parasitic delay unless you use > unbuffered RECFM. > > -- gil > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO IBM-MAIN > -- OREXXMan ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO IBM-MAIN
