Mike wrote: >Could something be developed similar to a SORTOUT exit that implements this switch? BatchPipes fittings are like sort exits on steroids: They can be applied to almost any DD, can use most Pipes filters, do not require compiled code, and they are not restricted to a single program. I plan to say more later. In a nutshell: I think BatchPipes, including the Pipe command, should be in the z/OS base. It would be the biggest enhancement to JCL ever, and would be of much more interest to production oriented management who care about stability.
OREXXMan Would you rather pass data in move mode (*nix piping) or locate mode (Pipes) or via disk (JCL)? Why do you think you rarely see *nix commands with more than a dozen filters, while Pipelines specifications are commonly over 100s of stages, and 1000s of stages are not uncommon. IBM has been looking for an HLL for program products; REXX is that language. On Mon, Sep 20, 2021 at 7:11 PM Mike Schwab <[email protected]> wrote: > So, in a sense, instead of pipes, the programs could be modified so > that instead of outputting a record, call the next program passing the > record as input. > > Could something be developed similar to a SORTOUT exit that implements > this switch? > > On Mon, Sep 20, 2021 at 4:27 PM Hobart Spitz <[email protected]> wrote: > > > > Phil; > > > > This is a great comment. I hadn't given that much thought to the > question. > > > > Not to split hairs, but I didn't say MIPS, I said hardware. > > > > If I had to guess, MIPS usage might actually increase slightly, because > the > > Pipes dispatcher has to switch between stages twice for every record that > > is passed. Access method overheard would drop. Buffered access methods, > > in most cases, only have to increment the pointer into the block buffer, > > check for end-of-buffer and return to the caller. I don't know for sure > > which is larger. Maybe someone more knowledgeable than I can shed some > > light. > > > > I would say the real savings would be in elapsed run time and working set > > size. Run time, due to eliminating something like 80-95% of I/O > operations > > for intra-JOB datasets. Working set reduction would save on real memory. > > (See below.) Run time is probably more of a concern to customers, > > especially those with tight batch windows. That said, working set size > > reduction would mean that processors would likely spend more, if not all, > > time pegged at 100% busy, because so many more address spaces (TSO and > JOB) > > would be in a swapped-in and ready state than before. Depending on what > > metrics the capacity planners are looking at, CPU sales might actually > > increase. As I think about it more, if thru-put increases, new data > could > > be generated more quickly and other times of hardware could be more in > > demand during peak load times. I just don't know enough to say for sure. > > > > Phil and others know what follows. > > > > For those who don't know, in the typical case, a record passes through > all > > possible stages before the next record begins the same trip. Each record > > stays in the working page set, at least partially, during the entire > time. > > Parts that are referenced have a good chance of staying cache resident > > between stages. > > > > Think of it this way: You can visualize UNIX piping as a series of > > hourglasses open at both ends and connected in a tower. Each grain of > sand > > must stop at every "stage" and wait its turn to go through the narrow > > opening at the waist of each hourglass. In Pipes, most stages have no > > delay and it's like a single tall hourglass tube with only one narrow > > point. My best guess is that Pipes, in this analogy, would have only > > 5%-15% of the narrow openings as an equivalent UNIX piping command, > meaning > > that the data (sand) would flow 85-95% faster in the Pipes "hourglass". > > > > > > OREXXMan > > Would you rather pass data in move mode (*nix piping) or locate mode > > (Pipes) or via disk (JCL)? Why do you think you rarely see *nix commands > > with more than a dozen filters, while Pipelines specifications are > commonly > > over 100s of stages, and 1000s of stages are not uncommon. > > IBM has been looking for an HLL for program products; REXX is that > language. > > > > > > On Mon, Sep 20, 2021 at 12:48 PM Phil Smith III <[email protected]> wrote: > > > > > Hobart Spitz wrote, in part: > > > > > > >The case *for *Pipes in the z/OS base.: > > > > > > > 2. Hardware usage would drop for customers. > > > > > > > > > > > > From IBM's perspective, that might not be a positive argument. It > should > > > be-they're hopefully not fooling themselves that they have a lock on > > > enterprise computing any more, so anything that makes life more > palatable > > > for the remaining faithful at low cost to IBM should be A Good > Thing-but I > > > can easily imagine someone saying, "We estimate this will reduce MIPS > > > purchases by x%, that's bad, don't do it". > > > > > > > > > > > > Just sayin'. > > > > > > > > > > > > ...phsiii > > > > > > > > > ---------------------------------------------------------------------- > > > For IBM-MAIN subscribe / signoff / archive access instructions, > > > send email to [email protected] with the message: INFO IBM-MAIN > > > > > > > ---------------------------------------------------------------------- > > For IBM-MAIN subscribe / signoff / archive access instructions, > > send email to [email protected] with the message: INFO IBM-MAIN > > > > -- > Mike A Schwab, Springfield IL USA > Where do Forest Rangers go to get away from it all? > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO IBM-MAIN > ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO IBM-MAIN
