BatchPipes was useless for any really large volume of data. My employer had a license to the product early in my time with them (OS390 era) and we looked at using it for efficient data passing across multiple batch stream jobs, but our very large data volumes precluded using it.
Also, BatchPipes was not really a PIPEs implementation – it was just a way to push data from one batch process to multiple other ones without a real file as the intermediary, the idea being that elapsed time could be reduced thus improving SLA compliance. When I think of PIPEs I mean a CMS-style pipe command implementation usable in a single (possibly) multitasking process, not BatchPipes sharing data across jobs. Your obvious prejudice that business application teams don’t know what they are doing is really irritating. We DO know the system, and we DO design to use the best-of-breed features that z/OS provides, and we DO consult with systems and capacity and storage and database and network teams to help us do our jobs better. Systems-level teams are NOT the ultimate deciders of what is acceptable performance design, and z/OS is NOT, as you put it, “. . . designed for sysprogs making choices”. Systems teams are PART of the business application implementation team. If a true multitasking application process (using real PIPEs or not) made business sense from a business perspective, then consultations with systems-level teams would help support that implementation, not bad-mouth the application team for even thinking about such an implementation. The job is to get the application running as best as we can to support the business effectively and efficiently, not to have arguments about which implementation technique is “better”. Systems teams do NOT rule the z/OS universe – business applications do, because that’s what the business pays all of us to implement. You say you are debunking the myth that PIPE’s are performant, but we’ve never seen a real and complete z/OS PIPEs implementation with a similar set of stages and filters as are available in CMS that would allow for a side-by-side comparison to prove or disprove the performance of an application using PIPEs versus other z/OS techniques. All this is frivolous watercooler talk anyway, since z/OS doesn’t have a real PIPEs implementation (and likely never will have one) outside of Netview, which effectively means none at all since AFAIK business applications can’t use Netview for business application purposes (i.e., not in the context of automated message processing but for real business data processing). Peter From: IBM Mainframe Discussion List <[email protected]> On Behalf Of Jon Perryman Sent: Monday, September 8, 2025 8:10 PM To: [email protected] Subject: Re: Pipelines = you don't understand z/OS On Mon, 8 Sep 2025 06:45:45 +0000, Farley, Peter <mailto:[email protected]> wrote: >A great deal of your arguments seem to me to be about the performance >impact of a multitasking program on other-system-critical applications, I'm discussing why TSO PIPEs no longer exist. I'm saying that people use them because they are perceived as performant. I'm debunking that myth. There are alternatives that could be made as performant that don't involve PIPEs. >How is the performance of one truly multitasking batch step any different from >any other WLM-managed batch job? Batchpipes involved 2 or more jobs. Should IBM implement batch enclaves? Or maybe IBM should implement multi-tasking JCL definitions? >You seem to be arguing that a multitasking batch job step in a production >batch stream could interfere >with the priority-CPU-I/O of the online and real-time and database activities >running on the same system(s). I'm saying that batchpipes most likely failed because it ignored performance. I'm saying that batchpipes removes control and understanding by the performance sysprog. >I just don’t understand why you are so dead set against pipes for "allowing" >multitasking >to be more easily created by "ordinary" programmers. I'm not against pipes, however PIPEs are designed for uncomplicated environments that don't have sysplex, srm, sysprogs and more. z/OS is designed for sysprogs making choices whereas Unix is about application programmers pretending to be in charge. How many times will Unix distributed computing be allowed to fail (now moving on from Kubernettes to micro services)? SYSPLEX has been around for at least 30 years while Unix develops a new distributed solution every couple of years. Doing things right the first time is uniquely mainframe. -- This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system. ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO IBM-MAIN
