This RFE was submitted through Share to IBM RFE:
DESCRIPT : A facility equivalent to the CMS PIPES product should be added to
TSO. PIPES allows the output of one TSO command to be directed as input to
another TSO command and the output from that command can be directed to the
next ... etc.
It is in
Uncommitted Candidate: This request may not be delivered within the release
currently under development, but the theme is aligned with the current
multi-year strategy. IBM may consider and evaluate any RFE Community feedback
for this request through activities such as voting. IBM will update this
request in the future.
>From the comments section:
Total comments (5)
z/OS NetView has support for PIPEs that is extremely useful but they are sadly
lacking from TSO/E.
Posted by MikeBike on 31 Mar 2017
Report abuse
Being a z/VM-er, forced to work on z/OS, I miss the functionality provided by
CMS/TSO Pipes.
I have the told CMS/TSO Pipes code available, and that is a huge productivity
booster, but the more recent (last couple of decades) enhancements are
seriously lacking, requiring often to recreate "standard" functions in private
REXX stages.
So getting a supported TSO/Pipes at the z/VM 6.4 level would be really
appreciated...
Posted by Ronald van der Laan on 7 Jul 2016
Report abuse
This requirement has been outstanding for more than 25 years. Clearly, support
for this requirement is forthcoming from the TSO user community.
One of the earliest SHARE presentations was SHARE 92 February 1990 "Plumbers
Who Use TSO and the Users Who Love Them: An Introduction to TSO Pipelines",
Session 2873, Hobart Spitz, Paine Webber.
"Plumbers Who Dump CVTs and the Control Blocks that Love Them: MVS/TSO
Pipelines for Systems Programmers", SHARE February 1999, Session 2875, Hobart
Spitz, Paine Webber
"Two Dimensional Pipes: How CMS Pipelines Differs from UNIX Pipes", SHARE 97,
Summer 2001, Session 9109, Marty Zimelis, Computer Associates
Posted by WilliamJ.Smith on 20 May 2016
Report abuse
This addition to the base z/OS environment would significantly improve the
productivity of those developing and delivery extensions and enhancements to
the platform. We know the code is available and that it isn't effectively
marketed. Since it is apparently stable why not integrate it and ship if for
free.
Posted by LionelB.Dyck on 19 May 2016
Lizette
> -----Original Message-----
> From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On
> Behalf Of Hobart Spitz
> Sent: Thursday, November 16, 2017 6:58 AM
> To: IBM-MAIN@LISTSERV.UA.EDU
> Subject: Re: [EXTERNAL] Re: Pipelines in the z/OS base.
>
> If permitted by policy, could someone post the text of RFE 47699, for those
> who don't have IBM ids?
>
> On Thu, Nov 16, 2017 at 8:56 AM, Dyck, Lionel B. (TRA) <lionel.d...@va.gov>
> wrote:
>
> > See RFE 47699 at https://www.ibm.com/developerworks/rfe/execute?
> > use_case=viewRfe&CR_ID=47699
> >
> > Currently sitting at 140 Votes and at #19 on the overall vote count.
> >
> > ----------------------------------------------------------------------
> > ----
> > Lionel B. Dyck <sdg><
> > Mainframe Systems Programmer - TRA
> >
> > -----Original Message-----
> > From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU]
> > On Behalf Of Lucas Rosalen
> > Sent: Thursday, November 16, 2017 7:49 AM
> > To: IBM-MAIN@LISTSERV.UA.EDU
> > Subject: [EXTERNAL] Re: Pipelines in the z/OS base.
> >
> > Hello Hobart,
> >
> > Do you have any RFE already opened for that? I would surely vote for
> > it (plus this might be a good place to broadcast its ref. number)!
> >
> > Thanks, Lucas
> >
> > On Nov 16, 2017 14:36, "Hobart Spitz" <orexx...@gmail.com> wrote:
> >
> > Someone on TSO-REXX wrote:
> > > Yes and if IBM would give us PIPES (BatchPipesWorks Product) then
> > > you can
> > use PIPE to read from a file into a STEM or STACK and vice-versa.
> >
> > There have been other such posts.
> >
> > For those who don't know TSO PIPEs is like an assembly line with
> > reusable, standardized stations. (You can create your own, on the fly
> > even.)
> >
> > IMHO, making TSO PIPEs part of the z/OS base is long, long overdue.
> > Over the last 20 years, I have been associated with two requirements
> > for Pipes in the OS base that have been submitted at SHARE, but which
> > produced no results. I have it on good authority that that if you
> > take all the requests for TSO Pipelines, sort in REXX, and other
> > requirements addressed by TSO PIPEs, that there would be about 75% positive
> RFE votes.
> >
> > Background on the term "piping":
> >
> > The early versions of UNIX were written on small machines (32K?!),
> > with slow/small disk drives. Writing temporary results out to disk
> > was too slow and inefficient, so they came up a pipe operator ( | ),
> > which connected the input of one command to the output of the previous
> > one, thus avoiding physical I/O.. E.g. *ls | wc*. The data passed
> > thru an in-memory buffer with both commands running, and UNIX
> > dispatching each as needed. It also allowed intermediate results to
> > be very much larger than the available disk space, since those intermediate
> results didn't have to be written to disk.
> >
> > CMS/TSO Pipelines does piping somewhat like this and much more. Once
> > difference is that data doesn't have to move, record descriptors do
> > that job for unchanged records.
> >
> > I was first exposed to this concept, when I was supporting a team of
> > previously UNIX-based developers working on TSO for the first time.
> > One of the team came to me and asked how to connect the output of one
> > program to the input of the next in JCL. On the whiteboard, I
> > proceeded to write the JCL for creating a temporary dataset in one job
> > step (DD with DSN, UNIT, DCB, DISP, etc.) and the JCL to read that dataset
> in the next job step.
> > When I turned around, the questioner was gone. A short time later,
> > his officemate came in with the same question. The original
> > questioner thought I was pulling his leg (since one could do that with
> > a single character in
> > UNIX) and he had left before I was done.
> >
> > For those that may not know:
> >
> > - TSO PIPEs implements dynamic, record-level, deterministic piping that
> > is significantly more advanced than that on UNIX, Linux, and USS.
> > (More on
> > that below.)
> > - If installed, TSO PIPEs can be used by invoking the "pipe": command
> > anywhere that a TSO command can be used, including: READY Prompt, ISPF
> > option 6, ISPF command line with the TSO prefix, REXX EXECs, CLISTs, TSO
> > batch, and the ISPF service "select cmd(pipe ... )".
> > - PIPEs is part of the z/VM base but is optional under z/OS. Hence this
> > post.
> > - TSO PIPEs, TSO Pipelines, and BatchPipesWorks, are all names for
> > essentially the same product, which is roughly a subset of CMS Pipes.
> > (My
> > understanding is that TSO PIPEs is not the same as the USS command
> > "pipe";
> > you need to direct the command to TSO.)
> > - Under z/VM, and under z/OS where available, it is well integrated into
> > both environments.
> > - On a very high-level conceptual level, PIPEs are somewhat similar to
> > SQL and 4GLs in that many common, low level processing details (such as
> > initialization, looping, and end of data) are taken care of internally.
> > Thus, many types of selections, matchings, transformations, etc. can be
> > done with compact commands and frequently without convention
> > programming in
> > languages like COBOL, PL/I, and C/C++. (It is different from SQL and
> > 4GLs
> > in that Pipes can process many kinds of data, not just one type of
> > data or
> > source.)
> > - PIPEs can read both USS text and most binary files and can convert
> > them to record orientation (address plus length descriptors).
> > - PIPEs can also be invoked from JCL thusly: // EXEC
> > PGM=PIPE,PARM='< IN.FILE|SORT|> OUT.FILE'
> >
> > PIPEs in the z/OS base would greatly improve the competitiveness of
> > mainframe systems, while cutting development, hardware and support
> > costs for IBM, third-parties and customers. By having Pipes available
> > to all internal and external developers, efficiency and productivity
> > would improve. This would help IBM competitiveness and bottom line,
> > as well as that of mainframe sites, and software vendors. If you've
> > been following IBM after quarterly reports are covered in the news,
> > you'll know that IBM needs a shot in the arm like this. Results have
> > been poor for many quarters. Over the long term it would mean
> > improved profitability for the entire mainframe division. There might
> > even be IBM customers who are saved from moving off mainframes
> > (because of costs) by such a move. In short, making PIPEs part of the
> > z/OS base would more than pay for itself in reduced costs, retained
> > customers, and additional sales. If I were an IBM stockholder, a
> > manager at a mainframe customer, or part of a third-party development
> > team, I would be asking for TSO PIPEs from anyone who would listen,
> > and maybe some who wouldn't. :-)
> >
> > Costs to IBM would not be an issue. TSO PIPEs already exists as a
> > subset of CMS PIPEs. Even in it's current subset state, it would
> > satisfy the vast majority of requirements (if not all) to deliver the
> > product with the base, thus minimizing the additional cost.
> >
> > UNIX and variants cannot achieve what PIPEs can. (I have some
> > experience with UNIX and Linux, but only a little exposure to USS, so
> > feel free to correct any misstatements or incorrect details,
> > especially in this
> > section.)
> >
> > - The *NIX byte stream model requires that every character of a text
> > file be inspected for end-of-record and forcing each and every
> > character to
> > the faster caches. There is no way to skip an entire record in a text
> > file. The PIPEs model of record orientation means that records can be
> > skipped, in their entirety, or by inspecting a part, without
> > paging-in and
> > cache-loading the whole record, should it span page or cache line
> > boundary. Certainly, the faster caches don't need to be flooded with
> > unneeded, unused bytes.
> > - In *NIX each and every byte must be copied from the sending process,
> > to the buffer, to the receiving process. In PIPEs, only the record
> > descriptor (address and length) gets copied, while the data remains in
> > place. (Actually, it's probable that the descriptor is passed by
> > address.)
> > - Where binary (non-text) files are used (e.g. for performance or random
> > access), this precludes the use of most *NIX/USS text-oriented built-in
> > filters, without extra effort. E.g., extra encoding/translating/
> > encapsulation
> > to prevent binary data from being interpreted as end-of-record
> > sequences,
> > to ensure the logical records appear as such.
> > - *NIX/USS pipeing byte stream can be split only with difficulty (think
> > about one job step creating two temporary datasets), and, because
> > the data
> > flow is not deterministic, they can't be rejoined without special
> > coding.
> > With PIPEs, splitting and rejoining streams is routine and useful,
> > because
> > the record flow characteristics are known and documented for built-in
> > stages.
> >
> > PIPEs make REXX an even more compelling tool in batch. I would
> > nominate PIPEs as a candidate for the JCL of the 21st century., but
> > that's a long story for another time. In both conventional multi-step
> > jobs and more contemporary applications (e.g. ETL, data warehousing,
> > etc.) multiple steps can be achieved in a single command more
> > efficiently and with less original code than is generally done in z/OS
> today.
> >
> > PIPEs has interfaces for DB2, PDS/Es, Byte Files, ISPF tables and
> > variables, as well as sequential files, to name a few.
> >
> > If you have any doubt about the usefulness of Pipes, take a poll of
> > your z/VM friends. They would tell you that they couldn't live
> > without Pipes, not because z/VM otherwise lacks anything, but because
> > Pipes is so powerful and efficient.
> >
> > I urge anyone and everyone who uses REXX or JCL to do what they can to
> > make bring PIpes to the z/OS base. Fee free to forward this post to
> > anyone who you think cares and/or who can help.
> >
> > I hope that someone more vocal and persuasive than I can take up this
> > issue and get some needed action.
> >
> > In the meantime, check to see if you have the TSO PIPE command already
> > installed. Type "[tso] pipe q"; if you get a version message, you
> > have it. Next step: "[tso] pipe help menu" and/or "[tso] pipe ahelp
> menu".
> > Failing that, you can convince your management to order and install
> > TSO PIPEs. Your site will benefit, and it might just be enough to get
> > the powers that be in IBM to wake up. The TSO PIPE command was
> > available separately from BatchPipes. I assume it still is.
> >
> > IBM needs to get its act together. The stock is right back to where
> > it was before the z encryption announcement, Buffet is selling his IBM
> > holding (possibly related), and the previous quarters have been a long
> > string of disappointments. Putting PIPEs into the z/OS base may be
> > just the step needed.
> >
> > My apologies for any errors, inaccuracies, or too harsh words.
> >
> > Whew! Kudos if you read the whole thing.
> >
> > - Hobart Spitz
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
