If permitted by policy, could someone post the text of RFE 47699, for those who don't have IBM ids?
On Thu, Nov 16, 2017 at 8:56 AM, Dyck, Lionel B. (TRA) <[email protected]> wrote: > See RFE 47699 at https://www.ibm.com/developerworks/rfe/execute? > use_case=viewRfe&CR_ID=47699 > > Currently sitting at 140 Votes and at #19 on the overall vote count. > > -------------------------------------------------------------------------- > Lionel B. Dyck <sdg>< > Mainframe Systems Programmer - TRA > > -----Original Message----- > From: IBM Mainframe Discussion List [mailto:[email protected]] On > Behalf Of Lucas Rosalen > Sent: Thursday, November 16, 2017 7:49 AM > To: [email protected] > Subject: [EXTERNAL] Re: Pipelines in the z/OS base. > > Hello Hobart, > > Do you have any RFE already opened for that? I would surely vote for it > (plus this might be a good place to broadcast its ref. number)! > > Thanks, Lucas > > On Nov 16, 2017 14:36, "Hobart Spitz" <[email protected]> wrote: > > Someone on TSO-REXX wrote: > > Yes and if IBM would give us PIPES (BatchPipesWorks Product) then you > > can > use PIPE to read from a file into a STEM or STACK and vice-versa. > > There have been other such posts. > > For those who don't know TSO PIPEs is like an assembly line with reusable, > standardized stations. (You can create your own, on the fly even.) > > IMHO, making TSO PIPEs part of the z/OS base is long, long overdue. Over > the last 20 years, I have been associated with two requirements for Pipes > in the OS base that have been submitted at SHARE, but which produced no > results. I have it on good authority that that if you take all the > requests for TSO Pipelines, sort in REXX, and other requirements addressed > by TSO PIPEs, that there would be about 75% positive RFE votes. > > Background on the term "piping": > > The early versions of UNIX were written on small machines (32K?!), with > slow/small disk drives. Writing temporary results out to disk was too slow > and inefficient, so they came up a pipe operator ( | ), which connected > the input of one command to the output of the previous one, thus avoiding > physical I/O.. E.g. *ls | wc*. The data passed thru an in-memory buffer > with both commands running, and UNIX dispatching each as needed. It also > allowed intermediate results to be very much larger than the available disk > space, since those intermediate results didn't have to be written to disk. > > CMS/TSO Pipelines does piping somewhat like this and much more. Once > difference is that data doesn't have to move, record descriptors do that > job for unchanged records. > > I was first exposed to this concept, when I was supporting a team of > previously UNIX-based developers working on TSO for the first time. One of > the team came to me and asked how to connect the output of one program to > the input of the next in JCL. On the whiteboard, I proceeded to write the > JCL for creating a temporary dataset in one job step (DD with DSN, UNIT, > DCB, DISP, etc.) and the JCL to read that dataset in the next job step. > When I turned around, the questioner was gone. A short time later, his > officemate came in with the same question. The original questioner thought > I was pulling his leg (since one could do that with a single character in > UNIX) and he had left before I was done. > > For those that may not know: > > - TSO PIPEs implements dynamic, record-level, deterministic piping that > is significantly more advanced than that on UNIX, Linux, and USS. > (More on > that below.) > - If installed, TSO PIPEs can be used by invoking the "pipe": command > anywhere that a TSO command can be used, including: READY Prompt, ISPF > option 6, ISPF command line with the TSO prefix, REXX EXECs, CLISTs, TSO > batch, and the ISPF service "select cmd(pipe ... )". > - PIPEs is part of the z/VM base but is optional under z/OS. Hence this > post. > - TSO PIPEs, TSO Pipelines, and BatchPipesWorks, are all names for > essentially the same product, which is roughly a subset of CMS Pipes. > (My > understanding is that TSO PIPEs is not the same as the USS command > "pipe"; > you need to direct the command to TSO.) > - Under z/VM, and under z/OS where available, it is well integrated into > both environments. > - On a very high-level conceptual level, PIPEs are somewhat similar to > SQL and 4GLs in that many common, low level processing details (such as > initialization, looping, and end of data) are taken care of internally. > Thus, many types of selections, matchings, transformations, etc. can be > done with compact commands and frequently without convention > programming in > languages like COBOL, PL/I, and C/C++. (It is different from SQL and > 4GLs > in that Pipes can process many kinds of data, not just one type of data > or > source.) > - PIPEs can read both USS text and most binary files and can convert > them to record orientation (address plus length descriptors). > - PIPEs can also be invoked from JCL thusly: // EXEC > PGM=PIPE,PARM='< IN.FILE|SORT|> OUT.FILE' > > PIPEs in the z/OS base would greatly improve the competitiveness of > mainframe systems, while cutting development, hardware and support costs > for IBM, third-parties and customers. By having Pipes available to all > internal and external developers, efficiency and productivity would > improve. This would help IBM competitiveness and bottom line, as well as > that of mainframe sites, and software vendors. If you've been following > IBM after quarterly reports are covered in the news, you'll know that IBM > needs a shot in the arm like this. Results have been poor for many > quarters. Over the long term it would mean improved profitability for the > entire mainframe division. There might even be IBM customers who are saved > from moving off mainframes (because of costs) by such a move. In short, > making PIPEs part of the z/OS base would more than pay for itself in > reduced costs, retained customers, and additional sales. If I were an IBM > stockholder, a manager at a mainframe customer, or part of a third-party > development team, I would be asking for TSO PIPEs from anyone who would > listen, and maybe some who wouldn't. :-) > > Costs to IBM would not be an issue. TSO PIPEs already exists as a subset > of CMS PIPEs. Even in it's current subset state, it would satisfy the vast > majority of requirements (if not all) to deliver the product with the base, > thus minimizing the additional cost. > > UNIX and variants cannot achieve what PIPEs can. (I have some experience > with UNIX and Linux, but only a little exposure to USS, so feel free to > correct any misstatements or incorrect details, especially in this > section.) > > - The *NIX byte stream model requires that every character of a text > file be inspected for end-of-record and forcing each and every > character to > the faster caches. There is no way to skip an entire record in a text > file. The PIPEs model of record orientation means that records can be > skipped, in their entirety, or by inspecting a part, without paging-in > and > cache-loading the whole record, should it span page or cache line > boundary. Certainly, the faster caches don't need to be flooded with > unneeded, unused bytes. > - In *NIX each and every byte must be copied from the sending process, > to the buffer, to the receiving process. In PIPEs, only the record > descriptor (address and length) gets copied, while the data remains in > place. (Actually, it's probable that the descriptor is passed by > address.) > - Where binary (non-text) files are used (e.g. for performance or random > access), this precludes the use of most *NIX/USS text-oriented built-in > filters, without extra effort. E.g., extra encoding/translating/ > encapsulation > to prevent binary data from being interpreted as end-of-record > sequences, > to ensure the logical records appear as such. > - *NIX/USS pipeing byte stream can be split only with difficulty (think > about one job step creating two temporary datasets), and, because the > data > flow is not deterministic, they can't be rejoined without special > coding. > With PIPEs, splitting and rejoining streams is routine and useful, > because > the record flow characteristics are known and documented for built-in > stages. > > PIPEs make REXX an even more compelling tool in batch. I would nominate > PIPEs as a candidate for the JCL of the 21st century., but that's a long > story for another time. In both conventional multi-step jobs and more > contemporary applications (e.g. ETL, data warehousing, etc.) multiple steps > can be achieved in a single command more efficiently and with less original > code than is generally done in z/OS today. > > PIPEs has interfaces for DB2, PDS/Es, Byte Files, ISPF tables and > variables, as well as sequential files, to name a few. > > If you have any doubt about the usefulness of Pipes, take a poll of your > z/VM friends. They would tell you that they couldn't live without Pipes, > not because z/VM otherwise lacks anything, but because Pipes is so powerful > and efficient. > > I urge anyone and everyone who uses REXX or JCL to do what they can to > make bring PIpes to the z/OS base. Fee free to forward this post to anyone > who you think cares and/or who can help. > > I hope that someone more vocal and persuasive than I can take up this > issue and get some needed action. > > In the meantime, check to see if you have the TSO PIPE command already > installed. Type "[tso] pipe q"; if you get a version message, you have > it. Next step: "[tso] pipe help menu" and/or "[tso] pipe ahelp menu". > Failing that, you can convince your management to order and install TSO > PIPEs. Your site will benefit, and it might just be enough to get the > powers that be in IBM to wake up. The TSO PIPE command was available > separately from BatchPipes. I assume it still is. > > IBM needs to get its act together. The stock is right back to where it > was before the z encryption announcement, Buffet is selling his IBM holding > (possibly related), and the previous quarters have been a long string of > disappointments. Putting PIPEs into the z/OS base may be just the step > needed. > > My apologies for any errors, inaccuracies, or too harsh words. > > Whew! Kudos if you read the whole thing. > > - Hobart Spitz > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, send email > to [email protected] with the message: INFO IBM-MAIN > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, send email > to [email protected] with the message: INFO IBM-MAIN > > ---------------------------------------------------------------------- > For IBM-MAIN subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO IBM-MAIN > -- OREXXMan ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO IBM-MAIN
