I think you have to remember that BatchPipes/MVS' origin story is
connecting existing batch job steps. I wouldn't want customers to have to
re-write e.g. in java.
One advantage is that it's easier to rework a pipeline - whether as a
fitting or between batch jobs - than rework some java code.
As someone who first proselytised Pipes in 1992 and first wrote about it
in a Redbook in 1997 (and wrote about it again in 2011 and 2013) you can
consider me a fan. Note: I don't have an enormous amount of influence on
those that make decisions about either IBM flavour of pipes. I do
recognise it's not as simple as "set the code free". There is the cost of
bringing it to market and supporting it. The latter in particular would be
much more costly than it's ever been before - if it were built in.
So, I hope we can make both Pipelines and BatchPipes/MVS available as part
of z/OS. I'd love to be talking about them again and supporting customers
using them. But I'm just a field guy.
Cheers, Martin
Martin Packer
WW z/OS Performance, Capacity and Architecture, IBM Technology Sales
+44-7802-245-584
email: [email protected]
Twitter / Facebook IDs: MartinPacker
Blog: https://mainframeperformancetopics.com
Mainframe, Performance, Topics Podcast Series (With Marna Walle):
https://anchor.fm/marna-walle
Youtube channel: https://www.youtube.com/channel/UCu_65HaYgksbF6Q8SQ4oOvA
From: "David Crayford" <[email protected]>
To: [email protected]
Date: 28/09/2021 08:17
Subject: [EXTERNAL] Re: The Business Case for BatchPipes in the
z/OS Base (was: ... Pipes ...)
Sent by: "IBM Mainframe Discussion List" <[email protected]>
This first question I would ask is does IBM actually "own" BatchPipes or
did the flog it off to an ISV like most of their other software? If it's
the latter then they will be in no position to make it freely available.
Secondly, rather than pine for something that isn't available why not
just switch technologies and use a language that supports functional
programming features that are similar to pipes. Even Java has supported
functional programming since Java 8 came with streams in 2014.
https://stackify.com/streams-guide-java-8/
On 28/09/2021 12:39 am, Hobart Spitz wrote:
Gil wrote:
On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:
I'm going to pivot here. I'm putting my support behind putting
BatchPipes
in the z/OS base (rather than just Pipes). If you agree, please
write/support such a requirement and/or educate your management to get
interested. BatchPipes includes BatchPipesWorks, a not so current,
but
still highly useful, version of TSO Pipelines.
Is an RFE for an update required? Conway's Law.
Great question. I guess it would depend on whether the were a lot of
current BatchPipes customers needed the missing builtin stages and/o
fix(es) or whether it was more important to get BatchPipes in the z/OS
base. It might also depend on how much additional work would be
required
to get a more current version of Pipes into BatchPipes, and whether the
staff, skills, and funding were available. IMHO, the update would delay
the benefits of BatchPipes in the base (especially global warming
mitigation), and not be important to current BatchPipes customers. In
my
estimation, BatchPipesWorks has 90% of the needed stages and 99% of the
fix(es) as would be in an up to date version of TSO Pipelines. I find
the
IBM decision making process obscure, so these may not be the
considerations
that affect the final decision.
AFAIK, there is only one person now working on Pipes, so I must be
missing
something in applying Conway's law.
Does BatchPipes support connecting two Classic modules with an
intervening
small Pipeline filter? How? Is a coordinated third job needed?
Let me clarify some terms, answer you questions in the process, and
clarify
my previous post:
Inter-JOB pipe - This is a ppe that let's two JOB pass records from one
to
the next thru memory. The BatchPipes subsystem(s) is/are required.
AFAIK: There are no obvious limitations to the topology. You can have
multiple JOB connected in a single "stream", split and/or join streams,
or
even have loop(s). In this respect, it is similar to the PIPE command,
except that the record flow with split and rejoined streams may not be
predictable between JOBs.
Half Pipe Fitting or Intra-STEP pipe - This is a series of Pipes stages
that operate between a program in a step and the storage medium.
Intra-JOB pipe - This would be similar to VIO; it doesn't exist
otherwise.
I think it would be a great feature. Hence I incorrectly used the term
to
refer to Half Pipe Fittings in a generic sense. Apologies for any
confusion.
AFAIK, and if I understood your question, connecting two classic
programs
in the same step with a series of Pipes stages may not be possible now,
at
least without resorting to Assembler. I think the main reason is the
potential for DD name conflict, both in the TIOT (DD name table) and in
JCL. Most COBOL programs require SYSOUT, so the output could be
intermixed. Much less tolerable, I suspect, would be a common DD (e.g.
OUTPUT) conflict. Putting a record that doesn't belong between two
records
that must be consecutive would probably be a disaster.
That said, it may be possible for a REXX filter to start two COBOL
(e.g.)
programs as subtasks (address attach ... , etc.) and feed and receive
their
records via an Assembler(?) Pipe Fitting. You would have to resolve any
DD
name conflicts. I don't think anyone has contemplated the capability
for
creating multiple TIOTs in a single step, let alone how to reference
them
in JCL. I think such things have been done under CMS using Assembler.
Also:
pipe - a generic connection, in memory, between two processes or tasks.
Pipes - the TSO/CMS Pipelines software, singular.
Pipe - a specific instance of Pipes.
PIPE - the command that runs Pipes.
OREXXMan
Would you rather pass data in move mode (*nix piping) or locate mode
(Pipes) or via disk (JCL)? Why do you think you rarely see *nix
commands
with more than a dozen filters, while Pipelines specifications are
commonly
over 100s of stages, and 1000s of stages are not uncommon.
IBM has been looking for an HLL for program products; REXX is that
language.
On Mon, Sep 27, 2021 at 9:10 AM Paul Gilmartin <
[email protected]> wrote:
On Mon, 27 Sep 2021 07:26:51 -0500, Hobart Spitz wrote:
I'm going to pivot here. I'm putting my support behind putting
BatchPipes
in the z/OS base (rather than just Pipes). If you agree, please
write/support such a requirement and/or educate your management to get
interested. BatchPipes includes BatchPipesWorks, a not so current,
but
still highly useful, version of TSO Pipelines.
Is an RFE for an update rrequired? Conway's Law.
Does BatchPipes support connecting two Classic modules with an
intervening
small Pipeline filter? How? Is a coordinated third job needed?
The reasons are: [Snip! See the archive.]
-- gil
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to [email protected] with the message: INFO IBM-MAIN
