Not sure it will help, but take a look at the COLLATE stage (IIRC, much like LOOKUP, but with the details and masters reversed).
And... you are running the CMS/TSO Pipelines Runtime Library version, right? It has VERY many improved and added stages that have never been and won't be included in the Endicott z/VM product version of Pipes. Mike Walter Hewitt Associates Mike Walter Hewitt Associates (Sent from the wee keyboard on a Blackberry.) ----- Original Message ----- From: "Ian S. Worthington" [[email protected]] Sent: 07/14/2009 07:30 AM EST To: [email protected] Subject: Re: Pipeline fanin problem A master does match multiple details. And the details can occur before the first master, though they don't always. I had tried it without the elastic on the details input and it stalled which didn't surprise me, though I don't know the exact reason for it. And sure, if I wanted to capture *every* matching detail I would *have* to use buffer. But given that I don't need to get every record, only a selection, and the memory requirements of buffer would be onerous, I'm quite happy with the magical properties of elastic, sacrilegious as they might be. i ------ Original Message ------ Received: 01:57 AM COT, 07/14/2009 From: Rob van der Heij <[email protected]> To: [email protected] Subject: Re: Pipeline fanin problem > Some plumbers will, when they are up to their knees in the water and > can't get the leak fixed, use "elastic" to fix the stalled pipeline. > > > Elastic: You're quite right in your guess as to how this got added. But, as I > > said in a follow up note to Rob, this is a huge file being read off of tape > > and buffer isn't going to work here. > > You use elastic when there's a variable amount of records to hold > during the run of the pipeline. If you need to buffer all (like you > have to with your lookup secondary input) you should use "buffer" > instead. But from what I read it's doable without buffering all. You > will need to buffer some records (for example the details when waiting > for a corresponding master record). Oh, and you did not say whether > the master matches more than one detail in your data - lookup expects > so, and will hold the entire master table until the end of your input. > > Melinda's "plunging on" may help on multi-stream plumbing. And John's > "CMS Pipelines Explained" has more about record delay. > > Rob > The information contained in this e-mail and any accompanying documents may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient of this message, or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message, including any attachments. Any dissemination, distribution or other use of the contents of this message by anyone other than the intended recipient is strictly prohibited. All messages sent to and from this e-mail address may be monitored as permitted by applicable law and regulations to ensure compliance with our internal policies and to protect our business. E-mails are not secure and cannot be guaranteed to be error free as they can be intercepted, amended, lost or destroyed, or contain viruses. You are deemed to have accepted these risks if you communicate with us by e-mail.
