A master does match multiple details. And the details can occur before the first master, though they don't always.
I had tried it without the elastic on the details input and it stalled which didn't surprise me, though I don't know the exact reason for it. And sure, if I wanted to capture *every* matching detail I would *have* to use buffer. But given that I don't need to get every record, only a selection, and the memory requirements of buffer would be onerous, I'm quite happy with the magical properties of elastic, sacrilegious as they might be. i ------ Original Message ------ Received: 01:57 AM COT, 07/14/2009 From: Rob van der Heij <[email protected]> To: [email protected] Subject: Re: Pipeline fanin problem > Some plumbers will, when they are up to their knees in the water and > can't get the leak fixed, use "elastic" to fix the stalled pipeline. > > > Elastic: You're quite right in your guess as to how this got added. But, as I > > said in a follow up note to Rob, this is a huge file being read off of tape > > and buffer isn't going to work here. > > You use elastic when there's a variable amount of records to hold > during the run of the pipeline. If you need to buffer all (like you > have to with your lookup secondary input) you should use "buffer" > instead. But from what I read it's doable without buffering all. You > will need to buffer some records (for example the details when waiting > for a corresponding master record). Oh, and you did not say whether > the master matches more than one detail in your data - lookup expects > so, and will hold the entire master table until the end of your input. > > Melinda's "plunging on" may help on multi-stream plumbing. And John's > "CMS Pipelines Explained" has more about record delay. > > Rob >
