Do I misunderstand the documentation then? I thought BUFFER only started to release records upon eof, whereas ELASTIC would release them as the following stage got ready for them.
i ------ Original Message ------ Received: 10:08 AM COT, 07/14/2009 From: Glenn Knickerbocker <[email protected]> To: [email protected] Subject: Re: Pipeline fanin problem > "Ian S. Worthington" wrote: > > Elastic: You're quite right in your guess as to how this got added. But, as I > > said in a follow up note to Rob, this is a huge file being read off of tape > > and buffer isn't going to work here. > > If it won't fit in BUFFER, it won't fit in ELASTIC. Either way, what > you're doing with this LOOKUP is going through the whole file twice, > once to pull out the 0000 START records, and once to match them up. > > So what do you know about the structure of the file that can allow you > to read it only once? > > * Do all the 0000 START records come at the beginning? If so, you can > use something like TOTARGET to split the file at the end of them. > > * If not, does each 0000 START record come before any other records for > the same LU? If so, you can feed them into input 2 of LOOKUP rather > than input 1, to add them to the master file as they arrive. > > * If the 0000 START record can come *after* the records you want to > match with it, you're stuck. You can't identify which records you > want until you get to the end of the file, so you'll have to read the > whole thing twice. > > --Glenn >
