Welcome. Lookup terminates, then getfiles, but as console does not propagate end-of-file backwards, you still see the list of files.
Use the STOPERROR global option to make the pipeline stop immediately a stage returns a nonzero return code. Or EOFBACK CONSOLE to make it terminate too. j. On 19 February 2010 01:07, Alan Winson <[email protected]> wrote: > I just joined the list today and haven't found any archives that address > these issues. > > I have a pipe that uses "getfiles" to read one or more files and feed their > contents to a "lookup" as detail records. Three things are happening that > I > can't explain. > > 1. The pipe does not abort when it runs out of memory. Here is the > relevant > part of the pipe: > > 'PIPE (name huge end ?) stem input_files.', /* fn ft (fm|dir) etc. */ > '| console', /* show details */ > '| getfiles', /* all records from all files */ > '| acc: lookup count autoadd w1', /* don't care about count, but */ > , /* forces dump of masters at end */ > > and here is the relevant (edited) part of the console output: > > 14:56:17 <fn> 20100209 <dir> F 1024 61468 15367 2010-02-10 04:27:55 > 14:57:34 <fn> 20100208 <dir> F 1024 61462 15366 2010-02-09 04:25:01 > 14:57:58 PIPSTO122E Insufficient free storage. > 14:57:58 PIPMSG004I ... Issued from stage 6 of pipeline 1 name "huge". > 14:57:58 PIPMSG001I ... Running "lookup count autoadd w1". > 14:57:58 <fn> 20100205 <dir> F 1024 61388 15347 2010-02-08 04:34:22 > 14:57:59 <fn> 20100204 <dir> F 1024 61337 15335 2010-02-05 04:27:43 > 14:57:59 <fn> 20100203 <dir> F 1024 61291 15323 2010-02-04 04:31:34 > 14:57:59 <fn> 20100202 <dir> F 1024 61248 15312 2010-02-03 04:25:37 > 14:57:59 <fn> 20100201 <dir> F 1024 61402 15351 2010-02-02 04:23:53 > > From the time stamps it appears that after memory is exhausted no records > are read from any of the remaining input files, since their display lines > all have the same time stamp. Still, I would expect the pipe to abort as > soon as it runs out of free storage. Shouldn't it? > > 2. I would expect the pipe's return code to be non-zero. But it is 0. > Why? > This is the most critical issue, since the program relies on the return > code to detect the failure. > > 3. I would expect the pipe to use the same amount of storage no matter how > many files are read. Those records should move through the pipe one at a > time, right? They are detail input to lookup, not master input. The pipe > works fine when only one or two files are read, but runs out of memory with > 11 inputs. It works with 11 inputs when I increase the amount of memory. > > The result is essentially the same with all versions of pipes that I have > tried, including these: > 1.0110 11 Oct 2005, 11 Rev 0A Mod 0028 > 1.0111 5 May 2009, 11 Rev 0B Mod 0021 > > All suggestions much appreciated. Thank you. >
