I would not dare to suggest John's paper http://vm.marist.edu/~pipeline/pipjarg.pdf would be meant for dummies, but you're not either. Melinda's papers are worth another read again: http://vm.marist.edu/~pipeline/#MWV in particular http://vm.marist.edu/~pipeline/mtpipe.pdf
Sir Rob the Plumber On 3 March 2015 at 23:44, Bob Cronin <[email protected]> wrote: > > Understanding record delay and how end-of-file propagates is important to > construct pipeline refineries that don't leak. > > Is there a "record delay for dummies" document available anywhere? Over > time I've developed a cursory understanding of the topic, but would really > like to crisp it up. > -- > bc > > On Tue, Mar 3, 2015 at 5:34 PM, Rob van der Heij <[email protected]> > wrote: > > > > 'PIPE stem dirstem.' , > > > '| find BUILD', > > > '| specs word -1;-1 1' , > > > '| var LastWord' > > > > > > And maybe throw in a 'take 1' to stop reading the stem if there's only > > one > > > occurrence of a line beginning with BUILD? > > > > > > > > The concerns expressed here are unfounded. As we know "var" (when used > like > > this) will only take the first record it gets. The stage knows that too, > > and severs its primary input when it has a record, the end-of-file will > > propagate back through "specs" and "find" because they have nothing else > to > > do either, and that will discourage "stem" from producing more records > than > > needed to satisfy "var". Whether there's just one match or multiple > ones. > > > > To experiment: add a "count" in the pipeline and replace the last "var" > by > > a "stem" stage to see the difference. Performance here is the least > concern > > (the cost of an added "take" is trivial). Understanding record delay and > > how end-of-file propagates is important to construct pipeline refineries > > that don't leak. > > > > The options for improvement are to move this function into the pipeline > > that produced the 'dirstem' array in the first place. If nothing else, > you > > could tag this on there and avoid reading the stem just to find this > > record. Depending on the rest of the code, you might not even need the > stem > > anymore. > > > > Sir Rob the Plumber > > >
