> Understanding record delay and how end-of-file propagates is important to construct pipeline refineries that don't leak.
Is there a "record delay for dummies" document available anywhere? Over time I've developed a cursory understanding of the topic, but would really like to crisp it up. -- bc On Tue, Mar 3, 2015 at 5:34 PM, Rob van der Heij <[email protected]> wrote: > > 'PIPE stem dirstem.' , > > '| find BUILD', > > '| specs word -1;-1 1' , > > '| var LastWord' > > > > And maybe throw in a 'take 1' to stop reading the stem if there's only > one > > occurrence of a line beginning with BUILD? > > > > > The concerns expressed here are unfounded. As we know "var" (when used like > this) will only take the first record it gets. The stage knows that too, > and severs its primary input when it has a record, the end-of-file will > propagate back through "specs" and "find" because they have nothing else to > do either, and that will discourage "stem" from producing more records than > needed to satisfy "var". Whether there's just one match or multiple ones. > > To experiment: add a "count" in the pipeline and replace the last "var" by > a "stem" stage to see the difference. Performance here is the least concern > (the cost of an added "take" is trivial). Understanding record delay and > how end-of-file propagates is important to construct pipeline refineries > that don't leak. > > The options for improvement are to move this function into the pipeline > that produced the 'dirstem' array in the first place. If nothing else, you > could tag this on there and avoid reading the stem just to find this > record. Depending on the rest of the code, you might not even need the stem > anymore. > > Sir Rob the Plumber >
