On Nov 7, 2006, at 8:40 AM, Arkady Borkovsky wrote:
Relationship with streaming?
when defining the implementation for speculative re-execution,
please take into account how Streaming is implemented.
Specifically, streaming tasks are supposed run in their own working
directories and read and write (temporary) (Unix) files there.
There may be some details that are important for speculative re-
execution implementation.
For the immediate term, Streaming probably won't support speculative
execution. I suspect there are some unexplored corner cases that will
make it hard. The "normal" Streaming data path via stdout should work
since it will use the framework's normal collector from the reduce.
-- Owen
