No, that's not good, we should fix that. Is it only in the SparkPipeline that the situation occurs?
On Mon, Jan 19, 2015 at 8:28 AM, Peter Dolan <[email protected]> wrote: > Hi Crunchers, > > At Nuna we've been using Crunch extensively, and I'm really thrilled with > it. It's excellent. There are of course some rough edges though. > > Today I ran into some exceptions being thrown in the Spark pipeline, and > am curious why they weren't resulting in the PipelineResult reporting > failure. In particular, my spark pipeline (running with a local spark > instance, that is with the spark master set to "local[16]") failed with an > IOException when the machine ran out of space in /tmp/. The PipelineResult > retrieved by Pipeline#done returned true from PipelineResult#succeeded. > > I've seen this in a couple other contexts, for example when a MapFn threw > an exception within MapFn#map, which did not result in a false success > value. > > Is this expected / intended behavior? Should I be getting at the success > or failure of the execution some other way? > > Thanks! > - Peter >
