Hi Crunchers, At Nuna we've been using Crunch extensively, and I'm really thrilled with it. It's excellent. There are of course some rough edges though.
Today I ran into some exceptions being thrown in the Spark pipeline, and am curious why they weren't resulting in the PipelineResult reporting failure. In particular, my spark pipeline (running with a local spark instance, that is with the spark master set to "local[16]") failed with an IOException when the machine ran out of space in /tmp/. The PipelineResult retrieved by Pipeline#done returned true from PipelineResult#succeeded. I've seen this in a couple other contexts, for example when a MapFn threw an exception within MapFn#map, which did not result in a false success value. Is this expected / intended behavior? Should I be getting at the success or failure of the execution some other way? Thanks! - Peter
