Yeah, I think that before, we would just fail catastrophically by throwing a CrunchRuntimeException, which I found annoying. Do you prefer that behavior? It's certainly something that could be configurable.
J On Wed, Jan 28, 2015 at 10:36 AM, Jinal Shah <jinalshah2...@gmail.com> wrote: > I think it was intented from these commits I see here > > https://github.com/apache/crunch/commit/3711cea61bded4c90b235a01163ae5f855089917 > and > > https://github.com/apache/crunch/commit/ded504eb133fa0814e2d90ff2a662e72a67e04bb > . > Josh can enhance on this more. > > On Wed, Jan 28, 2015 at 9:26 AM, Mārtiņš Kalvāns < > martins.kalv...@gmail.com> > wrote: > > > Hi. > > > > When pipeline fails on cluster with some exception, materialize() returns > > empty collection and just logs error message. > > > > I'm (very, very) puzzled about this behaviour: > > > > > https://github.com/apache/crunch/blob/master/crunch-core/src/main/java/org/apache/crunch/materialize/MaterializableIterable.java#L92 > > Is this really intended behaviour? > > > > If so, then some documentation for materialize() function about this > > behaviour would be really nice to have. :) > > > > > > -- > > Mārtiņš > > > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
