Hi Kamal, PipelineResult can be null. This will happen when there is a job failure in the pipeline(in almost all the cases I have seen). Not propagating to the launcher is because internally Crunch handles any exceptions and return a NULL object instead of throwing the exception. You could get the exception if you look at jobs details page. You could possibly raise the argument to throw an exception when a jobs from pipeline.run and pipeline.done.
Thanks, Jinal On Tue, Dec 30, 2014 at 8:38 PM, Kamal Ananthaneni < kamal.ananthan...@gmail.com> wrote: > What are the scenarios in which pipeline.run() or pipeline.done() returns a > PipelineResult of null? > > We have a pipeline that scans from HBase, does some processing runs the > pipeline and does some additional processing afterwords. > > We noticed that if the Scan from HBase failed due to Socket Timeouts or > other exceptions, the PipelineResult from pipeline.run() was null. > > We were not treating PipelineResult == null as an error scenario and were > only checking for PipelineResult.status if PipelineResult is != null and > hence the error was not getting propagated to the launcher job. > > In this case the error would have propagated and failed our launcher job > had we treated the PipelineResult == null as an Error scenario. > > Is the assumption that PipelineResult == null on Pipeline.run() or > Pipeline.done() indicates an error a valid one? In what other scenarios can > a PipelineResult be null? > > Thanks, > Kamal >
