Daniel Dai commented on PIG-943:

Hi, Dmitriy,
All this patch does is checking for nulls. The extra work in the patch is when 
counters object is null, we treat all counter as unknown, so we do not report 
confusing results to user. Which part do you think "hacky"? Also parseLong is 
there for a long time and currently the only problem we observe is 
RunningJob.getCounters is null. If this is your concern I can change it to use 

> Pig crash when it cannot get counter from hadoop
> ------------------------------------------------
>                 Key: PIG-943
>                 URL: https://issues.apache.org/jira/browse/PIG-943
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.3.0
>            Reporter: Daniel Dai
>             Fix For: 0.4.0
>         Attachments: PIG-943-1.patch
> We see following call stacks in Pig:
> Case 1:
> Caused by: java.lang.NullPointerException
>         at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390)
>         at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238)
>         at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265)
> Case 2:
> Caused by: java.lang.NullPointerException
>         at 
> org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150)
>         at 
> org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91)
>         at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192)
>         at 
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265)
> In both cases, hadoop jobs finishes without error. The cause of both problems 
> is RunningJob.getCounters() returns a null, and Pig do not currently check 
> for that. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to