[
https://issues.apache.org/jira/browse/PIG-943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12751210#action_12751210
]
Daniel Dai commented on PIG-943:
--------------------------------
Hi, Dmitriy,
All this patch does is checking for nulls. The extra work in the patch is when
counters object is null, we treat all counter as unknown, so we do not report
confusing results to user. Which part do you think "hacky"? Also parseLong is
there for a long time and currently the only problem we observe is
RunningJob.getCounters is null. If this is your concern I can change it to use
stringToLong.
> Pig crash when it cannot get counter from hadoop
> ------------------------------------------------
>
> Key: PIG-943
> URL: https://issues.apache.org/jira/browse/PIG-943
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: 0.3.0
> Reporter: Daniel Dai
> Fix For: 0.4.0
>
> Attachments: PIG-943-1.patch
>
>
> We see following call stacks in Pig:
> Case 1:
> Caused by: java.lang.NullPointerException
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.computeWarningAggregate(MapReduceLauncher.java:390)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:238)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265)
> Case 2:
> Caused by: java.lang.NullPointerException
> at
> org.apache.pig.tools.pigstats.PigStats.accumulateMRStats(PigStats.java:150)
> at
> org.apache.pig.tools.pigstats.PigStats.accumulateStats(PigStats.java:91)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:192)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265)
> In both cases, hadoop jobs finishes without error. The cause of both problems
> is RunningJob.getCounters() returns a null, and Pig do not currently check
> for that.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
