[
https://issues.apache.org/jira/browse/PIG-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-4164:
----------------------------
Attachment: PIG-4164-0.patch
According to [~vinodkv], Pig shall not reuse the Job object to pull the
taskReport/counters since it will keep trying AM. We shall construct a new Job
object with designated Job ID to do that.
Note this issue only happen in Hadoop 2 since Hadoop 1 does not have
AM/JobHistoryServer (will always query JobTracker).
Attach initial patch. In patch, I created a Cluster object every time which can
be further optimized.
> After Pig job finish, Pig client spend too much time retry to connect to AM
> ---------------------------------------------------------------------------
>
> Key: PIG-4164
> URL: https://issues.apache.org/jira/browse/PIG-4164
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4164-0.patch
>
>
> For some script, after job finish, Pig spend a lot time try to connect AM
> before get redirect to JobHistoryServer. Here is the message we saw:
> {code}
> 2014-09-10 15:13:55,370 [main] INFO org.apache.hadoop.ipc.Client - Retrying
> connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 0
> time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3,
> sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:56,371 [main] INFO org.apache.hadoop.ipc.Client - Retrying
> connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 1
> time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3,
> sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:57,372 [main] INFO org.apache.hadoop.ipc.Client - Retrying
> connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 2
> time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3,
> sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:57,476 [main] INFO
> org.apache.hadoop.mapred.ClientServiceDelegate - Application state is
> completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
