[
https://issues.apache.org/jira/browse/PIG-457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636261#action_12636261
]
Shravan Matthur Narayanamurthy commented on PIG-457:
----------------------------------------------------
Actually, the final progress check is exactly that. It doesn't report 100%
until it is also sure that the job completed successfully. However, there are 2
other changes: 1) to mask some issue with hadoop progress 2) When the issue (1)
is masked we would not see any kind of progress if the job is not showing
progress after a certain point of time. So instead appearing frozen, there is
some logic which displays the same percentage with an exponentially increasing
delay between each report of the same progress just to say that we are alive.
Once we get a different progress all counters are reset and it resumes normal
progress reporting until the progress is blocked again. (Not exactly right but
just to simplify understanding. It actually adds exponential delay between a
bunch of same progress reports).
> pig produces errors after a job is said to be 100% done
> -------------------------------------------------------
>
> Key: PIG-457
> URL: https://issues.apache.org/jira/browse/PIG-457
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Olga Natkovich
> Assignee: Shravan Matthur Narayanamurthy
> Fix For: types_branch
>
> Attachments: 457-2.patch, 457-3.patch, 457-4.patch, 457-part2.patch
>
>
> It is possible that we get errors for all tasks even the ones we retried.
> Need to look at the code that handles detecting end of processing and
> producing errors.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
