Oleksandr Shevchenko created MAPREDUCE-7134:
-----------------------------------------------
Summary: Setup and cleanup progress information are not available
after job completion
Key: MAPREDUCE-7134
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7134
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Oleksandr Shevchenko
Information about "setupProgress" and "cleanupProgress" are not available after
a job completion. These properties are ignored during creating JobReport since
they were not added to jhist file.
We can get this information during a job is running:
{code}
import org.apache.hadoop.mapred.JobClient
import org.apache.hadoop.conf.Configuration
val client = new JobClient(new Configuration())
val job = client.getJob("job_1535984902382_0006")
val setup = job.setupProgress()
{code}
When job runnin:
{code}
scala> val setup = job.setupProgress()
setup: Float = 1.0
{code}
When job already finished:
{code}
scala> val setup = job.setupProgress()
setup: Float = 0.0
{code}
JobReport interface includes getCleanupProgress and getSetupProgress methods.
But these methods ignore during creating JobReport in
CompletedJob#constructJobReport(). setupProgress and cleanupProgress properties
set in JobImpl class but JobFinishedEvent and JobUnsuccessfulCompletionEvent do
not include this information. As the result, these properties do not include in
jhist file.
We can add these properties to jhist file but of course, this is the
incompatible change. These changes should be applied for a major release.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
