[
https://issues.apache.org/jira/browse/CHUKWA-155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701330#action_12701330
]
Jerome Boulon commented on CHUKWA-155:
--------------------------------------
my 2 cents ...
Or we could have a more generic way:
The problem is that the mysql Job table does not contains columns for more than
one state.
What we care about in this table is the final Job state.
So why not create one additional key JOB_FINAL_STATE that will contain the
final Job state, aka, success, killed or failed.
the "JOB_FINAL_STATE" 's key could be created at the same time as the
finish-time parsing from JobHistory.
For example:
Job JOBID="job_200903310541_1200" FINISH_TIME="1238528943585"
JOB_STATUS="SUCCESS" will give JOB_FINAL_STATE="SUCCESS"
Then all others JOB_STATUS should remain unchanged, aka
JOB_STATUS="WHAT_EVER_THE_VALUE_IS", timestamp will be what ever is available
at that time.
This will give us the transition-states table.
> Job History status arrive out of order causing the status to update
> incorrectly.
> --------------------------------------------------------------------------------
>
> Key: CHUKWA-155
> URL: https://issues.apache.org/jira/browse/CHUKWA-155
> Project: Hadoop Chukwa
> Issue Type: Bug
> Components: data collection, Data Processors
> Environment: Redhat 5.1, Java 6
> Reporter: Eric Yang
> Assignee: Cheng
> Priority: Critical
> Attachments: chukwa-155.patch
>
>
> Job history contains lines like:
> Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
> ...
> Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308"
> JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760"
> FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .
> When pushing the data through collectors and demux, the data can arrive out
> of order. The database is updated with status "RUNNING" instead of
> "SUCCESS".
> Chukwa Sequence ID can be used to sort out of order data before the data is
> pumped to database.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
