[
https://issues.apache.org/jira/browse/MAPREDUCE-7220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuanbo Liu updated MAPREDUCE-7220:
----------------------------------
Issue Type: Bug (was: Improvement)
> Mapreduce jobhistory summary error if job name is very long
> -----------------------------------------------------------
>
> Key: MAPREDUCE-7220
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7220
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Yuanbo Liu
> Priority: Critical
>
> From JobHistoryEventHandler.java, we can see that mapreduce uses writeUTF to
> write summary.done file to hdfs. The code is here:
> {quote}summaryFileOut = doneDirFS.create(qualifiedSummaryDoneFile, true);
> summaryFileOut.writeUTF(mi.getJobSummary().getJobSummaryString());
> summaryFileOut.close();
> {quote}
> writeUTF uses first two bytes to record string length, hence the length of
> summary string cannot exceed 65535. But in the case of hive job, SQL string
> is part of job name. It's quite normal that SQL length is greater than 65535,
> then summary done file cannot be written successfully. In this case, hive
> client thinks such kind of mapreduce job is in the final state of failure
> sometimes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
