[
https://issues.apache.org/jira/browse/HADOOP-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622433#action_12622433
]
amareshwari edited comment on HADOOP-3937 at 8/13/08 9:02 PM:
--------------------------------------------------------------------------
Earlier we had trimming of the file name to MAX_FILENAME_SIZE, but that is
removed by HADOOP-3263.
bq. One can maintain the mapping of f(s) to jobtracker-hostname in some index
file along with the username and jobname information.
When DFS supports *appends*, we can have master-index file for job history that
was there earlier (till 0.16). Then user name, job name will be in master
index file, instead of part of filename.
Thoughts?
was (Author: amareshwari):
Earlier we had trimming of the file name to MAX_FILENAME_SIZE, but that is
removed by HADOOP-3263.
bq. One can maintain the mapping of f(s) to jobtracker-hostname in some index
file along with the username and jobname information.
Since DFS supports *appends* now, we can have master-index file for job history
that was there earlier (till 0.16). Then user name, job name will be in
master index file, instead of part of filename.
Thoughts?
> Job history may get disabled due to overly long job names
> ---------------------------------------------------------
>
> Key: HADOOP-3937
> URL: https://issues.apache.org/jira/browse/HADOOP-3937
> Project: Hadoop Core
> Issue Type: Bug
> Affects Versions: 0.17.0, 0.17.1, 0.18.0, 0.19.0
> Reporter: Matei Zaharia
> Attachments: HADOOP-3937.patch
>
>
> Since Hadoop 0.17, the job history logs include the job's name in the
> filename. However, this can lead to overly long filenames, because job names
> may be arbitrarily long. When a filename is too long for the underlying OS,
> file creation fails and the JobHistory class silently disables history from
> that point on. This can lead to days of lost history until somebody notices
> the error in the log.
> Proposed solution: Trim the job name to a reasonable length when selecting a
> filename for the history file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
