[
https://issues.apache.org/jira/browse/MAPREDUCE-4284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13283335#comment-13283335
]
Arun C Murthy commented on MAPREDUCE-4284:
------------------------------------------
@Tucu:
bq.Still, I would say this is a property to be use in development clusters.
If this is only needed for development clusters, then we just use the global
setting and make it very high (e.g. 3 days).
bq. Or, in order to make it more production friendly there should be a
MAX_TIME_TO_KEEP_FILES property in the NM and jobs can set any value up to that
time.
Then you pretty much have to have a limit on file-sizes, number of files etc.
which leads exactly to MAPREDUCE-1100, something which we've been trying to
avoid by durably storing logs in HDFS and not on the NM local disk.
----
To recap, if this is just for debugging, we can set the global limit very high
and not bother with per-job limits.
IAC, we have all task logs on HDFS - so I really don't see the need to reinvent
MAPREDUCE-1100.
----
@Ahmed - Your proposal doesn't work because the NodeManager doesn't load
jobConf of the container... this would require changes to ContainerManager
protocol.
> Allow setting yarn.nodemanager.delete.debug-delay-sec on a per-job basis
> ------------------------------------------------------------------------
>
> Key: MAPREDUCE-4284
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4284
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Reporter: Ahmed Radwan
> Assignee: Ahmed Radwan
>
> The yarn.nodemanager.delete.debug-delay-sec property is helpful in debugging
> jobs (inspecting container logs/local dirs after the job finishes). Currently
> it is a nodemanager property and changing it requires restarting the
> nodemanager. In a production cluster this can be a real problem. It is better
> to have this property set on a per-job basis and not requiring the restart of
> nodemanagers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
