[
https://issues.apache.org/jira/browse/MAPREDUCE-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385837#comment-15385837
]
Kai Sasaki commented on MAPREDUCE-6380:
---------------------------------------
[~varun_saxena] Thanks for detail explanation. What I'm trying to do in this
JIRA is below.
Currently {{listStatue}} is checking against root log dir. So returned list is
user log dirs like {{/tmp/logs/userA}}, {{/tmp/logs/userB}},
{{/tmp/logs/userC}}.
In this case, iterations is done for each user log dirs. (userA, userB, userC)
But if {{userB}} does not have {{/tmp/logs/userB/logs}} dir, an exception is
occurred and iteration will be stopped even if the later user log dirs have
proper log dir like {{/tmp/logs/userC/logs}}. So the patch makes iteration keep
to progress for {{userC}} even if {{/tmp/logs/userB}} does not have
{{/tmp/logs/userB/logs}} dir. Therefore checking whether {{/tmp/logs/userB}}
has {{logs}} dir and actual logs files are necessary.
{quote}
If extraneous directories have to be considered, I see a bigger issue while
listing app directories. If there is a spurious directory inside
/tmp/logs/user/logs which is not in app id format (i.e. a directory like
/tmp/logs/user/logs/dummy), it will be bigger problem.
{quote}
Yes, as you said the format of app logs should also be considered here.
> AggregatedLogDeletionService will throw exception when there are some other
> directories in remote-log-dir
> ---------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6380
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
> Reporter: Zhang Wei
> Assignee: Kai Sasaki
> Priority: Trivial
> Attachments: MAPREDUCE-6380.01.patch, MAPREDUCE-6380.02.patch,
> MAPREDUCE-6380.03.patch, MAPREDUCE-6380.04.patch, MAPREDUCE-6380.05.patch,
> MAPREDUCE-6380.06.patch, MAPREDUCE-6380.07.patch
>
>
> AggregatedLogDeletionService will throw FileNotFoundException when there are
> some extraneous directories put in remote-log-dir. The deletion function will
> try to listStatus against the "extraneous-dir + suffix" dir. I think it
> would be better if the function can ignore these directories.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
