[
https://issues.apache.org/jira/browse/YARN-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16144317#comment-16144317
]
Junping Du edited comment on YARN-7083 at 8/28/17 8:48 PM:
-----------------------------------------------------------
Trunk code has changed significantly since YARN-6876 so patch here doesn't
apply any more. From my quick look, the issue (not close file before
rename/delete) is still there but the fix is not straightforward as writer get
hidden behind for different formats. I would suggest to commit the patch fix
here to branch-2.8 and branch-2.8.2 only and create a separated jira tracking
for trunk/branch-2.
[~jlowe], what do you think?
was (Author: djp):
Trunk code has changed significantly since YARN-6877 so patch here doesn't
apply any more. From my quick look, the issue (not close file before
rename/delete) is still there but the fix is not straightforward as writer get
hidden behind for different formats. I would suggest to commit the patch fix
here to branch-2.8 and branch-2.8.2 only and create a separated jira tracking
for trunk/branch-2.
[~jlowe], what do you think?
> Log aggregation deletes/renames while file is open
> --------------------------------------------------
>
> Key: YARN-7083
> URL: https://issues.apache.org/jira/browse/YARN-7083
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.8.2
> Reporter: Daryn Sharp
> Assignee: Jason Lowe
> Priority: Critical
> Attachments: YARN-7083.001.patch
>
>
> YARN-6288 changes the log aggregation writer to be an autoclosable.
> Unfortunately the try-with-resources block for the writer will either rename
> or delete the log while open.
> Assuming the NM's behavior is correct, deleting open files only results in
> ominous WARNs in the nodemanager log and increases the rate of logging in the
> NN when the implicit try-with-resource close fails. These red herrings
> complicate debugging efforts.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
