[
https://issues.apache.org/jira/browse/YARN-9717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16995074#comment-16995074
]
Eric Badger commented on YARN-9717:
-----------------------------------
Would this still be a useful log to have? Right now, the logging only tells you
that the creation of the container directories failed, but it doesn't give any
information on what directories were trying to be created.
It also looks like create_container_directories will succeed if any directories
can be created, even if multiple are passed. Seems like this is because the
directories will already be created by the distributed cache, so we're ignoring
errors if the mkdir fails. Shouldn't we be checking the error we get back from
mkdir? If a disk goes bad and the mkdir fails, the container-executor will pass
the bad local dirs to the container.
> Add more logging to container-executor about issues with directory creation
> or permissions
> ------------------------------------------------------------------------------------------
>
> Key: YARN-9717
> URL: https://issues.apache.org/jira/browse/YARN-9717
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Szilard Nemeth
> Assignee: Peter Bacsko
> Priority: Major
>
> During some downstream testing we bumped into some problems with the
> container executor where an extra logging would be quite helpful when local
> files and directories could not be created (container-executor.c:1810).
> The most important log line could be the following:
> There's a function called create_container_directories in
> container-executor.c.
> We should place a log line like this:
> Before we're calling:
> We have:
> {code:java}
> if (mkdirs(container_dir, perms) == 0) {
> result = 0;
> }
> {code}
> We could add an else statement and add the following log, if creating the
> directory was not successful:
> {code:java}
> fprintf(LOGFILE, "Failed to create directory: %s, user: %s", container_dir,
> user);
> {code}
> This way, CE at least prints the directory itself if we have any permission
> issue while trying to create a subdirectory or file under it.
> If we want to be very precise, some logging into the mkdirs function could
> also be added as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
