[
https://issues.apache.org/jira/browse/YARN-9814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16929754#comment-16929754
]
Prabhu Joseph commented on YARN-9814:
-------------------------------------
[~adam.antal] Thanks for the patch. The patch looks good. Have found one issue.
1. With Hdfs, when yarn.nodemanager.remote-app-log-dir.group set to "" (default
value). the root log directory group is supergroup
(dfs.permissions.superusergroup) whereas without the patch it was yarn's
primaryGroupName. I think we have to handle "" similar to Null.
{code}
String primaryGroupName = conf.get(
YarnConfiguration.NM_REMOTE_APP_LOG_DIR_GROUP);
if (primaryGroupName == null || primaryGroupName.equals(""))
{code}
> JobHistoryServer can't delete aggregated files, if remote app root directory
> is created by NodeManager
> ------------------------------------------------------------------------------------------------------
>
> Key: YARN-9814
> URL: https://issues.apache.org/jira/browse/YARN-9814
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation, yarn
> Affects Versions: 3.1.2
> Reporter: Adam Antal
> Assignee: Adam Antal
> Priority: Minor
> Attachments: YARN-9814.001.patch, YARN-9814.002.patch,
> YARN-9814.003.patch
>
>
> If remote-app-log-dir is not created before starting Yarn processes, the
> NodeManager creates it during the init of AppLogAggregator service. In a
> custom system the primary group of the yarn user (which starts the NM/RM
> daemons) is not hadoop, but set to a more restricted group (say yarn). If
> NodeManager creates the folder it derives the group of the folder from the
> primary group of the login user (which is yarn:yarn in this case), thus
> setting the root log folder and all its subfolders to yarn group, ultimately
> making it unaccessible to other processes - e.g. the JobHistoryServer's
> AggregatedLogDeletionService.
> I suggest to make this group configurable. If this new configuration is not
> set then we can still stick to the existing behaviour.
> Creating the root app-log-dir each time during the setup of this system is a
> bit error prone, and an end user can easily forget it. I think the best to
> put this step is the LogAggregationService, which was responsible for
> creating the folder already.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
