Kevin W Monroe created BIGTOP-2603:
--------------------------------------
Summary: NN/RM charm should include a spark user/group
Key: BIGTOP-2603
URL: https://issues.apache.org/jira/browse/BIGTOP-2603
Project: Bigtop
Issue Type: Bug
Components: deployment
Affects Versions: 1.1.0
Reporter: Kevin W Monroe
Assignee: Kevin W Monroe
Priority: Minor
When in 'yarn-client' mode, the spark charm creates a directory in hdfs with
'spark' user/group permissions.
If spark is not colocated with the NN and RM, this user/group does not exist on
those machines. This causes warnings in the hdfs log like this:
WARN org.apache.hadoop.security.UserGroupInformation: No groups available for
user spark
It also prevents the RM history server from accessing spark job history files.
To combat this, always include a spark user/group in the NN/RM charms. This is
very similar to how we always create a 'mapred' user on the NN to prevent
warnings about mapred being unknown to the namenode.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
