Ottomata has submitted this change and it was merged.
Change subject: Create analytics-privatedata-users group
......................................................................
Create analytics-privatedata-users group
This group should be used to grant access to the Analytics (Hadoop)
cluster and to any private data within. Private data in HDFS
will be 640 hdfs:analytics-privatedata-users.
Change-Id: Iac08bcebbbdd7423eb2b3096eb6e0f4830343dea
---
M manifests/role/analytics.pp
M manifests/role/analytics/refinery.pp
M manifests/site.pp
M modules/admin/data/data.yaml
4 files changed, 36 insertions(+), 50 deletions(-)
Approvals:
Ottomata: Verified; Looks good to me, approved
Rush: Looks good to me, but someone else must approve
Alexandros Kosiaris: Looks good to me, but someone else must approve
diff --git a/manifests/role/analytics.pp b/manifests/role/analytics.pp
index cd59d82..4a78582 100644
--- a/manifests/role/analytics.pp
+++ b/manifests/role/analytics.pp
@@ -39,39 +39,3 @@
}
}
-# == Class role::analytics::users
-# Users that should be on analytics nodes.
-# This class is not included on *all* analytics
-# nodes, just ones where it is useful for users to
-# have accounts. I.e. hadoop related nodes.
-# Users do not need accounts on Kafka or Zookeeper nodes.
-class role::analytics::users {
-
- # Analytics user accounts will be added to the
- # 'stats' group which gets created by this class.
- require misc::statistics::user
-
- # NOTE: If you are filling an RT request for Hadoop access,
- # you will need to add the user to the list of accounts above,
- # as well as manually create the user's HDFS home directory.
- # Once the user's posix account is created on analytics1010
- # (the Hadoop NameNode), run these commands:
- #
- # sudo -u hdfs hadoop fs -mkdir /user/<username>
- # sudo -u hdfs hadoop fs -chown <username>:stats /user/<username>
- #
-
- # If hdfs user exists, then add it to the stats group.
- # I don't want to use puppet types to manage the hdfs
- # user, since it is installed by the cdh packages.
- exec { 'hdfs_user_in_stats_group':
- command => 'usermod hdfs -a -G stats',
- # only run this command if the hdfs user exists
- # and it is not already in the stats group
- # This command returns true if hdfs user does not exist,
- # or if hdfs user does exist and is in the stats group.
- unless => 'getent passwd hdfs > /dev/null; if [ $? != 0 ]; then true;
else groups hdfs | grep -q stats; fi',
- path => '/usr/sbin:/usr/bin:/bin',
- require => Group['stats'],
- }
-}
diff --git a/manifests/role/analytics/refinery.pp
b/manifests/role/analytics/refinery.pp
index d347915..0fe5882 100644
--- a/manifests/role/analytics/refinery.pp
+++ b/manifests/role/analytics/refinery.pp
@@ -3,9 +3,6 @@
# and using the analytics/refinery repository.
#
class role::analytics::refinery {
- # Require analytics users so we hdfs can write log files as stats user.
- require role::analytics::users
-
# Many Kraken python scripts use docopt for CLI parsing.
if !defined(Package['python-docopt']) {
package { 'python-docopt':
@@ -36,6 +33,8 @@
file { $log_dir:
ensure => 'directory',
owner => 'root',
+ # TODO: Change this to analytics-admins group after
+ # https://gerrit.wikimedia.org/r/#/c/150560 is merged.
group => 'stats',
# setgid bit here to make kraken log files writeable
# by users in the stats group.
diff --git a/manifests/site.pp b/manifests/site.pp
index 59421f3..4d11dcc 100644
--- a/manifests/site.pp
+++ b/manifests/site.pp
@@ -141,7 +141,12 @@
# ganglia cluster name.
$cluster = 'analytics'
- class { 'admin': groups => ['analytics-users'] }
+ class { 'admin':
+ groups => [
+ 'analytics-users',
+ 'analytics-privatedata-users',
+ ],
+ }
include standard
include role::analytics::hadoop::standby
@@ -156,7 +161,12 @@
# analytics1010 is analytics Ganglia aggregator for Row B
$ganglia_aggregator = true
- class { 'admin': groups => ['analytics-users'] }
+ class { 'admin':
+ groups => [
+ 'analytics-users',
+ 'analytics-privatedata-users',
+ ]
+ }
include standard
include role::analytics::hadoop::master
@@ -2464,8 +2474,13 @@
# Users should not use it for app development.
# Data processing on this machine is fine.
- class { 'admin': groups => ['statistics-privatedata-users',
- 'statistics-admins'] }
+ class { 'admin':
+ groups => [
+ 'statistics-privatedata-users',
+ 'statistics-admins',
+ 'analytics-privatedata-users',
+ ],
+ }
# include classes needed for storing and crunching
# private data on stat1002.
diff --git a/modules/admin/data/data.yaml b/modules/admin/data/data.yaml
index 3f373d4..4486899 100644
--- a/modules/admin/data/data.yaml
+++ b/modules/admin/data/data.yaml
@@ -125,13 +125,6 @@
description: users who have full root on jenkins servers
members: [hashar]
privs: ['ALL = NOPASSWD: ALL']
- analytics-users:
- posix_name: stats
- gid: 7080
- description: XXX:TEMP -- should not need to use posix_name attribute
- members: [dartar, erik, milimetric, yurik, maryana,
- halfak, dr0ptp4kt, qchris, tnegrin, ironholds, yuvipanda,
- kleduc, nuria, haithams]
ocg-render-admins:
gid: 721
description: admins for pdf render (rt 6468)
@@ -196,6 +189,21 @@
'ALL = (root) NOPASSWD: /usr/sbin/service mathoid start',
'ALL = (root) NOPASSWD: /usr/sbin/service mathoid restart',
'ALL = (root) NOPASSWD: /usr/sbin/service mathoid reload']
+ analytics-users:
+ gid: 7080
+ description: Gives generic client access to the Analytics (Hadoop) cluster.
+ This will grant shell access on Hadoop client nodes (stat1002) and on
+ Hadoop NameNodes.
+ members: []
+ analytics-privatedata-users:
+ gid: 731
+ description: Gives access to the Analytics (Hadoop) cluster as well as
prviate data within.
+ This will grant shell access on Hadoop client nodes (stat1002) and on
+ Hadoop NameNodes. Some files in HDFS have sensitive data in them.
+ Those files are group readable by the analytics-privatedata-users
group.
+ members: [dartar, erik, milimetric, yurik, maryana,
+ halfak, dr0ptp4kt, qchris, tnegrin, ironholds, yuvipanda,
+ kleduc, nuria, haithams, otto]
users:
rush:
ensure: present
--
To view, visit https://gerrit.wikimedia.org/r/151657
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Iac08bcebbbdd7423eb2b3096eb6e0f4830343dea
Gerrit-PatchSet: 6
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Ottomata <[email protected]>
Gerrit-Reviewer: Alexandros Kosiaris <[email protected]>
Gerrit-Reviewer: Chasemp <[email protected]>
Gerrit-Reviewer: Ottomata <[email protected]>
Gerrit-Reviewer: Rush <[email protected]>
Gerrit-Reviewer: jenkins-bot <>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
