-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35988/
-----------------------------------------------------------
Review request for Ambari and Ivan Kozlov.
Bugs: AMBARI-12191
https://issues.apache.org/jira/browse/AMBARI-12191
Repository: ambari
Description
-------
Attempting RU with non-root ambari server + umask 027 + non-root agent. Fails
when trying to restart first ZooKeeper.
If I stop my agent, and start it as root, and then click "retry" in the RU
dialog, it can make it thru the ZooKeeper restart.
Below are the errors when attempting as non-root agent, as well as my agent
user visudo setup. Are Ambari commands that need to be added and/or things
aren't running thru sudo like they should...or maybe it has to do with
something + umask 027?
Traceback (most recent call last):
File
"/var/lib/ambari-agent/cache/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py",
line 176, in <module>
ZookeeperServer().execute()
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 216, in execute
method(env)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 441, in restart
self.pre_rolling_restart(env)
File
"/var/lib/ambari-agent/cache/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/zookeeper_server.py",
line 79, in pre_rolling_restart
hdp_select.select("zookeeper-server", params.version)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/hdp_select.py",
line 107, in select
Execute(command)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py",
line 157, in __init__
self.env.run()
File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 152, in run
self.run_action(resource, action)
File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 118, in run_action
provider_action()
File
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
line 254, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70,
in inner
result = function(command, **kwargs)
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92,
in checked_call
tries=tries, try_sleep=try_sleep)
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 140,
in _call_wrapper
result = _call(command, **kwargs_copy)
File
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291,
in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 'hdp-select set
zookeeper-server 2.3.0.0-2474' returned 1. Traceback (most recent call last):
File "/usr/bin/hdp-select", line 373, in <module>
setPackages(pkgs, args[2], options.rpm_mode)
File "/usr/bin/hdp-select", line 265, in setPackages
os.remove(linkname)
OSError: [Errno 13] Permission denied: '/usr/hdp/current/zookeeper-server'
2015-06-26 00:04:56,968 - zookeeper-server is currently at version
2.2.6.0-2800
2015-06-26 00:04:56,996 - zookeeper-server is currently at version
2.2.6.0-2800
2015-06-26 00:04:57,021 - zookeeper-server is currently at version
2.2.6.0-2800
2015-06-26 00:04:57,046 - call['conf-select set-conf-dir --package hadoop
--stack-version 2.3.0.0-2474 --conf-version 0'] {'logoutput': False, 'quiet':
False}
2015-06-26 00:04:57,073 - call returned (1, '/etc/hadoop/2.3.0.0-2474/0does
not exist')
2015-06-26 00:04:57,096 - zookeeper-server is currently at version
2.2.6.0-2800
2015-06-26 00:04:57,282 - zookeeper-server is currently at version
2.2.6.0-2800
2015-06-26 00:04:57,305 - call['conf-select set-conf-dir --package hadoop
--stack-version 2.3.0.0-2474 --conf-version 0'] {'logoutput': False, 'quiet':
False}
2015-06-26 00:04:57,326 - call returned (1, '/etc/hadoop/2.3.0.0-2474/0does
not exist')
2015-06-26 00:04:57,349 - zookeeper-server is currently at version
2.2.6.0-2800
2015-06-26 00:04:57,352 -
Directory['/var/lib/ambari-agent/data/tmp/AMBARI-artifacts/'] {'recursive':
True}
2015-06-26 00:04:57,384 -
File['/var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip']
{'content':
DownloadSource('http://c6401.ambari.apache.org:8080/resources//UnlimitedJCEPolicyJDK7.zip')}
2015-06-26 00:04:57,412 - Not downloading the file from
http://c6401.ambari.apache.org:8080/resources//UnlimitedJCEPolicyJDK7.zip,
because /var/lib/ambari-agent/data/tmp/UnlimitedJCEPolicyJDK7.zip already exists
2015-06-26 00:04:57,436 - Group['jusers'] {'ignore_failures': False}
2015-06-26 00:04:57,439 - Group['jhadoop'] {'ignore_failures': False}
2015-06-26 00:04:57,439 - User['jams'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,440 - User['jhbase'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,440 - User['jyarn'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,442 - User['jhdfs'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,442 - User['jambari-qa'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jusers']}
2015-06-26 00:04:57,442 - User['jzookeeper'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,443 - User['jhive'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,444 - User['jtez'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jusers']}
2015-06-26 00:04:57,444 - User['jhcat'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,444 - User['jmapred'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,446 - User['jkafka'] {'gid': 'jhadoop',
'ignore_failures': False, 'groups': ['jhadoop']}
2015-06-26 00:04:57,447 -
File['/var/lib/ambari-agent/data/tmp/changeUid.sh'] {'content':
StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2015-06-26 00:04:57,527 -
Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh jambari-qa
/tmp/hadoop-jambari-qa,/tmp/hsperfdata_jambari-qa,/home/jambari-qa,/tmp/jambari-qa,/tmp/sqoop-jambari-qa']
{'not_if': '(test $(id -u jambari-qa) -gt 1000) || (false)'}
2015-06-26 00:04:57,535 - Skipping
Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh jambari-qa
/tmp/hadoop-jambari-qa,/tmp/hsperfdata_jambari-qa,/home/jambari-qa,/tmp/jambari-qa,/tmp/sqoop-jambari-qa']
due to not_if
2015-06-26 00:04:57,536 - Directory['/tmp/hbase-hbase'] {'owner': 'jhbase',
'recursive': True, 'mode': 0775, 'cd_access': 'a'}
2015-06-26 00:04:57,627 -
File['/var/lib/ambari-agent/data/tmp/changeUid.sh'] {'content':
StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2015-06-26 00:04:57,708 -
Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh jhbase
/home/jhbase,/tmp/jhbase,/usr/bin/jhbase,/var/log/jhbase,/tmp/hbase-hbase']
{'not_if': '(test $(id -u jhbase) -gt 1000) || (false)'}
2015-06-26 00:04:57,715 - Skipping
Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh jhbase
/home/jhbase,/tmp/jhbase,/usr/bin/jhbase,/var/log/jhbase,/tmp/hbase-hbase'] due
to not_if
2015-06-26 00:04:57,716 - Group['jhdfs'] {'ignore_failures': False}
2015-06-26 00:04:57,716 - User['jhdfs'] {'ignore_failures': False,
'groups': ['jhadoop', 'jhdfs']}
2015-06-26 00:04:57,717 - Directory['/etc/hadoop'] {'mode': 0755}
2015-06-26 00:04:57,783 -
File['/usr/hdp/2.3.0.0-2474/hadoop/conf/hadoop-env.sh'] {'content':
InlineTemplate(...), 'owner': 'jhdfs', 'group': 'jhadoop'}
2015-06-26 00:04:57,843 - Writing
File['/usr/hdp/2.3.0.0-2474/hadoop/conf/hadoop-env.sh'] because contents don't
match
2015-06-26 00:04:57,892 - Execute['('setenforce', '0')'] {'not_if': '(!
which getenforce ) || (which getenforce && getenforce | grep -q Disabled)',
'sudo': True, 'only_if': 'test -f /selinux/enforce'}
2015-06-26 00:04:57,904 - Skipping Execute['('setenforce', '0')'] due to
not_if
2015-06-26 00:04:57,905 - Directory['/var/log/hadoop'] {'owner': 'root',
'mode': 0775, 'group': 'jhadoop', 'recursive': True, 'cd_access': 'a'}
2015-06-26 00:04:58,034 - Directory['/var/run/hadoop'] {'owner': 'root',
'group': 'root', 'recursive': True, 'cd_access': 'a'}
2015-06-26 00:04:58,136 - Directory['/tmp/hadoop-jhdfs'] {'owner': 'jhdfs',
'recursive': True, 'cd_access': 'a'}
2015-06-26 00:04:58,256 -
File['/usr/hdp/2.3.0.0-2474/hadoop/conf/commons-logging.properties']
{'content': Template('commons-logging.properties.j2'), 'owner': 'jhdfs'}
2015-06-26 00:04:58,318 -
File['/usr/hdp/2.3.0.0-2474/hadoop/conf/health_check'] {'content':
Template('health_check.j2'), 'owner': 'jhdfs'}
2015-06-26 00:04:58,383 -
File['/usr/hdp/2.3.0.0-2474/hadoop/conf/log4j.properties'] {'content': '...',
'owner': 'jhdfs', 'group': 'jhadoop', 'mode': 0644}
2015-06-26 00:04:58,467 -
File['/usr/hdp/2.3.0.0-2474/hadoop/conf/hadoop-metrics2.properties']
{'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'jhdfs'}
2015-06-26 00:04:58,535 -
File['/usr/hdp/2.3.0.0-2474/hadoop/conf/task-log4j.properties'] {'content':
StaticFile('task-log4j.properties'), 'mode': 0755}
2015-06-26 00:04:58,618 -
File['/usr/hdp/2.3.0.0-2474/hadoop/conf/configuration.xsl'] {'owner': 'jhdfs',
'group': 'jhadoop'}
2015-06-26 00:04:58,930 - Execute['source
/usr/hdp/current/zookeeper-server/conf/zookeeper-env.sh ; env
ZOOCFGDIR=/usr/hdp/current/zookeeper-server/conf ZOOCFG=zoo.cfg
/usr/hdp/current/zookeeper-server/bin/zkServer.sh stop'] {'user': 'jzookeeper'}
2015-06-26 00:04:59,015 - Execute['rm -f
/var/run/zookeeper/zookeeper_server.pid'] {}
2015-06-26 00:04:59,028 - Executing Rolling Upgrade pre-restart
2015-06-26 00:04:59,061 - call['conf-select set-conf-dir --package
zookeeper --stack-version 2.3.0.0-2474 --conf-version 0'] {'logoutput': False,
'quiet': False}
2015-06-26 00:04:59,093 - call returned (1,
'/etc/zookeeper/2.3.0.0-2474/0does not exist')
2015-06-26 00:04:59,093 - Execute['hdp-select set zookeeper-server
2.3.0.0-2474'] {}
Defaults exempt_group = ambari
Defaults !env_reset,env_delete-=PATH
Defaults: ambari !requiretty
# Ambari Customizable Users
ambari ALL=(ALL) NOPASSWD:SETENV: /bin/su jhdfs *, /bin/su jzookeeper *,
/bin/su jknox *,/bin/su jfalcon *,/bin/su jflume *,/bin/su jhbase *,/bin/su
jhive *, /bin/su jhcat *,/bin/su jkafka *,/bin/su jmapred *,/bin/su joozie
*,/bin/su jsqoop *,/bin/su jstorm *,/bin/su jtez *,/bin/su jyarn *,/bin/su jams
*, /bin/su jambari-qa *, /bin/su jspark *, /bin/su jranger *
# Ambari Non-Customizable Users
ambari ALL=(ALL) NOPASSWD:SETENV: /bin/su mysql *
# Ambari Commands
ambari ALL=(ALL) NOPASSWD:SETENV:
/usr/bin/yum,/usr/bin/zypper,/usr/bin/apt-get, /bin/mkdir, /bin/ln,/bin/chown,
/bin/chmod, /bin/chgrp, /usr/sbin/groupadd,
/usr/sbin/groupmod,/usr/sbin/useradd, /usr/sbin/usermod, /bin/cp, /bin/sed,
/bin/mv, /bin/rm, /bin/kill,/usr/bin/unzip, /bin/tar, /usr/bin/hdp-select,
/usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh,/usr/lib/hadoop/bin/hadoop-daemon.sh,
/usr/lib/hadoop/sbin/hadoop-daemon.sh, /usr/sbin/service mysql *,/sbin/service
mysqld *, /sbin/service mysql *, /sbin/chkconfig gmond off,/sbin/chkconfig
gmetad off, /etc/init.d/httpd *, /sbin/service hdp-gmetad start, /sbin/service
hdp-gmond start, /usr/bin/tee, /usr/sbin/gmond, /usr/sbin/update-rc.d
ganglia-monitor *, /usr/sbin/update-rc.d gmetad *, /etc/init.d/apache2 *,
/usr/sbin/service hdp-gmond *, /usr/sbin/service hdpgmetad *, /usr/bin/test,
/bin/touch, /usr/bin/stat, /usr/sbin/setenforce,
/usr/hdp/current/ranger-admin/setup.sh,
/usr/hdp/current/ranger-usersync/setup.sh, /usr/bin/
ranger-usersync-start, /usr/bin/ranger-usersync-stop
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/hdp_select.py
f3b9b38
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode_upgrade.py
59fef7d
ambari-server/src/test/python/stacks/2.0.6/FLUME/test_flume.py 9a53af4
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_client.py aaeaf2d
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py cb4fd44
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_regionserver.py
11247e7
ambari-server/src/test/python/stacks/2.0.6/HBASE/test_phoenix_queryserver.py
c0a1686
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 7ee2126
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py 0f5b956
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_journalnode.py ba09c73
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py efe9881
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_client.py 7e1fcca
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py 4ae750f
ambari-server/src/test/python/stacks/2.0.6/HIVE/test_webhcat_server.py
c2ac340
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_client.py 1312239
ambari-server/src/test/python/stacks/2.0.6/OOZIE/test_oozie_server.py ce0f52e
ambari-server/src/test/python/stacks/2.0.6/PIG/test_pig_client.py ff6cc34
ambari-server/src/test/python/stacks/2.0.6/SQOOP/test_sqoop.py 4a17d63
ambari-server/src/test/python/stacks/2.0.6/YARN/test_historyserver.py 33f4956
ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py
d0cf44c
ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 7be252e
ambari-server/src/test/python/stacks/2.0.6/YARN/test_resourcemanager.py
fbde404
ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py b69be78
ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_client.py
c8736da
ambari-server/src/test/python/stacks/2.0.6/ZOOKEEPER/test_zookeeper_server.py
dc2d062
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_client.py abff777
ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 521f4a6
ambari-server/src/test/python/stacks/2.1/HIVE/test_hive_metastore.py 872372d
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_drpc_server.py
d2af9ac
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus.py aca528d
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_nimbus_prod.py
85a4a2c
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_rest_api_service.py
5c36377
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor.py
cc40b7b
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_supervisor_prod.py
1909078
ambari-server/src/test/python/stacks/2.1/STORM/test_storm_ui_server.py
dbe9a46
ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py 966ea43
ambari-server/src/test/python/stacks/2.1/YARN/test_apptimelineserver.py
037823c
ambari-server/src/test/python/stacks/2.2/ACCUMULO/test_accumulo_client.py
18c71e3
ambari-server/src/test/python/stacks/2.2/KAFKA/test_kafka_broker.py 04e904a
ambari-server/src/test/python/stacks/2.2/KNOX/test_knox_gateway.py 363a245
ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_admin.py 5413342
ambari-server/src/test/python/stacks/2.2/RANGER/test_ranger_usersync.py
48358cc
ambari-server/src/test/python/stacks/2.2/SLIDER/test_slider_client.py af876b9
ambari-server/src/test/python/stacks/2.2/SPARK/test_job_history_server.py
db83824
ambari-server/src/test/python/stacks/2.2/SPARK/test_spark_client.py d27aec2
ambari-server/src/test/python/stacks/2.3/MAHOUT/test_mahout_client.py 6b61b1a
Diff: https://reviews.apache.org/r/35988/diff/
Testing
-------
mvn clean test
Thanks,
Andrew Onischuk
