Saurabh Kumar created AMBARI-13255:
--------------------------------------
Summary: hdfs dfsadmin -rolledits timeout after 1200 secs
Key: AMBARI-13255
URL: https://issues.apache.org/jira/browse/AMBARI-13255
Project: Ambari
Issue Type: Bug
Components: ambari-admin
Affects Versions: 2.1.0
Reporter: Saurabh Kumar
2015-09-24 08:57:55,820 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:55,855 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:55,889 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:55,923 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:55,959 - checked_call['conf-select set-conf-dir --package
hadoop --stack-version 2.3.0.0-2557 --conf-version 0'] {'logoutput': False,
'sudo': True, 'quiet': False}
2015-09-24 08:57:55,994 - checked_call returned (0,
'/usr/hdp/2.3.0.0-2557/hadoop/conf -> /etc/hadoop/2.3.0.0-2557/0')
2015-09-24 08:57:56,158 - checked_call['conf-select set-conf-dir --package
hadoop --stack-version 2.3.0.0-2557 --conf-version 0'] {'logoutput': False,
'sudo': True, 'quiet': False}
2015-09-24 08:57:56,193 - checked_call returned (0,
'/usr/hdp/2.3.0.0-2557/hadoop/conf -> /etc/hadoop/2.3.0.0-2557/0')
2015-09-24 08:57:56,196 -
Directory['/var/lib/ambari-agent/data/tmp/AMBARI-artifacts/'] {'recursive':
True}
2015-09-24 08:57:56,198 -
File['/var/lib/ambari-agent/data/tmp/AMBARI-artifacts//UnlimitedJCEPolicyJDK7.zip']
{'content':
DownloadSource('http://eu-lamp-pre-prod-m-0043-hadoop-ambari:8080/resources//UnlimitedJCEPolicyJDK7.zip')}
2015-09-24 08:57:56,199 - Not downloading the file from
http://eu-lamp-pre-prod-m-0043-hadoop-ambari:8080/resources//UnlimitedJCEPolicyJDK7.zip,
because /var/lib/ambari-agent/data/tmp/UnlimitedJCEPolicyJDK7.zip already
exists
2015-09-24 08:57:56,199 - Group['hadoop'] {'ignore_failures': False}
2015-09-24 08:57:56,200 - Group['users'] {'ignore_failures': False}
2015-09-24 08:57:56,200 - Group['ranger'] {'ignore_failures': False}
2015-09-24 08:57:56,200 - Group['spark'] {'ignore_failures': False}
2015-09-24 08:57:56,201 - User['hive'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['hadoop']}
2015-09-24 08:57:56,202 - User['oozie'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['users']}
2015-09-24 08:57:56,202 - User['ambari-qa'] {'gid': 'hadoop',
'ignore_failures': False, 'groups': ['users']}
2015-09-24 08:57:56,203 - User['hdfs'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['hadoop']}
2015-09-24 08:57:56,204 - User['ranger'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['ranger']}
2015-09-24 08:57:56,205 - User['spark'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['hadoop']}
2015-09-24 08:57:56,206 - User['mapred'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['hadoop']}
2015-09-24 08:57:56,207 - User['hbase'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['hadoop']}
2015-09-24 08:57:56,207 - User['tez'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['users']}
2015-09-24 08:57:56,208 - User['zookeeper'] {'gid': 'hadoop',
'ignore_failures': False, 'groups': ['hadoop']}
2015-09-24 08:57:56,209 - User['yarn'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['hadoop']}
2015-09-24 08:57:56,210 - User['hcat'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['hadoop']}
2015-09-24 08:57:56,210 - User['ams'] {'gid': 'hadoop', 'ignore_failures':
False, 'groups': ['hadoop']}
2015-09-24 08:57:56,211 - File['/var/lib/ambari-agent/data/tmp/changeUid.sh']
{'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2015-09-24 08:57:56,212 - Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh
ambari-qa
/tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']
{'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2015-09-24 08:57:56,224 - Skipping
Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh ambari-qa
/tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa']
due to not_if
2015-09-24 08:57:56,225 - Directory['/tmp/hbase-hbase'] {'owner': 'hbase',
'recursive': True, 'mode': 0775, 'cd_access': 'a'}
2015-09-24 08:57:56,226 - File['/var/lib/ambari-agent/data/tmp/changeUid.sh']
{'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2015-09-24 08:57:56,227 - Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh
hbase /home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase']
{'not_if': '(test $(id -u hbase) -gt 1000) || (false)'}
2015-09-24 08:57:56,239 - Skipping
Execute['/var/lib/ambari-agent/data/tmp/changeUid.sh hbase
/home/hbase,/tmp/hbase,/usr/bin/hbase,/var/log/hbase,/tmp/hbase-hbase'] due to
not_if
2015-09-24 08:57:56,240 - Group['hdfs'] {'ignore_failures': False}
2015-09-24 08:57:56,240 - User['hdfs'] {'ignore_failures': False, 'groups':
['hadoop', 'hdfs']}
2015-09-24 08:57:56,241 - Directory['/etc/hadoop'] {'mode': 0755}
2015-09-24 08:57:56,254 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/hadoop-env.sh'] {'content':
InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2015-09-24 08:57:56,271 - Execute[('setenforce', '0')] {'not_if': '(! which
getenforce ) || (which getenforce && getenforce | grep -q Disabled)', 'sudo':
True, 'only_if': 'test -f /selinux/enforce'}
2015-09-24 08:57:56,286 - Skipping Execute[('setenforce', '0')] due to not_if
2015-09-24 08:57:56,287 - Directory['/var/log/hadoop'] {'owner': 'root',
'mode': 0775, 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2015-09-24 08:57:56,290 - Directory['/var/run/hadoop'] {'owner': 'root',
'group': 'root', 'recursive': True, 'cd_access': 'a'}
2015-09-24 08:57:56,291 - Directory['/tmp/hadoop-hdfs'] {'owner': 'hdfs',
'recursive': True, 'cd_access': 'a'}
2015-09-24 08:57:56,296 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/commons-logging.properties']
{'content': Template('commons-logging.properties.j2'), 'owner': 'hdfs'}
2015-09-24 08:57:56,298 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/health_check'] {'content':
Template('health_check.j2'), 'owner': 'hdfs'}
2015-09-24 08:57:56,299 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/log4j.properties'] {'content': ...,
'owner': 'hdfs', 'group': 'hadoop', 'mode': 0644}
2015-09-24 08:57:56,309 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/hadoop-metrics2.properties']
{'content': Template('hadoop-metrics2.properties.j2'), 'owner': 'hdfs'}
2015-09-24 08:57:56,310 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/task-log4j.properties'] {'content':
StaticFile('task-log4j.properties'), 'mode': 0755}
2015-09-24 08:57:56,315 - File['/etc/hadoop/conf/topology_mappings.data']
{'owner': 'hdfs', 'content': Template('topology_mappings.data.j2'), 'only_if':
'test -d /etc/hadoop/conf', 'group': 'hadoop'}
2015-09-24 08:57:56,325 - File['/etc/hadoop/conf/topology_script.py']
{'content': StaticFile('topology_script.py'), 'only_if': 'test -d
/etc/hadoop/conf', 'mode': 0755}
2015-09-24 08:57:56,744 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:56,778 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:56,812 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:56,847 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:56,882 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:56,917 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:56,953 - hadoop-hdfs-journalnode is currently at version
2.2.4.2-2
2015-09-24 08:57:56,962 -
File['/var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid'] {'action': ['delete'],
'not_if': 'ambari-sudo.sh -H -E test -f
/var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid && ambari-sudo.sh -H -E pgrep
-F /var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid'}
2015-09-24 08:57:56,981 - Skipping
File['/var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid'] due to not_if
2015-09-24 08:57:56,982 - Execute['ambari-sudo.sh su hdfs -l -s /bin/bash -c
'ulimit -c unlimited ; /usr/hdp/2.2.4.2-2/hadoop/sbin/hadoop-daemon.sh
--config /usr/hdp/2.2.4.2-2/hadoop/conf stop journalnode''] {'environment':
{'HADOOP_LIBEXEC_DIR': '/usr/hdp/2.2.4.2-2/hadoop/libexec'}, 'not_if': None}
2015-09-24 08:58:02,120 -
File['/var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid'] {'action': ['delete']}
2015-09-24 08:58:02,121 - Executing Rolling Upgrade pre-restart
2015-09-24 08:58:02,159 - checked_call['conf-select set-conf-dir --package
hadoop --stack-version 2.3.0.0-2557 --conf-version 0'] {'logoutput': False,
'sudo': True, 'quiet': False}
2015-09-24 08:58:02,196 - checked_call returned (0,
'/usr/hdp/2.3.0.0-2557/hadoop/conf -> /etc/hadoop/2.3.0.0-2557/0')
2015-09-24 08:58:02,197 - Execute[('hdp-select', 'set',
'hadoop-hdfs-journalnode', '2.3.0.0-2557')] {'sudo': True}
2015-09-24 08:58:02,269 - hadoop-hdfs-journalnode is currently at version
2.3.0.0-2557
2015-09-24 08:58:02,305 - checked_call['conf-select set-conf-dir --package
hadoop --stack-version 2.3.0.0-2557 --conf-version 0'] {'logoutput': False,
'sudo': True, 'quiet': False}
2015-09-24 08:58:02,343 - checked_call returned (0,
'/usr/hdp/2.3.0.0-2557/hadoop/conf -> /etc/hadoop/2.3.0.0-2557/0')
2015-09-24 08:58:02,344 - After hdp-select hadoop-hdfs-journalnode, reloaded
module status_params
2015-09-24 08:58:02,380 - hadoop-hdfs-journalnode is currently at version
2.3.0.0-2557
2015-09-24 08:58:02,415 - hadoop-hdfs-journalnode is currently at version
2.3.0.0-2557
2015-09-24 08:58:02,450 - hadoop-hdfs-journalnode is currently at version
2.3.0.0-2557
2015-09-24 08:58:02,484 - hadoop-hdfs-journalnode is currently at version
2.3.0.0-2557
2015-09-24 08:58:02,518 - hadoop-hdfs-journalnode is currently at version
2.3.0.0-2557
2015-09-24 08:58:02,554 - checked_call['conf-select set-conf-dir --package
hadoop --stack-version 2.3.0.0-2557 --conf-version 0'] {'logoutput': False,
'sudo': True, 'quiet': False}
2015-09-24 08:58:02,589 - checked_call returned (0,
'/usr/hdp/2.3.0.0-2557/hadoop/conf -> /etc/hadoop/2.3.0.0-2557/0')
2015-09-24 08:58:02,625 - hadoop-hdfs-journalnode is currently at version
2.3.0.0-2557
2015-09-24 08:58:02,632 - After hdp-select hadoop-hdfs-journalnode, reloaded
module params_linux
2015-09-24 08:58:02,633 - After hdp-select hadoop-hdfs-journalnode, reloaded
module params
2015-09-24 08:58:02,635 - Directory['/hadoop/hdfs/journal'] {'owner': 'hdfs',
'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2015-09-24 08:58:02,638 - Directory['/etc/security/limits.d'] {'owner': 'root',
'group': 'root', 'recursive': True}
2015-09-24 08:58:02,644 - File['/etc/security/limits.d/hdfs.conf'] {'content':
Template('hdfs.conf.j2'), 'owner': 'root', 'group': 'root', 'mode': 0644}
2015-09-24 08:58:02,645 - XmlConfig['hadoop-policy.xml'] {'owner': 'hdfs',
'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.0.0-2557/hadoop/conf',
'configuration_attributes': {}, 'configurations': ...}
2015-09-24 08:58:02,657 - Generating config:
/usr/hdp/2.3.0.0-2557/hadoop/conf/hadoop-policy.xml
2015-09-24 08:58:02,658 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/hadoop-policy.xml'] {'owner': 'hdfs',
'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding':
'UTF-8'}
2015-09-24 08:58:02,667 - Writing
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/hadoop-policy.xml'] because contents
don't match
2015-09-24 08:58:02,668 - XmlConfig['ssl-client.xml'] {'owner': 'hdfs',
'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.0.0-2557/hadoop/conf',
'configuration_attributes': {}, 'configurations': ...}
2015-09-24 08:58:02,678 - Generating config:
/usr/hdp/2.3.0.0-2557/hadoop/conf/ssl-client.xml
2015-09-24 08:58:02,678 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/ssl-client.xml'] {'owner': 'hdfs',
'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding':
'UTF-8'}
2015-09-24 08:58:02,684 - Writing
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/ssl-client.xml'] because contents don't
match
2015-09-24 08:58:02,685 - Directory['/usr/hdp/2.3.0.0-2557/hadoop/conf/secure']
{'owner': 'root', 'group': 'hadoop', 'recursive': True, 'cd_access': 'a'}
2015-09-24 08:58:02,685 - XmlConfig['ssl-client.xml'] {'owner': 'hdfs',
'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.0.0-2557/hadoop/conf/secure',
'configuration_attributes': {}, 'configurations': ...}
2015-09-24 08:58:02,696 - Generating config:
/usr/hdp/2.3.0.0-2557/hadoop/conf/secure/ssl-client.xml
2015-09-24 08:58:02,696 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/secure/ssl-client.xml'] {'owner':
'hdfs', 'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None,
'encoding': 'UTF-8'}
2015-09-24 08:58:02,702 - Writing
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/secure/ssl-client.xml'] because
contents don't match
2015-09-24 08:58:02,703 - XmlConfig['ssl-server.xml'] {'owner': 'hdfs',
'group': 'hadoop', 'conf_dir': '/usr/hdp/2.3.0.0-2557/hadoop/conf',
'configuration_attributes': {}, 'configurations': ...}
2015-09-24 08:58:02,713 - Generating config:
/usr/hdp/2.3.0.0-2557/hadoop/conf/ssl-server.xml
2015-09-24 08:58:02,713 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/ssl-server.xml'] {'owner': 'hdfs',
'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding':
'UTF-8'}
2015-09-24 08:58:02,720 - Writing
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/ssl-server.xml'] because contents don't
match
2015-09-24 08:58:02,721 - XmlConfig['hdfs-site.xml'] {'owner': 'hdfs', 'group':
'hadoop', 'conf_dir': '/usr/hdp/2.3.0.0-2557/hadoop/conf',
'configuration_attributes': {}, 'configurations': ...}
2015-09-24 08:58:02,731 - Generating config:
/usr/hdp/2.3.0.0-2557/hadoop/conf/hdfs-site.xml
2015-09-24 08:58:02,731 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/hdfs-site.xml'] {'owner': 'hdfs',
'content': InlineTemplate(...), 'group': 'hadoop', 'mode': None, 'encoding':
'UTF-8'}
2015-09-24 08:58:02,791 - Writing
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/hdfs-site.xml'] because contents don't
match
2015-09-24 08:58:02,791 - XmlConfig['core-site.xml'] {'group': 'hadoop',
'conf_dir': '/usr/hdp/2.3.0.0-2557/hadoop/conf', 'mode': 0644,
'configuration_attributes': {}, 'owner': 'hdfs', 'configurations': ...}
2015-09-24 08:58:02,801 - Generating config:
/usr/hdp/2.3.0.0-2557/hadoop/conf/core-site.xml
2015-09-24 08:58:02,802 -
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/core-site.xml'] {'owner': 'hdfs',
'content': InlineTemplate(...), 'group': 'hadoop', 'mode': 0644, 'encoding':
'UTF-8'}
2015-09-24 08:58:02,827 - Writing
File['/usr/hdp/2.3.0.0-2557/hadoop/conf/core-site.xml'] because contents don't
match
2015-09-24 08:58:02,829 - File['/usr/hdp/2.3.0.0-2557/hadoop/conf/slaves']
{'content': Template('slaves.j2'), 'owner': 'hdfs'}
2015-09-24 08:58:02,830 - Directory['/var/run/hadoop'] {'owner': 'hdfs',
'group': 'hadoop', 'mode': 0755}
2015-09-24 08:58:02,830 - Changing owner for /var/run/hadoop from 0 to hdfs
2015-09-24 08:58:02,830 - Changing group for /var/run/hadoop from 0 to hadoop
2015-09-24 08:58:02,830 - Directory['/var/run/hadoop/hdfs'] {'owner': 'hdfs',
'recursive': True}
2015-09-24 08:58:02,831 - Directory['/var/log/hadoop/hdfs'] {'owner': 'hdfs',
'recursive': True}
2015-09-24 08:58:02,831 -
File['/var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid'] {'action': ['delete'],
'not_if': 'ambari-sudo.sh -H -E test -f
/var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid && ambari-sudo.sh -H -E pgrep
-F /var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid'}
2015-09-24 08:58:02,844 - Execute['ambari-sudo.sh su hdfs -l -s /bin/bash -c
'ulimit -c unlimited ; /usr/hdp/2.3.0.0-2557/hadoop/sbin/hadoop-daemon.sh
--config /usr/hdp/2.3.0.0-2557/hadoop/conf start journalnode''] {'environment':
{'HADOOP_LIBEXEC_DIR': '/usr/hdp/2.3.0.0-2557/hadoop/libexec'}, 'not_if':
'ambari-sudo.sh -H -E test -f /var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid
&& ambari-sudo.sh -H -E pgrep -F
/var/run/hadoop/hdfs/hadoop-hdfs-journalnode.pid'}
2015-09-24 08:58:07,024 - Executing Rolling Upgrade post-restart
2015-09-24 08:58:07,027 - Ensuring Journalnode quorum is established
2015-09-24 08:58:12,031 - Execute['hdfs dfsadmin -rollEdits'] {'tries': 1,
'user': 'hdfs'}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
