[
https://issues.apache.org/jira/browse/AMBARI-22492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Tucker updated AMBARI-22492:
----------------------------------
Description:
When deploying a cluster with Hive (1.2.1000, in this case), Ambari issues an
incorrect WebHDFS request:
{code}
2017-11-20 20:21:21,839 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl
-sS -L -w '"'"'%{http_code}'"'"' -X GET
'"'"'http://dtucker-rjy277d.west.isilon.com:8082/webhdfs/v1?op=GETFILESTATUS&user.name=hdfs'"'"'
1>/tmp/tmpzipnFe 2>/tmp/tmpbGajx_''] {'logoutput': None, 'quiet': False}
{code}
That command should be:
{code}
curl -sS -L -w '%{http_code}' -X GET
'http://dtucker-rjy277d.west.isilon.com:8082/webhdfs/v1/?op=GETFILESTATUS&user.name=hdfs'
{code}
(Note the / after v1.)
On Apache, this results in a FileNotFoundException (404); however, this causes
problems with OneFS which returns an IllegalArgumentException (400) and
prevents the Metastore from starting.
Here is the full traceback:
{code}
Traceback (most recent call last):
File
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 203, in <module>
HiveMetastore().execute()
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 367, in execute
method(env)
File
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 54, in start
self.configure(env)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 120, in locking_configure
original_configure(obj, *args, **kw)
File
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
line 72, in configure
hive(name = 'metastore')
File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line
89, in thunk
return fn(*args, **kwargs)
File
"/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py",
line 343, in hive
mode = 01777)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line
166, in __init__
self.env.run()
File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 160, in run
self.run_action(resource, action)
File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 124, in run_action
provider_action()
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 604, in action_create_on_execute
self.action_delayed("create")
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 601, in action_delayed
self.get_hdfs_resource_executor().action_delayed(action_name, self)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 328, in action_delayed
self._assert_valid()
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 287, in _assert_valid
self.target_status = self._get_file_status(target)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 430, in _get_file_status
list_status = self.util.run_command(target, 'GETFILESTATUS', method='GET',
ignore_status_codes=['404'], assertable_result=False)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 177, in run_command
return self._run_command(*args, **kwargs)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 248, in _run_command
raise WebHDFSCallException(err_msg, result_dict)
{code}
was:
When deploying a cluster with Hive (1.2.1000, in this case), Ambari issues an
incorrect WebHDFS request:
{code}
2017-11-20 20:21:21,839 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl
-sS -L -w '"'"'%{http_code}'"'"' -X GET
'"'"'http://dtucker-rjy277d.west.isilon.com:8082/webhdfs/v1?op=GETFILESTATUS&user.name=hdfs'"'"'
1>/tmp/tmpzipnFe 2>/tmp/tmpbGajx_''] {'logoutput': None, 'quiet': False}
{code}
That command should be:
{code}
curl -sS -L -w '%{http_code}' -X GET
'http://dtucker-rjy277d.west.isilon.com:8082/webhdfs/v1/?op=GETFILESTATUS&user.name=hdfs'
{code}
(Note the / after v1.)
On Apache, this results in a FileNotFoundException (404); however, this causes
problems with OneFS which returns an IllegalArgumentException (400) and
prevents the Metastore from starting.
> A bad WebHDFS request is issued when starting Hive Metastore
> ------------------------------------------------------------
>
> Key: AMBARI-22492
> URL: https://issues.apache.org/jira/browse/AMBARI-22492
> Project: Ambari
> Issue Type: Bug
> Affects Versions: 2.6.0
> Reporter: David Tucker
>
> When deploying a cluster with Hive (1.2.1000, in this case), Ambari issues an
> incorrect WebHDFS request:
> {code}
> 2017-11-20 20:21:21,839 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c
> 'curl -sS -L -w '"'"'%{http_code}'"'"' -X GET
> '"'"'http://dtucker-rjy277d.west.isilon.com:8082/webhdfs/v1?op=GETFILESTATUS&user.name=hdfs'"'"'
> 1>/tmp/tmpzipnFe 2>/tmp/tmpbGajx_''] {'logoutput': None, 'quiet': False}
> {code}
> That command should be:
> {code}
> curl -sS -L -w '%{http_code}' -X GET
> 'http://dtucker-rjy277d.west.isilon.com:8082/webhdfs/v1/?op=GETFILESTATUS&user.name=hdfs'
> {code}
> (Note the / after v1.)
> On Apache, this results in a FileNotFoundException (404); however, this
> causes problems with OneFS which returns an IllegalArgumentException (400)
> and prevents the Metastore from starting.
> Here is the full traceback:
> {code}
> Traceback (most recent call last):
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
> line 203, in <module>
> HiveMetastore().execute()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 367, in execute
> method(env)
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
> line 54, in start
> self.configure(env)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
> line 120, in locking_configure
> original_configure(obj, *args, **kw)
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive_metastore.py",
> line 72, in configure
> hive(name = 'metastore')
> File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py",
> line 89, in thunk
> return fn(*args, **kwargs)
> File
> "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py",
> line 343, in hive
> mode = 01777)
> File "/usr/lib/python2.6/site-packages/resource_management/core/base.py",
> line 166, in __init__
> self.env.run()
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 160, in run
> self.run_action(resource, action)
> File
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
> line 124, in run_action
> provider_action()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
> line 604, in action_create_on_execute
> self.action_delayed("create")
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
> line 601, in action_delayed
> self.get_hdfs_resource_executor().action_delayed(action_name, self)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
> line 328, in action_delayed
> self._assert_valid()
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
> line 287, in _assert_valid
> self.target_status = self._get_file_status(target)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
> line 430, in _get_file_status
> list_status = self.util.run_command(target, 'GETFILESTATUS', method='GET',
> ignore_status_codes=['404'], assertable_result=False)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
> line 177, in run_command
> return self._run_command(*args, **kwargs)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
> line 248, in _run_command
> raise WebHDFSCallException(err_msg, result_dict)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
