-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57902/
-----------------------------------------------------------
Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
Bugs: AMBARI-20553
https://issues.apache.org/jira/browse/AMBARI-20553
Repository: ambari
Description
-------
Metrics collector startup scripts fail to handle s3 paths in configurations:
{
"hbase-site":{
"properties":{
"hbase.rootdir":"s3a://ss-datasets/apps/hbase/",
"hbase.wal.dir":"file:///usr/lib/ams-hbase/data"
}
}
Exception Trace
Traceback (most recent call last):
File
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
line 150, in <module>
AmsCollector().execute()
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 313, in execute
method(env)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 766, in restart
self.start(env)
File
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
line 48, in start
self.configure(env, action = 'start') # for security
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
line 116, in locking_configure
original_configure(obj, *args, **kw)
File
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py",
line 43, in configure
hbase('master', action)
File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py",
line 89, in thunk
return fn(*args, **kwargs)
File
"/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase.py",
line 213, in hbase
dfs_type=params.dfs_type
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py",
line 155, in __init__
self.env.run()
File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 160, in run
self.run_action(resource, action)
File
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py",
line 124, in run_action
provider_action()
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 555, in action_create_on_execute
self.action_delayed("create")
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 552, in action_delayed
self.get_hdfs_resource_executor().action_delayed(action_name, self)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 279, in action_delayed
self._assert_valid()
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 238, in _assert_valid
self.target_status = self._get_file_status(target)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 381, in _get_file_status
list_status = self.util.run_command(target, 'GETFILESTATUS', method='GET',
ignore_status_codes=['404'], assertable_result=False)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/providers/hdfs_resource.py",
line 186, in run_command
_, out, err = get_user_call_output(cmd, user=self.run_user,
logoutput=self.logoutput, quiet=False)
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/get_user_call_output.py",
line 61, in get_user_call_output
raise ExecutionFailed(err_msg, code, files_output[0], files_output[1])
resource_management.core.exceptions.ExecutionFailed: Execution of 'curl -sS -L
-w '%{http_code}' -X GET
'http://<host>:50070/webhdfs/v1s3a:/ss-datasets/apps/hbase?op=GETFILESTATUS&user.name=hdfs'
1>/tmp/tmpjkn3uB 2>/tmp/tmpCVb8Kl' returned 7. curl: (7) Failed to connect to
<host> port 50070: Connection refused
000
FIX
1. Made HBase rootdir creation on remote file system optional.
2. Added ability to specify wal dir for AMS HBase.
3. Added ability to specify additional classpath artifacts for Metrics
Collector.
Diffs
-----
ambari-metrics/ambari-metrics-timelineservice/conf/unix/ambari-metrics-collector
c106235
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-env.xml
81fa4c7
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/hbase.py
26b3502
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/params.py
3276cc1
Diff: https://reviews.apache.org/r/57902/diff/1/
Testing
-------
Manually tested.
Python unit tests pass.
Thanks,
Aravindan Vijayan
