[
https://issues.apache.org/jira/browse/AMBARI-25211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amarnath reddy pappu updated AMBARI-25211:
------------------------------------------
Description:
If transport mode is http then Spark2 Thrift alert does not work. it fails with
following exception.
{noformat}
Connection failed on host c2111-node2.hdp.com:10016 (Traceback (most recent
call last):
File
"/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/alerts/alert_spark2_thrift_port.py",
line 173, in execute
Execute(cmd, user=sparkuser, path=[beeline_cmd],
timeout=CHECK_COMMAND_TIMEOUT_DEFAULT)
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166,
in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py",
line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py",
line 124, in run_action
provider_action()
File
"/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line
263, in action_run
returns=self.resource.returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72,
in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102,
in checked_call
tries=tries, try_sleep=try_sleep,
timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150,
in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314,
in _call
raise ExecutionFailed(err_msg, code, out, err)
ExecutionFailed: Execution of '! /usr/hdp/current/spark2-client/bin/beeline -u
'jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http' -e ''
2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL' -e 'Error:
Could not open'' returned 1. Error: Could not open client transport with JDBC
Uri: jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http: Could
not create http connection to
jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http. HTTP
Response code: 405 (state=08S01,code=0)
Error: Could not open client transport with JDBC Uri:
jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http: Could not
create http connection to
jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http. HTTP
Response code: 405 (state=08S01,code=0)
)
{noformat}
To fix this "httpPath=cliservice" has to be specified in the connection.
was:
With HTTPS enabled for Spark2 Thrift, the Ambari alerts fail with:
{code}
Connection failed on host hdpmasterp01.hwp.int.domain.com:10016 (Traceback
(most recent call last):
File
"/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/alerts/alert_spark2_thrift_port.py",
line 147, in execute
...
ExecutionFailed: Execution of '! /usr/hdp/current/spark2-client/bin/beeline -u
'jdbc:hive2://hdpmasterp01.hwp.int.domain.com:10016/default;principal=spark/[email protected];transportMode=http'
-e '' 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL' -e
'Error: Could not open'' returned 1. Error: Could not open client transport
with JDBC Uri:
jdbc:hive2://hdpmasterp01.hwp.int.domain.com:10016/default;principal=spark/[email protected];transportMode=http:
Could not create http connection to
jdbc:hive2://hdpmasterp01.hwp.int.domain.com:10016/default;principal=spark/[email protected];transportMode=http.
org.apache.http.client.ClientProtocolException (state=08S01,code=0)
Error: Could not open client transport with JDBC Uri:
jdbc:hive2://hdpmasterp01.hwp.int.domain.com:10016/default;principal=spark/[email protected];transportMode=http:
Could not create http connection to
jdbc:hive2://hdpmasterp01.hwp.int.domain.com:10016/default;principal=spark/[email protected];transportMode=http.
org.apache.http.client.ClientProtocolException (state=08S01,code=0)
{code}
When HTTPS is enabled, the alert scripts should add
`;ssl=true;httpPath=cliservice` as I have below:
{code}
$ /usr/hdp/current/spark2-client/bin/beeline -u
'jdbc:hive2://hdpmasterp01.hwp.int.domain.com:10016/default;principal=spark/[email protected];transportMode=http;ssl=true;httpPath=cliservice'
Connecting to
jdbc:hive2://hdpmasterp01.hwp.int.domain.com:10016/default;principal=spark/[email protected];transportMode=http;ssl=true;httpPath=cliservice
...
Connected to: Spark SQL (version 2.3.2.3.1.0.0-78)
Driver: Hive JDBC (version 1.21.2.3.1.0.0-78)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.21.2.3.1.0.0-78 by Apache Hive
{code}
> Spark2 Thrift Server alert does not work if transfort mode is http
> ------------------------------------------------------------------
>
> Key: AMBARI-25211
> URL: https://issues.apache.org/jira/browse/AMBARI-25211
> Project: Ambari
> Issue Type: Bug
> Components: alerts
> Affects Versions: 2.6.0, 2.7.0, 2.7.1, 2.7.3
> Reporter: amarnath reddy pappu
> Assignee: amarnath reddy pappu
> Priority: Major
> Labels: pull-request-available
>
> If transport mode is http then Spark2 Thrift alert does not work. it fails
> with following exception.
> {noformat}
> Connection failed on host c2111-node2.hdp.com:10016 (Traceback (most recent
> call last):
> File
> "/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/alerts/alert_spark2_thrift_port.py",
> line 173, in execute
> Execute(cmd, user=sparkuser, path=[beeline_cmd],
> timeout=CHECK_COMMAND_TIMEOUT_DEFAULT)
> File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line
> 166, in __init__
> self.env.run()
> File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py",
> line 160, in run
> self.run_action(resource, action)
> File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py",
> line 124, in run_action
> provider_action()
> File
> "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py",
> line 263, in action_run
> returns=self.resource.returns)
> File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line
> 72, in inner
> result = function(command, **kwargs)
> File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line
> 102, in checked_call
> tries=tries, try_sleep=try_sleep,
> timeout_kill_strategy=timeout_kill_strategy, returns=returns)
> File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line
> 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
> File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line
> 314, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> ExecutionFailed: Execution of '! /usr/hdp/current/spark2-client/bin/beeline
> -u 'jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http' -e ''
> 2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL' -e
> 'Error: Could not open'' returned 1. Error: Could not open client transport
> with JDBC Uri:
> jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http: Could not
> create http connection to
> jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http. HTTP
> Response code: 405 (state=08S01,code=0)
> Error: Could not open client transport with JDBC Uri:
> jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http: Could not
> create http connection to
> jdbc:hive2://c2111-node2.hdp.com:10016/default;transportMode=http. HTTP
> Response code: 405 (state=08S01,code=0)
> )
> {noformat}
> To fix this "httpPath=cliservice" has to be specified in the connection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
