JaySenSharma created AMBARI-25328:
-------------------------------------
Summary: When spark_transport_mode is set to 'http' then STS
server fails to start in Ambari 2.7
Key: AMBARI-25328
URL: https://issues.apache.org/jira/browse/AMBARI-25328
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.7.3
Reporter: JaySenSharma
When the "hive.server2.transport.mode" is set tp "http" (instead of "binary")
inside the "spark2-hive-site-override" configuration then the "Spark2 Thrift
Server" start operation fails via Ambari UI with the following error:
{code}
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150,
in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314,
in _call
raise ExecutionFailed(err_msg, code, out, err)
ExecutionFailed: Execution of '!
/usr/hdp/current/spark2-thriftserver/bin/beeline -u
'jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http' -e ''
2>&1| awk '{print}'|grep -i -e 'Connection refused' -e 'Invalid URL' -e 'Error:
Could not open'' returned 1. Error: Could not open client transport with JDBC
Uri: jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http:
Could not create http connection to
jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http. HTTP
Response code: 405 (state=08S01,code=0)
Error: Could not open client transport with JDBC Uri:
jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http: Could not
create http connection to
jdbc:hive2://kerlatest4.example.com:10002/default;transportMode=http. HTTP
Response code: 405 (state=08S01,code=0)
{code}
This is because the "spark_service.py" script is not appending the
"httpPath=cliservice" when the "hive.server2.transport.mode" is set to "http"
in the "beeline_url" inside the
"/var/lib/ambari-server/resources/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_service.py"
script (On Ambari Agent
"/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_service.py").
*Manual Workaround:*
Update the
"/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SPARK2/package/scripts/spark_service.py"
on agent and amabri server as following with additional If else block to check
the spark_transport_mode == 'http':
{code}
146c146,149
< beeline_url =
["jdbc:hive2://{fqdn}:{spark_thrift_port}/default;principal={hive_kerberos_principal}","transportMode={spark_transport_mode}"]
---
> if params.spark_transport_mode == 'http':
> beeline_url =
> ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default;principal={hive_kerberos_principal}","transportMode={spark_transport_mode}","httpPath=cliservice"]
> else:
> beeline_url =
> ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default;principal={hive_kerberos_principal}","transportMode={spark_transport_mode}"]
148c151,154
< beeline_url =
["jdbc:hive2://{fqdn}:{spark_thrift_port}/default","transportMode={spark_transport_mode}"]
---
> if params.spark_transport_mode == 'http':
> beeline_url =
> ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default","transportMode={spark_transport_mode}","httpPath=cliservice"]
> else:
> beeline_url =
> ["jdbc:hive2://{fqdn}:{spark_thrift_port}/default","transportMode={spark_transport_mode}"]
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
