Thomas Marshall has uploaded this change for review. (
http://gerrit.cloudera.org:8080/13299
Change subject: [WIP] HS2 + HTTP(S) + BASIC/LDAP based thrift server endpoint
......................................................................
[WIP] HS2 + HTTP(S) + BASIC/LDAP based thrift server endpoint
This patch provides an option to use HTTP based transport for
HiveServer2 endpoint on coordinators that the clients can connect
to query. HTTP(S) also works when external TLS is enabled using
--ssl_server_certificate.
Implemented only for HS2 compatible thrift server since, unlike
beeswax, its session management does not need to be tied to the
underlying TCP conneciton.
Thirft's http transport is modified to support BASIC authentication
via ldap. For convenience of developing and reviewing, this patch
is based on another that copied THttpServer and THttpTransport into
Impala's codebase. Before this patch is committed, the intention is
to submit the changes to those files that are shown in this review as
a patch on Impala's native-toolchain Thrift.
TODO
=====
- IMPALA-1653: Don't close hiveserver2 session when the connection is
closed.
- Audit code to see if it can handle DOS kinda cases where the client
sends a huge payload that can potentially crash the server.
- Write more tests (be unittests for thrift-server and e2e tests)
- Implement SPNEGO for native Kerberos auth (in a followup patch).
Testing
=======
- Parameterized JdbcTest to work for HS2 + HTTP mode (no TLS).
Manual testing with Beeline client (from Apache Hive), which has
builtin support to connect to HTTP(S) based HS2 compatible endpoints.
Example
========
-- HTTP mode:
> start-impala-cluster.py
> JDBC_URL="jdbc:hive2://localhost:<port>/default;transportMode=http"
> beeline -u "$JDBC_URL"
-- HTTPS mode:
> cd $IMPALA_HOME
> SSL_ARGS="--ssl_client_ca_certificate=./be/src/testutil/server-cert.pem \
--ssl_server_certificate=./be/src/testutil/server-cert.pem \
--ssl_private_key=./be/src/testutil/server-key.pem --hostname=localhost"
> start-impala-cluster.py --impalad_args="$SSL_ARGS" \
--catalogd_args="$SSL_ARGS" --state_store_args="$SSL_ARGS"
- Create a local trust store using 'keytool' and import the certificate
from server-cert.pem (./clientkeystore in the example).
> JDBC_URL="jdbc:hive2://localhost:<port>/default;ssl=true;sslTrustStore= \
./clientkeystore;trustStorePassword=password;transportMode=http"
> beeline -u "$JDBC_URL"
-- BASIC Auth with LDAP:
> LDAP_ARGS="--enable_ldap_auth --ldap_uri='ldap://qa-openldap.ad.cloudera.com'
> \
--ldap_bind_pattern='cn=#UID,dc=ad,dc=cloudera,dc=com'
--ldap_passwords_in_clear_ok"
> start-impala-cluster.py --impalad_args="$LDAP_ARGS"
> JDBC_URL="jdbc:hive2://localhost:28000/default;user=admin;password=\
cloudera;transportMode=http"
> beeline -u "$JDBC_URL"
-- HTTPS mode with LDAP:
> start-impala-cluster.py --impalad_args="$LDAP_ARGS $SSL_ARGS" \
--catalogd_args="$SSL_ARGS" --state_store_args="$SSL_ARGS"
> JDBC_URL="jdbc:hive2://localhost:28000/default;user=admin;password=\
cloudera;ssl=true;sslTrustStore=./clientkeystore;trustStorePassword=\
password;transportMode=http"
> beeline -u "$JDBC_URL"
Change-Id: Ic5569ac62ef3af2868b5d0581f5029dac736b2ff
---
M be/src/rpc/auth-provider.h
M be/src/rpc/authentication-test.cc
M be/src/rpc/authentication.cc
M be/src/rpc/thrift-server.cc
M be/src/rpc/thrift-server.h
M be/src/service/impala-server.cc
M be/src/service/impala-server.h
M be/src/service/impalad-main.cc
M be/src/testutil/in-process-servers.cc
M be/src/testutil/in-process-servers.h
M be/src/transport/THttpServer.cpp
M be/src/transport/THttpServer.h
M be/src/transport/THttpTransport.cpp
M be/src/transport/THttpTransport.h
M bin/start-impala-cluster.py
M fe/src/test/java/org/apache/impala/service/JdbcTest.java
M fe/src/test/java/org/apache/impala/testutil/ImpalaJdbcClient.java
M tests/common/impala_cluster.py
18 files changed, 353 insertions(+), 105 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/99/13299/1
--
To view, visit http://gerrit.cloudera.org:8080/13299
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic5569ac62ef3af2868b5d0581f5029dac736b2ff
Gerrit-Change-Number: 13299
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall <[email protected]>
Gerrit-Reviewer: Bharath Vissapragada <[email protected]>
