[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-07-05 Thread Lars Volker (Code Review) 
Lars Volker has abandoned this change. ( http://gerrit.cloudera.org:8080/10764 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Abandoned

Not needed anymore.
--
To view, visit http://gerrit.cloudera.org:8080/10764
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: Icda4173ae0adbc12d167b9918e22b47fd460498c
Gerrit-Change-Number: 10764
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Sailesh Mukil

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-06-19 Thread Lars Volker (Code Review) 
Hello Sailesh Mukil, Impala Public Jenkins,

I'd like you to do a code review. Please visit

http://gerrit.cloudera.org:8080/10764

to review the following change.


Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..

IMPALA-6691: KRPC w/ kerberos fails on SLES11

The Kerberos version used in SLES 11 seems to have quite a few
undocumented bugs.
They have krb5-1.6 (krb5-client-1.6.3-133.49.112.1.x86_64).

With KRPC we see a new error "GSSAPI Error: A required input parameter
could not be read", which we've never seen before.

I looked into the krb5 codebase and between krb5-1.6 and krb5-1.7,
the code causing the above error (GSSAPI Error: A required input
parameter could not be read) has changed subtly without any
explanation as to why.

That error string corresponds to GSS_S_CALL_INACCESSIBLE_READ.

In 1.6, it returns an error if the 'input_token_buffer' string
is empty. krb5-1.6:
https://github.com/krb5/krb5/blob/krb5-1.6/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

In 1.7, it returns an error only if the 'input_token_buffer' string
is NULL. krb5-1.7:
https://github.com/krb5/krb5/blob/krb5-1.7/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

With KRPC, we test if Kerberos works by passing an empty string to SASL:
https://github.com/apache/impala/blob/master/be/src/kudu/rpc/server_negotiation.cc#L289

In 1.6, this is counted as an error, but in 1.7, this is completely
fine. I'm not sure why since they haven't documented it.

We can attempt to get KRPC working for SLES11 by removing the
PreflightGSSAPI() check for any kerberos version < 1.6. A function that
is unavailable on krb-1.6 is krb5_get_init_creds_opt_set_fast_ccache_name(),
and it is available from krb-1.7 onwards. The PreflightCheckGSSAPI() is
compiled in only if this function exists.

(However there may be more issues on SLES11 that we're not yet aware of)

Change-Id: Icda4173ae0adbc12d167b9918e22b47fd460498c
Reviewed-on: http://gerrit.cloudera.org:8080/9696
Reviewed-by: Sailesh Mukil 
Tested-by: Impala Public Jenkins
---
M be/src/kudu/rpc/messenger.cc
1 file changed, 13 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/10764/1
--
To view, visit http://gerrit.cloudera.org:8080/10764
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Icda4173ae0adbc12d167b9918e22b47fd460498c
Gerrit-Change-Number: 10764
Gerrit-PatchSet: 1
Gerrit-Owner: Lars Volker 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Sailesh Mukil

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-21 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..

IMPALA-6691: KRPC w/ kerberos fails on SLES11

The Kerberos version used in SLES 11 seems to have quite a few
undocumented bugs.
They have krb5-1.6 (krb5-client-1.6.3-133.49.112.1.x86_64).

With KRPC we see a new error "GSSAPI Error: A required input parameter
could not be read", which we've never seen before.

I looked into the krb5 codebase and between krb5-1.6 and krb5-1.7,
the code causing the above error (GSSAPI Error: A required input
parameter could not be read) has changed subtly without any
explanation as to why.

That error string corresponds to GSS_S_CALL_INACCESSIBLE_READ.

In 1.6, it returns an error if the 'input_token_buffer' string
is empty. krb5-1.6:
https://github.com/krb5/krb5/blob/krb5-1.6/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

In 1.7, it returns an error only if the 'input_token_buffer' string
is NULL. krb5-1.7:
https://github.com/krb5/krb5/blob/krb5-1.7/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

With KRPC, we test if Kerberos works by passing an empty string to SASL:
https://github.com/apache/impala/blob/master/be/src/kudu/rpc/server_negotiation.cc#L289

In 1.6, this is counted as an error, but in 1.7, this is completely
fine. I'm not sure why since they haven't documented it.

We can attempt to get KRPC working for SLES11 by removing the
PreflightGSSAPI() check for any kerberos version < 1.6. A function that
is unavailable on krb-1.6 is krb5_get_init_creds_opt_set_fast_ccache_name(),
and it is available from krb-1.7 onwards. The PreflightCheckGSSAPI() is
compiled in only if this function exists.

(However there may be more issues on SLES11 that we're not yet aware of)

Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Reviewed-on: http://gerrit.cloudera.org:8080/9696
Reviewed-by: Sailesh Mukil 
Tested-by: Impala Public Jenkins
---
M be/CMakeLists.txt
M be/src/common/config.h.in
M be/src/kudu/rpc/messenger.cc
3 files changed, 18 insertions(+), 0 deletions(-)

Approvals:
  Sailesh Mukil: Looks good to me, approved
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 5
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-21 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 4: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 4
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 21 Mar 2018 22:18:59 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-21 Thread Sailesh Mukil (Code Review) 
Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 4: Code-Review+2

(1 comment)

Thanks for the review!

Rebase, carry +2.

http://gerrit.cloudera.org:8080/#/c/9696/3/be/src/kudu/rpc/messenger.cc
File be/src/kudu/rpc/messenger.cc:

http://gerrit.cloudera.org:8080/#/c/9696/3/be/src/kudu/rpc/messenger.cc@290
PS3, Line 290: LOG(WARNING) << "Omitting Kerberos pre-flight check. 
Connection negotiations may fail"
> nit: indent wrong
Done



--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 4
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 21 Mar 2018 18:37:20 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-21 Thread Impala Public Jenkins (Code Review) 
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/2146/


--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 4
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 21 Mar 2018 18:37:59 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-21 Thread Sailesh Mukil (Code Review) 
Hello Michael Ho, Philip Zeyliger, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9696

to look at the new patch set (#4).

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..

IMPALA-6691: KRPC w/ kerberos fails on SLES11

The Kerberos version used in SLES 11 seems to have quite a few
undocumented bugs.
They have krb5-1.6 (krb5-client-1.6.3-133.49.112.1.x86_64).

With KRPC we see a new error "GSSAPI Error: A required input parameter
could not be read", which we've never seen before.

I looked into the krb5 codebase and between krb5-1.6 and krb5-1.7,
the code causing the above error (GSSAPI Error: A required input
parameter could not be read) has changed subtly without any
explanation as to why.

That error string corresponds to GSS_S_CALL_INACCESSIBLE_READ.

In 1.6, it returns an error if the 'input_token_buffer' string
is empty. krb5-1.6:
https://github.com/krb5/krb5/blob/krb5-1.6/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

In 1.7, it returns an error only if the 'input_token_buffer' string
is NULL. krb5-1.7:
https://github.com/krb5/krb5/blob/krb5-1.7/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

With KRPC, we test if Kerberos works by passing an empty string to SASL:
https://github.com/apache/impala/blob/master/be/src/kudu/rpc/server_negotiation.cc#L289

In 1.6, this is counted as an error, but in 1.7, this is completely
fine. I'm not sure why since they haven't documented it.

We can attempt to get KRPC working for SLES11 by removing the
PreflightGSSAPI() check for any kerberos version < 1.6. A function that
is unavailable on krb-1.6 is krb5_get_init_creds_opt_set_fast_ccache_name(),
and it is available from krb-1.7 onwards. The PreflightCheckGSSAPI() is
compiled in only if this function exists.

(However there may be more issues on SLES11 that we're not yet aware of)

Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
---
M be/CMakeLists.txt
M be/src/common/config.h.in
M be/src/kudu/rpc/messenger.cc
3 files changed, 18 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/9696/4
--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 4
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-21 Thread Michael Ho (Code Review) 
Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 3: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9696/3/be/src/kudu/rpc/messenger.cc
File be/src/kudu/rpc/messenger.cc:

http://gerrit.cloudera.org:8080/#/c/9696/3/be/src/kudu/rpc/messenger.cc@290
PS3, Line 290:   LOG(WARNING) << "Omitting Kerberos pre-flight check. 
Connection negotiations may fail "
nit: indent wrong



--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 3
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 21 Mar 2018 16:58:11 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-21 Thread Sailesh Mukil (Code Review) 
Hello Michael Ho, Philip Zeyliger, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9696

to look at the new patch set (#3).

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..

IMPALA-6691: KRPC w/ kerberos fails on SLES11

The Kerberos version used in SLES 11 seems to have quite a few
undocumented bugs.
They have krb5-1.6 (krb5-client-1.6.3-133.49.112.1.x86_64).

With KRPC we see a new error "GSSAPI Error: A required input parameter
could not be read", which we've never seen before.

I looked into the krb5 codebase and between krb5-1.6 and krb5-1.7,
the code causing the above error (GSSAPI Error: A required input
parameter could not be read) has changed subtly without any
explanation as to why.

That error string corresponds to GSS_S_CALL_INACCESSIBLE_READ.

In 1.6, it returns an error if the 'input_token_buffer' string
is empty. krb5-1.6:
https://github.com/krb5/krb5/blob/krb5-1.6/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

In 1.7, it returns an error only if the 'input_token_buffer' string
is NULL. krb5-1.7:
https://github.com/krb5/krb5/blob/krb5-1.7/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

With KRPC, we test if Kerberos works by passing an empty string to SASL:
https://github.com/apache/impala/blob/master/be/src/kudu/rpc/server_negotiation.cc#L289

In 1.6, this is counted as an error, but in 1.7, this is completely
fine. I'm not sure why since they haven't documented it.

We can attempt to get KRPC working for SLES11 by removing the
PreflightGSSAPI() check for any kerberos version < 1.6. A function that
is unavailable on krb-1.6 is krb5_get_init_creds_opt_set_fast_ccache_name(),
and it is available from krb-1.7 onwards. The PreflightCheckGSSAPI() is
compiled in only if this function exists.

(However there may be more issues on SLES11 that we're not yet aware of)

Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
---
M be/CMakeLists.txt
M be/src/common/config.h.in
M be/src/kudu/rpc/messenger.cc
3 files changed, 18 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/9696/3
--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 3
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-21 Thread Sailesh Mukil (Code Review) 
Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 3:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/9696/2/be/CMakeLists.txt
File be/CMakeLists.txt:

http://gerrit.cloudera.org:8080/#/c/9696/2/be/CMakeLists.txt@341
PS2, Line 341: krb5_get_init_creds_opt_set_fast_ccache_name
> I manually confirmed that this function is not defined in the source code h
Yup, I already checked it in SLES11. And I now checked it in CentOS 6.4 too.

It doesn't exist on SLES11 (as we expect) and it exists on CentOS 6.4.


http://gerrit.cloudera.org:8080/#/c/9696/2/be/src/kudu/rpc/messenger.cc
File be/src/kudu/rpc/messenger.cc:

http://gerrit.cloudera.org:8080/#/c/9696/2/be/src/kudu/rpc/messenger.cc@279
PS2, Line 279: We omit calling PreflightCheckGS
> We omit calling ...
Done


http://gerrit.cloudera.org:8080/#/c/9696/2/be/src/kudu/rpc/messenger.cc@282
PS2, Line 282: ll fail
> nit: omitting
Done


http://gerrit.cloudera.org:8080/#/c/9696/2/be/src/kudu/rpc/messenger.cc@290
PS2, Line 290: WARNING) << "Omitting Kerberos pre-flight check. Connection 
negotiations may fail "
> Is it worth adding a quick log statement to indicate that we are skipping P
Yes, I added a warning LOG message.



--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 3
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 21 Mar 2018 07:39:21 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-20 Thread Michael Ho (Code Review) 
Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 2: Code-Review+1

(4 comments)

http://gerrit.cloudera.org:8080/#/c/9696/2/be/CMakeLists.txt
File be/CMakeLists.txt:

http://gerrit.cloudera.org:8080/#/c/9696/2/be/CMakeLists.txt@341
PS2, Line 341: krb5_get_init_creds_opt_set_fast_ccache_name
I manually confirmed that this function is not defined in the source code 
https://github.com/krb5/krb5/tree/krb5-1.6/src but defined in 
https://github.com/krb5/krb5/tree/krb5-1.7/src.

It'd be nice to do an objdump of the kerberos shared library on vanilla SLES11 
and Centos6.4 to confirm they match our expectation.


http://gerrit.cloudera.org:8080/#/c/9696/2/be/src/kudu/rpc/messenger.cc
File be/src/kudu/rpc/messenger.cc:

http://gerrit.cloudera.org:8080/#/c/9696/2/be/src/kudu/rpc/messenger.cc@279
PS2, Line 279: Since we support SLES11, we omit
We omit calling ...


http://gerrit.cloudera.org:8080/#/c/9696/2/be/src/kudu/rpc/messenger.cc@282
PS2, Line 282: omiting
nit: omitting


http://gerrit.cloudera.org:8080/#/c/9696/2/be/src/kudu/rpc/messenger.cc@290
PS2, Line 290:
Is it worth adding a quick log statement to indicate that we are skipping 
PreflightCheckGSSAPI() ?



--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 2
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 21 Mar 2018 05:31:58 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-20 Thread Sailesh Mukil (Code Review) 
Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 2:

> Patch Set 1:
>
> (1 comment)

Thanks Todd and Phil! I've gone ahead and done the first fix Todd mentioned, 
since we have similar usages in other parts of our code base.

I confirmed that the code shows up when built with krb5 versions > 1.7, and 
that the code doesn't show up on krb5-1.6, by looking at the disassembly of the 
AddAcceptorPool() function in the binaries compiled with this fix.

I also ran the rpc-mgr-kerberized-test on an Ubuntu machine with this fix to 
ensure there's no regressions.


--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 2
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Wed, 21 Mar 2018 02:29:07 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-20 Thread Sailesh Mukil (Code Review) 
Hello Michael Ho, Philip Zeyliger, Todd Lipcon,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/9696

to look at the new patch set (#2).

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..

IMPALA-6691: KRPC w/ kerberos fails on SLES11

The Kerberos version used in SLES 11 seems to have quite a few
undocumented bugs.
They have krb5-1.6 (krb5-client-1.6.3-133.49.112.1.x86_64).

With KRPC we see a new error "GSSAPI Error: A required input parameter
could not be read", which we've never seen before.

I looked into the krb5 codebase and between krb5-1.6 and krb5-1.7,
the code causing the above error (GSSAPI Error: A required input
parameter could not be read) has changed subtly without any
explanation as to why.

That error string corresponds to GSS_S_CALL_INACCESSIBLE_READ.

In 1.6, it returns an error if the 'input_token_buffer' string
is empty. krb5-1.6:
https://github.com/krb5/krb5/blob/krb5-1.6/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

In 1.7, it returns an error only if the 'input_token_buffer' string
is NULL. krb5-1.7:
https://github.com/krb5/krb5/blob/krb5-1.7/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

With KRPC, we test if Kerberos works by passing an empty string to SASL:
https://github.com/apache/impala/blob/master/be/src/kudu/rpc/server_negotiation.cc#L289

In 1.6, this is counted as an error, but in 1.7, this is completely
fine. I'm not sure why since they haven't documented it.

We can attempt to get KRPC working for SLES11 by removing the
PreflightGSSAPI() check for any kerberos version < 1.6. A function that
is unavailable on krb-1.6 is krb5_get_init_creds_opt_set_fast_ccache_name(),
and it is available from krb-1.7 onwards. The PreflightCheckGSSAPI() is
compiled in only if this function exists.

(However there may be more issues on SLES11 that we're not yet aware of)

Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
---
M be/CMakeLists.txt
M be/src/common/config.h.in
M be/src/kudu/rpc/messenger.cc
3 files changed, 15 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/9696/2
--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 2
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-20 Thread Philip Zeyliger (Code Review) 
Philip Zeyliger has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9696/1/be/src/kudu/rpc/messenger.cc
File be/src/kudu/rpc/messenger.cc:

http://gerrit.cloudera.org:8080/#/c/9696/1/be/src/kudu/rpc/messenger.cc@284
PS1, Line 284:   if (!keytab_file_.empty()) {
> It's really a shame to lose this on all operating systems just due to the S
If you want to detect that you're in SLES11, you can also generate something at 
CMake time.



--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Philip Zeyliger 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Tue, 20 Mar 2018 21:50:26 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-20 Thread Todd Lipcon (Code Review) 
Todd Lipcon has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9696/1/be/src/kudu/rpc/messenger.cc
File be/src/kudu/rpc/messenger.cc:

http://gerrit.cloudera.org:8080/#/c/9696/1/be/src/kudu/rpc/messenger.cc@284
PS1, Line 284:   if (!keytab_file_.empty()) {
> Were you able to test this change on SLES 11 ?
It's really a shame to lose this on all operating systems just due to the 
SLES11 deficiency. The reason we added it is that we found it quite difficult 
to debug the issues if they happened at negotiation time rather than at service 
startup.

We've done a compile-time krb5 version detection in the past using an #ifdef 
based on some random constant defined in krb5.h that was added in a particular 
version. It's hacky, but since they don't provide any KRB5_VERSION macro or 
anythiing, it's the best we could do.  For example you could check for 
KRB5_NT_X500_PRINCIPAL which was added in 1.7.

Another option would be to use sasl_client_start to generate a token and pass 
that into sasl_server_start(). In other words, run a step or two of the 
negotiation in a "short circuited" configuration to see whether the server can 
init a connection to itself.



--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Reviewer: Todd Lipcon 
Gerrit-Comment-Date: Tue, 20 Mar 2018 21:47:08 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-19 Thread Sailesh Mukil (Code Review) 
Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 1:

> Patch Set 1:
>
> (1 comment)

I spent about 2 days trying to get a custom build on SLES11, but I couldn't get 
it to run.

After quite some trying, I got Impala to build on SLES11. However, even though 
I could build the impalad binary and the rpc-mgr-kerberized-test binary, I 
couldn't run the rpc-mgr-kerberized-test because one of the mini-KDC's 
dependencies 'kdb5_util' is not available on SLES11.

Also, I tried replacing a SLES11 remote cluster's impalad with the one I built, 
but it fails to start up since the frontend is unable to talk to the namenode 
(among other things).


--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Mon, 19 Mar 2018 06:40:28 +
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-16 Thread Michael Ho (Code Review) 
Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9696/1/be/src/kudu/rpc/messenger.cc
File be/src/kudu/rpc/messenger.cc:

http://gerrit.cloudera.org:8080/#/c/9696/1/be/src/kudu/rpc/messenger.cc@284
PS1, Line 284:   if (!keytab_file_.empty()) {
> I spent some time trying to disable it only for a specific kerberos version
Were you able to test this change on SLES 11 ?



--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Michael Ho 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Sat, 17 Mar 2018 00:17:16 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-16 Thread Sailesh Mukil (Code Review) 
Sailesh Mukil has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9696 )

Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/9696/1/be/src/kudu/rpc/messenger.cc
File be/src/kudu/rpc/messenger.cc:

http://gerrit.cloudera.org:8080/#/c/9696/1/be/src/kudu/rpc/messenger.cc@284
PS1, Line 284:   if (!keytab_file_.empty()) {
I spent some time trying to disable it only for a specific kerberos version, 
but I couldn't find an easy way to do that. So I disabled it in general for now.



--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil 
Gerrit-Reviewer: Sailesh Mukil 
Gerrit-Comment-Date: Fri, 16 Mar 2018 21:55:43 +
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6691: KRPC w/ kerberos fails on SLES11

2018-03-16 Thread Sailesh Mukil (Code Review) 
Sailesh Mukil has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/9696


Change subject: IMPALA-6691: KRPC w/ kerberos fails on SLES11
..

IMPALA-6691: KRPC w/ kerberos fails on SLES11

The Kerberos version used in SLES 11 seems to have quite a few
undocumented bugs.
They have krb5-1.6 (krb5-client-1.6.3-133.49.112.1.x86_64).

With KRPC we see a new error "GSSAPI Error: A required input parameter
could not be read", which we've never seen before.

I looked into the krb5 codebase and between krb5-1.6 and krb5-1.7,
the code causing the above error (GSSAPI Error: A required input
parameter could not be read) has changed subtly without any
explanation as to why.

That error string corresponds to GSS_S_CALL_INACCESSIBLE_READ.

In 1.6, it returns an error if the 'input_token_buffer' string
is empty. krb5-1.6:
https://github.com/krb5/krb5/blob/krb5-1.6/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

In 1.7, it returns an error only if the 'input_token_buffer' string
is NULL. krb5-1.7:
https://github.com/krb5/krb5/blob/krb5-1.7/src/lib/gssapi/mechglue/g_accept_sec_context.c#L149-L150

With KRPC, we test if Kerberos works by passing an empty string to SASL:
https://github.com/apache/impala/blob/master/be/src/kudu/rpc/server_negotiation.cc#L289

In 1.6, this is counted as an error, but in 1.7, this is completely
fine. I'm not sure why since they haven't documented it.

We can attempt to get KRPC working for SLES11 by removing the
PreflightGSSAPI() check.

(However there may be more issues on SLES11 that we're not yet aware of)

Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
---
M be/src/kudu/rpc/messenger.cc
1 file changed, 8 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/96/9696/1
--
To view, visit http://gerrit.cloudera.org:8080/9696
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic4cc7f0702f605fca02a2ff5d3d2735e6e080668
Gerrit-Change-Number: 9696
Gerrit-PatchSet: 1
Gerrit-Owner: Sailesh Mukil