[
https://issues.apache.org/jira/browse/HBASE-28757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17868765#comment-17868765
]
Andrew Kyle Purtell edited comment on HBASE-28757 at 7/25/24 7:09 PM:
----------------------------------------------------------------------
bq. You could definitely have TLS enabled and also use Kerberos for auth. With
TLS enabled, the netty client/server will do an SSL handshake. This may or may
not involve verifying of the certificates (depending on what you have enabled).
Once the SSL handshake is complete, it goes down a layer to where Kerberos is
and if it's enabled then you might also authenticate with kerberos.
Except what [~shahrs87] describes is a configuration that enables both client
and server TLS options to true, and although kerberos authentication does
succeed instead the server abnormally closes the connection. The server in this
scenario is the master, and the client is the regionserver, which therefore
cannot report for duty. "allowplaintext" being false here does prevent a non
TLS connection from succeeding but does not allow the client to negotiate TLS
and succeed, which the configuration option selection should support, at least
as a user would reasonably expect.
was (Author: apurtell):
bq. You could definitely have TLS enabled and also use Kerberos for auth. With
TLS enabled, the netty client/server will do an SSL handshake. This may or may
not involve verifying of the certificates (depending on what you have enabled).
Once the SSL handshake is complete, it goes down a layer to where Kerberos is
and if it's enabled then you might also authenticate with kerberos.
Except what [~shahrs87] describes is although kerberos authentication succeeded
the server abnormally closes the connection. The server in this scenario is the
master, and the client is the regionserver, which therefore cannot report for
duty. "allowplaintext" being false here does prevent a non TLS connection from
succeeding but does not allow the client to negotiate TLS and succeed, which
the configuration option selection should support, at least as a user would
reasonably expect.
> Understand how supportplaintext property works in TLS setup.
> ------------------------------------------------------------
>
> Key: HBASE-28757
> URL: https://issues.apache.org/jira/browse/HBASE-28757
> Project: HBase
> Issue Type: Improvement
> Components: security
> Affects Versions: 2.6.0
> Reporter: Rushabh Shah
> Priority: Major
>
> We are testing TLS feature and I am confused on how
> hbase.server.netty.tls.supportplaintext property works.
> Here is our current setup. This is a fresh cluster deployment.
> hbase.server.netty.tls.enabled --> true
> hbase.client.netty.tls.enabled --> true
> hbase.server.netty.tls.supportplaintext --> false (We don't want to fallback
> on kerberos)
> We still have our kerberos related configuration enabled.
> hbase.security.authentication --> kerberos
> *Our expectation:*
> During regionserver startup, regionserver will use TLS for authentication and
> the communication will succeed.
> *Actual observation*
> During regionserver startup, hmaster authenticates regionserver* via kerberos
> authentication*and *regionserver's reportForDuty RPC fails*.
> RS logs:
> {noformat}
> 2024-07-25 16:59:55,098 INFO [regionserver/regionserver-0:60020]
> regionserver.HRegionServer - reportForDuty to
> master=hmaster-0,60000,1721926791062 with
> isa=regionserver-0/<rs-ip-address>:60020, startcode=1721926793434
> 2024-07-25 16:59:55,548 DEBUG [RS-EventLoopGroup-1-2] ssl.SslHandler - [id:
> 0xa48e3487, L:/<rs-ip-address>:39837 -
> R:hmaster-0/<hmaster-ip-address>:60000] HANDSHAKEN: protocol:TLSv1.2 cipher
> suite:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
> 2024-07-25 16:59:55,578 DEBUG [RS-EventLoopGroup-1-2]
> security.UserGroupInformation - PrivilegedAction [as: hbase/regionserver-0.
> (auth:KERBEROS)][action:
> org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClientHandler$2@3769e55]
> java.lang.Exception
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1896)
> at
> org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClientHandler.channelRead0(NettyHBaseSaslRpcClientHandler.java:161)
> at
> org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClientHandler.channelRead0(NettyHBaseSaslRpcClientHandler.java:43)
> ...
> ...
> 2024-07-25 16:59:55,581 DEBUG [RS-EventLoopGroup-1-2]
> security.UserGroupInformation - PrivilegedAction [as: hbase/regionserver-0
> (auth:KERBEROS)][action:
> org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClientHandler$2@c6f0806]
> java.lang.Exception
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1896)
> at
> org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClientHandler.channelRead0(NettyHBaseSaslRpcClientHandler.java:161)
> at
> org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClientHandler.channelRead0(NettyHBaseSaslRpcClientHandler.java:43)
> at
> org.apache.hbase.thirdparty.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
> 2024-07-25 16:59:55,602 WARN [regionserver/regionserver-0:60020]
> regionserver.HRegionServer - error telling master we are up
> org.apache.hbase.thirdparty.com.google.protobuf.ServiceException:
> org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to
> address=hmaster-0:60000 failed on local exception:
> org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection
> closed
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:340)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$200(AbstractRpcClient.java:92)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:595)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:16398)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2997)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.lambda$run$2(HRegionServer.java:1084)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)
> at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1079)
> Caused by: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call
> to address=hmaster-0:60000 failed on local exception:
> org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection
> closed
> at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:233)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:391)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:92)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:425)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:420)
> at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:114)
> at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:129)
> at
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.cleanupCalls(NettyRpcDuplexHandler.java:231)
> at
> org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelInactive(NettyRpcDuplexHandler.java:239)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:303)
> {noformat}
> Hmaster logs
> {noformat}
> 2024-07-25 16:59:55,378 DEBUG [RS-EventLoopGroup-1-2] ipc.NettyRpcServer -
> SSL handler added for channel: [id: 0xd4071764, L:/<hmaster-ip>:60000 -
> R:regionserver-0/<rs-ip>:39837]
> 2024-07-25 16:59:55,526 DEBUG [RS-EventLoopGroup-1-2] ssl.SslHandler - [id:
> 0xd4071764, L:/<hmaster-ip>:60000 - R:regionserver-0/<rs-ip>:39837]
> HANDSHAKEN: protocol:TLSv1.2 cipher
> suite:TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
> 2024-07-25 16:59:55,583 INFO [RS-EventLoopGroup-1-2] hbase.Server - Auth
> successful for hbase/regionserver-0 (auth:KERBEROS)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
