Re: mTLS / x509 authentication
@Andrew Purtell @Bryan Beaudreault We've a PR for the proposal here: https://github.com/apache/hbase/pull/6447 Which adds the new RpcCoprocessorHost and RpcObserver to capture the login event. Also opened a PR to show an example of usage: https://github.com/anmolnar/hbase/pull/2 Once accepted I'll submit it to the hbase-examples project. PTAL. Andor On Fri, 2024-11-01 at 16:50 -0500, Andor Molnar wrote: > Filed a Jira ticket for this and I started to work on it. > > https://issues.apache.org/jira/browse/HBASE-28952 > > Andor > > > On Fri, 2024-09-20 at 15:03 -0700, Andrew Purtell wrote: > > I would accept a coprocessor hook addition for connection > > establishment, or rejecting same, provided the customary backwards > > compatibility provisions are made to avoid binary compatibility > > problems and minimize source level changes for interface > > implementers. I can’t speak for other committers but you would have > > my +1. > > > > > On Sep 19, 2024, at 7:07 AM, Charles Connell > > > wrote: > > > > > > Hi Andor, > > > > > > I don't believe there is a way for a coprocessor or other plugin > > > to > > > hook into connection establishment and reject connections. If you > > > were > > > interested, I imagine that contributing this change would be > > > appreciated by the core team, but I don't speak for them. > > > > > > As a workaround for now, you would need to block specific > > > operations > > > via coprocessor implementations of the various *Observer > > > interfaces > > > like RegionObserver and others. Once there, you can call > > > RpcServer.getCurrentCall() to get an RpcCallContext object. > > > > > > > > > Charles > > > > > > > On Tue, Aug 27, 2024 at 10:03 AM Andor Molnar > > > > > > > > wrote: > > > > > > > > Hi Andrew / Bryan, > > > > > > > > I'm looking at how to implement this client cert user/group > > > > verification stuff that we talked about here, but I have to > > > > admit > > > > I > > > > have no idea how to do this properly. Could you please give me > > > > some > > > > pointers? > > > > > > > > In @charlesconnel's PR [1] about adding client cert chain to > > > > RpcCallContext he mentions that it was needed for some kind of > > > > audit > > > > logging, though in the Jira ticket [2] we'd like to extend the > > > > authentication flow and prevent users from connecting with > > > > invalid > > > > certificates. > > > > > > > > The closest I've found so far is the AccessController > > > > coprocessor > > > > which > > > > implements _all_ interface methods to verify access. That is a > > > > bit > > > > scary. Is there a way to catch all login attempts somehow and > > > > access > > > > RpCallContext from there? > > > > > > > > Thanks, > > > > Andor > > > > > > > > [1] https://github.com/apache/hbase/pull/5644 > > > > [2] https://issues.apache.org/jira/browse/HBASE-27326 > > > > > > > > > > > > > > > > > > > > > On Mon, 2024-06-10 at 21:46 +0200, Andor Molnar wrote: > > > > > Awesome. > > > > > Thanks for all the input folks. > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > On Mon, 2024-06-10 at 15:24 -0400, Bryan Beaudreault wrote: > > > > > > I'm back at my computer now so was able to find the JIRA: > > > > > > https://issues.apache.org/jira/browse/HBASE-27326 > > > > > > > > > > > > I ended up not tackling it partially for the reason Andrew > > > > > > mentions > > > > > > -- I > > > > > > realized that even our use-case did not cleanly fit into > > > > > > the > > > > > > proposal > > > > > > there. So instead we ended up exposing the actual > > > > > > certificate > > > > > > chain > > > > > > [1] in > > > > > > the RpcCallContext, so that a user can make their own > > > > > > implementation > > > > > > using > > > > > > coprocessors, etc to do whatever they like. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-28317 > > > > > > > > > > > > On Mon, Jun 10, 2024 at 3:05 PM Andrew Purtell > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Cool. > > > > > > > I suggest hbase-examples because different organizations > > > > > > > will > > > > > > > embed > > > > > > > a user > > > > > > > name or potenitally other details into certificate fields > > > > > > > differently, so > > > > > > > there isn't a one size fits all solution. However HBase > > > > > > > could > > > > > > > provide a > > > > > > > worked example where the username is in CN. > > > > > > > > > > > > > > On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell > > > > > > > > wrote: > > > > > > > > > > > > > > > > > There is middle ground between your position and a > > > > > > > > > blanket > > > > > > > > > trust of > > > > > > > > > connection header parameters from any client with a > > > > > > > > > valid > > > > > > > > > certificate... > > > > > > > > > Certificates can carry additional information, >
Re: mTLS / x509 authentication
Filed a Jira ticket for this and I started to work on it. https://issues.apache.org/jira/browse/HBASE-28952 Andor On Fri, 2024-09-20 at 15:03 -0700, Andrew Purtell wrote: > I would accept a coprocessor hook addition for connection > establishment, or rejecting same, provided the customary backwards > compatibility provisions are made to avoid binary compatibility > problems and minimize source level changes for interface > implementers. I can’t speak for other committers but you would have > my +1. > > > On Sep 19, 2024, at 7:07 AM, Charles Connell > > wrote: > > > > Hi Andor, > > > > I don't believe there is a way for a coprocessor or other plugin to > > hook into connection establishment and reject connections. If you > > were > > interested, I imagine that contributing this change would be > > appreciated by the core team, but I don't speak for them. > > > > As a workaround for now, you would need to block specific > > operations > > via coprocessor implementations of the various *Observer interfaces > > like RegionObserver and others. Once there, you can call > > RpcServer.getCurrentCall() to get an RpcCallContext object. > > > > > > Charles > > > > > On Tue, Aug 27, 2024 at 10:03 AM Andor Molnar > > > wrote: > > > > > > Hi Andrew / Bryan, > > > > > > I'm looking at how to implement this client cert user/group > > > verification stuff that we talked about here, but I have to admit > > > I > > > have no idea how to do this properly. Could you please give me > > > some > > > pointers? > > > > > > In @charlesconnel's PR [1] about adding client cert chain to > > > RpcCallContext he mentions that it was needed for some kind of > > > audit > > > logging, though in the Jira ticket [2] we'd like to extend the > > > authentication flow and prevent users from connecting with > > > invalid > > > certificates. > > > > > > The closest I've found so far is the AccessController coprocessor > > > which > > > implements _all_ interface methods to verify access. That is a > > > bit > > > scary. Is there a way to catch all login attempts somehow and > > > access > > > RpCallContext from there? > > > > > > Thanks, > > > Andor > > > > > > [1] https://github.com/apache/hbase/pull/5644 > > > [2] https://issues.apache.org/jira/browse/HBASE-27326 > > > > > > > > > > > > > > > > On Mon, 2024-06-10 at 21:46 +0200, Andor Molnar wrote: > > > > Awesome. > > > > Thanks for all the input folks. > > > > > > > > Andor > > > > > > > > > > > > > > > > On Mon, 2024-06-10 at 15:24 -0400, Bryan Beaudreault wrote: > > > > > I'm back at my computer now so was able to find the JIRA: > > > > > https://issues.apache.org/jira/browse/HBASE-27326 > > > > > > > > > > I ended up not tackling it partially for the reason Andrew > > > > > mentions > > > > > -- I > > > > > realized that even our use-case did not cleanly fit into the > > > > > proposal > > > > > there. So instead we ended up exposing the actual certificate > > > > > chain > > > > > [1] in > > > > > the RpcCallContext, so that a user can make their own > > > > > implementation > > > > > using > > > > > coprocessors, etc to do whatever they like. > > > > > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-28317 > > > > > > > > > > On Mon, Jun 10, 2024 at 3:05 PM Andrew Purtell > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > Cool. > > > > > > I suggest hbase-examples because different organizations > > > > > > will > > > > > > embed > > > > > > a user > > > > > > name or potenitally other details into certificate fields > > > > > > differently, so > > > > > > there isn't a one size fits all solution. However HBase > > > > > > could > > > > > > provide a > > > > > > worked example where the username is in CN. > > > > > > > > > > > > On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar > > > > > > > > > > > > wrote: > > > > > > > > > > > > > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > > > > > > > > > > > > > > > There is middle ground between your position and a > > > > > > > > blanket > > > > > > > > trust of > > > > > > > > connection header parameters from any client with a > > > > > > > > valid > > > > > > > > certificate... > > > > > > > > Certificates can carry additional information, > > > > > > > > including > > > > > > > > usernames, > > > > > > > > which > > > > > > > > would be protected by the signature. Sometimes this is > > > > > > > > done. > > > > > > > > HBase > > > > > > > > doesn't > > > > > > > > implement it (yet) and if you have a trust architecture > > > > > > > > that > > > > > > > > demands > > > > > > > > it, it > > > > > > > > would establish the username as attested by an > > > > > > > > authority, > > > > > > > > like > > > > > > > > a > > > > > > > > kerberos > > > > > > > > ticket. You'd use a netty API presumably in a > > > > > > > > coprocessor > > > > > > > > hook > > > > > > > > to get > > > > > > > > it in > > > > > > > > order to hand off to UGI. Should someone decide this is > > > > > > > >
Re: mTLS / x509 authentication
I would accept a coprocessor hook addition for connection establishment, or rejecting same, provided the customary backwards compatibility provisions are made to avoid binary compatibility problems and minimize source level changes for interface implementers. I can’t speak for other committers but you would have my +1. > On Sep 19, 2024, at 7:07 AM, Charles Connell > wrote: > > Hi Andor, > > I don't believe there is a way for a coprocessor or other plugin to > hook into connection establishment and reject connections. If you were > interested, I imagine that contributing this change would be > appreciated by the core team, but I don't speak for them. > > As a workaround for now, you would need to block specific operations > via coprocessor implementations of the various *Observer interfaces > like RegionObserver and others. Once there, you can call > RpcServer.getCurrentCall() to get an RpcCallContext object. > > > Charles > >> On Tue, Aug 27, 2024 at 10:03 AM Andor Molnar wrote: >> >> Hi Andrew / Bryan, >> >> I'm looking at how to implement this client cert user/group >> verification stuff that we talked about here, but I have to admit I >> have no idea how to do this properly. Could you please give me some >> pointers? >> >> In @charlesconnel's PR [1] about adding client cert chain to >> RpcCallContext he mentions that it was needed for some kind of audit >> logging, though in the Jira ticket [2] we'd like to extend the >> authentication flow and prevent users from connecting with invalid >> certificates. >> >> The closest I've found so far is the AccessController coprocessor which >> implements _all_ interface methods to verify access. That is a bit >> scary. Is there a way to catch all login attempts somehow and access >> RpCallContext from there? >> >> Thanks, >> Andor >> >> [1] https://github.com/apache/hbase/pull/5644 >> [2] https://issues.apache.org/jira/browse/HBASE-27326 >> >> >> >> >>> On Mon, 2024-06-10 at 21:46 +0200, Andor Molnar wrote: >>> Awesome. >>> Thanks for all the input folks. >>> >>> Andor >>> >>> >>> >>> On Mon, 2024-06-10 at 15:24 -0400, Bryan Beaudreault wrote: I'm back at my computer now so was able to find the JIRA: https://issues.apache.org/jira/browse/HBASE-27326 I ended up not tackling it partially for the reason Andrew mentions -- I realized that even our use-case did not cleanly fit into the proposal there. So instead we ended up exposing the actual certificate chain [1] in the RpcCallContext, so that a user can make their own implementation using coprocessors, etc to do whatever they like. [1] https://issues.apache.org/jira/browse/HBASE-28317 On Mon, Jun 10, 2024 at 3:05 PM Andrew Purtell wrote: > Cool. > I suggest hbase-examples because different organizations will > embed > a user > name or potenitally other details into certificate fields > differently, so > there isn't a one size fits all solution. However HBase could > provide a > worked example where the username is in CN. > > On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar > wrote: > >> On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: >> >>> There is middle ground between your position and a blanket >>> trust of >>> connection header parameters from any client with a valid >>> certificate... >>> Certificates can carry additional information, including >>> usernames, >>> which >>> would be protected by the signature. Sometimes this is done. >>> HBase >>> doesn't >>> implement it (yet) and if you have a trust architecture that >>> demands >>> it, it >>> would establish the username as attested by an authority, >>> like >>> a >>> kerberos >>> ticket. You'd use a netty API presumably in a coprocessor >>> hook >>> to get >>> it in >>> order to hand off to UGI. Should someone decide this is >>> needed >>> it >>> would >>> make a good contribution back to the project, somewhere in >>> hbase- >>> examples >>> perhaps. >> >> Exactly. That's what I wanted to clarify and pointed what other >> projects (like ZooKeeper) are doing under the hood of x509 >> authentication. I definitely would like to contribute something >> similar >> for HBase too. >> >> Thanks for the pointer of hbase-examples. >> >> Andor >> >> >> >> >>> On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell < >>> [email protected]> >>> wrote: >>> Like I said the successful verification of the certificate implies the client is trustworthy, including what the client supplies in the header. Now, if within your organization, you are distributing trusted certificates to potentially
Re: mTLS / x509 authentication
Hi Charles, Thanks for coming back on this. I see you mentioned that you wanted to log something in the coprocessor based on the cert information. I get you now. Thanks for the help. Andor On Thu, 2024-09-19 at 10:04 -0400, Charles Connell wrote: > Hi Andor, > > I don't believe there is a way for a coprocessor or other plugin to > hook into connection establishment and reject connections. If you > were > interested, I imagine that contributing this change would be > appreciated by the core team, but I don't speak for them. > > As a workaround for now, you would need to block specific operations > via coprocessor implementations of the various *Observer interfaces > like RegionObserver and others. Once there, you can call > RpcServer.getCurrentCall() to get an RpcCallContext object. > > > Charles > > On Tue, Aug 27, 2024 at 10:03 AM Andor Molnar > wrote: > > Hi Andrew / Bryan, > > > > I'm looking at how to implement this client cert user/group > > verification stuff that we talked about here, but I have to admit I > > have no idea how to do this properly. Could you please give me some > > pointers? > > > > In @charlesconnel's PR [1] about adding client cert chain to > > RpcCallContext he mentions that it was needed for some kind of > > audit > > logging, though in the Jira ticket [2] we'd like to extend the > > authentication flow and prevent users from connecting with invalid > > certificates. > > > > The closest I've found so far is the AccessController coprocessor > > which > > implements _all_ interface methods to verify access. That is a bit > > scary. Is there a way to catch all login attempts somehow and > > access > > RpCallContext from there? > > > > Thanks, > > Andor > > > > [1] https://github.com/apache/hbase/pull/5644 > > [2] https://issues.apache.org/jira/browse/HBASE-27326 > > > > > > > > > > On Mon, 2024-06-10 at 21:46 +0200, Andor Molnar wrote: > > > Awesome. > > > Thanks for all the input folks. > > > > > > Andor > > > > > > > > > > > > On Mon, 2024-06-10 at 15:24 -0400, Bryan Beaudreault wrote: > > > > I'm back at my computer now so was able to find the JIRA: > > > > https://issues.apache.org/jira/browse/HBASE-27326 > > > > > > > > I ended up not tackling it partially for the reason Andrew > > > > mentions > > > > -- I > > > > realized that even our use-case did not cleanly fit into the > > > > proposal > > > > there. So instead we ended up exposing the actual certificate > > > > chain > > > > [1] in > > > > the RpcCallContext, so that a user can make their own > > > > implementation > > > > using > > > > coprocessors, etc to do whatever they like. > > > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-28317 > > > > > > > > On Mon, Jun 10, 2024 at 3:05 PM Andrew Purtell < > > > > [email protected] > > > > wrote: > > > > > > > > > Cool. > > > > > I suggest hbase-examples because different organizations will > > > > > embed > > > > > a user > > > > > name or potenitally other details into certificate fields > > > > > differently, so > > > > > there isn't a one size fits all solution. However HBase could > > > > > provide a > > > > > worked example where the username is in CN. > > > > > > > > > > On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar < > > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > > > > > > > > > > > > > There is middle ground between your position and a > > > > > > > blanket > > > > > > > trust of > > > > > > > connection header parameters from any client with a valid > > > > > > > certificate... > > > > > > > Certificates can carry additional information, including > > > > > > > usernames, > > > > > > > which > > > > > > > would be protected by the signature. Sometimes this is > > > > > > > done. > > > > > > > HBase > > > > > > > doesn't > > > > > > > implement it (yet) and if you have a trust architecture > > > > > > > that > > > > > > > demands > > > > > > > it, it > > > > > > > would establish the username as attested by an authority, > > > > > > > like > > > > > > > a > > > > > > > kerberos > > > > > > > ticket. You'd use a netty API presumably in a coprocessor > > > > > > > hook > > > > > > > to get > > > > > > > it in > > > > > > > order to hand off to UGI. Should someone decide this is > > > > > > > needed > > > > > > > it > > > > > > > would > > > > > > > make a good contribution back to the project, somewhere > > > > > > > in > > > > > > > hbase- > > > > > > > examples > > > > > > > perhaps. > > > > > > > > > > > > Exactly. That's what I wanted to clarify and pointed what > > > > > > other > > > > > > projects (like ZooKeeper) are doing under the hood of x509 > > > > > > authentication. I definitely would like to contribute > > > > > > something > > > > > > similar > > > > > > for HBase too. > > > > > > > > > > > > Thanks for the pointer of hbase-examples. > > > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
Re: mTLS / x509 authentication
Hi Andor, I don't believe there is a way for a coprocessor or other plugin to hook into connection establishment and reject connections. If you were interested, I imagine that contributing this change would be appreciated by the core team, but I don't speak for them. As a workaround for now, you would need to block specific operations via coprocessor implementations of the various *Observer interfaces like RegionObserver and others. Once there, you can call RpcServer.getCurrentCall() to get an RpcCallContext object. Charles On Tue, Aug 27, 2024 at 10:03 AM Andor Molnar wrote: > > Hi Andrew / Bryan, > > I'm looking at how to implement this client cert user/group > verification stuff that we talked about here, but I have to admit I > have no idea how to do this properly. Could you please give me some > pointers? > > In @charlesconnel's PR [1] about adding client cert chain to > RpcCallContext he mentions that it was needed for some kind of audit > logging, though in the Jira ticket [2] we'd like to extend the > authentication flow and prevent users from connecting with invalid > certificates. > > The closest I've found so far is the AccessController coprocessor which > implements _all_ interface methods to verify access. That is a bit > scary. Is there a way to catch all login attempts somehow and access > RpCallContext from there? > > Thanks, > Andor > > [1] https://github.com/apache/hbase/pull/5644 > [2] https://issues.apache.org/jira/browse/HBASE-27326 > > > > > On Mon, 2024-06-10 at 21:46 +0200, Andor Molnar wrote: > > Awesome. > > Thanks for all the input folks. > > > > Andor > > > > > > > > On Mon, 2024-06-10 at 15:24 -0400, Bryan Beaudreault wrote: > > > I'm back at my computer now so was able to find the JIRA: > > > https://issues.apache.org/jira/browse/HBASE-27326 > > > > > > I ended up not tackling it partially for the reason Andrew mentions > > > -- I > > > realized that even our use-case did not cleanly fit into the > > > proposal > > > there. So instead we ended up exposing the actual certificate chain > > > [1] in > > > the RpcCallContext, so that a user can make their own > > > implementation > > > using > > > coprocessors, etc to do whatever they like. > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-28317 > > > > > > On Mon, Jun 10, 2024 at 3:05 PM Andrew Purtell > > > > > > wrote: > > > > > > > Cool. > > > > I suggest hbase-examples because different organizations will > > > > embed > > > > a user > > > > name or potenitally other details into certificate fields > > > > differently, so > > > > there isn't a one size fits all solution. However HBase could > > > > provide a > > > > worked example where the username is in CN. > > > > > > > > On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar > > > > wrote: > > > > > > > > > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > > > > > > > > > > > There is middle ground between your position and a blanket > > > > > > trust of > > > > > > connection header parameters from any client with a valid > > > > > > certificate... > > > > > > Certificates can carry additional information, including > > > > > > usernames, > > > > > > which > > > > > > would be protected by the signature. Sometimes this is done. > > > > > > HBase > > > > > > doesn't > > > > > > implement it (yet) and if you have a trust architecture that > > > > > > demands > > > > > > it, it > > > > > > would establish the username as attested by an authority, > > > > > > like > > > > > > a > > > > > > kerberos > > > > > > ticket. You'd use a netty API presumably in a coprocessor > > > > > > hook > > > > > > to get > > > > > > it in > > > > > > order to hand off to UGI. Should someone decide this is > > > > > > needed > > > > > > it > > > > > > would > > > > > > make a good contribution back to the project, somewhere in > > > > > > hbase- > > > > > > examples > > > > > > perhaps. > > > > > > > > > > Exactly. That's what I wanted to clarify and pointed what other > > > > > projects (like ZooKeeper) are doing under the hood of x509 > > > > > authentication. I definitely would like to contribute something > > > > > similar > > > > > for HBase too. > > > > > > > > > > Thanks for the pointer of hbase-examples. > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell < > > > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Like I said the successful verification of the certificate > > > > > > > implies > > > > > > > the > > > > > > > client is trustworthy, including what the client supplies > > > > > > > in > > > > > > > the > > > > > > > header. > > > > > > > Now, if within your organization, you are distributing > > > > > > > trusted > > > > > > > certificates > > > > > > > to potentially untrustworthy software, that is a you > > > > > > > problem, > > > > > > > as > > > > > > > they say. > > > > > > > > > > > > > > It is absolutely not true that kerberos authentication is > > > > > >
Re: mTLS / x509 authentication
Hi Andrew / Bryan, I'm looking at how to implement this client cert user/group verification stuff that we talked about here, but I have to admit I have no idea how to do this properly. Could you please give me some pointers? In @charlesconnel's PR [1] about adding client cert chain to RpcCallContext he mentions that it was needed for some kind of audit logging, though in the Jira ticket [2] we'd like to extend the authentication flow and prevent users from connecting with invalid certificates. The closest I've found so far is the AccessController coprocessor which implements _all_ interface methods to verify access. That is a bit scary. Is there a way to catch all login attempts somehow and access RpCallContext from there? Thanks, Andor [1] https://github.com/apache/hbase/pull/5644 [2] https://issues.apache.org/jira/browse/HBASE-27326 On Mon, 2024-06-10 at 21:46 +0200, Andor Molnar wrote: > Awesome. > Thanks for all the input folks. > > Andor > > > > On Mon, 2024-06-10 at 15:24 -0400, Bryan Beaudreault wrote: > > I'm back at my computer now so was able to find the JIRA: > > https://issues.apache.org/jira/browse/HBASE-27326 > > > > I ended up not tackling it partially for the reason Andrew mentions > > -- I > > realized that even our use-case did not cleanly fit into the > > proposal > > there. So instead we ended up exposing the actual certificate chain > > [1] in > > the RpcCallContext, so that a user can make their own > > implementation > > using > > coprocessors, etc to do whatever they like. > > > > [1] https://issues.apache.org/jira/browse/HBASE-28317 > > > > On Mon, Jun 10, 2024 at 3:05 PM Andrew Purtell > > > > wrote: > > > > > Cool. > > > I suggest hbase-examples because different organizations will > > > embed > > > a user > > > name or potenitally other details into certificate fields > > > differently, so > > > there isn't a one size fits all solution. However HBase could > > > provide a > > > worked example where the username is in CN. > > > > > > On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar > > > wrote: > > > > > > > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > > > > > > > > > There is middle ground between your position and a blanket > > > > > trust of > > > > > connection header parameters from any client with a valid > > > > > certificate... > > > > > Certificates can carry additional information, including > > > > > usernames, > > > > > which > > > > > would be protected by the signature. Sometimes this is done. > > > > > HBase > > > > > doesn't > > > > > implement it (yet) and if you have a trust architecture that > > > > > demands > > > > > it, it > > > > > would establish the username as attested by an authority, > > > > > like > > > > > a > > > > > kerberos > > > > > ticket. You'd use a netty API presumably in a coprocessor > > > > > hook > > > > > to get > > > > > it in > > > > > order to hand off to UGI. Should someone decide this is > > > > > needed > > > > > it > > > > > would > > > > > make a good contribution back to the project, somewhere in > > > > > hbase- > > > > > examples > > > > > perhaps. > > > > > > > > Exactly. That's what I wanted to clarify and pointed what other > > > > projects (like ZooKeeper) are doing under the hood of x509 > > > > authentication. I definitely would like to contribute something > > > > similar > > > > for HBase too. > > > > > > > > Thanks for the pointer of hbase-examples. > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell < > > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > Like I said the successful verification of the certificate > > > > > > implies > > > > > > the > > > > > > client is trustworthy, including what the client supplies > > > > > > in > > > > > > the > > > > > > header. > > > > > > Now, if within your organization, you are distributing > > > > > > trusted > > > > > > certificates > > > > > > to potentially untrustworthy software, that is a you > > > > > > problem, > > > > > > as > > > > > > they say. > > > > > > > > > > > > It is absolutely not true that kerberos authentication is > > > > > > *required* when > > > > > > using mTLS. Frankly I'd prefer you expand on such comments > > > > > > to > > > > > > explain why > > > > > > you might have untrustworthy clients operating in your > > > > > > production > > > > > > as > > > > > > opposed to another organization with competent operations > > > > > > and > > > > > > better > > > > > > controls. Otherwise this flirts with F.U.D. > > > > > > > > > > > > > > > > > > On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar < > > > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Yeah. I think the key point here is that the client > > > > > > > certificate > > > > > > > identifies the originating "host" and not the "user", > > > > > > > hence > > > > > > > we > > > > > > > have the > > > > > > > client hostname verification built-in. The only thing >
Re: mTLS / x509 authentication
Awesome. Thanks for all the input folks. Andor On Mon, 2024-06-10 at 15:24 -0400, Bryan Beaudreault wrote: > I'm back at my computer now so was able to find the JIRA: > https://issues.apache.org/jira/browse/HBASE-27326 > > I ended up not tackling it partially for the reason Andrew mentions > -- I > realized that even our use-case did not cleanly fit into the proposal > there. So instead we ended up exposing the actual certificate chain > [1] in > the RpcCallContext, so that a user can make their own implementation > using > coprocessors, etc to do whatever they like. > > [1] https://issues.apache.org/jira/browse/HBASE-28317 > > On Mon, Jun 10, 2024 at 3:05 PM Andrew Purtell > wrote: > > > Cool. > > I suggest hbase-examples because different organizations will embed > > a user > > name or potenitally other details into certificate fields > > differently, so > > there isn't a one size fits all solution. However HBase could > > provide a > > worked example where the username is in CN. > > > > On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar > > wrote: > > > > > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > > > > > > > There is middle ground between your position and a blanket > > > > trust of > > > > connection header parameters from any client with a valid > > > > certificate... > > > > Certificates can carry additional information, including > > > > usernames, > > > > which > > > > would be protected by the signature. Sometimes this is done. > > > > HBase > > > > doesn't > > > > implement it (yet) and if you have a trust architecture that > > > > demands > > > > it, it > > > > would establish the username as attested by an authority, like > > > > a > > > > kerberos > > > > ticket. You'd use a netty API presumably in a coprocessor hook > > > > to get > > > > it in > > > > order to hand off to UGI. Should someone decide this is needed > > > > it > > > > would > > > > make a good contribution back to the project, somewhere in > > > > hbase- > > > > examples > > > > perhaps. > > > > > > Exactly. That's what I wanted to clarify and pointed what other > > > projects (like ZooKeeper) are doing under the hood of x509 > > > authentication. I definitely would like to contribute something > > > similar > > > for HBase too. > > > > > > Thanks for the pointer of hbase-examples. > > > > > > Andor > > > > > > > > > > > > > > > > On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell < > > > > [email protected]> > > > > wrote: > > > > > > > > > Like I said the successful verification of the certificate > > > > > implies > > > > > the > > > > > client is trustworthy, including what the client supplies in > > > > > the > > > > > header. > > > > > Now, if within your organization, you are distributing > > > > > trusted > > > > > certificates > > > > > to potentially untrustworthy software, that is a you problem, > > > > > as > > > > > they say. > > > > > > > > > > It is absolutely not true that kerberos authentication is > > > > > *required* when > > > > > using mTLS. Frankly I'd prefer you expand on such comments to > > > > > explain why > > > > > you might have untrustworthy clients operating in your > > > > > production > > > > > as > > > > > opposed to another organization with competent operations and > > > > > better > > > > > controls. Otherwise this flirts with F.U.D. > > > > > > > > > > > > > > > On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar < > > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > Yeah. I think the key point here is that the client > > > > > > certificate > > > > > > identifies the originating "host" and not the "user", hence > > > > > > we > > > > > > have the > > > > > > client hostname verification built-in. The only thing that > > > > > > you > > > > > > can be > > > > > > sure about when you do mTLS is that the request is coming > > > > > > from a > > > > > > legitimate host. > > > > > > > > > > > > Therefore you still need a secure authentication method in > > > > > > order > > > > > > to > > > > > > prevent legitimate users to impersonate each other. > > > > > > > > > > > > For instance, you have Alice and Bob valid users both > > > > > > having > > > > > > trusted > > > > > > client certificates. They both can make requests from > > > > > > legitimate > > > > > > hosts, > > > > > > but with SIMPLE auth they're free to lie about their valid > > > > > > usernames. > > > > > > > > > > > > As far as I know, the only secure authentication option for > > > > > > HBase > > > > > > is > > > > > > Kerberos, so you still have to use it. Using mTLS will > > > > > > prevent > > > > > > attackers from making requests from ordinary hosts by > > > > > > stoling > > > > > > Kerberos > > > > > > tokens. > > > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > > > > On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: > > > > > > > mTLS is totally unrelated to the username. It's whatever > > > > > > > you'd > > > > > > > typically > > > > > >
Re: mTLS / x509 authentication
I'm back at my computer now so was able to find the JIRA: https://issues.apache.org/jira/browse/HBASE-27326 I ended up not tackling it partially for the reason Andrew mentions -- I realized that even our use-case did not cleanly fit into the proposal there. So instead we ended up exposing the actual certificate chain [1] in the RpcCallContext, so that a user can make their own implementation using coprocessors, etc to do whatever they like. [1] https://issues.apache.org/jira/browse/HBASE-28317 On Mon, Jun 10, 2024 at 3:05 PM Andrew Purtell wrote: > Cool. > I suggest hbase-examples because different organizations will embed a user > name or potenitally other details into certificate fields differently, so > there isn't a one size fits all solution. However HBase could provide a > worked example where the username is in CN. > > On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar wrote: > > > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > > > > > There is middle ground between your position and a blanket trust of > > > connection header parameters from any client with a valid > > > certificate... > > > Certificates can carry additional information, including usernames, > > > which > > > would be protected by the signature. Sometimes this is done. HBase > > > doesn't > > > implement it (yet) and if you have a trust architecture that demands > > > it, it > > > would establish the username as attested by an authority, like a > > > kerberos > > > ticket. You'd use a netty API presumably in a coprocessor hook to get > > > it in > > > order to hand off to UGI. Should someone decide this is needed it > > > would > > > make a good contribution back to the project, somewhere in hbase- > > > examples > > > perhaps. > > > > Exactly. That's what I wanted to clarify and pointed what other > > projects (like ZooKeeper) are doing under the hood of x509 > > authentication. I definitely would like to contribute something similar > > for HBase too. > > > > Thanks for the pointer of hbase-examples. > > > > Andor > > > > > > > > > > > > > > On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell > > > wrote: > > > > > > > Like I said the successful verification of the certificate implies > > > > the > > > > client is trustworthy, including what the client supplies in the > > > > header. > > > > Now, if within your organization, you are distributing trusted > > > > certificates > > > > to potentially untrustworthy software, that is a you problem, as > > > > they say. > > > > > > > > It is absolutely not true that kerberos authentication is > > > > *required* when > > > > using mTLS. Frankly I'd prefer you expand on such comments to > > > > explain why > > > > you might have untrustworthy clients operating in your production > > > > as > > > > opposed to another organization with competent operations and > > > > better > > > > controls. Otherwise this flirts with F.U.D. > > > > > > > > > > > > On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar > > > > wrote: > > > > > > > > > Yeah. I think the key point here is that the client certificate > > > > > identifies the originating "host" and not the "user", hence we > > > > > have the > > > > > client hostname verification built-in. The only thing that you > > > > > can be > > > > > sure about when you do mTLS is that the request is coming from a > > > > > legitimate host. > > > > > > > > > > Therefore you still need a secure authentication method in order > > > > > to > > > > > prevent legitimate users to impersonate each other. > > > > > > > > > > For instance, you have Alice and Bob valid users both having > > > > > trusted > > > > > client certificates. They both can make requests from legitimate > > > > > hosts, > > > > > but with SIMPLE auth they're free to lie about their valid > > > > > usernames. > > > > > > > > > > As far as I know, the only secure authentication option for HBase > > > > > is > > > > > Kerberos, so you still have to use it. Using mTLS will prevent > > > > > attackers from making requests from ordinary hosts by stoling > > > > > Kerberos > > > > > tokens. > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: > > > > > > mTLS is totally unrelated to the username. It's whatever you'd > > > > > > typically > > > > > > have without mTLS. > > > > > > > > > > > > On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar > > > > > > wrote: > > > > > > > > > > > > > That is a completely fair point and I agree that from > > > > > > > security > > > > > > > perspective, the approach is safe enough. > > > > > > > > > > > > > > I'd just like to figure out what is the username in this > > > > > > > case? > > > > > > > Linux > > > > > > > user id? Anything that comes from SASL layer based on the > > > > > > > Hadoop > > > > > > > stack? > > > > > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: > > > > > > > > Most us
Re: mTLS / x509 authentication
Cool. I suggest hbase-examples because different organizations will embed a user name or potenitally other details into certificate fields differently, so there isn't a one size fits all solution. However HBase could provide a worked example where the username is in CN. On Mon, Jun 10, 2024 at 12:32 AM Andor Molnar wrote: > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > > > There is middle ground between your position and a blanket trust of > > connection header parameters from any client with a valid > > certificate... > > Certificates can carry additional information, including usernames, > > which > > would be protected by the signature. Sometimes this is done. HBase > > doesn't > > implement it (yet) and if you have a trust architecture that demands > > it, it > > would establish the username as attested by an authority, like a > > kerberos > > ticket. You'd use a netty API presumably in a coprocessor hook to get > > it in > > order to hand off to UGI. Should someone decide this is needed it > > would > > make a good contribution back to the project, somewhere in hbase- > > examples > > perhaps. > > Exactly. That's what I wanted to clarify and pointed what other > projects (like ZooKeeper) are doing under the hood of x509 > authentication. I definitely would like to contribute something similar > for HBase too. > > Thanks for the pointer of hbase-examples. > > Andor > > > > > > > > On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell > > wrote: > > > > > Like I said the successful verification of the certificate implies > > > the > > > client is trustworthy, including what the client supplies in the > > > header. > > > Now, if within your organization, you are distributing trusted > > > certificates > > > to potentially untrustworthy software, that is a you problem, as > > > they say. > > > > > > It is absolutely not true that kerberos authentication is > > > *required* when > > > using mTLS. Frankly I'd prefer you expand on such comments to > > > explain why > > > you might have untrustworthy clients operating in your production > > > as > > > opposed to another organization with competent operations and > > > better > > > controls. Otherwise this flirts with F.U.D. > > > > > > > > > On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar > > > wrote: > > > > > > > Yeah. I think the key point here is that the client certificate > > > > identifies the originating "host" and not the "user", hence we > > > > have the > > > > client hostname verification built-in. The only thing that you > > > > can be > > > > sure about when you do mTLS is that the request is coming from a > > > > legitimate host. > > > > > > > > Therefore you still need a secure authentication method in order > > > > to > > > > prevent legitimate users to impersonate each other. > > > > > > > > For instance, you have Alice and Bob valid users both having > > > > trusted > > > > client certificates. They both can make requests from legitimate > > > > hosts, > > > > but with SIMPLE auth they're free to lie about their valid > > > > usernames. > > > > > > > > As far as I know, the only secure authentication option for HBase > > > > is > > > > Kerberos, so you still have to use it. Using mTLS will prevent > > > > attackers from making requests from ordinary hosts by stoling > > > > Kerberos > > > > tokens. > > > > > > > > Andor > > > > > > > > > > > > > > > > On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: > > > > > mTLS is totally unrelated to the username. It's whatever you'd > > > > > typically > > > > > have without mTLS. > > > > > > > > > > On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar > > > > > wrote: > > > > > > > > > > > That is a completely fair point and I agree that from > > > > > > security > > > > > > perspective, the approach is safe enough. > > > > > > > > > > > > I'd just like to figure out what is the username in this > > > > > > case? > > > > > > Linux > > > > > > user id? Anything that comes from SASL layer based on the > > > > > > Hadoop > > > > > > stack? > > > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: > > > > > > > Most users who would employ a mTLS authentication scheme > > > > > > > would > > > > > > > operate with this trust model. The fact the client has a > > > > > > > valid > > > > > > > signed > > > > > > > certificate means it can be trusted, and that trust > > > > > > > includes > > > > > > > supplied > > > > > > > connection metadata like username. Or, if not, then not. > > > > > > > So then a lot of security engineering effort goes in to > > > > > > > protecting > > > > > > > the trust established by certificate distribution, like > > > > > > > using > > > > > > > short > > > > > > > lived certs, and secure distribution methods. > > > > > > > > > > > > > > > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < > > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > > > You're sort of
Re: mTLS / x509 authentication
It probably helps here to recognize the similarities in management of client certs in the usual mTLS architectures and the Kerberos TGT. Both are short lived credentials that the client obtains from an authority using some kind of secret that establishes its right to receive the credential and then that credential is a proxy for this trust for as long as it is valid. The cert or ticket is presented when a trusted connection is required and validated and this validation is a local cryptographic operation. The expenses related to granting trust are amortized over the short lifetime of the cert or ticket and especially the server doesn’t need to do something expensive to determine if the client is trustworthy. The risks are similar relating to an attacker stealing the cert or ticket for impersonation as are the countermeasures. > On Jun 10, 2024, at 12:27 AM, Andor Molnar wrote: > > Generally speaking clients are not trustworthy, otherwise we didn't > need security at all, right? > > Even if they're trustworthy, certificates and security credentials can > potentially be stolen. Following my previous example, Charlie is an > attacker and was able to break into Bob's account and has stolen his > credentials and certificates. With SIMPLE auth this is alone was enough > for Charlie to compromise _all_ accounts in the system, not just Bob's. > > Fixing myself: strictly speaking Kerberos is not *required*, but adds > additional level of security. > > Andor > > > >> On Sun, 2024-06-09 at 22:57 -0700, Andrew Purtell wrote: >> Like I said the successful verification of the certificate implies >> the >> client is trustworthy, including what the client supplies in the >> header. >> Now, if within your organization, you are distributing trusted >> certificates >> to potentially untrustworthy software, that is a you problem, as they >> say. >> >> It is absolutely not true that kerberos authentication is *required* >> when >> using mTLS. Frankly I'd prefer you expand on such comments to explain >> why >> you might have untrustworthy clients operating in your production as >> opposed to another organization with competent operations and better >> controls. Otherwise this flirts with F.U.D. >> >> >> On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar >> wrote: >> >>> Yeah. I think the key point here is that the client certificate >>> identifies the originating "host" and not the "user", hence we have >>> the >>> client hostname verification built-in. The only thing that you can >>> be >>> sure about when you do mTLS is that the request is coming from a >>> legitimate host. >>> >>> Therefore you still need a secure authentication method in order to >>> prevent legitimate users to impersonate each other. >>> >>> For instance, you have Alice and Bob valid users both having >>> trusted >>> client certificates. They both can make requests from legitimate >>> hosts, >>> but with SIMPLE auth they're free to lie about their valid >>> usernames. >>> >>> As far as I know, the only secure authentication option for HBase >>> is >>> Kerberos, so you still have to use it. Using mTLS will prevent >>> attackers from making requests from ordinary hosts by stoling >>> Kerberos >>> tokens. >>> >>> Andor >>> >>> >>> >>> On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: mTLS is totally unrelated to the username. It's whatever you'd typically have without mTLS. On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar wrote: > That is a completely fair point and I agree that from security > perspective, the approach is safe enough. > > I'd just like to figure out what is the username in this case? > Linux > user id? Anything that comes from SASL layer based on the > Hadoop > stack? > > Andor > > > > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: >> Most users who would employ a mTLS authentication scheme >> would >> operate with this trust model. The fact the client has a >> valid >> signed >> certificate means it can be trusted, and that trust includes >> supplied >> connection metadata like username. Or, if not, then not. >> So then a lot of security engineering effort goes in to >> protecting >> the trust established by certificate distribution, like using >> short >> lived certs, and secure distribution methods. >> >>> On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < >>> [email protected]> wrote: >>> >>> You're sort of correct. We've been using mTLS in prod for >>> a >>> while >>> now, ever >>> since the feature was committed. It's true that the actual >>> HBase >>> username >>> is not verified with mTLS, however you still can >>> authenticate >>> the >>> connection. The idea behind mTLS is that the certificate >>> carries >>> the >>> authentication -- so a client will need a
Re: mTLS / x509 authentication
We use mTLS with short lived certificates which are provided via an authenticated certificate authority service (CAS) and only work for a specific cluster. In order for someone to get a certificate, they must make a request to CAS which verifies they are allowed to have a certificate for the specific cluster. Once you have a certificate, it only works for a few hours. The client side will periodically provision a new one in a thread, and FileWatcher will swap it in. Further, with host verification the certificate only works on the given host it was provisioned on (a secure host generally) As I said, when I first planned this work I had considered a feature like you said. I decided not to do it for many of the reasons Andrew said. With the above design, we can trust our connections enough to just trust the username. The username on hbase isn’t used for much beyond table ACL, quotas, and bulk load verification. The only one that mattered for us is table ACL and actually it’s somewhat easier to manage this outside of hbase. We have a coprocessor which inspects the certificate for audit logging, and we can add a table field to the certificate which we could validate. There’s already a field on the RpcCall to get the current certificate. You could use that in a coprocessor to validate the username in your application, as Andrew mentions. On Mon, Jun 10, 2024 at 3:31 AM Andor Molnar wrote: > On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > > > There is middle ground between your position and a blanket trust of > > connection header parameters from any client with a valid > > certificate... > > Certificates can carry additional information, including usernames, > > which > > would be protected by the signature. Sometimes this is done. HBase > > doesn't > > implement it (yet) and if you have a trust architecture that demands > > it, it > > would establish the username as attested by an authority, like a > > kerberos > > ticket. You'd use a netty API presumably in a coprocessor hook to get > > it in > > order to hand off to UGI. Should someone decide this is needed it > > would > > make a good contribution back to the project, somewhere in hbase- > > examples > > perhaps. > > Exactly. That's what I wanted to clarify and pointed what other > projects (like ZooKeeper) are doing under the hood of x509 > authentication. I definitely would like to contribute something similar > for HBase too. > > Thanks for the pointer of hbase-examples. > > Andor > > > > > > > > On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell > > wrote: > > > > > Like I said the successful verification of the certificate implies > > > the > > > client is trustworthy, including what the client supplies in the > > > header. > > > Now, if within your organization, you are distributing trusted > > > certificates > > > to potentially untrustworthy software, that is a you problem, as > > > they say. > > > > > > It is absolutely not true that kerberos authentication is > > > *required* when > > > using mTLS. Frankly I'd prefer you expand on such comments to > > > explain why > > > you might have untrustworthy clients operating in your production > > > as > > > opposed to another organization with competent operations and > > > better > > > controls. Otherwise this flirts with F.U.D. > > > > > > > > > On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar > > > wrote: > > > > > > > Yeah. I think the key point here is that the client certificate > > > > identifies the originating "host" and not the "user", hence we > > > > have the > > > > client hostname verification built-in. The only thing that you > > > > can be > > > > sure about when you do mTLS is that the request is coming from a > > > > legitimate host. > > > > > > > > Therefore you still need a secure authentication method in order > > > > to > > > > prevent legitimate users to impersonate each other. > > > > > > > > For instance, you have Alice and Bob valid users both having > > > > trusted > > > > client certificates. They both can make requests from legitimate > > > > hosts, > > > > but with SIMPLE auth they're free to lie about their valid > > > > usernames. > > > > > > > > As far as I know, the only secure authentication option for HBase > > > > is > > > > Kerberos, so you still have to use it. Using mTLS will prevent > > > > attackers from making requests from ordinary hosts by stoling > > > > Kerberos > > > > tokens. > > > > > > > > Andor > > > > > > > > > > > > > > > > On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: > > > > > mTLS is totally unrelated to the username. It's whatever you'd > > > > > typically > > > > > have without mTLS. > > > > > > > > > > On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar > > > > > wrote: > > > > > > > > > > > That is a completely fair point and I agree that from > > > > > > security > > > > > > perspective, the approach is safe enough. > > > > > > > > > > > > I'd just like to figure out what is the username in this > > > > > > case? > > > > > >
Re: mTLS / x509 authentication
On Sun, 2024-06-09 at 23:10 -0700, Andrew Purtell wrote: > There is middle ground between your position and a blanket trust of > connection header parameters from any client with a valid > certificate... > Certificates can carry additional information, including usernames, > which > would be protected by the signature. Sometimes this is done. HBase > doesn't > implement it (yet) and if you have a trust architecture that demands > it, it > would establish the username as attested by an authority, like a > kerberos > ticket. You'd use a netty API presumably in a coprocessor hook to get > it in > order to hand off to UGI. Should someone decide this is needed it > would > make a good contribution back to the project, somewhere in hbase- > examples > perhaps. Exactly. That's what I wanted to clarify and pointed what other projects (like ZooKeeper) are doing under the hood of x509 authentication. I definitely would like to contribute something similar for HBase too. Thanks for the pointer of hbase-examples. Andor > > On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell > wrote: > > > Like I said the successful verification of the certificate implies > > the > > client is trustworthy, including what the client supplies in the > > header. > > Now, if within your organization, you are distributing trusted > > certificates > > to potentially untrustworthy software, that is a you problem, as > > they say. > > > > It is absolutely not true that kerberos authentication is > > *required* when > > using mTLS. Frankly I'd prefer you expand on such comments to > > explain why > > you might have untrustworthy clients operating in your production > > as > > opposed to another organization with competent operations and > > better > > controls. Otherwise this flirts with F.U.D. > > > > > > On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar > > wrote: > > > > > Yeah. I think the key point here is that the client certificate > > > identifies the originating "host" and not the "user", hence we > > > have the > > > client hostname verification built-in. The only thing that you > > > can be > > > sure about when you do mTLS is that the request is coming from a > > > legitimate host. > > > > > > Therefore you still need a secure authentication method in order > > > to > > > prevent legitimate users to impersonate each other. > > > > > > For instance, you have Alice and Bob valid users both having > > > trusted > > > client certificates. They both can make requests from legitimate > > > hosts, > > > but with SIMPLE auth they're free to lie about their valid > > > usernames. > > > > > > As far as I know, the only secure authentication option for HBase > > > is > > > Kerberos, so you still have to use it. Using mTLS will prevent > > > attackers from making requests from ordinary hosts by stoling > > > Kerberos > > > tokens. > > > > > > Andor > > > > > > > > > > > > On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: > > > > mTLS is totally unrelated to the username. It's whatever you'd > > > > typically > > > > have without mTLS. > > > > > > > > On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar > > > > wrote: > > > > > > > > > That is a completely fair point and I agree that from > > > > > security > > > > > perspective, the approach is safe enough. > > > > > > > > > > I'd just like to figure out what is the username in this > > > > > case? > > > > > Linux > > > > > user id? Anything that comes from SASL layer based on the > > > > > Hadoop > > > > > stack? > > > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: > > > > > > Most users who would employ a mTLS authentication scheme > > > > > > would > > > > > > operate with this trust model. The fact the client has a > > > > > > valid > > > > > > signed > > > > > > certificate means it can be trusted, and that trust > > > > > > includes > > > > > > supplied > > > > > > connection metadata like username. Or, if not, then not. > > > > > > So then a lot of security engineering effort goes in to > > > > > > protecting > > > > > > the trust established by certificate distribution, like > > > > > > using > > > > > > short > > > > > > lived certs, and secure distribution methods. > > > > > > > > > > > > > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < > > > > > > > [email protected]> wrote: > > > > > > > > > > > > > > You're sort of correct. We've been using mTLS in prod > > > > > > > for a > > > > > > > while > > > > > > > now, ever > > > > > > > since the feature was committed. It's true that the > > > > > > > actual > > > > > > > HBase > > > > > > > username > > > > > > > is not verified with mTLS, however you still can > > > > > > > authenticate > > > > > > > the > > > > > > > connection. The idea behind mTLS is that the certificate > > > > > > > carries > > > > > > > the > > > > > > > authentication -- so a client will need a certificate > > > > > > > which has > > > > > > > been sig
Re: mTLS / x509 authentication
Generally speaking clients are not trustworthy, otherwise we didn't need security at all, right? Even if they're trustworthy, certificates and security credentials can potentially be stolen. Following my previous example, Charlie is an attacker and was able to break into Bob's account and has stolen his credentials and certificates. With SIMPLE auth this is alone was enough for Charlie to compromise _all_ accounts in the system, not just Bob's. Fixing myself: strictly speaking Kerberos is not *required*, but adds additional level of security. Andor On Sun, 2024-06-09 at 22:57 -0700, Andrew Purtell wrote: > Like I said the successful verification of the certificate implies > the > client is trustworthy, including what the client supplies in the > header. > Now, if within your organization, you are distributing trusted > certificates > to potentially untrustworthy software, that is a you problem, as they > say. > > It is absolutely not true that kerberos authentication is *required* > when > using mTLS. Frankly I'd prefer you expand on such comments to explain > why > you might have untrustworthy clients operating in your production as > opposed to another organization with competent operations and better > controls. Otherwise this flirts with F.U.D. > > > On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar > wrote: > > > Yeah. I think the key point here is that the client certificate > > identifies the originating "host" and not the "user", hence we have > > the > > client hostname verification built-in. The only thing that you can > > be > > sure about when you do mTLS is that the request is coming from a > > legitimate host. > > > > Therefore you still need a secure authentication method in order to > > prevent legitimate users to impersonate each other. > > > > For instance, you have Alice and Bob valid users both having > > trusted > > client certificates. They both can make requests from legitimate > > hosts, > > but with SIMPLE auth they're free to lie about their valid > > usernames. > > > > As far as I know, the only secure authentication option for HBase > > is > > Kerberos, so you still have to use it. Using mTLS will prevent > > attackers from making requests from ordinary hosts by stoling > > Kerberos > > tokens. > > > > Andor > > > > > > > > On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: > > > mTLS is totally unrelated to the username. It's whatever you'd > > > typically > > > have without mTLS. > > > > > > On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar > > > wrote: > > > > > > > That is a completely fair point and I agree that from security > > > > perspective, the approach is safe enough. > > > > > > > > I'd just like to figure out what is the username in this case? > > > > Linux > > > > user id? Anything that comes from SASL layer based on the > > > > Hadoop > > > > stack? > > > > > > > > Andor > > > > > > > > > > > > > > > > > > > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: > > > > > Most users who would employ a mTLS authentication scheme > > > > > would > > > > > operate with this trust model. The fact the client has a > > > > > valid > > > > > signed > > > > > certificate means it can be trusted, and that trust includes > > > > > supplied > > > > > connection metadata like username. Or, if not, then not. > > > > > So then a lot of security engineering effort goes in to > > > > > protecting > > > > > the trust established by certificate distribution, like using > > > > > short > > > > > lived certs, and secure distribution methods. > > > > > > > > > > > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < > > > > > > [email protected]> wrote: > > > > > > > > > > > > You're sort of correct. We've been using mTLS in prod for > > > > > > a > > > > > > while > > > > > > now, ever > > > > > > since the feature was committed. It's true that the actual > > > > > > HBase > > > > > > username > > > > > > is not verified with mTLS, however you still can > > > > > > authenticate > > > > > > the > > > > > > connection. The idea behind mTLS is that the certificate > > > > > > carries > > > > > > the > > > > > > authentication -- so a client will need a certificate which > > > > > > has > > > > > > been signed > > > > > > by the same CA (or at least within the CA chain) which > > > > > > signed > > > > > > the > > > > > > server's > > > > > > certificate, and vise versa. > > > > > > > > > > > > For us, if someone has a valid certificate and the mTLS > > > > > > authentication > > > > > > succeeds, then we just trust their username. Based on how > > > > > > we > > > > > > use > > > > > > HBase in > > > > > > our environment, this is perfectly secure for our use-case. > > > > > > That > > > > > > may not > > > > > > work for everyone, and I did file a jira to add a feature > > > > > > for > > > > > > validating > > > > > > the username (perhaps pulling from a custom certificate > > > > > > property). > > > > > > But I > > > > > > haven't actually implemen
Re: mTLS / x509 authentication
Also, before this thread goes off on this particular tangent, I'd like to go back to something Bryan mentioned in his earlier response. > Based on how we use HBase in > our environment, this is perfectly secure for our use-case. That may not > work for everyone, and I did file a jira to add a feature for validating > the username (perhaps pulling from a custom certificate property). There is middle ground between your position and a blanket trust of connection header parameters from any client with a valid certificate... Certificates can carry additional information, including usernames, which would be protected by the signature. Sometimes this is done. HBase doesn't implement it (yet) and if you have a trust architecture that demands it, it would establish the username as attested by an authority, like a kerberos ticket. You'd use a netty API presumably in a coprocessor hook to get it in order to hand off to UGI. Should someone decide this is needed it would make a good contribution back to the project, somewhere in hbase-examples perhaps. On Sun, Jun 9, 2024 at 10:57 PM Andrew Purtell wrote: > Like I said the successful verification of the certificate implies the > client is trustworthy, including what the client supplies in the header. > Now, if within your organization, you are distributing trusted certificates > to potentially untrustworthy software, that is a you problem, as they say. > > It is absolutely not true that kerberos authentication is *required* when > using mTLS. Frankly I'd prefer you expand on such comments to explain why > you might have untrustworthy clients operating in your production as > opposed to another organization with competent operations and better > controls. Otherwise this flirts with F.U.D. > > > On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar wrote: > >> Yeah. I think the key point here is that the client certificate >> identifies the originating "host" and not the "user", hence we have the >> client hostname verification built-in. The only thing that you can be >> sure about when you do mTLS is that the request is coming from a >> legitimate host. >> >> Therefore you still need a secure authentication method in order to >> prevent legitimate users to impersonate each other. >> >> For instance, you have Alice and Bob valid users both having trusted >> client certificates. They both can make requests from legitimate hosts, >> but with SIMPLE auth they're free to lie about their valid usernames. >> >> As far as I know, the only secure authentication option for HBase is >> Kerberos, so you still have to use it. Using mTLS will prevent >> attackers from making requests from ordinary hosts by stoling Kerberos >> tokens. >> >> Andor >> >> >> >> On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: >> > mTLS is totally unrelated to the username. It's whatever you'd >> > typically >> > have without mTLS. >> > >> > On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar wrote: >> > >> > > That is a completely fair point and I agree that from security >> > > perspective, the approach is safe enough. >> > > >> > > I'd just like to figure out what is the username in this case? >> > > Linux >> > > user id? Anything that comes from SASL layer based on the Hadoop >> > > stack? >> > > >> > > Andor >> > > >> > > >> > > >> > > >> > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: >> > > > Most users who would employ a mTLS authentication scheme would >> > > > operate with this trust model. The fact the client has a valid >> > > > signed >> > > > certificate means it can be trusted, and that trust includes >> > > > supplied >> > > > connection metadata like username. Or, if not, then not. >> > > > So then a lot of security engineering effort goes in to >> > > > protecting >> > > > the trust established by certificate distribution, like using >> > > > short >> > > > lived certs, and secure distribution methods. >> > > > >> > > > > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < >> > > > > [email protected]> wrote: >> > > > > >> > > > > You're sort of correct. We've been using mTLS in prod for a >> > > > > while >> > > > > now, ever >> > > > > since the feature was committed. It's true that the actual >> > > > > HBase >> > > > > username >> > > > > is not verified with mTLS, however you still can authenticate >> > > > > the >> > > > > connection. The idea behind mTLS is that the certificate >> > > > > carries >> > > > > the >> > > > > authentication -- so a client will need a certificate which has >> > > > > been signed >> > > > > by the same CA (or at least within the CA chain) which signed >> > > > > the >> > > > > server's >> > > > > certificate, and vise versa. >> > > > > >> > > > > For us, if someone has a valid certificate and the mTLS >> > > > > authentication >> > > > > succeeds, then we just trust their username. Based on how we >> > > > > use >> > > > > HBase in >> > > > > our environment, this is perfectly secure for our use-case. >> > > > > That >> > > > > may no
Re: mTLS / x509 authentication
Like I said the successful verification of the certificate implies the client is trustworthy, including what the client supplies in the header. Now, if within your organization, you are distributing trusted certificates to potentially untrustworthy software, that is a you problem, as they say. It is absolutely not true that kerberos authentication is *required* when using mTLS. Frankly I'd prefer you expand on such comments to explain why you might have untrustworthy clients operating in your production as opposed to another organization with competent operations and better controls. Otherwise this flirts with F.U.D. On Sun, Jun 9, 2024 at 10:07 PM Andor Molnar wrote: > Yeah. I think the key point here is that the client certificate > identifies the originating "host" and not the "user", hence we have the > client hostname verification built-in. The only thing that you can be > sure about when you do mTLS is that the request is coming from a > legitimate host. > > Therefore you still need a secure authentication method in order to > prevent legitimate users to impersonate each other. > > For instance, you have Alice and Bob valid users both having trusted > client certificates. They both can make requests from legitimate hosts, > but with SIMPLE auth they're free to lie about their valid usernames. > > As far as I know, the only secure authentication option for HBase is > Kerberos, so you still have to use it. Using mTLS will prevent > attackers from making requests from ordinary hosts by stoling Kerberos > tokens. > > Andor > > > > On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: > > mTLS is totally unrelated to the username. It's whatever you'd > > typically > > have without mTLS. > > > > On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar wrote: > > > > > That is a completely fair point and I agree that from security > > > perspective, the approach is safe enough. > > > > > > I'd just like to figure out what is the username in this case? > > > Linux > > > user id? Anything that comes from SASL layer based on the Hadoop > > > stack? > > > > > > Andor > > > > > > > > > > > > > > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: > > > > Most users who would employ a mTLS authentication scheme would > > > > operate with this trust model. The fact the client has a valid > > > > signed > > > > certificate means it can be trusted, and that trust includes > > > > supplied > > > > connection metadata like username. Or, if not, then not. > > > > So then a lot of security engineering effort goes in to > > > > protecting > > > > the trust established by certificate distribution, like using > > > > short > > > > lived certs, and secure distribution methods. > > > > > > > > > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < > > > > > [email protected]> wrote: > > > > > > > > > > You're sort of correct. We've been using mTLS in prod for a > > > > > while > > > > > now, ever > > > > > since the feature was committed. It's true that the actual > > > > > HBase > > > > > username > > > > > is not verified with mTLS, however you still can authenticate > > > > > the > > > > > connection. The idea behind mTLS is that the certificate > > > > > carries > > > > > the > > > > > authentication -- so a client will need a certificate which has > > > > > been signed > > > > > by the same CA (or at least within the CA chain) which signed > > > > > the > > > > > server's > > > > > certificate, and vise versa. > > > > > > > > > > For us, if someone has a valid certificate and the mTLS > > > > > authentication > > > > > succeeds, then we just trust their username. Based on how we > > > > > use > > > > > HBase in > > > > > our environment, this is perfectly secure for our use-case. > > > > > That > > > > > may not > > > > > work for everyone, and I did file a jira to add a feature for > > > > > validating > > > > > the username (perhaps pulling from a custom certificate > > > > > property). > > > > > But I > > > > > haven't actually implemented that, and not sure that I will > > > > > since > > > > > it works > > > > > as-is for us. > > > > > > > > > > I'm on mobile now so I can't find it, but it should be findable > > > > > in > > > > > jira if > > > > > you search the tls-related tickets > > > > > > > > > > > On Fri, Jun 7, 2024 at 8:53 AM Andor Molnar > > > > > > > > > > > > wrote: > > > > > > > > > > > > Hi Bryan / Hbase devs, > > > > > > > > > > > > Based on the changes when you added mTLS support in HBASE- > > > > > > 27280 > > > > > > [1], > > > > > > only the certificate and hostname verification part were > > > > > > added to > > > > > > the > > > > > > codebase. HBase doesn't actually authenticates the user when > > > > > > mTLS > > > > > > is > > > > > > being used. > > > > > > > > > > > > In other words some other auth method Simple or Kerberos is > > > > > > still > > > > > > needed to identify the HBase user, because mTLS doesn't > > > > > > extract > > > > > > identity information from the client c
Re: mTLS / x509 authentication
Yeah. I think the key point here is that the client certificate identifies the originating "host" and not the "user", hence we have the client hostname verification built-in. The only thing that you can be sure about when you do mTLS is that the request is coming from a legitimate host. Therefore you still need a secure authentication method in order to prevent legitimate users to impersonate each other. For instance, you have Alice and Bob valid users both having trusted client certificates. They both can make requests from legitimate hosts, but with SIMPLE auth they're free to lie about their valid usernames. As far as I know, the only secure authentication option for HBase is Kerberos, so you still have to use it. Using mTLS will prevent attackers from making requests from ordinary hosts by stoling Kerberos tokens. Andor On Sun, 2024-06-09 at 13:45 -0400, Bryan Beaudreault wrote: > mTLS is totally unrelated to the username. It's whatever you'd > typically > have without mTLS. > > On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar wrote: > > > That is a completely fair point and I agree that from security > > perspective, the approach is safe enough. > > > > I'd just like to figure out what is the username in this case? > > Linux > > user id? Anything that comes from SASL layer based on the Hadoop > > stack? > > > > Andor > > > > > > > > > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: > > > Most users who would employ a mTLS authentication scheme would > > > operate with this trust model. The fact the client has a valid > > > signed > > > certificate means it can be trusted, and that trust includes > > > supplied > > > connection metadata like username. Or, if not, then not. > > > So then a lot of security engineering effort goes in to > > > protecting > > > the trust established by certificate distribution, like using > > > short > > > lived certs, and secure distribution methods. > > > > > > > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < > > > > [email protected]> wrote: > > > > > > > > You're sort of correct. We've been using mTLS in prod for a > > > > while > > > > now, ever > > > > since the feature was committed. It's true that the actual > > > > HBase > > > > username > > > > is not verified with mTLS, however you still can authenticate > > > > the > > > > connection. The idea behind mTLS is that the certificate > > > > carries > > > > the > > > > authentication -- so a client will need a certificate which has > > > > been signed > > > > by the same CA (or at least within the CA chain) which signed > > > > the > > > > server's > > > > certificate, and vise versa. > > > > > > > > For us, if someone has a valid certificate and the mTLS > > > > authentication > > > > succeeds, then we just trust their username. Based on how we > > > > use > > > > HBase in > > > > our environment, this is perfectly secure for our use-case. > > > > That > > > > may not > > > > work for everyone, and I did file a jira to add a feature for > > > > validating > > > > the username (perhaps pulling from a custom certificate > > > > property). > > > > But I > > > > haven't actually implemented that, and not sure that I will > > > > since > > > > it works > > > > as-is for us. > > > > > > > > I'm on mobile now so I can't find it, but it should be findable > > > > in > > > > jira if > > > > you search the tls-related tickets > > > > > > > > > On Fri, Jun 7, 2024 at 8:53 AM Andor Molnar > > > > > > > > > > wrote: > > > > > > > > > > Hi Bryan / Hbase devs, > > > > > > > > > > Based on the changes when you added mTLS support in HBASE- > > > > > 27280 > > > > > [1], > > > > > only the certificate and hostname verification part were > > > > > added to > > > > > the > > > > > codebase. HBase doesn't actually authenticates the user when > > > > > mTLS > > > > > is > > > > > being used. > > > > > > > > > > In other words some other auth method Simple or Kerberos is > > > > > still > > > > > needed to identify the HBase user, because mTLS doesn't > > > > > extract > > > > > identity information from the client certificate and doesn't > > > > > map > > > > > it to > > > > > an active HBase user. > > > > > > > > > > Is that correct? > > > > > > > > > > Regards, > > > > > Andor > > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-27280 > > > > > > > > > > > > > > > > > > > >
Re: mTLS / x509 authentication
mTLS is totally unrelated to the username. It's whatever you'd typically have without mTLS. On Sun, Jun 9, 2024 at 1:38 PM Andor Molnar wrote: > That is a completely fair point and I agree that from security > perspective, the approach is safe enough. > > I'd just like to figure out what is the username in this case? Linux > user id? Anything that comes from SASL layer based on the Hadoop stack? > > Andor > > > > > On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: > > Most users who would employ a mTLS authentication scheme would > > operate with this trust model. The fact the client has a valid signed > > certificate means it can be trusted, and that trust includes supplied > > connection metadata like username. Or, if not, then not. > > So then a lot of security engineering effort goes in to protecting > > the trust established by certificate distribution, like using short > > lived certs, and secure distribution methods. > > > > > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < > > > [email protected]> wrote: > > > > > > You're sort of correct. We've been using mTLS in prod for a while > > > now, ever > > > since the feature was committed. It's true that the actual HBase > > > username > > > is not verified with mTLS, however you still can authenticate the > > > connection. The idea behind mTLS is that the certificate carries > > > the > > > authentication -- so a client will need a certificate which has > > > been signed > > > by the same CA (or at least within the CA chain) which signed the > > > server's > > > certificate, and vise versa. > > > > > > For us, if someone has a valid certificate and the mTLS > > > authentication > > > succeeds, then we just trust their username. Based on how we use > > > HBase in > > > our environment, this is perfectly secure for our use-case. That > > > may not > > > work for everyone, and I did file a jira to add a feature for > > > validating > > > the username (perhaps pulling from a custom certificate property). > > > But I > > > haven't actually implemented that, and not sure that I will since > > > it works > > > as-is for us. > > > > > > I'm on mobile now so I can't find it, but it should be findable in > > > jira if > > > you search the tls-related tickets > > > > > > > On Fri, Jun 7, 2024 at 8:53 AM Andor Molnar > > > > wrote: > > > > > > > > Hi Bryan / Hbase devs, > > > > > > > > Based on the changes when you added mTLS support in HBASE-27280 > > > > [1], > > > > only the certificate and hostname verification part were added to > > > > the > > > > codebase. HBase doesn't actually authenticates the user when mTLS > > > > is > > > > being used. > > > > > > > > In other words some other auth method Simple or Kerberos is still > > > > needed to identify the HBase user, because mTLS doesn't extract > > > > identity information from the client certificate and doesn't map > > > > it to > > > > an active HBase user. > > > > > > > > Is that correct? > > > > > > > > Regards, > > > > Andor > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-27280 > > > > > > > > > > > > > > > > > >
Re: mTLS / x509 authentication
That is a completely fair point and I agree that from security perspective, the approach is safe enough. I'd just like to figure out what is the username in this case? Linux user id? Anything that comes from SASL layer based on the Hadoop stack? Andor On Fri, 2024-06-07 at 09:30 -0700, Andrew Purtell wrote: > Most users who would employ a mTLS authentication scheme would > operate with this trust model. The fact the client has a valid signed > certificate means it can be trusted, and that trust includes supplied > connection metadata like username. Or, if not, then not. > So then a lot of security engineering effort goes in to protecting > the trust established by certificate distribution, like using short > lived certs, and secure distribution methods. > > > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault < > > [email protected]> wrote: > > > > You're sort of correct. We've been using mTLS in prod for a while > > now, ever > > since the feature was committed. It's true that the actual HBase > > username > > is not verified with mTLS, however you still can authenticate the > > connection. The idea behind mTLS is that the certificate carries > > the > > authentication -- so a client will need a certificate which has > > been signed > > by the same CA (or at least within the CA chain) which signed the > > server's > > certificate, and vise versa. > > > > For us, if someone has a valid certificate and the mTLS > > authentication > > succeeds, then we just trust their username. Based on how we use > > HBase in > > our environment, this is perfectly secure for our use-case. That > > may not > > work for everyone, and I did file a jira to add a feature for > > validating > > the username (perhaps pulling from a custom certificate property). > > But I > > haven't actually implemented that, and not sure that I will since > > it works > > as-is for us. > > > > I'm on mobile now so I can't find it, but it should be findable in > > jira if > > you search the tls-related tickets > > > > > On Fri, Jun 7, 2024 at 8:53 AM Andor Molnar > > > wrote: > > > > > > Hi Bryan / Hbase devs, > > > > > > Based on the changes when you added mTLS support in HBASE-27280 > > > [1], > > > only the certificate and hostname verification part were added to > > > the > > > codebase. HBase doesn't actually authenticates the user when mTLS > > > is > > > being used. > > > > > > In other words some other auth method Simple or Kerberos is still > > > needed to identify the HBase user, because mTLS doesn't extract > > > identity information from the client certificate and doesn't map > > > it to > > > an active HBase user. > > > > > > Is that correct? > > > > > > Regards, > > > Andor > > > > > > > > > [1] https://issues.apache.org/jira/browse/HBASE-27280 > > > > > > > > > > > >
Re: mTLS / x509 authentication
Most users who would employ a mTLS authentication scheme would operate with this trust model. The fact the client has a valid signed certificate means it can be trusted, and that trust includes supplied connection metadata like username. Or, if not, then not. So then a lot of security engineering effort goes in to protecting the trust established by certificate distribution, like using short lived certs, and secure distribution methods. > On Jun 7, 2024, at 6:34 AM, Bryan Beaudreault wrote: > > You're sort of correct. We've been using mTLS in prod for a while now, ever > since the feature was committed. It's true that the actual HBase username > is not verified with mTLS, however you still can authenticate the > connection. The idea behind mTLS is that the certificate carries the > authentication -- so a client will need a certificate which has been signed > by the same CA (or at least within the CA chain) which signed the server's > certificate, and vise versa. > > For us, if someone has a valid certificate and the mTLS authentication > succeeds, then we just trust their username. Based on how we use HBase in > our environment, this is perfectly secure for our use-case. That may not > work for everyone, and I did file a jira to add a feature for validating > the username (perhaps pulling from a custom certificate property). But I > haven't actually implemented that, and not sure that I will since it works > as-is for us. > > I'm on mobile now so I can't find it, but it should be findable in jira if > you search the tls-related tickets > >> On Fri, Jun 7, 2024 at 8:53 AM Andor Molnar wrote: >> >> Hi Bryan / Hbase devs, >> >> Based on the changes when you added mTLS support in HBASE-27280 [1], >> only the certificate and hostname verification part were added to the >> codebase. HBase doesn't actually authenticates the user when mTLS is >> being used. >> >> In other words some other auth method Simple or Kerberos is still >> needed to identify the HBase user, because mTLS doesn't extract >> identity information from the client certificate and doesn't map it to >> an active HBase user. >> >> Is that correct? >> >> Regards, >> Andor >> >> >> [1] https://issues.apache.org/jira/browse/HBASE-27280 >> >> >> >>
Re: mTLS / x509 authentication
You're sort of correct. We've been using mTLS in prod for a while now, ever since the feature was committed. It's true that the actual HBase username is not verified with mTLS, however you still can authenticate the connection. The idea behind mTLS is that the certificate carries the authentication -- so a client will need a certificate which has been signed by the same CA (or at least within the CA chain) which signed the server's certificate, and vise versa. For us, if someone has a valid certificate and the mTLS authentication succeeds, then we just trust their username. Based on how we use HBase in our environment, this is perfectly secure for our use-case. That may not work for everyone, and I did file a jira to add a feature for validating the username (perhaps pulling from a custom certificate property). But I haven't actually implemented that, and not sure that I will since it works as-is for us. I'm on mobile now so I can't find it, but it should be findable in jira if you search the tls-related tickets On Fri, Jun 7, 2024 at 8:53 AM Andor Molnar wrote: > Hi Bryan / Hbase devs, > > Based on the changes when you added mTLS support in HBASE-27280 [1], > only the certificate and hostname verification part were added to the > codebase. HBase doesn't actually authenticates the user when mTLS is > being used. > > In other words some other auth method Simple or Kerberos is still > needed to identify the HBase user, because mTLS doesn't extract > identity information from the client certificate and doesn't map it to > an active HBase user. > > Is that correct? > > Regards, > Andor > > > [1] https://issues.apache.org/jira/browse/HBASE-27280 > > > >
