In terms of TLS: - All of our clients (many thousands) in production are using the NettyRpcConnection with TLS enabled. However, these clients are currently connecting to the RegionServer/HMaster through an haproxy process local to each server which handles SSL termination. So not quite end-to-end yet. - On the server side, most of our QA environment (a thousand regionservers and ~200 hmasters) are running it. So these are accepting TLS from clients and using TLS for intra-cluster communication.
The migration is tricky for us due to the scale and the fact that we need to migrate off haproxy at the same time. Hopefully we should have some of production running end-to-end TLS within the next month or so. >From what we've seen in QA so far, there have not been any major issues. We also couldn't discern any performance issues in testing, though we were comparing against our legacy haproxy setup and can't really compare against kerberos. One outstanding issue is https://issues.apache.org/jira/browse/HBASE-27782, which we still see periodically. It doesn't seem to cause actual issues, since the RpcClient still handles it gracefully, but it does cause noise and may have implications. On Fri, Jun 16, 2023 at 11:41 AM 张铎(Duo Zhang) <[email protected]> wrote: > So any updates here? > > Do we have any good news about the TLS usage in production so we can > move forward on release 2.6.x? > > Thanks. > > Andrew Purtell <[email protected]> 于2023年4月7日周五 09:37写道： > > > > Agreed, that sounds like a good plan. > > > > On Wed, Mar 29, 2023 at 7:31 AM 张铎(Duo Zhang) <[email protected]> > wrote: > > > > > I think we could follow the old pattern when we cut a new release > branch. > > > That is, after the new release branch is cut and the new minor release > is > > > out, we will do a final release of the oldest release line and then > mark it > > > as EOL. > > > > > > So here, I think once we cut branch-2.6 and release 2.6.0, we can do a > > > final release for 2.4.x and mark 2.4.x as EOL. > > > > > > Thanks. > > > > > > Bryan Beaudreault <[email protected]> 于2023年3月27日周一 09:57写道： > > > > > > > Primary development on hbase-backup and TLS is complete. There are a > > > couple > > > > minor things I may want to add to TLS in the future, such as > pluggable > > > cert > > > > verification. But those are not needed for initial release IMO. > > > > > > > > We are almost ready integrating hbase-backup in production. We’ve > fixed a > > > > few minor things (all committed) but otherwise it’s worked well so > far in > > > > tests. > > > > > > > > We are a bit delayed in integrating TLS. I’m hopeful it will happen > in > > > the > > > > next 2-3 months. It’s a big project for us, so not quick, but > definitely > > > on > > > > the roadmap. > > > > > > > > It seems like cloudera may be closer to integrating TLS in > production. > > > > Balazs recently filed and fixed HBASE-27673 related to mTLS. Maybe > he can > > > > chime in on his status, or let me know if I am totally off base :) > > > > > > > > On Sun, Mar 26, 2023 at 9:25 PM Andrew Purtell < > [email protected] > > > > > > > > wrote: > > > > > > > > > Before we open a new code line should we discuss EOL of 2.4? After > the > > > > > first 2.6 release? It’s not required of course but cuts down the > amount > > > > of > > > > > labor to have two 2.x code lines (presumably, one as stable and > one as > > > > > next) rather than three. Perhaps even before that, should we move > the > > > > > stable pointer to the latest 2.5 release? > > > > > > > > > > > > > > > > > On Mar 26, 2023, at 5:59 PM, 张铎 <[email protected]> wrote: > > > > > > > > > > > > Bump. > > > > > > > > > > > > I believe the mTLS and backup related code have all been > finished on > > > > > > branch-2? > > > > > > > > > > > > Are there any other things which block us making the branch-2.6 > > > branch? > > > > > > > > > > > > Thanks. > > > > > > > > > > > > Mallikarjun <[email protected]> 于2022年10月17日周一 02:09写道： > > > > > > > > > > > >> On hbase-backup, we are using in production for more then 1 > year. I > > > > can > > > > > >> vouch for it to be stable enough to be in a release version so > that > > > > more > > > > > >> people can use it and polished it further. > > > > > >> > > > > > >>> On Sun, Oct 16, 2022, 11:25 PM Andrew Purtell < > > > > > [email protected]> > > > > > >>> wrote: > > > > > >>> > > > > > >>> My understanding is some folks evaluating and polishing TLS for > > > their > > > > > >>> production are also considering hbase-backup in the same way, > which > > > > is > > > > > >> why > > > > > >>> I linked them together. If that is incorrect then they both are > > > still > > > > > >> worth > > > > > >>> considering in my opinion but would have a more tenuous link. > > > > > >>> > > > > > >>> Where we are with hbase-backup is it should probably be ported > to > > > > where > > > > > >>> more people would be inclined to evaluate it, in order for it > to > > > make > > > > > >> more > > > > > >>> progress. A new minor releasing line would fit. On the other > hand > > > if > > > > it > > > > > >> is > > > > > >>> too unpolished then the experience would be poor. > > > > > >>> > > > > > >>> > > > > > >>>> On Oct 16, 2022, at 5:35 AM, 张铎 <[email protected]> > wrote: > > > > > >>>> > > > > > >>>> I believe the second one is still ongoing? > > > > > >>>> > > > > > >>>> Andrew Purtell <[email protected]> 于2022年10月14日周五 05:37写道： > > > > > >>>>> > > > > > >>>>> We will begin releasing activity for the 2.6 code line and > as a > > > > > >>>>> prerequisite to that we shall need to make a new branch > > > branch-2.6 > > > > > >> from > > > > > >>>>> branch-2. > > > > > >>>>> > > > > > >>>>> Before we do that let's make sure all commits for the key > > > features > > > > of > > > > > >>> 2.6 > > > > > >>>>> are settled in branch-2 before the branching point. Those key > > > > > features > > > > > >>> are: > > > > > >>>>> - mTLS RPC > > > > > >>>>> - hbase-backup backport > > > > > >>>>> > > > > > >>>>> -- > > > > > >>>>> Best regards, > > > > > >>>>> Andrew > > > > > >>> > > > > > >> > > > > > > > > > > > > > > > > > > -- > > Best regards, > > Andrew > > > > Unrest, ignorance distilled, nihilistic imbeciles - > > It's what we’ve earned > > Welcome, apocalypse, what’s taken you so long? > > Bring us the fitting end that we’ve been counting on > > - A23, Welcome, Apocalypse >
