Are there any more important or serious issues with branch-2 which are not resolved?
Planning to cut branch-2.6 from branch-2 on 9/5 and stabilize it for a 2.6.0 release from there. On Wed, Aug 23, 2023 at 7:41 PM 张铎(Duo Zhang) <[email protected]> wrote: > Let's get HBASE-28028 in too. Since Andrew is still on vacation, let > me find another committer to help review the PR. > > Thanks. > > Bryan Beaudreault <[email protected]> 于2023年8月24日周四 02:23写道： > > > > We resolved HBASE-28010, so we should be good to go there now. > > > > On Sun, Aug 20, 2023 at 7:22 AM Bryan Beaudreault < > [email protected]> > > wrote: > > > > > I made > > > https://issues.apache.org/jira/browse/HBASE-28010 a blocker so we > don’t > > > release a broken new feature. We will have it fixed early this week. > > > > > > On Fri, Aug 18, 2023 at 11:08 AM Andrew Purtell < > [email protected]> > > > wrote: > > > > > >> Let’s do it. We can use the new branch to stabilize for release. > > >> > > >> I am back from vacation in two weeks and can then lend time and AWS > > >> resources for testing (and presumably fixing). > > >> > > >> > On Aug 18, 2023, at 10:53 AM, 张铎 <[email protected]> wrote: > > >> > > > >> > HBASE-27947 has been resolved. > > >> > > > >> > So I think now we are good to cut branch-2.6? > > >> > > > >> > Another good news is we are also close to fix the WAL value > > >> > compression issue in HBASE-28028. > > >> > > > >> > Thanks. > > >> > > > >> > Bryan Beaudreault <[email protected]> 于2023年6月23日周五 02:47写道： > > >> >> > > >> >> Thanks! > > >> >> > > >> >> We're looking into one other emergent issue that we uncovered > during > > >> the > > >> >> rollout of server side TLS on RegionServers. It seems > > >> nettyDirectMemory has > > >> >> increased substantially when under load with it enabled. Details in > > >> >> https://issues.apache.org/jira/browse/HBASE-27947. > > >> >> > > >> >> > > >> >>> On Thu, Jun 22, 2023 at 12:02 PM 张铎(Duo Zhang) < > [email protected] > > >> > > > >> >>> wrote: > > >> >>> > > >> >>> PR is ready > > >> >>> > > >> >>> https://github.com/apache/hbase/pull/5305 > > >> >>> > > >> >>> PTAL. > > >> >>> > > >> >>> Thanks. > > >> >>> > > >> >>> 张铎(Duo Zhang) <[email protected]> 于2023年6月22日周四 21:40写道： > > >> >>>> > > >> >>>> Ah, missed your last comment on HBASE-27782. > > >> >>>> > > >> >>>> Let me take a look. > > >> >>>> > > >> >>>> Netty has some rules about how the exceptions are passed through > the > > >> >>>> pipeline(especially the order, forward or backward...) but > honestly I > > >> >>>> always forget it just a day later after I finished the code... > > >> >>>> > > >> >>>> Bryan Beaudreault <[email protected]> 于2023年6月17日周六 > 00:43写道： > > >> >>>>> > > >> >>>>> In terms of TLS: > > >> >>>>> > > >> >>>>> - All of our clients (many thousands) in production are using > the > > >> >>>>> NettyRpcConnection with TLS enabled. However, these clients are > > >> >>> currently > > >> >>>>> connecting to the RegionServer/HMaster through an haproxy > process > > >> >>> local to > > >> >>>>> each server which handles SSL termination. So not quite > end-to-end > > >> yet. > > >> >>>>> - On the server side, most of our QA environment (a thousand > > >> >>> regionservers > > >> >>>>> and ~200 hmasters) are running it. So these are accepting TLS > from > > >> >>> clients > > >> >>>>> and using TLS for intra-cluster communication. > > >> >>>>> > > >> >>>>> The migration is tricky for us due to the scale and the fact > that we > > >> >>> need > > >> >>>>> to migrate off haproxy at the same time. Hopefully we should > have > > >> some > > >> >>> of > > >> >>>>> production running end-to-end TLS within the next month or so. > > >> >>>>> > > >> >>>>> From what we've seen in QA so far, there have not been any major > > >> >>> issues. We > > >> >>>>> also couldn't discern any performance issues in testing, though > we > > >> were > > >> >>>>> comparing against our legacy haproxy setup and can't really > compare > > >> >>> against > > >> >>>>> kerberos. > > >> >>>>> > > >> >>>>> One outstanding issue is > > >> >>> https://issues.apache.org/jira/browse/HBASE-27782, > > >> >>>>> which we still see periodically. It doesn't seem to cause actual > > >> >>> issues, > > >> >>>>> since the RpcClient still handles it gracefully, but it does > cause > > >> >>> noise > > >> >>>>> and may have implications. > > >> >>>>> > > >> >>>>> On Fri, Jun 16, 2023 at 11:41 AM 张铎(Duo Zhang) < > > >> [email protected]> > > >> >>>>> wrote: > > >> >>>>> > > >> >>>>>> So any updates here? > > >> >>>>>> > > >> >>>>>> Do we have any good news about the TLS usage in production so > we > > >> can > > >> >>>>>> move forward on release 2.6.x? > > >> >>>>>> > > >> >>>>>> Thanks. > > >> >>>>>> > > >> >>>>>> Andrew Purtell <[email protected]> 于2023年4月7日周五 09:37写道： > > >> >>>>>>> > > >> >>>>>>> Agreed, that sounds like a good plan. > > >> >>>>>>> > > >> >>>>>>> On Wed, Mar 29, 2023 at 7:31 AM 张铎(Duo Zhang) < > > >> >>> [email protected]> > > >> >>>>>> wrote: > > >> >>>>>>> > > >> >>>>>>>> I think we could follow the old pattern when we cut a new > release > > >> >>>>>> branch. > > >> >>>>>>>> That is, after the new release branch is cut and the new > minor > > >> >>> release > > >> >>>>>> is > > >> >>>>>>>> out, we will do a final release of the oldest release line > and > > >> >>> then > > >> >>>>>> mark it > > >> >>>>>>>> as EOL. > > >> >>>>>>>> > > >> >>>>>>>> So here, I think once we cut branch-2.6 and release 2.6.0, we > > >> >>> can do a > > >> >>>>>>>> final release for 2.4.x and mark 2.4.x as EOL. > > >> >>>>>>>> > > >> >>>>>>>> Thanks. > > >> >>>>>>>> > > >> >>>>>>>> Bryan Beaudreault <[email protected]> 于2023年3月27日周一 > > >> >>> 09:57写道： > > >> >>>>>>>> > > >> >>>>>>>>> Primary development on hbase-backup and TLS is complete. > There > > >> >>> are a > > >> >>>>>>>> couple > > >> >>>>>>>>> minor things I may want to add to TLS in the future, such as > > >> >>>>>> pluggable > > >> >>>>>>>> cert > > >> >>>>>>>>> verification. But those are not needed for initial release > IMO. > > >> >>>>>>>>> > > >> >>>>>>>>> We are almost ready integrating hbase-backup in production. > > >> >>> We’ve > > >> >>>>>> fixed a > > >> >>>>>>>>> few minor things (all committed) but otherwise it’s worked > > >> >>> well so > > >> >>>>>> far in > > >> >>>>>>>>> tests. > > >> >>>>>>>>> > > >> >>>>>>>>> We are a bit delayed in integrating TLS. I’m hopeful it will > > >> >>> happen > > >> >>>>>> in > > >> >>>>>>>> the > > >> >>>>>>>>> next 2-3 months. It’s a big project for us, so not quick, > but > > >> >>>>>> definitely > > >> >>>>>>>> on > > >> >>>>>>>>> the roadmap. > > >> >>>>>>>>> > > >> >>>>>>>>> It seems like cloudera may be closer to integrating TLS in > > >> >>>>>> production. > > >> >>>>>>>>> Balazs recently filed and fixed HBASE-27673 related to mTLS. > > >> >>> Maybe > > >> >>>>>> he can > > >> >>>>>>>>> chime in on his status, or let me know if I am totally off > > >> >>> base :) > > >> >>>>>>>>> > > >> >>>>>>>>> On Sun, Mar 26, 2023 at 9:25 PM Andrew Purtell < > > >> >>>>>> [email protected] > > >> >>>>>>>>> > > >> >>>>>>>>> wrote: > > >> >>>>>>>>> > > >> >>>>>>>>>> Before we open a new code line should we discuss EOL of > 2.4? > > >> >>> After > > >> >>>>>> the > > >> >>>>>>>>>> first 2.6 release? It’s not required of course but cuts > down > > >> >>> the > > >> >>>>>> amount > > >> >>>>>>>>> of > > >> >>>>>>>>>> labor to have two 2.x code lines (presumably, one as stable > > >> >>> and > > >> >>>>>> one as > > >> >>>>>>>>>> next) rather than three. Perhaps even before that, should > we > > >> >>> move > > >> >>>>>> the > > >> >>>>>>>>>> stable pointer to the latest 2.5 release? > > >> >>>>>>>>>> > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> On Mar 26, 2023, at 5:59 PM, 张铎 <[email protected]> > > >> >>> wrote: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Bump. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> I believe the mTLS and backup related code have all been > > >> >>>>>> finished on > > >> >>>>>>>>>>> branch-2? > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Are there any other things which block us making the > > >> >>> branch-2.6 > > >> >>>>>>>> branch? > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Thanks. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> Mallikarjun <[email protected]> 于2022年10月17日周一 > > >> >>> 02:09写道： > > >> >>>>>>>>>>> > > >> >>>>>>>>>>>> On hbase-backup, we are using in production for more then > > >> >>> 1 > > >> >>>>>> year. I > > >> >>>>>>>>> can > > >> >>>>>>>>>>>> vouch for it to be stable enough to be in a release > > >> >>> version so > > >> >>>>>> that > > >> >>>>>>>>> more > > >> >>>>>>>>>>>> people can use it and polished it further. > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>>> On Sun, Oct 16, 2022, 11:25 PM Andrew Purtell < > > >> >>>>>>>>>> [email protected]> > > >> >>>>>>>>>>>>> wrote: > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> My understanding is some folks evaluating and polishing > > >> >>> TLS for > > >> >>>>>>>> their > > >> >>>>>>>>>>>>> production are also considering hbase-backup in the same > > >> >>> way, > > >> >>>>>> which > > >> >>>>>>>>> is > > >> >>>>>>>>>>>> why > > >> >>>>>>>>>>>>> I linked them together. If that is incorrect then they > > >> >>> both are > > >> >>>>>>>> still > > >> >>>>>>>>>>>> worth > > >> >>>>>>>>>>>>> considering in my opinion but would have a more tenuous > > >> >>> link. > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> Where we are with hbase-backup is it should probably be > > >> >>> ported > > >> >>>>>> to > > >> >>>>>>>>> where > > >> >>>>>>>>>>>>> more people would be inclined to evaluate it, in order > > >> >>> for it > > >> >>>>>> to > > >> >>>>>>>> make > > >> >>>>>>>>>>>> more > > >> >>>>>>>>>>>>> progress. A new minor releasing line would fit. On the > > >> >>> other > > >> >>>>>> hand > > >> >>>>>>>> if > > >> >>>>>>>>> it > > >> >>>>>>>>>>>> is > > >> >>>>>>>>>>>>> too unpolished then the experience would be poor. > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> On Oct 16, 2022, at 5:35 AM, 张铎 <[email protected] > > > > >> >>>>>> wrote: > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> I believe the second one is still ongoing? > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> Andrew Purtell <[email protected]> 于2022年10月14日周五 > > >> >>> 05:37写道： > > >> >>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>> We will begin releasing activity for the 2.6 code line > > >> >>> and > > >> >>>>>> as a > > >> >>>>>>>>>>>>>>> prerequisite to that we shall need to make a new > branch > > >> >>>>>>>> branch-2.6 > > >> >>>>>>>>>>>> from > > >> >>>>>>>>>>>>>>> branch-2. > > >> >>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>> Before we do that let's make sure all commits for the > > >> >>> key > > >> >>>>>>>> features > > >> >>>>>>>>> of > > >> >>>>>>>>>>>>> 2.6 > > >> >>>>>>>>>>>>>>> are settled in branch-2 before the branching point. > > >> >>> Those key > > >> >>>>>>>>>> features > > >> >>>>>>>>>>>>> are: > > >> >>>>>>>>>>>>>>> - mTLS RPC > > >> >>>>>>>>>>>>>>> - hbase-backup backport > > >> >>>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>>> -- > > >> >>>>>>>>>>>>>>> Best regards, > > >> >>>>>>>>>>>>>>> Andrew > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> > > >> >>>>>>> > > >> >>>>>>> -- > > >> >>>>>>> Best regards, > > >> >>>>>>> Andrew > > >> >>>>>>> > > >> >>>>>>> Unrest, ignorance distilled, nihilistic imbeciles - > > >> >>>>>>> It's what we’ve earned > > >> >>>>>>> Welcome, apocalypse, what’s taken you so long? > > >> >>>>>>> Bring us the fitting end that we’ve been counting on > > >> >>>>>>> - A23, Welcome, Apocalypse > > >> >>>>>> > > >> >>> > > >> > > > > -- Best regards, Andrew Unrest, ignorance distilled, nihilistic imbeciles - It's what we’ve earned Welcome, apocalypse, what’s taken you so long? Bring us the fitting end that we’ve been counting on - A23, Welcome, Apocalypse
