I like the last idea suggested by Stack. This way the core idea of keeping both the dev process separate stays intact and also the operators have a version of hbck to fix their clusters without worrying about compatibility issues.
On Thu, Aug 29, 2019 at 2:03 PM Stack <[email protected]> wrote: > Could also just make a release now of hbase-operator-tools (or in a week or > so when we should have hbck1+ coverage in place) built against an > up-to-date hbase release. It has the check version before running a feature > in place where it matters. Operators could use this on all currently > shipping hbase2s? > > S > > On Thu, Aug 29, 2019 at 11:42 AM Stack <[email protected]> wrote: > > > On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < > > [email protected]> wrote: > > > >> > > >> > I do not think we need to compile HBCK2 with every releases? > >> > > >> Well, not with every release, was thinking in doing it whenever an hbase > >> release breaks compatibility. > >> > >> We just need > >> > make sure that it can work with all the releases. > >> > >> This could be a solution as well, but I believe it would be harder to > >> guarantee. Here the problem is: > >> 1) A new hbase release changes one or more interfaces currently used by > >> hbck2; > >> 2) We update hbck2 to depend on this new hbase release, and change hbck2 > >> accordingly; > >> 3) Operators need to run hbck2 to a previous hbase release. If they try > to > >> build hbck2 against that version, it won't compile. If they build it > with > >> latest hbase version, it may give a runtime error, and now they have no > >> working tool to fix the problem. > >> To avoid #3, we would need to add extra checks on the changes applied on > >> #2. Might become too complex. > >> > >> Thanks Wellington for above. I see issue now. > > > > So, we should make retroactive releases of hbase-operator-tools at points > > just before compat broke? Release could be named for the hbase2 versions > > supported. Releases would make it easier on operators making it so they > > don't have to build themselves? There'd be one only? Two maybe? > > > > Looking at changes to the Hbck Interface -- using this as gauge for > > possible breakage points -- there aren't many. One release? Maybe two? > > > > S > > > > > > > >> If there are missing > >> > methods, we just tell users you can not use several features. > >> > >> Fine for new fix options added, but what if some changes break basic, > >> already working hbck2 methods. > >> > >> > >> Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) < > [email protected]> > >> escreveu: > >> > >> > I do not think we need to compile HBCK2 with every releases? We just > >> need > >> > make sure that it can work with all the releases. If there are missing > >> > methods, we just tell users you can not use several features. > >> > > >> > Wellington Chevreuil <[email protected]> 于2019年8月29日周四 > >> > 下午9:39写道： > >> > > >> > > > > >> > > > bq. what would folks think about going with an hbck2 alpha > release? > >> > > > I'm fine with an alpha release but since "HBCK2 should > continuously > >> > > evolve" > >> > > > it might be better to always use the latest codebase whenever you > >> need > >> > to > >> > > > use the tool. > >> > > > > >> > > Ideally yes, but that might not always be possible, as hbase API > might > >> > > change ahead of hbck2. Operators could then have problems to get a > >> > working > >> > > version of hbck2. Since hbck2 already has now many equivalent > options > >> for > >> > > the ones from hbck1, I guess a first release would provide a working > >> > hbck2 > >> > > that already brings a considerable number of fix methods to help > with > >> > most > >> > > common inconsistencies issues seen in hbase 2 so far. And we could > >> still > >> > > fulfil the "HBCK2 should continuously evolve" principle by building > >> from > >> > > its master branch, in scenarios where a new fix was needed and > >> > implemented > >> > > back into hbck2. > >> > > > >> > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi < > >> [email protected]> > >> > > escreveu: > >> > > > >> > > > bq. Is it possible to put some hacks into HBCK2 to work around > >> > > > the compatibility to fix the current state > >> > > > > >> > > > There are some classes around Replication which were introduced in > >> > 2.1.0+ > >> > > > so I don't think we could easily solve it for 2.0. > >> > > > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, > >> > > probably > >> > > > that could be solved with some workarounds or using reflection. > >> > > > > >> > > > bq. focus more on automation to let us know the next time we > >> inevitably > >> > > > break it again? ;) > >> > > > > >> > > > Sure! Based on this I think it should be a strong goal. We can set > >> up > >> > > > nightly builds for hbase-operator-tools repo that builds against > the > >> > > latest > >> > > > development branches as well as checking compatibility with > released > >> > > > versions. > >> > > > > >> > > > bq. what would folks think about going with an hbck2 alpha > release? > >> > > > I'm fine with an alpha release but since "HBCK2 should > continuously > >> > > evolve" > >> > > > it might be better to always use the latest codebase whenever you > >> need > >> > to > >> > > > use the tool. > >> > > > > >> > > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < > >> > > > [email protected]> wrote: > >> > > > > >> > > > > I would favour having hbck2 release branches. Temporary > >> compatibility > >> > > > > breaks at compile time may be inevitable if we always point to > the > >> > > latest > >> > > > > release. That could cause problems for operators trying to build > >> > hbck2 > >> > > > (we > >> > > > > are already seeing this happening with our support team). > Another > >> > > > argument > >> > > > > for starting having hbck2 releases is that we already have > quite a > >> > few > >> > > > > hbase 2 releases, yet, the main tool to fix inconsistencies is > not > >> > > easily > >> > > > > available. And there's been considerable efforts lately to bring > >> many > >> > > of > >> > > > > the fix options from hbck1 into hbck2, so what would folks think > >> > about > >> > > > > going with an hbck2 alpha release? > >> > > > > > >> > > > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser < > [email protected] > >> > > >> > > > > escreveu: > >> > > > > > >> > > > > > I still like one HBCK2 release as the goal. > >> > > > > > > >> > > > > > Is it possible to put some hacks into HBCK2 to work around the > >> > > > > > compatibility to fix the current state and focus more on > >> automation > >> > > to > >> > > > > > let us know the next time we inevitably break it again? ;) > >> > > > > > > >> > > > > > On 8/29/19 8:12 AM, Peter Somogyi wrote: > >> > > > > > > Hi everyone, > >> > > > > > > > >> > > > > > > This topic came up a couple of times already but now we > >> reached a > >> > > > point > >> > > > > > > when the recent HBCK2 is incompatible with older HBase > >> releases, > >> > > > > > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 > >> against > >> > > one > >> > > > > of > >> > > > > > > the previously mentioned releases you will get compilation > >> > errors. > >> > > > (mvn > >> > > > > > > clean install -DskipTests -Dhbase.version=2.0.6) > >> > > > > > > > >> > > > > > > Our previous goal was to maintain compatibility with HBCK2 > and > >> > all > >> > > > > HBase > >> > > > > > 2 > >> > > > > > > releases. Now we missed this target. > >> > > > > > > > >> > > > > > > One option that we could do is to have different > >> > branches/releases > >> > > of > >> > > > > > HBCK2 > >> > > > > > > targeted for specific HBase releases (e.g. branch-2.0 > version > >> of > >> > > > > HBCK2). > >> > > > > > > This probably makes the development on HBCK2 a bit harder > >> since > >> > > we'll > >> > > > > > have > >> > > > > > > to take care of multiple branches. > >> > > > > > > > >> > > > > > > Another option I could think of is to always build HBCK2 > with > >> the > >> > > > > latest > >> > > > > > > HBase release but have version checks on individual commands > >> > where > >> > > we > >> > > > > > could > >> > > > > > > decide if it is supported on that release line. > >> > > > > > > > >> > > > > > > What are your opinions on this? > >> > > > > > > > >> > > > > > > Thanks, > >> > > > > > > Peter > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
