I filed, https://issues.apache.org/jira/browse/HBASE-22956, for a first release of hbck2. S
On Fri, Aug 30, 2019 at 8:31 AM Peter Somogyi <psomo...@apache.org> wrote: > Thanks for all the feedback! > > Looks like my information about compatibility was incorrect and we do not > need to build HBCK2 targeted for the actual HBase version. Having > command-specific version checks that Stack mentioned looks good enough to > deal with compatibility. > Currently there are some issues with Replication on 2.0 releases. I'll work > on some corrections and once it's done with hbck1+ coverage we can start to > work on the first release! > > Peter > > On Thu, Aug 29, 2019 at 11:07 PM Sakthi <sak...@apache.org> wrote: > > > I like the last idea suggested by Stack. This way the core idea of > keeping > > both the dev process separate stays intact and also the operators have a > > version of hbck to fix their clusters without worrying about > compatibility > > issues. > > > > On Thu, Aug 29, 2019 at 2:03 PM Stack <st...@duboce.net> wrote: > > > > > Could also just make a release now of hbase-operator-tools (or in a > week > > or > > > so when we should have hbck1+ coverage in place) built against an > > > up-to-date hbase release. It has the check version before running a > > feature > > > in place where it matters. Operators could use this on all currently > > > shipping hbase2s? > > > > > > S > > > > > > On Thu, Aug 29, 2019 at 11:42 AM Stack <st...@duboce.net> wrote: > > > > > > > On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < > > > > wellington.chevre...@gmail.com> wrote: > > > > > > > >> > > > > >> > I do not think we need to compile HBCK2 with every releases? > > > >> > > > > >> Well, not with every release, was thinking in doing it whenever an > > hbase > > > >> release breaks compatibility. > > > >> > > > >> We just need > > > >> > make sure that it can work with all the releases. > > > >> > > > >> This could be a solution as well, but I believe it would be harder > to > > > >> guarantee. Here the problem is: > > > >> 1) A new hbase release changes one or more interfaces currently used > > by > > > >> hbck2; > > > >> 2) We update hbck2 to depend on this new hbase release, and change > > hbck2 > > > >> accordingly; > > > >> 3) Operators need to run hbck2 to a previous hbase release. If they > > try > > > to > > > >> build hbck2 against that version, it won't compile. If they build it > > > with > > > >> latest hbase version, it may give a runtime error, and now they have > > no > > > >> working tool to fix the problem. > > > >> To avoid #3, we would need to add extra checks on the changes > applied > > on > > > >> #2. Might become too complex. > > > >> > > > >> Thanks Wellington for above. I see issue now. > > > > > > > > So, we should make retroactive releases of hbase-operator-tools at > > points > > > > just before compat broke? Release could be named for the hbase2 > > versions > > > > supported. Releases would make it easier on operators making it so > they > > > > don't have to build themselves? There'd be one only? Two maybe? > > > > > > > > Looking at changes to the Hbck Interface -- using this as gauge for > > > > possible breakage points -- there aren't many. One release? Maybe > two? > > > > > > > > S > > > > > > > > > > > > > > > >> If there are missing > > > >> > methods, we just tell users you can not use several features. > > > >> > > > >> Fine for new fix options added, but what if some changes break > basic, > > > >> already working hbck2 methods. > > > >> > > > >> > > > >> Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) < > > > palomino...@gmail.com> > > > >> escreveu: > > > >> > > > >> > I do not think we need to compile HBCK2 with every releases? We > just > > > >> need > > > >> > make sure that it can work with all the releases. If there are > > missing > > > >> > methods, we just tell users you can not use several features. > > > >> > > > > >> > Wellington Chevreuil <wellington.chevre...@gmail.com> > 于2019年8月29日周四 > > > >> > 下午9:39写道： > > > >> > > > > >> > > > > > > >> > > > bq. what would folks think about going with an hbck2 alpha > > > release? > > > >> > > > I'm fine with an alpha release but since "HBCK2 should > > > continuously > > > >> > > evolve" > > > >> > > > it might be better to always use the latest codebase whenever > > you > > > >> need > > > >> > to > > > >> > > > use the tool. > > > >> > > > > > > >> > > Ideally yes, but that might not always be possible, as hbase API > > > might > > > >> > > change ahead of hbck2. Operators could then have problems to > get a > > > >> > working > > > >> > > version of hbck2. Since hbck2 already has now many equivalent > > > options > > > >> for > > > >> > > the ones from hbck1, I guess a first release would provide a > > working > > > >> > hbck2 > > > >> > > that already brings a considerable number of fix methods to help > > > with > > > >> > most > > > >> > > common inconsistencies issues seen in hbase 2 so far. And we > could > > > >> still > > > >> > > fulfil the "HBCK2 should continuously evolve" principle by > > building > > > >> from > > > >> > > its master branch, in scenarios where a new fix was needed and > > > >> > implemented > > > >> > > back into hbck2. > > > >> > > > > > >> > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi < > > > >> psomo...@apache.org> > > > >> > > escreveu: > > > >> > > > > > >> > > > bq. Is it possible to put some hacks into HBCK2 to work around > > > >> > > > the compatibility to fix the current state > > > >> > > > > > > >> > > > There are some classes around Replication which were > introduced > > in > > > >> > 2.1.0+ > > > >> > > > so I don't think we could easily solve it for 2.0. > > > >> > > > For 2.1.1 the missing method is > > Hbck#scheduleServerCrashProcedure, > > > >> > > probably > > > >> > > > that could be solved with some workarounds or using > reflection. > > > >> > > > > > > >> > > > bq. focus more on automation to let us know the next time we > > > >> inevitably > > > >> > > > break it again? ;) > > > >> > > > > > > >> > > > Sure! Based on this I think it should be a strong goal. We can > > set > > > >> up > > > >> > > > nightly builds for hbase-operator-tools repo that builds > against > > > the > > > >> > > latest > > > >> > > > development branches as well as checking compatibility with > > > released > > > >> > > > versions. > > > >> > > > > > > >> > > > bq. what would folks think about going with an hbck2 alpha > > > release? > > > >> > > > I'm fine with an alpha release but since "HBCK2 should > > > continuously > > > >> > > evolve" > > > >> > > > it might be better to always use the latest codebase whenever > > you > > > >> need > > > >> > to > > > >> > > > use the tool. > > > >> > > > > > > >> > > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < > > > >> > > > wellington.chevre...@gmail.com> wrote: > > > >> > > > > > > >> > > > > I would favour having hbck2 release branches. Temporary > > > >> compatibility > > > >> > > > > breaks at compile time may be inevitable if we always point > to > > > the > > > >> > > latest > > > >> > > > > release. That could cause problems for operators trying to > > build > > > >> > hbck2 > > > >> > > > (we > > > >> > > > > are already seeing this happening with our support team). > > > Another > > > >> > > > argument > > > >> > > > > for starting having hbck2 releases is that we already have > > > quite a > > > >> > few > > > >> > > > > hbase 2 releases, yet, the main tool to fix inconsistencies > is > > > not > > > >> > > easily > > > >> > > > > available. And there's been considerable efforts lately to > > bring > > > >> many > > > >> > > of > > > >> > > > > the fix options from hbck1 into hbck2, so what would folks > > think > > > >> > about > > > >> > > > > going with an hbck2 alpha release? > > > >> > > > > > > > >> > > > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser < > > > els...@apache.org > > > >> > > > > >> > > > > escreveu: > > > >> > > > > > > > >> > > > > > I still like one HBCK2 release as the goal. > > > >> > > > > > > > > >> > > > > > Is it possible to put some hacks into HBCK2 to work around > > the > > > >> > > > > > compatibility to fix the current state and focus more on > > > >> automation > > > >> > > to > > > >> > > > > > let us know the next time we inevitably break it again? ;) > > > >> > > > > > > > > >> > > > > > On 8/29/19 8:12 AM, Peter Somogyi wrote: > > > >> > > > > > > Hi everyone, > > > >> > > > > > > > > > >> > > > > > > This topic came up a couple of times already but now we > > > >> reached a > > > >> > > > point > > > >> > > > > > > when the recent HBCK2 is incompatible with older HBase > > > >> releases, > > > >> > > > > > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build > HBCK2 > > > >> against > > > >> > > one > > > >> > > > > of > > > >> > > > > > > the previously mentioned releases you will get > compilation > > > >> > errors. > > > >> > > > (mvn > > > >> > > > > > > clean install -DskipTests -Dhbase.version=2.0.6) > > > >> > > > > > > > > > >> > > > > > > Our previous goal was to maintain compatibility with > HBCK2 > > > and > > > >> > all > > > >> > > > > HBase > > > >> > > > > > 2 > > > >> > > > > > > releases. Now we missed this target. > > > >> > > > > > > > > > >> > > > > > > One option that we could do is to have different > > > >> > branches/releases > > > >> > > of > > > >> > > > > > HBCK2 > > > >> > > > > > > targeted for specific HBase releases (e.g. branch-2.0 > > > version > > > >> of > > > >> > > > > HBCK2). > > > >> > > > > > > This probably makes the development on HBCK2 a bit > harder > > > >> since > > > >> > > we'll > > > >> > > > > > have > > > >> > > > > > > to take care of multiple branches. > > > >> > > > > > > > > > >> > > > > > > Another option I could think of is to always build HBCK2 > > > with > > > >> the > > > >> > > > > latest > > > >> > > > > > > HBase release but have version checks on individual > > commands > > > >> > where > > > >> > > we > > > >> > > > > > could > > > >> > > > > > > decide if it is supported on that release line. > > > >> > > > > > > > > > >> > > > > > > What are your opinions on this? > > > >> > > > > > > > > > >> > > > > > > Thanks, > > > >> > > > > > > Peter > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > >
