Could also just make a release now of hbase-operator-tools (or in a week or so when we should have hbck1+ coverage in place) built against an up-to-date hbase release. It has the check version before running a feature in place where it matters. Operators could use this on all currently shipping hbase2s?
S On Thu, Aug 29, 2019 at 11:42 AM Stack <[email protected]> wrote: > On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < > [email protected]> wrote: > >> > >> > I do not think we need to compile HBCK2 with every releases? >> > >> Well, not with every release, was thinking in doing it whenever an hbase >> release breaks compatibility. >> >> We just need >> > make sure that it can work with all the releases. >> >> This could be a solution as well, but I believe it would be harder to >> guarantee. Here the problem is: >> 1) A new hbase release changes one or more interfaces currently used by >> hbck2; >> 2) We update hbck2 to depend on this new hbase release, and change hbck2 >> accordingly; >> 3) Operators need to run hbck2 to a previous hbase release. If they try to >> build hbck2 against that version, it won't compile. If they build it with >> latest hbase version, it may give a runtime error, and now they have no >> working tool to fix the problem. >> To avoid #3, we would need to add extra checks on the changes applied on >> #2. Might become too complex. >> >> Thanks Wellington for above. I see issue now. > > So, we should make retroactive releases of hbase-operator-tools at points > just before compat broke? Release could be named for the hbase2 versions > supported. Releases would make it easier on operators making it so they > don't have to build themselves? There'd be one only? Two maybe? > > Looking at changes to the Hbck Interface -- using this as gauge for > possible breakage points -- there aren't many. One release? Maybe two? > > S > > > >> If there are missing >> > methods, we just tell users you can not use several features. >> >> Fine for new fix options added, but what if some changes break basic, >> already working hbck2 methods. >> >> >> Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) <[email protected]> >> escreveu: >> >> > I do not think we need to compile HBCK2 with every releases? We just >> need >> > make sure that it can work with all the releases. If there are missing >> > methods, we just tell users you can not use several features. >> > >> > Wellington Chevreuil <[email protected]> 于2019年8月29日周四 >> > 下午9:39写道： >> > >> > > > >> > > > bq. what would folks think about going with an hbck2 alpha release? >> > > > I'm fine with an alpha release but since "HBCK2 should continuously >> > > evolve" >> > > > it might be better to always use the latest codebase whenever you >> need >> > to >> > > > use the tool. >> > > > >> > > Ideally yes, but that might not always be possible, as hbase API might >> > > change ahead of hbck2. Operators could then have problems to get a >> > working >> > > version of hbck2. Since hbck2 already has now many equivalent options >> for >> > > the ones from hbck1, I guess a first release would provide a working >> > hbck2 >> > > that already brings a considerable number of fix methods to help with >> > most >> > > common inconsistencies issues seen in hbase 2 so far. And we could >> still >> > > fulfil the "HBCK2 should continuously evolve" principle by building >> from >> > > its master branch, in scenarios where a new fix was needed and >> > implemented >> > > back into hbck2. >> > > >> > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi < >> [email protected]> >> > > escreveu: >> > > >> > > > bq. Is it possible to put some hacks into HBCK2 to work around >> > > > the compatibility to fix the current state >> > > > >> > > > There are some classes around Replication which were introduced in >> > 2.1.0+ >> > > > so I don't think we could easily solve it for 2.0. >> > > > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, >> > > probably >> > > > that could be solved with some workarounds or using reflection. >> > > > >> > > > bq. focus more on automation to let us know the next time we >> inevitably >> > > > break it again? ;) >> > > > >> > > > Sure! Based on this I think it should be a strong goal. We can set >> up >> > > > nightly builds for hbase-operator-tools repo that builds against the >> > > latest >> > > > development branches as well as checking compatibility with released >> > > > versions. >> > > > >> > > > bq. what would folks think about going with an hbck2 alpha release? >> > > > I'm fine with an alpha release but since "HBCK2 should continuously >> > > evolve" >> > > > it might be better to always use the latest codebase whenever you >> need >> > to >> > > > use the tool. >> > > > >> > > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < >> > > > [email protected]> wrote: >> > > > >> > > > > I would favour having hbck2 release branches. Temporary >> compatibility >> > > > > breaks at compile time may be inevitable if we always point to the >> > > latest >> > > > > release. That could cause problems for operators trying to build >> > hbck2 >> > > > (we >> > > > > are already seeing this happening with our support team). Another >> > > > argument >> > > > > for starting having hbck2 releases is that we already have quite a >> > few >> > > > > hbase 2 releases, yet, the main tool to fix inconsistencies is not >> > > easily >> > > > > available. And there's been considerable efforts lately to bring >> many >> > > of >> > > > > the fix options from hbck1 into hbck2, so what would folks think >> > about >> > > > > going with an hbck2 alpha release? >> > > > > >> > > > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser <[email protected] >> > >> > > > > escreveu: >> > > > > >> > > > > > I still like one HBCK2 release as the goal. >> > > > > > >> > > > > > Is it possible to put some hacks into HBCK2 to work around the >> > > > > > compatibility to fix the current state and focus more on >> automation >> > > to >> > > > > > let us know the next time we inevitably break it again? ;) >> > > > > > >> > > > > > On 8/29/19 8:12 AM, Peter Somogyi wrote: >> > > > > > > Hi everyone, >> > > > > > > >> > > > > > > This topic came up a couple of times already but now we >> reached a >> > > > point >> > > > > > > when the recent HBCK2 is incompatible with older HBase >> releases, >> > > > > > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 >> against >> > > one >> > > > > of >> > > > > > > the previously mentioned releases you will get compilation >> > errors. >> > > > (mvn >> > > > > > > clean install -DskipTests -Dhbase.version=2.0.6) >> > > > > > > >> > > > > > > Our previous goal was to maintain compatibility with HBCK2 and >> > all >> > > > > HBase >> > > > > > 2 >> > > > > > > releases. Now we missed this target. >> > > > > > > >> > > > > > > One option that we could do is to have different >> > branches/releases >> > > of >> > > > > > HBCK2 >> > > > > > > targeted for specific HBase releases (e.g. branch-2.0 version >> of >> > > > > HBCK2). >> > > > > > > This probably makes the development on HBCK2 a bit harder >> since >> > > we'll >> > > > > > have >> > > > > > > to take care of multiple branches. >> > > > > > > >> > > > > > > Another option I could think of is to always build HBCK2 with >> the >> > > > > latest >> > > > > > > HBase release but have version checks on individual commands >> > where >> > > we >> > > > > > could >> > > > > > > decide if it is supported on that release line. >> > > > > > > >> > > > > > > What are your opinions on this? >> > > > > > > >> > > > > > > Thanks, >> > > > > > > Peter >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >
