Re: [DISCUSS] HBCK2 compatibility
I filed, https://issues.apache.org/jira/browse/HBASE-22956, for a first release of hbck2. S On Fri, Aug 30, 2019 at 8:31 AM Peter Somogyi wrote: > Thanks for all the feedback! > > Looks like my information about compatibility was incorrect and we do not > need to build HBCK2 targeted for the actual HBase version. Having > command-specific version checks that Stack mentioned looks good enough to > deal with compatibility. > Currently there are some issues with Replication on 2.0 releases. I'll work > on some corrections and once it's done with hbck1+ coverage we can start to > work on the first release! > > Peter > > On Thu, Aug 29, 2019 at 11:07 PM Sakthi wrote: > > > I like the last idea suggested by Stack. This way the core idea of > keeping > > both the dev process separate stays intact and also the operators have a > > version of hbck to fix their clusters without worrying about > compatibility > > issues. > > > > On Thu, Aug 29, 2019 at 2:03 PM Stack wrote: > > > > > Could also just make a release now of hbase-operator-tools (or in a > week > > or > > > so when we should have hbck1+ coverage in place) built against an > > > up-to-date hbase release. It has the check version before running a > > feature > > > in place where it matters. Operators could use this on all currently > > > shipping hbase2s? > > > > > > S > > > > > > On Thu, Aug 29, 2019 at 11:42 AM Stack wrote: > > > > > > > On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < > > > > wellington.chevre...@gmail.com> wrote: > > > > > > > >> > > > > >> > I do not think we need to compile HBCK2 with every releases? > > > >> > > > > >> Well, not with every release, was thinking in doing it whenever an > > hbase > > > >> release breaks compatibility. > > > >> > > > >> We just need > > > >> > make sure that it can work with all the releases. > > > >> > > > >> This could be a solution as well, but I believe it would be harder > to > > > >> guarantee. Here the problem is: > > > >> 1) A new hbase release changes one or more interfaces currently used > > by > > > >> hbck2; > > > >> 2) We update hbck2 to depend on this new hbase release, and change > > hbck2 > > > >> accordingly; > > > >> 3) Operators need to run hbck2 to a previous hbase release. If they > > try > > > to > > > >> build hbck2 against that version, it won't compile. If they build it > > > with > > > >> latest hbase version, it may give a runtime error, and now they have > > no > > > >> working tool to fix the problem. > > > >> To avoid #3, we would need to add extra checks on the changes > applied > > on > > > >> #2. Might become too complex. > > > >> > > > >> Thanks Wellington for above. I see issue now. > > > > > > > > So, we should make retroactive releases of hbase-operator-tools at > > points > > > > just before compat broke? Release could be named for the hbase2 > > versions > > > > supported. Releases would make it easier on operators making it so > they > > > > don't have to build themselves? There'd be one only? Two maybe? > > > > > > > > Looking at changes to the Hbck Interface -- using this as gauge for > > > > possible breakage points -- there aren't many. One release? Maybe > two? > > > > > > > > S > > > > > > > > > > > > > > > >> If there are missing > > > >> > methods, we just tell users you can not use several features. > > > >> > > > >> Fine for new fix options added, but what if some changes break > basic, > > > >> already working hbck2 methods. > > > >> > > > >> > > > >> Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) < > > > palomino...@gmail.com> > > > >> escreveu: > > > >> > > > >> > I do not think we need to compile HBCK2 with every releases? We > just > > > >> need > > > >> > make sure that it can work with all the releases. If there are > > missing > > > >> > methods, we just tell users you can not use several features. > > > >> > > > > >> > Wellington Chevreuil > 于2019年8月29日周四 > > > >> > 下午9:39写道: > > > >> > > > > >> > > > > > > >> > > > bq. what would folks think about going with an hbck2 alpha > > > release? > > > >> > > > I'm fine with an alpha release but since "HBCK2 should > > > continuously > > > >> > > evolve" > > > >> > > > it might be better to always use the latest codebase whenever > > you > > > >> need > > > >> > to > > > >> > > > use the tool. > > > >> > > > > > > >> > > Ideally yes, but that might not always be possible, as hbase API > > > might > > > >> > > change ahead of hbck2. Operators could then have problems to > get a > > > >> > working > > > >> > > version of hbck2. Since hbck2 already has now many equivalent > > > options > > > >> for > > > >> > > the ones from hbck1, I guess a first release would provide a > > working > > > >> > hbck2 > > > >> > > that already brings a considerable number of fix methods to help > > > with > > > >> > most > > > >> > > common inconsistencies issues seen in hbase 2 so far. And we > could > > > >> still > > > >> > > fulfil the "HBCK2 should continuously evolve" principle by > > building > > > >>
Re: [DISCUSS] HBCK2 compatibility
Thanks for all the feedback! Looks like my information about compatibility was incorrect and we do not need to build HBCK2 targeted for the actual HBase version. Having command-specific version checks that Stack mentioned looks good enough to deal with compatibility. Currently there are some issues with Replication on 2.0 releases. I'll work on some corrections and once it's done with hbck1+ coverage we can start to work on the first release! Peter On Thu, Aug 29, 2019 at 11:07 PM Sakthi wrote: > I like the last idea suggested by Stack. This way the core idea of keeping > both the dev process separate stays intact and also the operators have a > version of hbck to fix their clusters without worrying about compatibility > issues. > > On Thu, Aug 29, 2019 at 2:03 PM Stack wrote: > > > Could also just make a release now of hbase-operator-tools (or in a week > or > > so when we should have hbck1+ coverage in place) built against an > > up-to-date hbase release. It has the check version before running a > feature > > in place where it matters. Operators could use this on all currently > > shipping hbase2s? > > > > S > > > > On Thu, Aug 29, 2019 at 11:42 AM Stack wrote: > > > > > On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < > > > wellington.chevre...@gmail.com> wrote: > > > > > >> > > > >> > I do not think we need to compile HBCK2 with every releases? > > >> > > > >> Well, not with every release, was thinking in doing it whenever an > hbase > > >> release breaks compatibility. > > >> > > >> We just need > > >> > make sure that it can work with all the releases. > > >> > > >> This could be a solution as well, but I believe it would be harder to > > >> guarantee. Here the problem is: > > >> 1) A new hbase release changes one or more interfaces currently used > by > > >> hbck2; > > >> 2) We update hbck2 to depend on this new hbase release, and change > hbck2 > > >> accordingly; > > >> 3) Operators need to run hbck2 to a previous hbase release. If they > try > > to > > >> build hbck2 against that version, it won't compile. If they build it > > with > > >> latest hbase version, it may give a runtime error, and now they have > no > > >> working tool to fix the problem. > > >> To avoid #3, we would need to add extra checks on the changes applied > on > > >> #2. Might become too complex. > > >> > > >> Thanks Wellington for above. I see issue now. > > > > > > So, we should make retroactive releases of hbase-operator-tools at > points > > > just before compat broke? Release could be named for the hbase2 > versions > > > supported. Releases would make it easier on operators making it so they > > > don't have to build themselves? There'd be one only? Two maybe? > > > > > > Looking at changes to the Hbck Interface -- using this as gauge for > > > possible breakage points -- there aren't many. One release? Maybe two? > > > > > > S > > > > > > > > > > > >> If there are missing > > >> > methods, we just tell users you can not use several features. > > >> > > >> Fine for new fix options added, but what if some changes break basic, > > >> already working hbck2 methods. > > >> > > >> > > >> Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) < > > palomino...@gmail.com> > > >> escreveu: > > >> > > >> > I do not think we need to compile HBCK2 with every releases? We just > > >> need > > >> > make sure that it can work with all the releases. If there are > missing > > >> > methods, we just tell users you can not use several features. > > >> > > > >> > Wellington Chevreuil 于2019年8月29日周四 > > >> > 下午9:39写道: > > >> > > > >> > > > > > >> > > > bq. what would folks think about going with an hbck2 alpha > > release? > > >> > > > I'm fine with an alpha release but since "HBCK2 should > > continuously > > >> > > evolve" > > >> > > > it might be better to always use the latest codebase whenever > you > > >> need > > >> > to > > >> > > > use the tool. > > >> > > > > > >> > > Ideally yes, but that might not always be possible, as hbase API > > might > > >> > > change ahead of hbck2. Operators could then have problems to get a > > >> > working > > >> > > version of hbck2. Since hbck2 already has now many equivalent > > options > > >> for > > >> > > the ones from hbck1, I guess a first release would provide a > working > > >> > hbck2 > > >> > > that already brings a considerable number of fix methods to help > > with > > >> > most > > >> > > common inconsistencies issues seen in hbase 2 so far. And we could > > >> still > > >> > > fulfil the "HBCK2 should continuously evolve" principle by > building > > >> from > > >> > > its master branch, in scenarios where a new fix was needed and > > >> > implemented > > >> > > back into hbck2. > > >> > > > > >> > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi < > > >> psomo...@apache.org> > > >> > > escreveu: > > >> > > > > >> > > > bq. Is it possible to put some hacks into HBCK2 to work around > > >> > > > the compatibility to fix the current state > > >> > > > > > >> > > > There are
Re: [DISCUSS] HBCK2 compatibility
> > Could also just make a release now of hbase-operator-tools (or in a week or > so when we should have hbck1+ coverage in place) built against an > up-to-date hbase release. +1, I think current version has a reasonable number of fix options, and if we can ensure it runs for valid 2.x releases, would be of great help for users/operators. Em qui, 29 de ago de 2019 às 22:07, Sakthi escreveu: > I like the last idea suggested by Stack. This way the core idea of keeping > both the dev process separate stays intact and also the operators have a > version of hbck to fix their clusters without worrying about compatibility > issues. > > On Thu, Aug 29, 2019 at 2:03 PM Stack wrote: > > > Could also just make a release now of hbase-operator-tools (or in a week > or > > so when we should have hbck1+ coverage in place) built against an > > up-to-date hbase release. It has the check version before running a > feature > > in place where it matters. Operators could use this on all currently > > shipping hbase2s? > > > > S > > > > On Thu, Aug 29, 2019 at 11:42 AM Stack wrote: > > > > > On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < > > > wellington.chevre...@gmail.com> wrote: > > > > > >> > > > >> > I do not think we need to compile HBCK2 with every releases? > > >> > > > >> Well, not with every release, was thinking in doing it whenever an > hbase > > >> release breaks compatibility. > > >> > > >> We just need > > >> > make sure that it can work with all the releases. > > >> > > >> This could be a solution as well, but I believe it would be harder to > > >> guarantee. Here the problem is: > > >> 1) A new hbase release changes one or more interfaces currently used > by > > >> hbck2; > > >> 2) We update hbck2 to depend on this new hbase release, and change > hbck2 > > >> accordingly; > > >> 3) Operators need to run hbck2 to a previous hbase release. If they > try > > to > > >> build hbck2 against that version, it won't compile. If they build it > > with > > >> latest hbase version, it may give a runtime error, and now they have > no > > >> working tool to fix the problem. > > >> To avoid #3, we would need to add extra checks on the changes applied > on > > >> #2. Might become too complex. > > >> > > >> Thanks Wellington for above. I see issue now. > > > > > > So, we should make retroactive releases of hbase-operator-tools at > points > > > just before compat broke? Release could be named for the hbase2 > versions > > > supported. Releases would make it easier on operators making it so they > > > don't have to build themselves? There'd be one only? Two maybe? > > > > > > Looking at changes to the Hbck Interface -- using this as gauge for > > > possible breakage points -- there aren't many. One release? Maybe two? > > > > > > S > > > > > > > > > > > >> If there are missing > > >> > methods, we just tell users you can not use several features. > > >> > > >> Fine for new fix options added, but what if some changes break basic, > > >> already working hbck2 methods. > > >> > > >> > > >> Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) < > > palomino...@gmail.com> > > >> escreveu: > > >> > > >> > I do not think we need to compile HBCK2 with every releases? We just > > >> need > > >> > make sure that it can work with all the releases. If there are > missing > > >> > methods, we just tell users you can not use several features. > > >> > > > >> > Wellington Chevreuil 于2019年8月29日周四 > > >> > 下午9:39写道: > > >> > > > >> > > > > > >> > > > bq. what would folks think about going with an hbck2 alpha > > release? > > >> > > > I'm fine with an alpha release but since "HBCK2 should > > continuously > > >> > > evolve" > > >> > > > it might be better to always use the latest codebase whenever > you > > >> need > > >> > to > > >> > > > use the tool. > > >> > > > > > >> > > Ideally yes, but that might not always be possible, as hbase API > > might > > >> > > change ahead of hbck2. Operators could then have problems to get a > > >> > working > > >> > > version of hbck2. Since hbck2 already has now many equivalent > > options > > >> for > > >> > > the ones from hbck1, I guess a first release would provide a > working > > >> > hbck2 > > >> > > that already brings a considerable number of fix methods to help > > with > > >> > most > > >> > > common inconsistencies issues seen in hbase 2 so far. And we could > > >> still > > >> > > fulfil the "HBCK2 should continuously evolve" principle by > building > > >> from > > >> > > its master branch, in scenarios where a new fix was needed and > > >> > implemented > > >> > > back into hbck2. > > >> > > > > >> > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi < > > >> psomo...@apache.org> > > >> > > escreveu: > > >> > > > > >> > > > bq. Is it possible to put some hacks into HBCK2 to work around > > >> > > > the compatibility to fix the current state > > >> > > > > > >> > > > There are some classes around Replication which were introduced > in > > >> > 2.1.0+ > > >> > > > so I don't think we
Re: [DISCUSS] HBCK2 compatibility
I like the last idea suggested by Stack. This way the core idea of keeping both the dev process separate stays intact and also the operators have a version of hbck to fix their clusters without worrying about compatibility issues. On Thu, Aug 29, 2019 at 2:03 PM Stack wrote: > Could also just make a release now of hbase-operator-tools (or in a week or > so when we should have hbck1+ coverage in place) built against an > up-to-date hbase release. It has the check version before running a feature > in place where it matters. Operators could use this on all currently > shipping hbase2s? > > S > > On Thu, Aug 29, 2019 at 11:42 AM Stack wrote: > > > On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < > > wellington.chevre...@gmail.com> wrote: > > > >> > > >> > I do not think we need to compile HBCK2 with every releases? > >> > > >> Well, not with every release, was thinking in doing it whenever an hbase > >> release breaks compatibility. > >> > >> We just need > >> > make sure that it can work with all the releases. > >> > >> This could be a solution as well, but I believe it would be harder to > >> guarantee. Here the problem is: > >> 1) A new hbase release changes one or more interfaces currently used by > >> hbck2; > >> 2) We update hbck2 to depend on this new hbase release, and change hbck2 > >> accordingly; > >> 3) Operators need to run hbck2 to a previous hbase release. If they try > to > >> build hbck2 against that version, it won't compile. If they build it > with > >> latest hbase version, it may give a runtime error, and now they have no > >> working tool to fix the problem. > >> To avoid #3, we would need to add extra checks on the changes applied on > >> #2. Might become too complex. > >> > >> Thanks Wellington for above. I see issue now. > > > > So, we should make retroactive releases of hbase-operator-tools at points > > just before compat broke? Release could be named for the hbase2 versions > > supported. Releases would make it easier on operators making it so they > > don't have to build themselves? There'd be one only? Two maybe? > > > > Looking at changes to the Hbck Interface -- using this as gauge for > > possible breakage points -- there aren't many. One release? Maybe two? > > > > S > > > > > > > >> If there are missing > >> > methods, we just tell users you can not use several features. > >> > >> Fine for new fix options added, but what if some changes break basic, > >> already working hbck2 methods. > >> > >> > >> Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) < > palomino...@gmail.com> > >> escreveu: > >> > >> > I do not think we need to compile HBCK2 with every releases? We just > >> need > >> > make sure that it can work with all the releases. If there are missing > >> > methods, we just tell users you can not use several features. > >> > > >> > Wellington Chevreuil 于2019年8月29日周四 > >> > 下午9:39写道: > >> > > >> > > > > >> > > > bq. what would folks think about going with an hbck2 alpha > release? > >> > > > I'm fine with an alpha release but since "HBCK2 should > continuously > >> > > evolve" > >> > > > it might be better to always use the latest codebase whenever you > >> need > >> > to > >> > > > use the tool. > >> > > > > >> > > Ideally yes, but that might not always be possible, as hbase API > might > >> > > change ahead of hbck2. Operators could then have problems to get a > >> > working > >> > > version of hbck2. Since hbck2 already has now many equivalent > options > >> for > >> > > the ones from hbck1, I guess a first release would provide a working > >> > hbck2 > >> > > that already brings a considerable number of fix methods to help > with > >> > most > >> > > common inconsistencies issues seen in hbase 2 so far. And we could > >> still > >> > > fulfil the "HBCK2 should continuously evolve" principle by building > >> from > >> > > its master branch, in scenarios where a new fix was needed and > >> > implemented > >> > > back into hbck2. > >> > > > >> > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi < > >> psomo...@apache.org> > >> > > escreveu: > >> > > > >> > > > bq. Is it possible to put some hacks into HBCK2 to work around > >> > > > the compatibility to fix the current state > >> > > > > >> > > > There are some classes around Replication which were introduced in > >> > 2.1.0+ > >> > > > so I don't think we could easily solve it for 2.0. > >> > > > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, > >> > > probably > >> > > > that could be solved with some workarounds or using reflection. > >> > > > > >> > > > bq. focus more on automation to let us know the next time we > >> inevitably > >> > > > break it again? ;) > >> > > > > >> > > > Sure! Based on this I think it should be a strong goal. We can set > >> up > >> > > > nightly builds for hbase-operator-tools repo that builds against > the > >> > > latest > >> > > > development branches as well as checking compatibility with > released > >> > > > versions. > >> > > > > >> > > > bq. what
Re: [DISCUSS] HBCK2 compatibility
Could also just make a release now of hbase-operator-tools (or in a week or so when we should have hbck1+ coverage in place) built against an up-to-date hbase release. It has the check version before running a feature in place where it matters. Operators could use this on all currently shipping hbase2s? S On Thu, Aug 29, 2019 at 11:42 AM Stack wrote: > On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < > wellington.chevre...@gmail.com> wrote: > >> > >> > I do not think we need to compile HBCK2 with every releases? >> > >> Well, not with every release, was thinking in doing it whenever an hbase >> release breaks compatibility. >> >> We just need >> > make sure that it can work with all the releases. >> >> This could be a solution as well, but I believe it would be harder to >> guarantee. Here the problem is: >> 1) A new hbase release changes one or more interfaces currently used by >> hbck2; >> 2) We update hbck2 to depend on this new hbase release, and change hbck2 >> accordingly; >> 3) Operators need to run hbck2 to a previous hbase release. If they try to >> build hbck2 against that version, it won't compile. If they build it with >> latest hbase version, it may give a runtime error, and now they have no >> working tool to fix the problem. >> To avoid #3, we would need to add extra checks on the changes applied on >> #2. Might become too complex. >> >> Thanks Wellington for above. I see issue now. > > So, we should make retroactive releases of hbase-operator-tools at points > just before compat broke? Release could be named for the hbase2 versions > supported. Releases would make it easier on operators making it so they > don't have to build themselves? There'd be one only? Two maybe? > > Looking at changes to the Hbck Interface -- using this as gauge for > possible breakage points -- there aren't many. One release? Maybe two? > > S > > > >> If there are missing >> > methods, we just tell users you can not use several features. >> >> Fine for new fix options added, but what if some changes break basic, >> already working hbck2 methods. >> >> >> Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) >> escreveu: >> >> > I do not think we need to compile HBCK2 with every releases? We just >> need >> > make sure that it can work with all the releases. If there are missing >> > methods, we just tell users you can not use several features. >> > >> > Wellington Chevreuil 于2019年8月29日周四 >> > 下午9:39写道: >> > >> > > > >> > > > bq. what would folks think about going with an hbck2 alpha release? >> > > > I'm fine with an alpha release but since "HBCK2 should continuously >> > > evolve" >> > > > it might be better to always use the latest codebase whenever you >> need >> > to >> > > > use the tool. >> > > > >> > > Ideally yes, but that might not always be possible, as hbase API might >> > > change ahead of hbck2. Operators could then have problems to get a >> > working >> > > version of hbck2. Since hbck2 already has now many equivalent options >> for >> > > the ones from hbck1, I guess a first release would provide a working >> > hbck2 >> > > that already brings a considerable number of fix methods to help with >> > most >> > > common inconsistencies issues seen in hbase 2 so far. And we could >> still >> > > fulfil the "HBCK2 should continuously evolve" principle by building >> from >> > > its master branch, in scenarios where a new fix was needed and >> > implemented >> > > back into hbck2. >> > > >> > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi < >> psomo...@apache.org> >> > > escreveu: >> > > >> > > > bq. Is it possible to put some hacks into HBCK2 to work around >> > > > the compatibility to fix the current state >> > > > >> > > > There are some classes around Replication which were introduced in >> > 2.1.0+ >> > > > so I don't think we could easily solve it for 2.0. >> > > > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, >> > > probably >> > > > that could be solved with some workarounds or using reflection. >> > > > >> > > > bq. focus more on automation to let us know the next time we >> inevitably >> > > > break it again? ;) >> > > > >> > > > Sure! Based on this I think it should be a strong goal. We can set >> up >> > > > nightly builds for hbase-operator-tools repo that builds against the >> > > latest >> > > > development branches as well as checking compatibility with released >> > > > versions. >> > > > >> > > > bq. what would folks think about going with an hbck2 alpha release? >> > > > I'm fine with an alpha release but since "HBCK2 should continuously >> > > evolve" >> > > > it might be better to always use the latest codebase whenever you >> need >> > to >> > > > use the tool. >> > > > >> > > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < >> > > > wellington.chevre...@gmail.com> wrote: >> > > > >> > > > > I would favour having hbck2 release branches. Temporary >> compatibility >> > > > > breaks at compile time may be inevitable if we always point to the >>
Re: [DISCUSS] HBCK2 compatibility
It does seem better to put it back in tree with core because core will be branched, and would allow for the usual way we vary features and compatibility by code line. > On Aug 29, 2019, at 11:14 AM, Stack wrote: > >> On Thu, Aug 29, 2019 at 5:12 AM Peter Somogyi wrote: >> >> Hi everyone, >> >> This topic came up a couple of times already but now we reached a point >> when the recent HBCK2 is incompatible with older HBase releases, >> specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 against one of >> the previously mentioned releases you will get compilation errors. (mvn >> clean install -DskipTests -Dhbase.version=2.0.6) >> >> Our previous goal was to maintain compatibility with HBCK2 and all HBase 2 >> releases. Now we missed this target. >> >> > Was being able to build against all hbase2s a target? Didn't know. > > > >> One option that we could do is to have different branches/releases of HBCK2 >> targeted for specific HBase releases (e.g. branch-2.0 version of HBCK2). >> This probably makes the development on HBCK2 a bit harder since we'll have >> to take care of multiple branches. >> >> > This would defeat the whole reason for moving hbck2 out of core. Better to > just have hbck2 in hbase rather than do this. > > >> Another option I could think of is to always build HBCK2 with the latest >> HBase release but have version checks on individual commands where we could >> decide if it is supported on that release line. >> >> > This is what we have now [1]. > > S > > 1. > https://github.com/apache/hbase-operator-tools/blob/master/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java#L105 > > >> What are your opinions on this? >> >> Thanks, >> Peter >>
Re: [DISCUSS] HBCK2 compatibility
On Thu, Aug 29, 2019 at 7:34 AM Wellington Chevreuil < wellington.chevre...@gmail.com> wrote: > > > > I do not think we need to compile HBCK2 with every releases? > > > Well, not with every release, was thinking in doing it whenever an hbase > release breaks compatibility. > > We just need > > make sure that it can work with all the releases. > > This could be a solution as well, but I believe it would be harder to > guarantee. Here the problem is: > 1) A new hbase release changes one or more interfaces currently used by > hbck2; > 2) We update hbck2 to depend on this new hbase release, and change hbck2 > accordingly; > 3) Operators need to run hbck2 to a previous hbase release. If they try to > build hbck2 against that version, it won't compile. If they build it with > latest hbase version, it may give a runtime error, and now they have no > working tool to fix the problem. > To avoid #3, we would need to add extra checks on the changes applied on > #2. Might become too complex. > > Thanks Wellington for above. I see issue now. So, we should make retroactive releases of hbase-operator-tools at points just before compat broke? Release could be named for the hbase2 versions supported. Releases would make it easier on operators making it so they don't have to build themselves? There'd be one only? Two maybe? Looking at changes to the Hbck Interface -- using this as gauge for possible breakage points -- there aren't many. One release? Maybe two? S > If there are missing > > methods, we just tell users you can not use several features. > > Fine for new fix options added, but what if some changes break basic, > already working hbck2 methods. > > > Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) > escreveu: > > > I do not think we need to compile HBCK2 with every releases? We just need > > make sure that it can work with all the releases. If there are missing > > methods, we just tell users you can not use several features. > > > > Wellington Chevreuil 于2019年8月29日周四 > > 下午9:39写道: > > > > > > > > > > bq. what would folks think about going with an hbck2 alpha release? > > > > I'm fine with an alpha release but since "HBCK2 should continuously > > > evolve" > > > > it might be better to always use the latest codebase whenever you > need > > to > > > > use the tool. > > > > > > > Ideally yes, but that might not always be possible, as hbase API might > > > change ahead of hbck2. Operators could then have problems to get a > > working > > > version of hbck2. Since hbck2 already has now many equivalent options > for > > > the ones from hbck1, I guess a first release would provide a working > > hbck2 > > > that already brings a considerable number of fix methods to help with > > most > > > common inconsistencies issues seen in hbase 2 so far. And we could > still > > > fulfil the "HBCK2 should continuously evolve" principle by building > from > > > its master branch, in scenarios where a new fix was needed and > > implemented > > > back into hbck2. > > > > > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi > > > > escreveu: > > > > > > > bq. Is it possible to put some hacks into HBCK2 to work around > > > > the compatibility to fix the current state > > > > > > > > There are some classes around Replication which were introduced in > > 2.1.0+ > > > > so I don't think we could easily solve it for 2.0. > > > > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, > > > probably > > > > that could be solved with some workarounds or using reflection. > > > > > > > > bq. focus more on automation to let us know the next time we > inevitably > > > > break it again? ;) > > > > > > > > Sure! Based on this I think it should be a strong goal. We can set up > > > > nightly builds for hbase-operator-tools repo that builds against the > > > latest > > > > development branches as well as checking compatibility with released > > > > versions. > > > > > > > > bq. what would folks think about going with an hbck2 alpha release? > > > > I'm fine with an alpha release but since "HBCK2 should continuously > > > evolve" > > > > it might be better to always use the latest codebase whenever you > need > > to > > > > use the tool. > > > > > > > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < > > > > wellington.chevre...@gmail.com> wrote: > > > > > > > > > I would favour having hbck2 release branches. Temporary > compatibility > > > > > breaks at compile time may be inevitable if we always point to the > > > latest > > > > > release. That could cause problems for operators trying to build > > hbck2 > > > > (we > > > > > are already seeing this happening with our support team). Another > > > > argument > > > > > for starting having hbck2 releases is that we already have quite a > > few > > > > > hbase 2 releases, yet, the main tool to fix inconsistencies is not > > > easily > > > > > available. And there's been considerable efforts lately to bring > many > > > of > > > > > the fix options from hbck1 into hbck2,
Re: [DISCUSS] HBCK2 compatibility
> > what would folks think about going with an hbck2 alpha release? > > I'm fine with an alpha release but since "HBCK2 should continuously evolve" > it might be better to always use the latest codebase whenever you need to > use the tool. We can't do this, as a matter of ASF policy. Anything that's supposed to be used by folks outside of the dev@hbase mailing list needs to have artifacts that the PMC has signed off on and placed into dist.a.o. It's fine if they're just source releases and we expect folks to always build it. But it can't be pointing them at the git repo. On Thu, Aug 29, 2019 at 8:09 AM Peter Somogyi wrote: > > bq. Is it possible to put some hacks into HBCK2 to work around > the compatibility to fix the current state > > There are some classes around Replication which were introduced in 2.1.0+ > so I don't think we could easily solve it for 2.0. > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, probably > that could be solved with some workarounds or using reflection. > > bq. focus more on automation to let us know the next time we inevitably > break it again? ;) > > Sure! Based on this I think it should be a strong goal. We can set up > nightly builds for hbase-operator-tools repo that builds against the latest > development branches as well as checking compatibility with released > versions. > > bq. what would folks think about going with an hbck2 alpha release? > I'm fine with an alpha release but since "HBCK2 should continuously evolve" > it might be better to always use the latest codebase whenever you need to > use the tool. > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < > wellington.chevre...@gmail.com> wrote: > > > I would favour having hbck2 release branches. Temporary compatibility > > breaks at compile time may be inevitable if we always point to the latest > > release. That could cause problems for operators trying to build hbck2 (we > > are already seeing this happening with our support team). Another argument > > for starting having hbck2 releases is that we already have quite a few > > hbase 2 releases, yet, the main tool to fix inconsistencies is not easily > > available. And there's been considerable efforts lately to bring many of > > the fix options from hbck1 into hbck2, so what would folks think about > > going with an hbck2 alpha release? > > > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser > > escreveu: > > > > > I still like one HBCK2 release as the goal. > > > > > > Is it possible to put some hacks into HBCK2 to work around the > > > compatibility to fix the current state and focus more on automation to > > > let us know the next time we inevitably break it again? ;) > > > > > > On 8/29/19 8:12 AM, Peter Somogyi wrote: > > > > Hi everyone, > > > > > > > > This topic came up a couple of times already but now we reached a point > > > > when the recent HBCK2 is incompatible with older HBase releases, > > > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 against one > > of > > > > the previously mentioned releases you will get compilation errors. (mvn > > > > clean install -DskipTests -Dhbase.version=2.0.6) > > > > > > > > Our previous goal was to maintain compatibility with HBCK2 and all > > HBase > > > 2 > > > > releases. Now we missed this target. > > > > > > > > One option that we could do is to have different branches/releases of > > > HBCK2 > > > > targeted for specific HBase releases (e.g. branch-2.0 version of > > HBCK2). > > > > This probably makes the development on HBCK2 a bit harder since we'll > > > have > > > > to take care of multiple branches. > > > > > > > > Another option I could think of is to always build HBCK2 with the > > latest > > > > HBase release but have version checks on individual commands where we > > > could > > > > decide if it is supported on that release line. > > > > > > > > What are your opinions on this? > > > > > > > > Thanks, > > > > Peter > > > > > > > > >
Re: [DISCUSS] HBCK2 compatibility
On Thu, Aug 29, 2019 at 5:12 AM Peter Somogyi wrote: > Hi everyone, > > This topic came up a couple of times already but now we reached a point > when the recent HBCK2 is incompatible with older HBase releases, > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 against one of > the previously mentioned releases you will get compilation errors. (mvn > clean install -DskipTests -Dhbase.version=2.0.6) > > Our previous goal was to maintain compatibility with HBCK2 and all HBase 2 > releases. Now we missed this target. > > Was being able to build against all hbase2s a target? Didn't know. > One option that we could do is to have different branches/releases of HBCK2 > targeted for specific HBase releases (e.g. branch-2.0 version of HBCK2). > This probably makes the development on HBCK2 a bit harder since we'll have > to take care of multiple branches. > > This would defeat the whole reason for moving hbck2 out of core. Better to just have hbck2 in hbase rather than do this. > Another option I could think of is to always build HBCK2 with the latest > HBase release but have version checks on individual commands where we could > decide if it is supported on that release line. > > This is what we have now [1]. S 1. https://github.com/apache/hbase-operator-tools/blob/master/hbase-hbck2/src/main/java/org/apache/hbase/HBCK2.java#L105 > What are your opinions on this? > > Thanks, > Peter >
Re: [DISCUSS] HBCK2 compatibility
bq. I do not think we need to compile HBCK2 with every releases? We just need make sure that it can work with all the releases. If there are missing methods, we just tell users you can not use several features. We could do this but in this case we need to put version-specific checks for each HBCK2 commands. On Thu, Aug 29, 2019 at 4:34 PM Wellington Chevreuil < wellington.chevre...@gmail.com> wrote: > > > > I do not think we need to compile HBCK2 with every releases? > > > Well, not with every release, was thinking in doing it whenever an hbase > release breaks compatibility. > > We just need > > make sure that it can work with all the releases. > > This could be a solution as well, but I believe it would be harder to > guarantee. Here the problem is: > 1) A new hbase release changes one or more interfaces currently used by > hbck2; > 2) We update hbck2 to depend on this new hbase release, and change hbck2 > accordingly; > 3) Operators need to run hbck2 to a previous hbase release. If they try to > build hbck2 against that version, it won't compile. If they build it with > latest hbase version, it may give a runtime error, and now they have no > working tool to fix the problem. > To avoid #3, we would need to add extra checks on the changes applied on > #2. Might become too complex. > > If there are missing > > methods, we just tell users you can not use several features. > > Fine for new fix options added, but what if some changes break basic, > already working hbck2 methods. > > > Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) > escreveu: > > > I do not think we need to compile HBCK2 with every releases? We just need > > make sure that it can work with all the releases. If there are missing > > methods, we just tell users you can not use several features. > > > > Wellington Chevreuil 于2019年8月29日周四 > > 下午9:39写道: > > > > > > > > > > bq. what would folks think about going with an hbck2 alpha release? > > > > I'm fine with an alpha release but since "HBCK2 should continuously > > > evolve" > > > > it might be better to always use the latest codebase whenever you > need > > to > > > > use the tool. > > > > > > > Ideally yes, but that might not always be possible, as hbase API might > > > change ahead of hbck2. Operators could then have problems to get a > > working > > > version of hbck2. Since hbck2 already has now many equivalent options > for > > > the ones from hbck1, I guess a first release would provide a working > > hbck2 > > > that already brings a considerable number of fix methods to help with > > most > > > common inconsistencies issues seen in hbase 2 so far. And we could > still > > > fulfil the "HBCK2 should continuously evolve" principle by building > from > > > its master branch, in scenarios where a new fix was needed and > > implemented > > > back into hbck2. > > > > > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi > > > > escreveu: > > > > > > > bq. Is it possible to put some hacks into HBCK2 to work around > > > > the compatibility to fix the current state > > > > > > > > There are some classes around Replication which were introduced in > > 2.1.0+ > > > > so I don't think we could easily solve it for 2.0. > > > > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, > > > probably > > > > that could be solved with some workarounds or using reflection. > > > > > > > > bq. focus more on automation to let us know the next time we > inevitably > > > > break it again? ;) > > > > > > > > Sure! Based on this I think it should be a strong goal. We can set up > > > > nightly builds for hbase-operator-tools repo that builds against the > > > latest > > > > development branches as well as checking compatibility with released > > > > versions. > > > > > > > > bq. what would folks think about going with an hbck2 alpha release? > > > > I'm fine with an alpha release but since "HBCK2 should continuously > > > evolve" > > > > it might be better to always use the latest codebase whenever you > need > > to > > > > use the tool. > > > > > > > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < > > > > wellington.chevre...@gmail.com> wrote: > > > > > > > > > I would favour having hbck2 release branches. Temporary > compatibility > > > > > breaks at compile time may be inevitable if we always point to the > > > latest > > > > > release. That could cause problems for operators trying to build > > hbck2 > > > > (we > > > > > are already seeing this happening with our support team). Another > > > > argument > > > > > for starting having hbck2 releases is that we already have quite a > > few > > > > > hbase 2 releases, yet, the main tool to fix inconsistencies is not > > > easily > > > > > available. And there's been considerable efforts lately to bring > many > > > of > > > > > the fix options from hbck1 into hbck2, so what would folks think > > about > > > > > going with an hbck2 alpha release? > > > > > > > > > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser > > > > >
Re: [DISCUSS] HBCK2 compatibility
> > I do not think we need to compile HBCK2 with every releases? > Well, not with every release, was thinking in doing it whenever an hbase release breaks compatibility. We just need > make sure that it can work with all the releases. This could be a solution as well, but I believe it would be harder to guarantee. Here the problem is: 1) A new hbase release changes one or more interfaces currently used by hbck2; 2) We update hbck2 to depend on this new hbase release, and change hbck2 accordingly; 3) Operators need to run hbck2 to a previous hbase release. If they try to build hbck2 against that version, it won't compile. If they build it with latest hbase version, it may give a runtime error, and now they have no working tool to fix the problem. To avoid #3, we would need to add extra checks on the changes applied on #2. Might become too complex. If there are missing > methods, we just tell users you can not use several features. Fine for new fix options added, but what if some changes break basic, already working hbck2 methods. Em qui, 29 de ago de 2019 às 14:51, 张铎(Duo Zhang) escreveu: > I do not think we need to compile HBCK2 with every releases? We just need > make sure that it can work with all the releases. If there are missing > methods, we just tell users you can not use several features. > > Wellington Chevreuil 于2019年8月29日周四 > 下午9:39写道: > > > > > > > bq. what would folks think about going with an hbck2 alpha release? > > > I'm fine with an alpha release but since "HBCK2 should continuously > > evolve" > > > it might be better to always use the latest codebase whenever you need > to > > > use the tool. > > > > > Ideally yes, but that might not always be possible, as hbase API might > > change ahead of hbck2. Operators could then have problems to get a > working > > version of hbck2. Since hbck2 already has now many equivalent options for > > the ones from hbck1, I guess a first release would provide a working > hbck2 > > that already brings a considerable number of fix methods to help with > most > > common inconsistencies issues seen in hbase 2 so far. And we could still > > fulfil the "HBCK2 should continuously evolve" principle by building from > > its master branch, in scenarios where a new fix was needed and > implemented > > back into hbck2. > > > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi > > escreveu: > > > > > bq. Is it possible to put some hacks into HBCK2 to work around > > > the compatibility to fix the current state > > > > > > There are some classes around Replication which were introduced in > 2.1.0+ > > > so I don't think we could easily solve it for 2.0. > > > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, > > probably > > > that could be solved with some workarounds or using reflection. > > > > > > bq. focus more on automation to let us know the next time we inevitably > > > break it again? ;) > > > > > > Sure! Based on this I think it should be a strong goal. We can set up > > > nightly builds for hbase-operator-tools repo that builds against the > > latest > > > development branches as well as checking compatibility with released > > > versions. > > > > > > bq. what would folks think about going with an hbck2 alpha release? > > > I'm fine with an alpha release but since "HBCK2 should continuously > > evolve" > > > it might be better to always use the latest codebase whenever you need > to > > > use the tool. > > > > > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < > > > wellington.chevre...@gmail.com> wrote: > > > > > > > I would favour having hbck2 release branches. Temporary compatibility > > > > breaks at compile time may be inevitable if we always point to the > > latest > > > > release. That could cause problems for operators trying to build > hbck2 > > > (we > > > > are already seeing this happening with our support team). Another > > > argument > > > > for starting having hbck2 releases is that we already have quite a > few > > > > hbase 2 releases, yet, the main tool to fix inconsistencies is not > > easily > > > > available. And there's been considerable efforts lately to bring many > > of > > > > the fix options from hbck1 into hbck2, so what would folks think > about > > > > going with an hbck2 alpha release? > > > > > > > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser > > > > escreveu: > > > > > > > > > I still like one HBCK2 release as the goal. > > > > > > > > > > Is it possible to put some hacks into HBCK2 to work around the > > > > > compatibility to fix the current state and focus more on automation > > to > > > > > let us know the next time we inevitably break it again? ;) > > > > > > > > > > On 8/29/19 8:12 AM, Peter Somogyi wrote: > > > > > > Hi everyone, > > > > > > > > > > > > This topic came up a couple of times already but now we reached a > > > point > > > > > > when the recent HBCK2 is incompatible with older HBase releases, > > > > > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2
Re: [DISCUSS] HBCK2 compatibility
I do not think we need to compile HBCK2 with every releases? We just need make sure that it can work with all the releases. If there are missing methods, we just tell users you can not use several features. Wellington Chevreuil 于2019年8月29日周四 下午9:39写道: > > > > bq. what would folks think about going with an hbck2 alpha release? > > I'm fine with an alpha release but since "HBCK2 should continuously > evolve" > > it might be better to always use the latest codebase whenever you need to > > use the tool. > > > Ideally yes, but that might not always be possible, as hbase API might > change ahead of hbck2. Operators could then have problems to get a working > version of hbck2. Since hbck2 already has now many equivalent options for > the ones from hbck1, I guess a first release would provide a working hbck2 > that already brings a considerable number of fix methods to help with most > common inconsistencies issues seen in hbase 2 so far. And we could still > fulfil the "HBCK2 should continuously evolve" principle by building from > its master branch, in scenarios where a new fix was needed and implemented > back into hbck2. > > Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi > escreveu: > > > bq. Is it possible to put some hacks into HBCK2 to work around > > the compatibility to fix the current state > > > > There are some classes around Replication which were introduced in 2.1.0+ > > so I don't think we could easily solve it for 2.0. > > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, > probably > > that could be solved with some workarounds or using reflection. > > > > bq. focus more on automation to let us know the next time we inevitably > > break it again? ;) > > > > Sure! Based on this I think it should be a strong goal. We can set up > > nightly builds for hbase-operator-tools repo that builds against the > latest > > development branches as well as checking compatibility with released > > versions. > > > > bq. what would folks think about going with an hbck2 alpha release? > > I'm fine with an alpha release but since "HBCK2 should continuously > evolve" > > it might be better to always use the latest codebase whenever you need to > > use the tool. > > > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < > > wellington.chevre...@gmail.com> wrote: > > > > > I would favour having hbck2 release branches. Temporary compatibility > > > breaks at compile time may be inevitable if we always point to the > latest > > > release. That could cause problems for operators trying to build hbck2 > > (we > > > are already seeing this happening with our support team). Another > > argument > > > for starting having hbck2 releases is that we already have quite a few > > > hbase 2 releases, yet, the main tool to fix inconsistencies is not > easily > > > available. And there's been considerable efforts lately to bring many > of > > > the fix options from hbck1 into hbck2, so what would folks think about > > > going with an hbck2 alpha release? > > > > > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser > > > escreveu: > > > > > > > I still like one HBCK2 release as the goal. > > > > > > > > Is it possible to put some hacks into HBCK2 to work around the > > > > compatibility to fix the current state and focus more on automation > to > > > > let us know the next time we inevitably break it again? ;) > > > > > > > > On 8/29/19 8:12 AM, Peter Somogyi wrote: > > > > > Hi everyone, > > > > > > > > > > This topic came up a couple of times already but now we reached a > > point > > > > > when the recent HBCK2 is incompatible with older HBase releases, > > > > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 against > one > > > of > > > > > the previously mentioned releases you will get compilation errors. > > (mvn > > > > > clean install -DskipTests -Dhbase.version=2.0.6) > > > > > > > > > > Our previous goal was to maintain compatibility with HBCK2 and all > > > HBase > > > > 2 > > > > > releases. Now we missed this target. > > > > > > > > > > One option that we could do is to have different branches/releases > of > > > > HBCK2 > > > > > targeted for specific HBase releases (e.g. branch-2.0 version of > > > HBCK2). > > > > > This probably makes the development on HBCK2 a bit harder since > we'll > > > > have > > > > > to take care of multiple branches. > > > > > > > > > > Another option I could think of is to always build HBCK2 with the > > > latest > > > > > HBase release but have version checks on individual commands where > we > > > > could > > > > > decide if it is supported on that release line. > > > > > > > > > > What are your opinions on this? > > > > > > > > > > Thanks, > > > > > Peter > > > > > > > > > > > > > > >
Re: [DISCUSS] HBCK2 compatibility
> > bq. what would folks think about going with an hbck2 alpha release? > I'm fine with an alpha release but since "HBCK2 should continuously evolve" > it might be better to always use the latest codebase whenever you need to > use the tool. > Ideally yes, but that might not always be possible, as hbase API might change ahead of hbck2. Operators could then have problems to get a working version of hbck2. Since hbck2 already has now many equivalent options for the ones from hbck1, I guess a first release would provide a working hbck2 that already brings a considerable number of fix methods to help with most common inconsistencies issues seen in hbase 2 so far. And we could still fulfil the "HBCK2 should continuously evolve" principle by building from its master branch, in scenarios where a new fix was needed and implemented back into hbck2. Em qui, 29 de ago de 2019 às 14:09, Peter Somogyi escreveu: > bq. Is it possible to put some hacks into HBCK2 to work around > the compatibility to fix the current state > > There are some classes around Replication which were introduced in 2.1.0+ > so I don't think we could easily solve it for 2.0. > For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, probably > that could be solved with some workarounds or using reflection. > > bq. focus more on automation to let us know the next time we inevitably > break it again? ;) > > Sure! Based on this I think it should be a strong goal. We can set up > nightly builds for hbase-operator-tools repo that builds against the latest > development branches as well as checking compatibility with released > versions. > > bq. what would folks think about going with an hbck2 alpha release? > I'm fine with an alpha release but since "HBCK2 should continuously evolve" > it might be better to always use the latest codebase whenever you need to > use the tool. > > On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < > wellington.chevre...@gmail.com> wrote: > > > I would favour having hbck2 release branches. Temporary compatibility > > breaks at compile time may be inevitable if we always point to the latest > > release. That could cause problems for operators trying to build hbck2 > (we > > are already seeing this happening with our support team). Another > argument > > for starting having hbck2 releases is that we already have quite a few > > hbase 2 releases, yet, the main tool to fix inconsistencies is not easily > > available. And there's been considerable efforts lately to bring many of > > the fix options from hbck1 into hbck2, so what would folks think about > > going with an hbck2 alpha release? > > > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser > > escreveu: > > > > > I still like one HBCK2 release as the goal. > > > > > > Is it possible to put some hacks into HBCK2 to work around the > > > compatibility to fix the current state and focus more on automation to > > > let us know the next time we inevitably break it again? ;) > > > > > > On 8/29/19 8:12 AM, Peter Somogyi wrote: > > > > Hi everyone, > > > > > > > > This topic came up a couple of times already but now we reached a > point > > > > when the recent HBCK2 is incompatible with older HBase releases, > > > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 against one > > of > > > > the previously mentioned releases you will get compilation errors. > (mvn > > > > clean install -DskipTests -Dhbase.version=2.0.6) > > > > > > > > Our previous goal was to maintain compatibility with HBCK2 and all > > HBase > > > 2 > > > > releases. Now we missed this target. > > > > > > > > One option that we could do is to have different branches/releases of > > > HBCK2 > > > > targeted for specific HBase releases (e.g. branch-2.0 version of > > HBCK2). > > > > This probably makes the development on HBCK2 a bit harder since we'll > > > have > > > > to take care of multiple branches. > > > > > > > > Another option I could think of is to always build HBCK2 with the > > latest > > > > HBase release but have version checks on individual commands where we > > > could > > > > decide if it is supported on that release line. > > > > > > > > What are your opinions on this? > > > > > > > > Thanks, > > > > Peter > > > > > > > > > >
Re: [DISCUSS] HBCK2 compatibility
bq. Is it possible to put some hacks into HBCK2 to work around the compatibility to fix the current state There are some classes around Replication which were introduced in 2.1.0+ so I don't think we could easily solve it for 2.0. For 2.1.1 the missing method is Hbck#scheduleServerCrashProcedure, probably that could be solved with some workarounds or using reflection. bq. focus more on automation to let us know the next time we inevitably break it again? ;) Sure! Based on this I think it should be a strong goal. We can set up nightly builds for hbase-operator-tools repo that builds against the latest development branches as well as checking compatibility with released versions. bq. what would folks think about going with an hbck2 alpha release? I'm fine with an alpha release but since "HBCK2 should continuously evolve" it might be better to always use the latest codebase whenever you need to use the tool. On Thu, Aug 29, 2019 at 2:30 PM Wellington Chevreuil < wellington.chevre...@gmail.com> wrote: > I would favour having hbck2 release branches. Temporary compatibility > breaks at compile time may be inevitable if we always point to the latest > release. That could cause problems for operators trying to build hbck2 (we > are already seeing this happening with our support team). Another argument > for starting having hbck2 releases is that we already have quite a few > hbase 2 releases, yet, the main tool to fix inconsistencies is not easily > available. And there's been considerable efforts lately to bring many of > the fix options from hbck1 into hbck2, so what would folks think about > going with an hbck2 alpha release? > > Em qui, 29 de ago de 2019 às 13:20, Josh Elser > escreveu: > > > I still like one HBCK2 release as the goal. > > > > Is it possible to put some hacks into HBCK2 to work around the > > compatibility to fix the current state and focus more on automation to > > let us know the next time we inevitably break it again? ;) > > > > On 8/29/19 8:12 AM, Peter Somogyi wrote: > > > Hi everyone, > > > > > > This topic came up a couple of times already but now we reached a point > > > when the recent HBCK2 is incompatible with older HBase releases, > > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 against one > of > > > the previously mentioned releases you will get compilation errors. (mvn > > > clean install -DskipTests -Dhbase.version=2.0.6) > > > > > > Our previous goal was to maintain compatibility with HBCK2 and all > HBase > > 2 > > > releases. Now we missed this target. > > > > > > One option that we could do is to have different branches/releases of > > HBCK2 > > > targeted for specific HBase releases (e.g. branch-2.0 version of > HBCK2). > > > This probably makes the development on HBCK2 a bit harder since we'll > > have > > > to take care of multiple branches. > > > > > > Another option I could think of is to always build HBCK2 with the > latest > > > HBase release but have version checks on individual commands where we > > could > > > decide if it is supported on that release line. > > > > > > What are your opinions on this? > > > > > > Thanks, > > > Peter > > > > > >
Re: [DISCUSS] HBCK2 compatibility
I would favour having hbck2 release branches. Temporary compatibility breaks at compile time may be inevitable if we always point to the latest release. That could cause problems for operators trying to build hbck2 (we are already seeing this happening with our support team). Another argument for starting having hbck2 releases is that we already have quite a few hbase 2 releases, yet, the main tool to fix inconsistencies is not easily available. And there's been considerable efforts lately to bring many of the fix options from hbck1 into hbck2, so what would folks think about going with an hbck2 alpha release? Em qui, 29 de ago de 2019 às 13:20, Josh Elser escreveu: > I still like one HBCK2 release as the goal. > > Is it possible to put some hacks into HBCK2 to work around the > compatibility to fix the current state and focus more on automation to > let us know the next time we inevitably break it again? ;) > > On 8/29/19 8:12 AM, Peter Somogyi wrote: > > Hi everyone, > > > > This topic came up a couple of times already but now we reached a point > > when the recent HBCK2 is incompatible with older HBase releases, > > specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 against one of > > the previously mentioned releases you will get compilation errors. (mvn > > clean install -DskipTests -Dhbase.version=2.0.6) > > > > Our previous goal was to maintain compatibility with HBCK2 and all HBase > 2 > > releases. Now we missed this target. > > > > One option that we could do is to have different branches/releases of > HBCK2 > > targeted for specific HBase releases (e.g. branch-2.0 version of HBCK2). > > This probably makes the development on HBCK2 a bit harder since we'll > have > > to take care of multiple branches. > > > > Another option I could think of is to always build HBCK2 with the latest > > HBase release but have version checks on individual commands where we > could > > decide if it is supported on that release line. > > > > What are your opinions on this? > > > > Thanks, > > Peter > > >
Re: [DISCUSS] HBCK2 compatibility
I still like one HBCK2 release as the goal. Is it possible to put some hacks into HBCK2 to work around the compatibility to fix the current state and focus more on automation to let us know the next time we inevitably break it again? ;) On 8/29/19 8:12 AM, Peter Somogyi wrote: Hi everyone, This topic came up a couple of times already but now we reached a point when the recent HBCK2 is incompatible with older HBase releases, specifically 2.0.x, 2.1.0 and 2.1.1. When you build HBCK2 against one of the previously mentioned releases you will get compilation errors. (mvn clean install -DskipTests -Dhbase.version=2.0.6) Our previous goal was to maintain compatibility with HBCK2 and all HBase 2 releases. Now we missed this target. One option that we could do is to have different branches/releases of HBCK2 targeted for specific HBase releases (e.g. branch-2.0 version of HBCK2). This probably makes the development on HBCK2 a bit harder since we'll have to take care of multiple branches. Another option I could think of is to always build HBCK2 with the latest HBase release but have version checks on individual commands where we could decide if it is supported on that release line. What are your opinions on this? Thanks, Peter