Re: [DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
Yeah, the API is a good idea for sure, but it does not solve problems of specific version dependencies in compile time. This HBASE-21393 is a good example, if we don't raise hbase version to 2.1.2, project does not even compile. So unless we come up with an approach to call master rpc methods using reflection, for example, we would still need to track version numbers dependency between hbase/hbck2. Em qui, 4 de abr de 2019 às 15:41, Sean Busbey escreveu: > HBASE-21393 already hit a case where the maintenance version will matter > (e.g. works in 2.1.2 but not 2.1.1). > > I don't see how tracking ever more complicated sets of version numbers in > HBCK2 is going to be simpler than an API in the service that can send back > some strings. > > On Wed, Apr 3, 2019 at 5:46 AM Wellington Chevreuil < > wellington.chevre...@gmail.com> wrote: > > > Unfortunately, if we plan to have an hbck2 capable of evolving ahead of > > server/master API, version mappings between hbck2/hbase project seems > > unavoidable. I guess it would need to rely on best practices guidelines, > > such as making sure to document minimal hbase version required for new > > methods relying on server/master API. Maybe a framework for commands > > implementation, where remote methods would be invoked reflectively, to > > avoid compile time headache of finding the most recent hbck2 version that > > compiles against target cluster hbase version. > > > > Em ter, 2 de abr de 2019 às 18:53, Stack escreveu: > > > > > On Tue, Apr 2, 2019 at 7:14 AM Sean Busbey wrote: > > > > > > > Should we instead make sure the hbck service in master is more > > pluggable? > > > > Is it poking at stuff going to be too version specific? > > > > > > > > > > > I like the sentiment. > > > > > > Up to this, we've been adding API as we figure what we need. Meantime > > > hbase's ship and are missing the latest API additions. > > > > > > More universal API-types would help I suppose though hard in practice > or > > we > > > could work on a means of loading code into Master or enabling an > > > interpreter or some such (Or something like Allan's servlet loading > > trick). > > > > > > > > > > Instead of version checking gymnastics in hbck2 could we have the > hbck > > > > service report available feature set? > > > > > > > > > > > Strikes me as overkill at the moment but would be a nice addition. > > > > > > S > > > > > > > > > > On Mon, Apr 1, 2019 at 1:21 PM Stack wrote: > > > > > > > > > On Mon, Apr 1, 2019 at 9:56 AM Wellington Chevreuil < > > > > > wellington.chevre...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Am interested to hear what's the general opinion on the way to > move > > > > > forward > > > > > > with hbck2? Should we completely discourage client bound logic > > > patches > > > > > such > > > > > > as the one from HBASE-22143 ? > > > > > > > > > > > > > > > > > > > > > No. If in a bind -- need to change a state but don't have the > > facility > > > on > > > > > the master to do it in the kosher way -- then you have no recourse > > but > > > to > > > > > do 'dirty tricks' as HBASE-22143 does making use of 'private' > > > Interfaces. > > > > > > > > > > What you think Wellington of checking master version in the > > client-side > > > > and > > > > > doing 'dirty tricks' version if master version is old but taking > the > > > > > master-route when it provides support; i.e. when making a fix, we'd > > do > > > > two > > > > > versions: one via the ordained route of asking the master to make > the > > > > > change for us, and then a workaround second version for those > deploys > > > > that > > > > > don't currently have the master-side support? > > > > > > > > > > Thanks for bringing up this topic, > > > > > S > > > > > > > > > > > > > > >
Re: [DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
HBASE-21393 already hit a case where the maintenance version will matter (e.g. works in 2.1.2 but not 2.1.1). I don't see how tracking ever more complicated sets of version numbers in HBCK2 is going to be simpler than an API in the service that can send back some strings. On Wed, Apr 3, 2019 at 5:46 AM Wellington Chevreuil < wellington.chevre...@gmail.com> wrote: > Unfortunately, if we plan to have an hbck2 capable of evolving ahead of > server/master API, version mappings between hbck2/hbase project seems > unavoidable. I guess it would need to rely on best practices guidelines, > such as making sure to document minimal hbase version required for new > methods relying on server/master API. Maybe a framework for commands > implementation, where remote methods would be invoked reflectively, to > avoid compile time headache of finding the most recent hbck2 version that > compiles against target cluster hbase version. > > Em ter, 2 de abr de 2019 às 18:53, Stack escreveu: > > > On Tue, Apr 2, 2019 at 7:14 AM Sean Busbey wrote: > > > > > Should we instead make sure the hbck service in master is more > pluggable? > > > Is it poking at stuff going to be too version specific? > > > > > > > > I like the sentiment. > > > > Up to this, we've been adding API as we figure what we need. Meantime > > hbase's ship and are missing the latest API additions. > > > > More universal API-types would help I suppose though hard in practice or > we > > could work on a means of loading code into Master or enabling an > > interpreter or some such (Or something like Allan's servlet loading > trick). > > > > > > > Instead of version checking gymnastics in hbck2 could we have the hbck > > > service report available feature set? > > > > > > > > Strikes me as overkill at the moment but would be a nice addition. > > > > S > > > > > > > On Mon, Apr 1, 2019 at 1:21 PM Stack wrote: > > > > > > > On Mon, Apr 1, 2019 at 9:56 AM Wellington Chevreuil < > > > > wellington.chevre...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Am interested to hear what's the general opinion on the way to move > > > > forward > > > > > with hbck2? Should we completely discourage client bound logic > > patches > > > > such > > > > > as the one from HBASE-22143 ? > > > > > > > > > > > > > > > > > No. If in a bind -- need to change a state but don't have the > facility > > on > > > > the master to do it in the kosher way -- then you have no recourse > but > > to > > > > do 'dirty tricks' as HBASE-22143 does making use of 'private' > > Interfaces. > > > > > > > > What you think Wellington of checking master version in the > client-side > > > and > > > > doing 'dirty tricks' version if master version is old but taking the > > > > master-route when it provides support; i.e. when making a fix, we'd > do > > > two > > > > versions: one via the ordained route of asking the master to make the > > > > change for us, and then a workaround second version for those deploys > > > that > > > > don't currently have the master-side support? > > > > > > > > Thanks for bringing up this topic, > > > > S > > > > > > > > > >
Re: [DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
Unfortunately, if we plan to have an hbck2 capable of evolving ahead of server/master API, version mappings between hbck2/hbase project seems unavoidable. I guess it would need to rely on best practices guidelines, such as making sure to document minimal hbase version required for new methods relying on server/master API. Maybe a framework for commands implementation, where remote methods would be invoked reflectively, to avoid compile time headache of finding the most recent hbck2 version that compiles against target cluster hbase version. Em ter, 2 de abr de 2019 às 18:53, Stack escreveu: > On Tue, Apr 2, 2019 at 7:14 AM Sean Busbey wrote: > > > Should we instead make sure the hbck service in master is more pluggable? > > Is it poking at stuff going to be too version specific? > > > > > I like the sentiment. > > Up to this, we've been adding API as we figure what we need. Meantime > hbase's ship and are missing the latest API additions. > > More universal API-types would help I suppose though hard in practice or we > could work on a means of loading code into Master or enabling an > interpreter or some such (Or something like Allan's servlet loading trick). > > > > Instead of version checking gymnastics in hbck2 could we have the hbck > > service report available feature set? > > > > > Strikes me as overkill at the moment but would be a nice addition. > > S > > > > On Mon, Apr 1, 2019 at 1:21 PM Stack wrote: > > > > > On Mon, Apr 1, 2019 at 9:56 AM Wellington Chevreuil < > > > wellington.chevre...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Am interested to hear what's the general opinion on the way to move > > > forward > > > > with hbck2? Should we completely discourage client bound logic > patches > > > such > > > > as the one from HBASE-22143 ? > > > > > > > > > > > > > No. If in a bind -- need to change a state but don't have the facility > on > > > the master to do it in the kosher way -- then you have no recourse but > to > > > do 'dirty tricks' as HBASE-22143 does making use of 'private' > Interfaces. > > > > > > What you think Wellington of checking master version in the client-side > > and > > > doing 'dirty tricks' version if master version is old but taking the > > > master-route when it provides support; i.e. when making a fix, we'd do > > two > > > versions: one via the ordained route of asking the master to make the > > > change for us, and then a workaround second version for those deploys > > that > > > don't currently have the master-side support? > > > > > > Thanks for bringing up this topic, > > > S > > > > > >
Re: [DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
On Tue, Apr 2, 2019 at 7:14 AM Sean Busbey wrote: > Should we instead make sure the hbck service in master is more pluggable? > Is it poking at stuff going to be too version specific? > > I like the sentiment. Up to this, we've been adding API as we figure what we need. Meantime hbase's ship and are missing the latest API additions. More universal API-types would help I suppose though hard in practice or we could work on a means of loading code into Master or enabling an interpreter or some such (Or something like Allan's servlet loading trick). > Instead of version checking gymnastics in hbck2 could we have the hbck > service report available feature set? > > Strikes me as overkill at the moment but would be a nice addition. S > On Mon, Apr 1, 2019 at 1:21 PM Stack wrote: > > > On Mon, Apr 1, 2019 at 9:56 AM Wellington Chevreuil < > > wellington.chevre...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > Am interested to hear what's the general opinion on the way to move > > forward > > > with hbck2? Should we completely discourage client bound logic patches > > such > > > as the one from HBASE-22143 ? > > > > > > > > > No. If in a bind -- need to change a state but don't have the facility on > > the master to do it in the kosher way -- then you have no recourse but to > > do 'dirty tricks' as HBASE-22143 does making use of 'private' Interfaces. > > > > What you think Wellington of checking master version in the client-side > and > > doing 'dirty tricks' version if master version is old but taking the > > master-route when it provides support; i.e. when making a fix, we'd do > two > > versions: one via the ordained route of asking the master to make the > > change for us, and then a workaround second version for those deploys > that > > don't currently have the master-side support? > > > > Thanks for bringing up this topic, > > S > > >
Re: [DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
Should we instead make sure the hbck service in master is more pluggable? Is it poking at stuff going to be too version specific? Instead of version checking gymnastics in hbck2 could we have the hbck service report available feature set? On Mon, Apr 1, 2019 at 1:21 PM Stack wrote: > On Mon, Apr 1, 2019 at 9:56 AM Wellington Chevreuil < > wellington.chevre...@gmail.com> wrote: > > > > > > > > > > > Am interested to hear what's the general opinion on the way to move > forward > > with hbck2? Should we completely discourage client bound logic patches > such > > as the one from HBASE-22143 ? > > > > > No. If in a bind -- need to change a state but don't have the facility on > the master to do it in the kosher way -- then you have no recourse but to > do 'dirty tricks' as HBASE-22143 does making use of 'private' Interfaces. > > What you think Wellington of checking master version in the client-side and > doing 'dirty tricks' version if master version is old but taking the > master-route when it provides support; i.e. when making a fix, we'd do two > versions: one via the ordained route of asking the master to make the > change for us, and then a workaround second version for those deploys that > don't currently have the master-side support? > > Thanks for bringing up this topic, > S >
Re: [DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
On Mon, Apr 1, 2019 at 9:56 AM Wellington Chevreuil < wellington.chevre...@gmail.com> wrote: > > > > Am interested to hear what's the general opinion on the way to move forward > with hbck2? Should we completely discourage client bound logic patches such > as the one from HBASE-22143 ? > No. If in a bind -- need to change a state but don't have the facility on the master to do it in the kosher way -- then you have no recourse but to do 'dirty tricks' as HBASE-22143 does making use of 'private' Interfaces. What you think Wellington of checking master version in the client-side and doing 'dirty tricks' version if master version is old but taking the master-route when it provides support; i.e. when making a fix, we'd do two versions: one via the ordained route of asking the master to make the change for us, and then a workaround second version for those deploys that don't currently have the master-side support? Thanks for bringing up this topic, S
[DISCUSS] hbck2 evolve independently X avoiding having client side logic on hbck2 commands
My understanding on the motivation of moving hbck2 (and potentially other operation related tools) to a separate project was to allow it to evolve on its own cadence, as new interventions to fix previously unforeseen problems were identified, we would now be able to deliver updated tool versions without requiring a new release on the main projects, therefore avoiding the need of a cluster upgrade. At same time, the current pattern for hbck2 at least is to avoid giving too much control of regions assignment inconsistencies over to the client, favouring an approach where hbck2 client mainly act as a proxy for fix methods exposed at master RPC interface. That way, master is able to keep track of all procedures manipulating regions, where an independent process manipulating regions at same time could cause more problems than fix it (we saw this happening many times with misuse of hbck v1 methods). However, with such restriction we are again limiting the ability to evolve independently of main project releases and cluster upgrades. With growing adoption of branch-2 based releases by our customer base, there's been a surge on assignment issues, which can't be fixed solely with the existing hbck2 commands. One example I had is what motivated HBASE-22143. I would favour the approach of allowing more complex client methods, mainly because it allows for adding new fixes on demand, but there's also value on keeping things owned by master. Am interested to hear what's the general opinion on the way to move forward with hbck2? Should we completely discourage client bound logic patches such as the one from HBASE-22143 ?
