Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Thanks. I filed a JIRA to track it. https://issues.apache.org/jira/browse/HBASE-19145 On Mon, Oct 30, 2017 at 9:36 PM, Stackwrote: > On Mon, Oct 30, 2017 at 11:46 AM, Jerry He wrote: > >> Hi, Stack >> >> Coming back to this thread, i have meant to ask this question long ago. >> Do we support hbase-2 client going against hbase-1 server? >> > > It has not been an objective. It might work doing basic Client API on a > later branch-1 but will fail doing Admin functions (and figuring if a Table > is online). I've not tried it Jerry. If it was a small thing to make it > work, lets get it in. > > St.Ack > > > >> We seem to be fine mix-and-match the clients and servers within the >> hbase-1 releases. And IIRC hbase-1 client is ok against 0.98 server. >> Suppose I have a product that depends and bundles HBase client. I >> want to upgrade the dependency to hbase-2 so that it can take >> advantages of and claim support of hbase-2. >> But does it mean that I will need drop the claims that the new version >> of the product support any hbase-1 backend? >> >> Thanks. >> >> >> >> On Tue, Sep 12, 2017 at 10:21 AM, Stack wrote: >> > Let me put this one on this thread, G1GC on by default in hbase2? >> > St.Ack >> > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: >> > >> >> What are our expectations regards compatibility between hbase1 and >> hbase2? >> >> >> >> Lets have a chat about it. Here are some goal posts. >> >> >> >> + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No >> >> migration from < hbase-1 (Is this too onerous? Should we support 0.98 => >> >> 2.0?). >> >> + You do NOT have to upgrade to the latest release of hbase1 to migrate >> to >> >> hbase2; being up on hbase-1.0.0+ will be sufficient. >> >> + You'll have to update your hbase1 coprocessors to deploy them on >> hbase2. >> >> A bunch of CP API has/will change by the time hbase2 comes out; e.g. >> >> watching for region split on RegionServer no longer makes sense given >> >> Master runs all splits now. >> >> + An hbase1 client can run against an hbase2 cluster but it will only be >> >> able to do DML (Get/Put/Scan, etc.). We do not allow being able to do >> admin >> >> ops using an hbase1 Admin client against an hbase2 cluster. We have some >> >> egregious API violations in branch-1; e.g. we have protobuf in our API >> (See >> >> HBASE-15607). The notion is that we can't afford a deprecation cycle >> >> purging this stuff from our Admin API. >> >> >> >> What you all think? >> >> >> >> St.Ack >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Mon, Oct 30, 2017 at 11:46 AM, Jerry Hewrote: > Hi, Stack > > Coming back to this thread, i have meant to ask this question long ago. > Do we support hbase-2 client going against hbase-1 server? > It has not been an objective. It might work doing basic Client API on a later branch-1 but will fail doing Admin functions (and figuring if a Table is online). I've not tried it Jerry. If it was a small thing to make it work, lets get it in. St.Ack > We seem to be fine mix-and-match the clients and servers within the > hbase-1 releases. And IIRC hbase-1 client is ok against 0.98 server. > Suppose I have a product that depends and bundles HBase client. I > want to upgrade the dependency to hbase-2 so that it can take > advantages of and claim support of hbase-2. > But does it mean that I will need drop the claims that the new version > of the product support any hbase-1 backend? > > Thanks. > > > > On Tue, Sep 12, 2017 at 10:21 AM, Stack wrote: > > Let me put this one on this thread, G1GC on by default in hbase2? > > St.Ack > > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > >> What are our expectations regards compatibility between hbase1 and > hbase2? > >> > >> Lets have a chat about it. Here are some goal posts. > >> > >> + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No > >> migration from < hbase-1 (Is this too onerous? Should we support 0.98 => > >> 2.0?). > >> + You do NOT have to upgrade to the latest release of hbase1 to migrate > to > >> hbase2; being up on hbase-1.0.0+ will be sufficient. > >> + You'll have to update your hbase1 coprocessors to deploy them on > hbase2. > >> A bunch of CP API has/will change by the time hbase2 comes out; e.g. > >> watching for region split on RegionServer no longer makes sense given > >> Master runs all splits now. > >> + An hbase1 client can run against an hbase2 cluster but it will only be > >> able to do DML (Get/Put/Scan, etc.). We do not allow being able to do > admin > >> ops using an hbase1 Admin client against an hbase2 cluster. We have some > >> egregious API violations in branch-1; e.g. we have protobuf in our API > (See > >> HBASE-15607). The notion is that we can't afford a deprecation cycle > >> purging this stuff from our Admin API. > >> > >> What you all think? > >> > >> St.Ack > >> > >> > >> > >> > >> > >> > >> > >> > >> >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Hi, Stack Coming back to this thread, i have meant to ask this question long ago. Do we support hbase-2 client going against hbase-1 server? We seem to be fine mix-and-match the clients and servers within the hbase-1 releases. And IIRC hbase-1 client is ok against 0.98 server. Suppose I have a product that depends and bundles HBase client. I want to upgrade the dependency to hbase-2 so that it can take advantages of and claim support of hbase-2. But does it mean that I will need drop the claims that the new version of the product support any hbase-1 backend? Thanks. On Tue, Sep 12, 2017 at 10:21 AM, Stackwrote: > Let me put this one on this thread, G1GC on by default in hbase2? > St.Ack > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > >> What are our expectations regards compatibility between hbase1 and hbase2? >> >> Lets have a chat about it. Here are some goal posts. >> >> + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No >> migration from < hbase-1 (Is this too onerous? Should we support 0.98 => >> 2.0?). >> + You do NOT have to upgrade to the latest release of hbase1 to migrate to >> hbase2; being up on hbase-1.0.0+ will be sufficient. >> + You'll have to update your hbase1 coprocessors to deploy them on hbase2. >> A bunch of CP API has/will change by the time hbase2 comes out; e.g. >> watching for region split on RegionServer no longer makes sense given >> Master runs all splits now. >> + An hbase1 client can run against an hbase2 cluster but it will only be >> able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin >> ops using an hbase1 Admin client against an hbase2 cluster. We have some >> egregious API violations in branch-1; e.g. we have protobuf in our API (See >> HBASE-15607). The notion is that we can't afford a deprecation cycle >> purging this stuff from our Admin API. >> >> What you all think? >> >> St.Ack >> >> >> >> >> >> >> >> >>
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Let me put this one on this thread, G1GC on by default in hbase2? St.Ack On Wed, Aug 2, 2017 at 4:38 PM, Stackwrote: > What are our expectations regards compatibility between hbase1 and hbase2? > > Lets have a chat about it. Here are some goal posts. > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No > migration from < hbase-1 (Is this too onerous? Should we support 0.98 => > 2.0?). > + You do NOT have to upgrade to the latest release of hbase1 to migrate to > hbase2; being up on hbase-1.0.0+ will be sufficient. > + You'll have to update your hbase1 coprocessors to deploy them on hbase2. > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > watching for region split on RegionServer no longer makes sense given > Master runs all splits now. > + An hbase1 client can run against an hbase2 cluster but it will only be > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin > ops using an hbase1 Admin client against an hbase2 cluster. We have some > egregious API violations in branch-1; e.g. we have protobuf in our API (See > HBASE-15607). The notion is that we can't afford a deprecation cycle > purging this stuff from our Admin API. > > What you all think? > > St.Ack > > > > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Mon, Sep 4, 2017 at 6:55 PM, Francis Christopher Liu < toffer@gmail.com> wrote: > Just some high-level thoughts on rolling upgradeability (I'm repeating a > few things already said). > > 1. No service interruption for 1.x clients while a cluster is being > ugpraded to 2.x. - This includes support for apis commonly used in a running system: > delegation tokens, getting region locations, etc. > Agreed. Having hbase-1 clients read the basic info they need to operate seems doable w/ a little bit of work (having meta table edits mirrored in zk). Let me add note to test delegation tokens continue to work. > 2. The somewhat reverse of #1 is also necessary for Master DMLs to meta. > Come again. hbase1 won't be able to admin a hbase2 (basic read-only admin function will work like checking if table is online or not, etc.) > 3. No interruption for replication between 1.x and 2.x cluster > 4. Restart master first before any other service sounds reasonable to me. > Will we support RU for zkless mode? > I'll not be testing RU for ZKLESS mode. Any one up for this? > 5. 2.x should be either able to understand formats written by 1.x or online > migration is done as a preparation step. > Yep. Auto-migration preferred. > 6. Data generated by 2.x daemons should be readable in 1.x (ie HFile, > zookeeper, etc). Some deploys may have large amounts of data could be > cumbersome/impractical (ie cost, time, etc) to copy the data. > > Thanks Francis, S > Thanks, > Francis > > > > On Fri, Sep 1, 2017 at 10:01 PM Chia-Ping Tsai> wrote: > > > The most of issues use hadoop flag, so the filter looks like this. > > project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, 2.0.0-alpha-2, > > 2.0.0-alpha-3, > > 3.0) AND (labels in (incompatibleChange, incompatible, incompatibility) > OR > > "Hadoop Flags" in ("Incompatible change")) > > > > On 2017-08-04 05:46, Ted Yu wrote: > > > I expanded the condition in the filter like this: > > > > > > project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, > 2.0.0-alpha-2, > > > 3.0) AND labels in (incompatibleChange, incompatible, incompatibility) > > > > > > Still there're only two showing up. > > > > > > On Thu, Aug 3, 2017 at 9:07 AM, Zach York < > zyork.contribut...@gmail.com> > > > wrote: > > > > > > > I tried to filter based on imcompatible labels and there were only > two > > > > JIRAs returned [1]. I have a hard time believing that there were only > > two > > > > breaking changes from 1.x to 2.x. > > > > > > > > [1] > > > > https://issues.apache.org/jira/browse/HBASE-17957?jql= > > > > project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.0.0% > > > > 20AND%20labels%20in%20(incompatibleChange%2C%20incompatible%2C% > > > > 20incompatibility) > > > > > > > > On Thu, Aug 3, 2017 at 8:54 AM, Zach York < > > zyork.contribut...@gmail.com> > > > > wrote: > > > > > > > > > This kinda helps, but these seem more like expectations. I was > going > > more > > > > > for things like HFile format changed, meta table structure changed, > > > > > coprocessor implementations changed (these are just examples, I > don't > > > > know > > > > > if any of these actually changed). > > > > > > > > > > More technical differences between branch-1 and branch-2 which then > > can > > > > > help us get the right expectations for compatibility. > > > > > > > > > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > > > > > > > > >> On Wed, Aug 2, 2017 at 5:25 PM, Zach York < > > zyork.contribut...@gmail.com > > > > > > > > > >> wrote: > > > > >> > > > > >> > Do we know what the major pain points for migration are? Can we > > > > discuss > > > > >> > that/get a list going? > > > > >> > > > > > >> > > > > > >> Here's a few in outline: > > > > >> > > > > >> + There is issue of formats, of hbase-2.x being able to read > > hbase-1.x > > > > >> data > > > > >> whether from HDFS or ZooKeeper or off the wire. > > > > >> + An hbase-1.x client should be able to Get/Put and Scan an > > hbase-2.x > > > > >> cluster; no holes in the API or unintelligible serializations. > > > > >> + There is then the little dance that has us rolling restart from > an > > > > >> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then > > it > > > > will > > > > >> assign regions to the new hbase-2.x regionservers as they come on > > line. > > > > >> TBD. > > > > >> > > > > >> Is this what you mean sir? > > > > >> > > > > >> S > > > > >> > > > > >> > > > > >> > I think without that knowledge it is hard (for me at least :) ) > to > > > > >> > determine where we should set our sights in terms of migration. > > > > >> > > > > > >> > Thanks, > > > > >> > Zach > > > > >> > > > > > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > >> > > > > > >> > > What are our expectations regards compatibility between hbase1 > > and > > > > >> > hbase2? > > > > >> > > > > > > >> > > Lets have a chat about it. Here are some goal posts. > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Fri, Sep 1, 2017 at 3:12 PM, Sean Busbeywrote: > Giving rolling-upgradable ability within branch-1, I'm inclined to > push for our current stable release line,1.2. > > I like this suggestion. Any other opinions on minimum version from which we'd support rolling upgrade? S > On Fri, Sep 1, 2017 at 2:24 PM, Stack wrote: > > On Fri, Sep 1, 2017 at 2:23 PM, Stack wrote: > > > >> What versions should you be able to upgrade from? 1.0.0? 0.98.x? 1.2? > I'm > >> thinking 0.98 unless it means more work. > >> > > > > Scratch that. Minimum is 1.0.0 I think given that is what I'm doing all > > compat compares against and given we did API cleanup before we shipped > > 1.0.0. > > St.Ack > > > > > > > >> Thanks, > >> St.Ack > >> > >> On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > >> > >>> What are our expectations regards compatibility between hbase1 and > hbase2? > >>> > >>> Lets have a chat about it. Here are some goal posts. > >>> > >>> + You have to upgrade to hbase-1.x before you can migrate to hbase-2. > No > >>> migration from < hbase-1 (Is this too onerous? Should we support 0.98 > => > >>> 2.0?). > >>> + You do NOT have to upgrade to the latest release of hbase1 to migrate > >>> to hbase2; being up on hbase-1.0.0+ will be sufficient. > >>> + You'll have to update your hbase1 coprocessors to deploy them on > >>> hbase2. A bunch of CP API has/will change by the time hbase2 comes out; > >>> e.g. watching for region split on RegionServer no longer makes sense > given > >>> Master runs all splits now. > >>> + An hbase1 client can run against an hbase2 cluster but it will only > be > >>> able to do DML (Get/Put/Scan, etc.). We do not allow being able to do > admin > >>> ops using an hbase1 Admin client against an hbase2 cluster. We have > some > >>> egregious API violations in branch-1; e.g. we have protobuf in our API > (See > >>> HBASE-15607). The notion is that we can't afford a deprecation cycle > >>> purging this stuff from our Admin API. > >>> > >>> What you all think? > >>> > >>> St.Ack > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >> >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Just some high-level thoughts on rolling upgradeability (I'm repeating a few things already said). 1. No service interruption for 1.x clients while a cluster is being ugpraded to 2.x. - This includes support for apis commonly used in a running system: delegation tokens, getting region locations, etc. 2. The somewhat reverse of #1 is also necessary for Master DMLs to meta. 3. No interruption for replication between 1.x and 2.x cluster 4. Restart master first before any other service sounds reasonable to me. Will we support RU for zkless mode? 5. 2.x should be either able to understand formats written by 1.x or online migration is done as a preparation step. 6. Data generated by 2.x daemons should be readable in 1.x (ie HFile, zookeeper, etc). Some deploys may have large amounts of data could be cumbersome/impractical (ie cost, time, etc) to copy the data. Thanks, Francis On Fri, Sep 1, 2017 at 10:01 PM Chia-Ping Tsaiwrote: > The most of issues use hadoop flag, so the filter looks like this. > project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, 2.0.0-alpha-2, > 2.0.0-alpha-3, > 3.0) AND (labels in (incompatibleChange, incompatible, incompatibility) OR > "Hadoop Flags" in ("Incompatible change")) > > On 2017-08-04 05:46, Ted Yu wrote: > > I expanded the condition in the filter like this: > > > > project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, 2.0.0-alpha-2, > > 3.0) AND labels in (incompatibleChange, incompatible, incompatibility) > > > > Still there're only two showing up. > > > > On Thu, Aug 3, 2017 at 9:07 AM, Zach York > > wrote: > > > > > I tried to filter based on imcompatible labels and there were only two > > > JIRAs returned [1]. I have a hard time believing that there were only > two > > > breaking changes from 1.x to 2.x. > > > > > > [1] > > > https://issues.apache.org/jira/browse/HBASE-17957?jql= > > > project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.0.0% > > > 20AND%20labels%20in%20(incompatibleChange%2C%20incompatible%2C% > > > 20incompatibility) > > > > > > On Thu, Aug 3, 2017 at 8:54 AM, Zach York < > zyork.contribut...@gmail.com> > > > wrote: > > > > > > > This kinda helps, but these seem more like expectations. I was going > more > > > > for things like HFile format changed, meta table structure changed, > > > > coprocessor implementations changed (these are just examples, I don't > > > know > > > > if any of these actually changed). > > > > > > > > More technical differences between branch-1 and branch-2 which then > can > > > > help us get the right expectations for compatibility. > > > > > > > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > > > > > > >> On Wed, Aug 2, 2017 at 5:25 PM, Zach York < > zyork.contribut...@gmail.com > > > > > > > >> wrote: > > > >> > > > >> > Do we know what the major pain points for migration are? Can we > > > discuss > > > >> > that/get a list going? > > > >> > > > > >> > > > > >> Here's a few in outline: > > > >> > > > >> + There is issue of formats, of hbase-2.x being able to read > hbase-1.x > > > >> data > > > >> whether from HDFS or ZooKeeper or off the wire. > > > >> + An hbase-1.x client should be able to Get/Put and Scan an > hbase-2.x > > > >> cluster; no holes in the API or unintelligible serializations. > > > >> + There is then the little dance that has us rolling restart from an > > > >> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then > it > > > will > > > >> assign regions to the new hbase-2.x regionservers as they come on > line. > > > >> TBD. > > > >> > > > >> Is this what you mean sir? > > > >> > > > >> S > > > >> > > > >> > > > >> > I think without that knowledge it is hard (for me at least :) ) to > > > >> > determine where we should set our sights in terms of migration. > > > >> > > > > >> > Thanks, > > > >> > Zach > > > >> > > > > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > >> > > > > >> > > What are our expectations regards compatibility between hbase1 > and > > > >> > hbase2? > > > >> > > > > > >> > > Lets have a chat about it. Here are some goal posts. > > > >> > > > > > >> > > + You have to upgrade to hbase-1.x before you can migrate to > > > hbase-2. > > > >> No > > > >> > > migration from < hbase-1 (Is this too onerous? Should we support > > > 0.98 > > > >> => > > > >> > > 2.0?). > > > >> > > + You do NOT have to upgrade to the latest release of hbase1 to > > > >> migrate > > > >> > to > > > >> > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > > >> > > + You'll have to update your hbase1 coprocessors to deploy them > on > > > >> > hbase2. > > > >> > > A bunch of CP API has/will change by the time hbase2 comes out; > e.g. > > > >> > > watching for region split on RegionServer no longer makes sense > > > given > > > >> > > Master runs all splits now. > > > >> > > + An hbase1 client can run against an hbase2 cluster but it will > > > only > > > >> be > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
The most of issues use hadoop flag, so the filter looks like this. project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, 2.0.0-alpha-2, 2.0.0-alpha-3, 3.0) AND (labels in (incompatibleChange, incompatible, incompatibility) OR "Hadoop Flags" in ("Incompatible change")) On 2017-08-04 05:46, Ted Yuwrote: > I expanded the condition in the filter like this: > > project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, 2.0.0-alpha-2, > 3.0) AND labels in (incompatibleChange, incompatible, incompatibility) > > Still there're only two showing up. > > On Thu, Aug 3, 2017 at 9:07 AM, Zach York > wrote: > > > I tried to filter based on imcompatible labels and there were only two > > JIRAs returned [1]. I have a hard time believing that there were only two > > breaking changes from 1.x to 2.x. > > > > [1] > > https://issues.apache.org/jira/browse/HBASE-17957?jql= > > project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.0.0% > > 20AND%20labels%20in%20(incompatibleChange%2C%20incompatible%2C% > > 20incompatibility) > > > > On Thu, Aug 3, 2017 at 8:54 AM, Zach York > > wrote: > > > > > This kinda helps, but these seem more like expectations. I was going more > > > for things like HFile format changed, meta table structure changed, > > > coprocessor implementations changed (these are just examples, I don't > > know > > > if any of these actually changed). > > > > > > More technical differences between branch-1 and branch-2 which then can > > > help us get the right expectations for compatibility. > > > > > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > > > > >> On Wed, Aug 2, 2017 at 5:25 PM, Zach York > > > > >> wrote: > > >> > > >> > Do we know what the major pain points for migration are? Can we > > discuss > > >> > that/get a list going? > > >> > > > >> > > > >> Here's a few in outline: > > >> > > >> + There is issue of formats, of hbase-2.x being able to read hbase-1.x > > >> data > > >> whether from HDFS or ZooKeeper or off the wire. > > >> + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > > >> cluster; no holes in the API or unintelligible serializations. > > >> + There is then the little dance that has us rolling restart from an > > >> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it > > will > > >> assign regions to the new hbase-2.x regionservers as they come on line. > > >> TBD. > > >> > > >> Is this what you mean sir? > > >> > > >> S > > >> > > >> > > >> > I think without that knowledge it is hard (for me at least :) ) to > > >> > determine where we should set our sights in terms of migration. > > >> > > > >> > Thanks, > > >> > Zach > > >> > > > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > >> > > > >> > > What are our expectations regards compatibility between hbase1 and > > >> > hbase2? > > >> > > > > >> > > Lets have a chat about it. Here are some goal posts. > > >> > > > > >> > > + You have to upgrade to hbase-1.x before you can migrate to > > hbase-2. > > >> No > > >> > > migration from < hbase-1 (Is this too onerous? Should we support > > 0.98 > > >> => > > >> > > 2.0?). > > >> > > + You do NOT have to upgrade to the latest release of hbase1 to > > >> migrate > > >> > to > > >> > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > >> > > + You'll have to update your hbase1 coprocessors to deploy them on > > >> > hbase2. > > >> > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > > >> > > watching for region split on RegionServer no longer makes sense > > given > > >> > > Master runs all splits now. > > >> > > + An hbase1 client can run against an hbase2 cluster but it will > > only > > >> be > > >> > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to > > do > > >> > admin > > >> > > ops using an hbase1 Admin client against an hbase2 cluster. We have > > >> some > > >> > > egregious API violations in branch-1; e.g. we have protobuf in our > > API > > >> > (See > > >> > > HBASE-15607). The notion is that we can't afford a deprecation cycle > > >> > > purging this stuff from our Admin API. > > >> > > > > >> > > What you all think? > > >> > > > > >> > > St.Ack > > >> > > > > >> > > > >> > > > > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Giving rolling-upgradable ability within branch-1, I'm inclined to push for our current stable release line,1.2. On Fri, Sep 1, 2017 at 2:24 PM, Stackwrote: > On Fri, Sep 1, 2017 at 2:23 PM, Stack wrote: > >> What versions should you be able to upgrade from? 1.0.0? 0.98.x? 1.2? I'm >> thinking 0.98 unless it means more work. >> > > Scratch that. Minimum is 1.0.0 I think given that is what I'm doing all > compat compares against and given we did API cleanup before we shipped > 1.0.0. > St.Ack > > > >> Thanks, >> St.Ack >> >> On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: >> >>> What are our expectations regards compatibility between hbase1 and hbase2? >>> >>> Lets have a chat about it. Here are some goal posts. >>> >>> + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No >>> migration from < hbase-1 (Is this too onerous? Should we support 0.98 => >>> 2.0?). >>> + You do NOT have to upgrade to the latest release of hbase1 to migrate >>> to hbase2; being up on hbase-1.0.0+ will be sufficient. >>> + You'll have to update your hbase1 coprocessors to deploy them on >>> hbase2. A bunch of CP API has/will change by the time hbase2 comes out; >>> e.g. watching for region split on RegionServer no longer makes sense given >>> Master runs all splits now. >>> + An hbase1 client can run against an hbase2 cluster but it will only be >>> able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin >>> ops using an hbase1 Admin client against an hbase2 cluster. We have some >>> egregious API violations in branch-1; e.g. we have protobuf in our API (See >>> HBASE-15607). The notion is that we can't afford a deprecation cycle >>> purging this stuff from our Admin API. >>> >>> What you all think? >>> >>> St.Ack >>> >>> >>> >>> >>> >>> >>> >>> >>> >>
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Fri, Sep 1, 2017 at 2:23 PM, Stackwrote: > What versions should you be able to upgrade from? 1.0.0? 0.98.x? 1.2? I'm > thinking 0.98 unless it means more work. > Scratch that. Minimum is 1.0.0 I think given that is what I'm doing all compat compares against and given we did API cleanup before we shipped 1.0.0. St.Ack > Thanks, > St.Ack > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > >> What are our expectations regards compatibility between hbase1 and hbase2? >> >> Lets have a chat about it. Here are some goal posts. >> >> + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No >> migration from < hbase-1 (Is this too onerous? Should we support 0.98 => >> 2.0?). >> + You do NOT have to upgrade to the latest release of hbase1 to migrate >> to hbase2; being up on hbase-1.0.0+ will be sufficient. >> + You'll have to update your hbase1 coprocessors to deploy them on >> hbase2. A bunch of CP API has/will change by the time hbase2 comes out; >> e.g. watching for region split on RegionServer no longer makes sense given >> Master runs all splits now. >> + An hbase1 client can run against an hbase2 cluster but it will only be >> able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin >> ops using an hbase1 Admin client against an hbase2 cluster. We have some >> egregious API violations in branch-1; e.g. we have protobuf in our API (See >> HBASE-15607). The notion is that we can't afford a deprecation cycle >> purging this stuff from our Admin API. >> >> What you all think? >> >> St.Ack >> >> >> >> >> >> >> >> >> >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
What versions should you be able to upgrade from? 1.0.0? 0.98.x? 1.2? I'm thinking 0.98 unless it means more work. Thanks, St.Ack On Wed, Aug 2, 2017 at 4:38 PM, Stackwrote: > What are our expectations regards compatibility between hbase1 and hbase2? > > Lets have a chat about it. Here are some goal posts. > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No > migration from < hbase-1 (Is this too onerous? Should we support 0.98 => > 2.0?). > + You do NOT have to upgrade to the latest release of hbase1 to migrate to > hbase2; being up on hbase-1.0.0+ will be sufficient. > + You'll have to update your hbase1 coprocessors to deploy them on hbase2. > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > watching for region split on RegionServer no longer makes sense given > Master runs all splits now. > + An hbase1 client can run against an hbase2 cluster but it will only be > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin > ops using an hbase1 Admin client against an hbase2 cluster. We have some > egregious API violations in branch-1; e.g. we have protobuf in our API (See > HBASE-15607). The notion is that we can't afford a deprecation cycle > purging this stuff from our Admin API. > > What you all think? > > St.Ack > > > > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Thu, Aug 24, 2017 at 9:13 AM, stackwrote: > On Thu, Aug 3, 2017 at 11:05 PM, stack wrote: > >> Thanks Zach for clarification. Let me work up a list and then come back >> to this thread. Jira needs an edit pass to. >> >> > JIRA editing reflecting incompatibles has been coming along. It is still a > work-in-progress though. Meantime, let me air out here the incompatibles as > I encounter them. The complete list may never be known (smile) and the best > knowledge will only be had on the day before release which will be giving > notice too late for folks to do much about it. > > Here is a known breaking incompatible that just went in: HBASE-15982 > "Interface ReplicationEndpoint extends Guava's Service". See the release > note for why the breakage unavoidable. Anyone who has implemented our > Replication Endpoint will have a refactor on their hands deploying their > replication machine atop hbase2 (Side note: Estebans' initial work has it > that hbase native replication seems to work going from hbase1 to hbase2 and > even the other way around; more to do). Upside, if there is the will, > now is the time for HBASE-10504 "Define Replication Interface". It is late > in the game but am up for helping out if interest. > > Here is a breaking compatible we *could* (maybe) avoid but the amount of > work involved in the workaround and how the patch-up would muddy our > messaging around protobufs in the HBase public API is not worth the price > IMO. What do you all think? The issue is HBASE-15607 "Remove PB references > from Admin for 2.0". The plan is to (re-)commit Ram's change. We'd then > tell users that you cannot administer an hbase2 cluster using an hbase1 > client. > > Ok. The "could" above has changed to a "can't" on review of changes in Admin Interface since hbase1.0.0. The amount of work involved would be too much and the methods we'd be restoring compatibility on are wonky in the first place; i.e. async calls that gave you no means of querying state of the call and protobuf Messages that would require 'translation' from our internal protobuf to protobuf 2.5. Admin Interface in 2.0 will be incompatible with hbase 1.x Admin. I've a WIP incompatibles list as a messy subsection of the hbase2 doc [1]. St.Ack 1. https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.723jjn18p2jr > Coprocessors will require a refactor to deploy on hbase2. More detail to > follow. > > St.Ack > > > > > > >> S >> >> On Aug 3, 2017 23:54, "Zach York" wrote: >> >> This kinda helps, but these seem more like expectations. I was going more >> for things like HFile format changed, meta table structure changed, >> coprocessor implementations changed (these are just examples, I don't know >> if any of these actually changed). >> >> More technical differences between branch-1 and branch-2 which then can >> help us get the right expectations for compatibility. >> >> On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: >> >> > On Wed, Aug 2, 2017 at 5:25 PM, Zach York > > >> > wrote: >> > >> > > Do we know what the major pain points for migration are? Can we >> discuss >> > > that/get a list going? >> > > >> > > >> > Here's a few in outline: >> > >> > + There is issue of formats, of hbase-2.x being able to read hbase-1.x >> data >> > whether from HDFS or ZooKeeper or off the wire. >> > + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x >> > cluster; no holes in the API or unintelligible serializations. >> > + There is then the little dance that has us rolling restart from an >> > hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it >> will >> > assign regions to the new hbase-2.x regionservers as they come on line. >> > TBD. >> > >> > Is this what you mean sir? >> > >> > S >> > >> > >> > > I think without that knowledge it is hard (for me at least :) ) to >> > > determine where we should set our sights in terms of migration. >> > > >> > > Thanks, >> > > Zach >> > > >> > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: >> > > >> > > > What are our expectations regards compatibility between hbase1 and >> > > hbase2? >> > > > >> > > > Lets have a chat about it. Here are some goal posts. >> > > > >> > > > + You have to upgrade to hbase-1.x before you can migrate to >> hbase-2. >> > No >> > > > migration from < hbase-1 (Is this too onerous? Should we support >> 0.98 >> > => >> > > > 2.0?). >> > > > + You do NOT have to upgrade to the latest release of hbase1 to >> migrate >> > > to >> > > > hbase2; being up on hbase-1.0.0+ will be sufficient. >> > > > + You'll have to update your hbase1 coprocessors to deploy them on >> > > hbase2. >> > > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. >> > > > watching for region split on RegionServer no longer makes sense >> given >> > > > Master runs all splits now.
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Thu, Aug 3, 2017 at 11:05 PM, stackwrote: > Thanks Zach for clarification. Let me work up a list and then come back to > this thread. Jira needs an edit pass to. > > JIRA editing reflecting incompatibles has been coming along. It is still a work-in-progress though. Meantime, let me air out here the incompatibles as I encounter them. The complete list may never be known (smile) and the best knowledge will only be had on the day before release which will be giving notice too late for folks to do much about it. Here is a known breaking incompatible that just went in: HBASE-15982 "Interface ReplicationEndpoint extends Guava's Service". See the release note for why the breakage unavoidable. Anyone who has implemented our Replication Endpoint will have a refactor on their hands deploying their replication machine atop hbase2 (Side note: Estebans' initial work has it that hbase native replication seems to work going from hbase1 to hbase2 and even the other way around; more to do). Upside, if there is the will, now is the time for HBASE-10504 "Define Replication Interface". It is late in the game but am up for helping out if interest. Here is a breaking compatible we *could* (maybe) avoid but the amount of work involved in the workaround and how the patch-up would muddy our messaging around protobufs in the HBase public API is not worth the price IMO. What do you all think? The issue is HBASE-15607 "Remove PB references from Admin for 2.0". The plan is to (re-)commit Ram's change. We'd then tell users that you cannot administer an hbase2 cluster using an hbase1 client. Coprocessors will require a refactor to deploy on hbase2. More detail to follow. St.Ack > S > > On Aug 3, 2017 23:54, "Zach York" wrote: > > This kinda helps, but these seem more like expectations. I was going more > for things like HFile format changed, meta table structure changed, > coprocessor implementations changed (these are just examples, I don't know > if any of these actually changed). > > More technical differences between branch-1 and branch-2 which then can > help us get the right expectations for compatibility. > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > > On Wed, Aug 2, 2017 at 5:25 PM, Zach York > > wrote: > > > > > Do we know what the major pain points for migration are? Can we discuss > > > that/get a list going? > > > > > > > > Here's a few in outline: > > > > + There is issue of formats, of hbase-2.x being able to read hbase-1.x > data > > whether from HDFS or ZooKeeper or off the wire. > > + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > > cluster; no holes in the API or unintelligible serializations. > > + There is then the little dance that has us rolling restart from an > > hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it > will > > assign regions to the new hbase-2.x regionservers as they come on line. > > TBD. > > > > Is this what you mean sir? > > > > S > > > > > > > I think without that knowledge it is hard (for me at least :) ) to > > > determine where we should set our sights in terms of migration. > > > > > > Thanks, > > > Zach > > > > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > > > > What are our expectations regards compatibility between hbase1 and > > > hbase2? > > > > > > > > Lets have a chat about it. Here are some goal posts. > > > > > > > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. > > No > > > > migration from < hbase-1 (Is this too onerous? Should we support 0.98 > > => > > > > 2.0?). > > > > + You do NOT have to upgrade to the latest release of hbase1 to > migrate > > > to > > > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > > > + You'll have to update your hbase1 coprocessors to deploy them on > > > hbase2. > > > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > > > > watching for region split on RegionServer no longer makes sense given > > > > Master runs all splits now. > > > > + An hbase1 client can run against an hbase2 cluster but it will only > > be > > > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do > > > admin > > > > ops using an hbase1 Admin client against an hbase2 cluster. We have > > some > > > > egregious API violations in branch-1; e.g. we have protobuf in our > API > > > (See > > > > HBASE-15607). The notion is that we can't afford a deprecation cycle > > > > purging this stuff from our Admin API. > > > > > > > > What you all think? > > > > > > > > St.Ack > > > > > > > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Thanks Stack! -- Cloudera, Inc. On Mon, Aug 14, 2017 at 1:05 PM, Stackwrote: > On Fri, Aug 4, 2017 at 10:00 AM, Esteban Gutierrez > wrote: > > > Should we add additional details around replication as well? for > instance, > > shall we consider a hbase-1.x cluster as a client for a hbase-2.x > cluster? > > > > > Yes. I'd say this should be a blocker Esteban. Filed HBASE-18596. > Thanks, > S > > > > > > > Thanks for starting this discussion Stack, > > > > esteban. > > > > -- > > Cloudera, Inc. > > > > > > On Fri, Aug 4, 2017 at 1:05 AM, stack wrote: > > > > > Thanks Zach for clarification. Let me work up a list and then come back > > to > > > this thread. Jira needs an edit pass to. > > > > > > S > > > > > > On Aug 3, 2017 23:54, "Zach York" > wrote: > > > > > > This kinda helps, but these seem more like expectations. I was going > more > > > for things like HFile format changed, meta table structure changed, > > > coprocessor implementations changed (these are just examples, I don't > > know > > > if any of these actually changed). > > > > > > More technical differences between branch-1 and branch-2 which then can > > > help us get the right expectations for compatibility. > > > > > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > > > > > > On Wed, Aug 2, 2017 at 5:25 PM, Zach York < > > zyork.contribut...@gmail.com> > > > > wrote: > > > > > > > > > Do we know what the major pain points for migration are? Can we > > discuss > > > > > that/get a list going? > > > > > > > > > > > > > > Here's a few in outline: > > > > > > > > + There is issue of formats, of hbase-2.x being able to read > hbase-1.x > > > data > > > > whether from HDFS or ZooKeeper or off the wire. > > > > + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > > > > cluster; no holes in the API or unintelligible serializations. > > > > + There is then the little dance that has us rolling restart from an > > > > hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it > > > will > > > > assign regions to the new hbase-2.x regionservers as they come on > line. > > > > TBD. > > > > > > > > Is this what you mean sir? > > > > > > > > S > > > > > > > > > > > > > I think without that knowledge it is hard (for me at least :) ) to > > > > > determine where we should set our sights in terms of migration. > > > > > > > > > > Thanks, > > > > > Zach > > > > > > > > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > > > > > > > > What are our expectations regards compatibility between hbase1 > and > > > > > hbase2? > > > > > > > > > > > > Lets have a chat about it. Here are some goal posts. > > > > > > > > > > > > + You have to upgrade to hbase-1.x before you can migrate to > > hbase-2. > > > > No > > > > > > migration from < hbase-1 (Is this too onerous? Should we support > > 0.98 > > > > => > > > > > > 2.0?). > > > > > > + You do NOT have to upgrade to the latest release of hbase1 to > > > migrate > > > > > to > > > > > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > > > > > + You'll have to update your hbase1 coprocessors to deploy them > on > > > > > hbase2. > > > > > > A bunch of CP API has/will change by the time hbase2 comes out; > > e.g. > > > > > > watching for region split on RegionServer no longer makes sense > > given > > > > > > Master runs all splits now. > > > > > > + An hbase1 client can run against an hbase2 cluster but it will > > only > > > > be > > > > > > able to do DML (Get/Put/Scan, etc.). We do not allow being able > to > > do > > > > > admin > > > > > > ops using an hbase1 Admin client against an hbase2 cluster. We > have > > > > some > > > > > > egregious API violations in branch-1; e.g. we have protobuf in > our > > > API > > > > > (See > > > > > > HBASE-15607). The notion is that we can't afford a deprecation > > cycle > > > > > > purging this stuff from our Admin API. > > > > > > > > > > > > What you all think? > > > > > > > > > > > > St.Ack > > > > > > > > > > > > > > > > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Fri, Aug 11, 2017 at 5:07 PM, Apekshit Sharmawrote: > Ref: deleting/deprecating some methods in HBaseAdmin > https://github.com/apache/hbase/commit/de696cf6b653749c6bf105ef3d62d7 > a6c6923c57 > From first message: > *" An hbase1 client can run against an hbase2 cluster but it will only be > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin > ops using an hbase1 Admin client against an hbase2 cluster."* > > Question 1: > These guarantees are with what version of jars? > Does the client has to keep using corresponding 1.x jars with which it was > compiled, OR can the 2.x jars just replace earlier ones and client is still > expected to work correctly? (i would say no!) > > Dropping hbase2 jars into an hbase1 deploy? No. No guarantee of binary compat. I was thinking of wire compatibility here Appy. > Question second: > How do we handle clients doing deprecated admin ops? > a) make admin functions no-op by using empty method. > b) throw exception. May/maynot crash the client - ideally shouldn't since > methods are define with 'throws IOException'. > I think throwing exception the way to go. Its clean signal of change. > c) delete the method - will crash the client > But the question only makes sense if expect clients to work with 2.x jars. > If not, we can just delete them, simple! > > I'm not sure I follow. If a hbase1 client calls a deprecated/unimplemented method, I was thinking it'd get an exception. > Question 3: AMv2 related > There are ops in old HBaseAdmin that'll be destructive for 2.0 > (closeRegion() fn). > If clients continue to use 1.x jars and make these calls, that's not > acceptable. We need to handle such deprecation on server sider itself. > If only there was way of versioning the rpc service! > Another way, but bad one is, deprecate existing ones and move logic to a > new one. I think there are just 1-2 that need this treatment. > Either way, need to come up with a solution! > > Throw an exception pointing at the alternative API? Thanks Appy, S > > On Fri, Aug 4, 2017 at 10:08 AM, Andrew Purtell > wrote: > > > This is a really good question. I think many operators will give a lot of > > leeway to data format changes as long as data can be copied from A to B > > (perhaps with batch rewrite to upgrade (ideally, not required)) and > > replication can be enabled to sync up to the current moment for cut over. > > > > > > > On Aug 4, 2017, at 10:00 AM, Esteban Gutierrez > > wrote: > > > > > > Should we add additional details around replication as well? for > > instance, > > > shall we consider a hbase-1.x cluster as a client for a hbase-2.x > > cluster? > > > > > > Thanks for starting this discussion Stack, > > > > > > esteban. > > > > > > -- > > > Cloudera, Inc. > > > > > > > > >> On Fri, Aug 4, 2017 at 1:05 AM, stack wrote: > > >> > > >> Thanks Zach for clarification. Let me work up a list and then come > back > > to > > >> this thread. Jira needs an edit pass to. > > >> > > >> S > > >> > > >> On Aug 3, 2017 23:54, "Zach York" > wrote: > > >> > > >> This kinda helps, but these seem more like expectations. I was going > > more > > >> for things like HFile format changed, meta table structure changed, > > >> coprocessor implementations changed (these are just examples, I don't > > know > > >> if any of these actually changed). > > >> > > >> More technical differences between branch-1 and branch-2 which then > can > > >> help us get the right expectations for compatibility. > > >> > > >>> On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > >>> > > >>> On Wed, Aug 2, 2017 at 5:25 PM, Zach York < > > zyork.contribut...@gmail.com> > > >>> wrote: > > >>> > > Do we know what the major pain points for migration are? Can we > > discuss > > that/get a list going? > > > > > > >>> Here's a few in outline: > > >>> > > >>> + There is issue of formats, of hbase-2.x being able to read > hbase-1.x > > >> data > > >>> whether from HDFS or ZooKeeper or off the wire. > > >>> + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > > >>> cluster; no holes in the API or unintelligible serializations. > > >>> + There is then the little dance that has us rolling restart from an > > >>> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it > > >> will > > >>> assign regions to the new hbase-2.x regionservers as they come on > line. > > >>> TBD. > > >>> > > >>> Is this what you mean sir? > > >>> > > >>> S > > >>> > > >>> > > I think without that knowledge it is hard (for me at least :) ) to > > determine where we should set our sights in terms of migration. > > > > Thanks, > > Zach > > > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > > > What are our expectations regards compatibility between hbase1 and > > hbase2? >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Fri, Aug 4, 2017 at 10:00 AM, Esteban Gutierrezwrote: > Should we add additional details around replication as well? for instance, > shall we consider a hbase-1.x cluster as a client for a hbase-2.x cluster? > > Yes. I'd say this should be a blocker Esteban. Filed HBASE-18596. Thanks, S > Thanks for starting this discussion Stack, > > esteban. > > -- > Cloudera, Inc. > > > On Fri, Aug 4, 2017 at 1:05 AM, stack wrote: > > > Thanks Zach for clarification. Let me work up a list and then come back > to > > this thread. Jira needs an edit pass to. > > > > S > > > > On Aug 3, 2017 23:54, "Zach York" wrote: > > > > This kinda helps, but these seem more like expectations. I was going more > > for things like HFile format changed, meta table structure changed, > > coprocessor implementations changed (these are just examples, I don't > know > > if any of these actually changed). > > > > More technical differences between branch-1 and branch-2 which then can > > help us get the right expectations for compatibility. > > > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > > > > On Wed, Aug 2, 2017 at 5:25 PM, Zach York < > zyork.contribut...@gmail.com> > > > wrote: > > > > > > > Do we know what the major pain points for migration are? Can we > discuss > > > > that/get a list going? > > > > > > > > > > > Here's a few in outline: > > > > > > + There is issue of formats, of hbase-2.x being able to read hbase-1.x > > data > > > whether from HDFS or ZooKeeper or off the wire. > > > + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > > > cluster; no holes in the API or unintelligible serializations. > > > + There is then the little dance that has us rolling restart from an > > > hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it > > will > > > assign regions to the new hbase-2.x regionservers as they come on line. > > > TBD. > > > > > > Is this what you mean sir? > > > > > > S > > > > > > > > > > I think without that knowledge it is hard (for me at least :) ) to > > > > determine where we should set our sights in terms of migration. > > > > > > > > Thanks, > > > > Zach > > > > > > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > > > > > > What are our expectations regards compatibility between hbase1 and > > > > hbase2? > > > > > > > > > > Lets have a chat about it. Here are some goal posts. > > > > > > > > > > + You have to upgrade to hbase-1.x before you can migrate to > hbase-2. > > > No > > > > > migration from < hbase-1 (Is this too onerous? Should we support > 0.98 > > > => > > > > > 2.0?). > > > > > + You do NOT have to upgrade to the latest release of hbase1 to > > migrate > > > > to > > > > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > > > > + You'll have to update your hbase1 coprocessors to deploy them on > > > > hbase2. > > > > > A bunch of CP API has/will change by the time hbase2 comes out; > e.g. > > > > > watching for region split on RegionServer no longer makes sense > given > > > > > Master runs all splits now. > > > > > + An hbase1 client can run against an hbase2 cluster but it will > only > > > be > > > > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to > do > > > > admin > > > > > ops using an hbase1 Admin client against an hbase2 cluster. We have > > > some > > > > > egregious API violations in branch-1; e.g. we have protobuf in our > > API > > > > (See > > > > > HBASE-15607). The notion is that we can't afford a deprecation > cycle > > > > > purging this stuff from our Admin API. > > > > > > > > > > What you all think? > > > > > > > > > > St.Ack > > > > > > > > > > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Ref: deleting/deprecating some methods in HBaseAdmin https://github.com/apache/hbase/commit/de696cf6b653749c6bf105ef3d62d7a6c6923c57 >From first message: *" An hbase1 client can run against an hbase2 cluster but it will only be able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin ops using an hbase1 Admin client against an hbase2 cluster."* Question 1: These guarantees are with what version of jars? Does the client has to keep using corresponding 1.x jars with which it was compiled, OR can the 2.x jars just replace earlier ones and client is still expected to work correctly? (i would say no!) Question second: How do we handle clients doing deprecated admin ops? a) make admin functions no-op by using empty method. b) throw exception. May/maynot crash the client - ideally shouldn't since methods are define with 'throws IOException'. c) delete the method - will crash the client But the question only makes sense if expect clients to work with 2.x jars. If not, we can just delete them, simple! Question 3: AMv2 related There are ops in old HBaseAdmin that'll be destructive for 2.0 (closeRegion() fn). If clients continue to use 1.x jars and make these calls, that's not acceptable. We need to handle such deprecation on server sider itself. If only there was way of versioning the rpc service! Another way, but bad one is, deprecate existing ones and move logic to a new one. I think there are just 1-2 that need this treatment. Either way, need to come up with a solution! On Fri, Aug 4, 2017 at 10:08 AM, Andrew Purtellwrote: > This is a really good question. I think many operators will give a lot of > leeway to data format changes as long as data can be copied from A to B > (perhaps with batch rewrite to upgrade (ideally, not required)) and > replication can be enabled to sync up to the current moment for cut over. > > > > On Aug 4, 2017, at 10:00 AM, Esteban Gutierrez > wrote: > > > > Should we add additional details around replication as well? for > instance, > > shall we consider a hbase-1.x cluster as a client for a hbase-2.x > cluster? > > > > Thanks for starting this discussion Stack, > > > > esteban. > > > > -- > > Cloudera, Inc. > > > > > >> On Fri, Aug 4, 2017 at 1:05 AM, stack wrote: > >> > >> Thanks Zach for clarification. Let me work up a list and then come back > to > >> this thread. Jira needs an edit pass to. > >> > >> S > >> > >> On Aug 3, 2017 23:54, "Zach York" wrote: > >> > >> This kinda helps, but these seem more like expectations. I was going > more > >> for things like HFile format changed, meta table structure changed, > >> coprocessor implementations changed (these are just examples, I don't > know > >> if any of these actually changed). > >> > >> More technical differences between branch-1 and branch-2 which then can > >> help us get the right expectations for compatibility. > >> > >>> On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > >>> > >>> On Wed, Aug 2, 2017 at 5:25 PM, Zach York < > zyork.contribut...@gmail.com> > >>> wrote: > >>> > Do we know what the major pain points for migration are? Can we > discuss > that/get a list going? > > > >>> Here's a few in outline: > >>> > >>> + There is issue of formats, of hbase-2.x being able to read hbase-1.x > >> data > >>> whether from HDFS or ZooKeeper or off the wire. > >>> + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > >>> cluster; no holes in the API or unintelligible serializations. > >>> + There is then the little dance that has us rolling restart from an > >>> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it > >> will > >>> assign regions to the new hbase-2.x regionservers as they come on line. > >>> TBD. > >>> > >>> Is this what you mean sir? > >>> > >>> S > >>> > >>> > I think without that knowledge it is hard (for me at least :) ) to > determine where we should set our sights in terms of migration. > > Thanks, > Zach > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > What are our expectations regards compatibility between hbase1 and > hbase2? > > > > Lets have a chat about it. Here are some goal posts. > > > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. > >>> No > > migration from < hbase-1 (Is this too onerous? Should we support 0.98 > >>> => > > 2.0?). > > + You do NOT have to upgrade to the latest release of hbase1 to > >> migrate > to > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > + You'll have to update your hbase1 coprocessors to deploy them on > hbase2. > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > > watching for region split on RegionServer no longer makes sense given > > Master runs all splits now. > > + An hbase1
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
This is a really good question. I think many operators will give a lot of leeway to data format changes as long as data can be copied from A to B (perhaps with batch rewrite to upgrade (ideally, not required)) and replication can be enabled to sync up to the current moment for cut over. > On Aug 4, 2017, at 10:00 AM, Esteban Gutierrezwrote: > > Should we add additional details around replication as well? for instance, > shall we consider a hbase-1.x cluster as a client for a hbase-2.x cluster? > > Thanks for starting this discussion Stack, > > esteban. > > -- > Cloudera, Inc. > > >> On Fri, Aug 4, 2017 at 1:05 AM, stack wrote: >> >> Thanks Zach for clarification. Let me work up a list and then come back to >> this thread. Jira needs an edit pass to. >> >> S >> >> On Aug 3, 2017 23:54, "Zach York" wrote: >> >> This kinda helps, but these seem more like expectations. I was going more >> for things like HFile format changed, meta table structure changed, >> coprocessor implementations changed (these are just examples, I don't know >> if any of these actually changed). >> >> More technical differences between branch-1 and branch-2 which then can >> help us get the right expectations for compatibility. >> >>> On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: >>> >>> On Wed, Aug 2, 2017 at 5:25 PM, Zach York >>> wrote: >>> Do we know what the major pain points for migration are? Can we discuss that/get a list going? >>> Here's a few in outline: >>> >>> + There is issue of formats, of hbase-2.x being able to read hbase-1.x >> data >>> whether from HDFS or ZooKeeper or off the wire. >>> + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x >>> cluster; no holes in the API or unintelligible serializations. >>> + There is then the little dance that has us rolling restart from an >>> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it >> will >>> assign regions to the new hbase-2.x regionservers as they come on line. >>> TBD. >>> >>> Is this what you mean sir? >>> >>> S >>> >>> I think without that knowledge it is hard (for me at least :) ) to determine where we should set our sights in terms of migration. Thanks, Zach > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > What are our expectations regards compatibility between hbase1 and hbase2? > > Lets have a chat about it. Here are some goal posts. > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. >>> No > migration from < hbase-1 (Is this too onerous? Should we support 0.98 >>> => > 2.0?). > + You do NOT have to upgrade to the latest release of hbase1 to >> migrate to > hbase2; being up on hbase-1.0.0+ will be sufficient. > + You'll have to update your hbase1 coprocessors to deploy them on hbase2. > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > watching for region split on RegionServer no longer makes sense given > Master runs all splits now. > + An hbase1 client can run against an hbase2 cluster but it will only >>> be > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do admin > ops using an hbase1 Admin client against an hbase2 cluster. We have >>> some > egregious API violations in branch-1; e.g. we have protobuf in our >> API (See > HBASE-15607). The notion is that we can't afford a deprecation cycle > purging this stuff from our Admin API. > > What you all think? > > St.Ack > >>> >>
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Should we add additional details around replication as well? for instance, shall we consider a hbase-1.x cluster as a client for a hbase-2.x cluster? Thanks for starting this discussion Stack, esteban. -- Cloudera, Inc. On Fri, Aug 4, 2017 at 1:05 AM, stackwrote: > Thanks Zach for clarification. Let me work up a list and then come back to > this thread. Jira needs an edit pass to. > > S > > On Aug 3, 2017 23:54, "Zach York" wrote: > > This kinda helps, but these seem more like expectations. I was going more > for things like HFile format changed, meta table structure changed, > coprocessor implementations changed (these are just examples, I don't know > if any of these actually changed). > > More technical differences between branch-1 and branch-2 which then can > help us get the right expectations for compatibility. > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > > On Wed, Aug 2, 2017 at 5:25 PM, Zach York > > wrote: > > > > > Do we know what the major pain points for migration are? Can we discuss > > > that/get a list going? > > > > > > > > Here's a few in outline: > > > > + There is issue of formats, of hbase-2.x being able to read hbase-1.x > data > > whether from HDFS or ZooKeeper or off the wire. > > + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > > cluster; no holes in the API or unintelligible serializations. > > + There is then the little dance that has us rolling restart from an > > hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it > will > > assign regions to the new hbase-2.x regionservers as they come on line. > > TBD. > > > > Is this what you mean sir? > > > > S > > > > > > > I think without that knowledge it is hard (for me at least :) ) to > > > determine where we should set our sights in terms of migration. > > > > > > Thanks, > > > Zach > > > > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > > > > What are our expectations regards compatibility between hbase1 and > > > hbase2? > > > > > > > > Lets have a chat about it. Here are some goal posts. > > > > > > > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. > > No > > > > migration from < hbase-1 (Is this too onerous? Should we support 0.98 > > => > > > > 2.0?). > > > > + You do NOT have to upgrade to the latest release of hbase1 to > migrate > > > to > > > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > > > + You'll have to update your hbase1 coprocessors to deploy them on > > > hbase2. > > > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > > > > watching for region split on RegionServer no longer makes sense given > > > > Master runs all splits now. > > > > + An hbase1 client can run against an hbase2 cluster but it will only > > be > > > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do > > > admin > > > > ops using an hbase1 Admin client against an hbase2 cluster. We have > > some > > > > egregious API violations in branch-1; e.g. we have protobuf in our > API > > > (See > > > > HBASE-15607). The notion is that we can't afford a deprecation cycle > > > > purging this stuff from our Admin API. > > > > > > > > What you all think? > > > > > > > > St.Ack > > > > > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
Thanks Zach for clarification. Let me work up a list and then come back to this thread. Jira needs an edit pass to. S On Aug 3, 2017 23:54, "Zach York"wrote: This kinda helps, but these seem more like expectations. I was going more for things like HFile format changed, meta table structure changed, coprocessor implementations changed (these are just examples, I don't know if any of these actually changed). More technical differences between branch-1 and branch-2 which then can help us get the right expectations for compatibility. On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > On Wed, Aug 2, 2017 at 5:25 PM, Zach York > wrote: > > > Do we know what the major pain points for migration are? Can we discuss > > that/get a list going? > > > > > Here's a few in outline: > > + There is issue of formats, of hbase-2.x being able to read hbase-1.x data > whether from HDFS or ZooKeeper or off the wire. > + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > cluster; no holes in the API or unintelligible serializations. > + There is then the little dance that has us rolling restart from an > hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it will > assign regions to the new hbase-2.x regionservers as they come on line. > TBD. > > Is this what you mean sir? > > S > > > > I think without that knowledge it is hard (for me at least :) ) to > > determine where we should set our sights in terms of migration. > > > > Thanks, > > Zach > > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > > What are our expectations regards compatibility between hbase1 and > > hbase2? > > > > > > Lets have a chat about it. Here are some goal posts. > > > > > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. > No > > > migration from < hbase-1 (Is this too onerous? Should we support 0.98 > => > > > 2.0?). > > > + You do NOT have to upgrade to the latest release of hbase1 to migrate > > to > > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > > + You'll have to update your hbase1 coprocessors to deploy them on > > hbase2. > > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > > > watching for region split on RegionServer no longer makes sense given > > > Master runs all splits now. > > > + An hbase1 client can run against an hbase2 cluster but it will only > be > > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do > > admin > > > ops using an hbase1 Admin client against an hbase2 cluster. We have > some > > > egregious API violations in branch-1; e.g. we have protobuf in our API > > (See > > > HBASE-15607). The notion is that we can't afford a deprecation cycle > > > purging this stuff from our Admin API. > > > > > > What you all think? > > > > > > St.Ack > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
I expanded the condition in the filter like this: project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, 2.0.0-alpha-2, 3.0) AND labels in (incompatibleChange, incompatible, incompatibility) Still there're only two showing up. On Thu, Aug 3, 2017 at 9:07 AM, Zach Yorkwrote: > I tried to filter based on imcompatible labels and there were only two > JIRAs returned [1]. I have a hard time believing that there were only two > breaking changes from 1.x to 2.x. > > [1] > https://issues.apache.org/jira/browse/HBASE-17957?jql= > project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.0.0% > 20AND%20labels%20in%20(incompatibleChange%2C%20incompatible%2C% > 20incompatibility) > > On Thu, Aug 3, 2017 at 8:54 AM, Zach York > wrote: > > > This kinda helps, but these seem more like expectations. I was going more > > for things like HFile format changed, meta table structure changed, > > coprocessor implementations changed (these are just examples, I don't > know > > if any of these actually changed). > > > > More technical differences between branch-1 and branch-2 which then can > > help us get the right expectations for compatibility. > > > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > > > >> On Wed, Aug 2, 2017 at 5:25 PM, Zach York > > >> wrote: > >> > >> > Do we know what the major pain points for migration are? Can we > discuss > >> > that/get a list going? > >> > > >> > > >> Here's a few in outline: > >> > >> + There is issue of formats, of hbase-2.x being able to read hbase-1.x > >> data > >> whether from HDFS or ZooKeeper or off the wire. > >> + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > >> cluster; no holes in the API or unintelligible serializations. > >> + There is then the little dance that has us rolling restart from an > >> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it > will > >> assign regions to the new hbase-2.x regionservers as they come on line. > >> TBD. > >> > >> Is this what you mean sir? > >> > >> S > >> > >> > >> > I think without that knowledge it is hard (for me at least :) ) to > >> > determine where we should set our sights in terms of migration. > >> > > >> > Thanks, > >> > Zach > >> > > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > >> > > >> > > What are our expectations regards compatibility between hbase1 and > >> > hbase2? > >> > > > >> > > Lets have a chat about it. Here are some goal posts. > >> > > > >> > > + You have to upgrade to hbase-1.x before you can migrate to > hbase-2. > >> No > >> > > migration from < hbase-1 (Is this too onerous? Should we support > 0.98 > >> => > >> > > 2.0?). > >> > > + You do NOT have to upgrade to the latest release of hbase1 to > >> migrate > >> > to > >> > > hbase2; being up on hbase-1.0.0+ will be sufficient. > >> > > + You'll have to update your hbase1 coprocessors to deploy them on > >> > hbase2. > >> > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > >> > > watching for region split on RegionServer no longer makes sense > given > >> > > Master runs all splits now. > >> > > + An hbase1 client can run against an hbase2 cluster but it will > only > >> be > >> > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to > do > >> > admin > >> > > ops using an hbase1 Admin client against an hbase2 cluster. We have > >> some > >> > > egregious API violations in branch-1; e.g. we have protobuf in our > API > >> > (See > >> > > HBASE-15607). The notion is that we can't afford a deprecation cycle > >> > > purging this stuff from our Admin API. > >> > > > >> > > What you all think? > >> > > > >> > > St.Ack > >> > > > >> > > >> > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
I tried to filter based on imcompatible labels and there were only two JIRAs returned [1]. I have a hard time believing that there were only two breaking changes from 1.x to 2.x. [1] https://issues.apache.org/jira/browse/HBASE-17957?jql=project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.0.0%20AND%20labels%20in%20(incompatibleChange%2C%20incompatible%2C%20incompatibility) On Thu, Aug 3, 2017 at 8:54 AM, Zach Yorkwrote: > This kinda helps, but these seem more like expectations. I was going more > for things like HFile format changed, meta table structure changed, > coprocessor implementations changed (these are just examples, I don't know > if any of these actually changed). > > More technical differences between branch-1 and branch-2 which then can > help us get the right expectations for compatibility. > > On Wed, Aug 2, 2017 at 6:34 PM, Stack wrote: > >> On Wed, Aug 2, 2017 at 5:25 PM, Zach York >> wrote: >> >> > Do we know what the major pain points for migration are? Can we discuss >> > that/get a list going? >> > >> > >> Here's a few in outline: >> >> + There is issue of formats, of hbase-2.x being able to read hbase-1.x >> data >> whether from HDFS or ZooKeeper or off the wire. >> + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x >> cluster; no holes in the API or unintelligible serializations. >> + There is then the little dance that has us rolling restart from an >> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it will >> assign regions to the new hbase-2.x regionservers as they come on line. >> TBD. >> >> Is this what you mean sir? >> >> S >> >> >> > I think without that knowledge it is hard (for me at least :) ) to >> > determine where we should set our sights in terms of migration. >> > >> > Thanks, >> > Zach >> > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: >> > >> > > What are our expectations regards compatibility between hbase1 and >> > hbase2? >> > > >> > > Lets have a chat about it. Here are some goal posts. >> > > >> > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. >> No >> > > migration from < hbase-1 (Is this too onerous? Should we support 0.98 >> => >> > > 2.0?). >> > > + You do NOT have to upgrade to the latest release of hbase1 to >> migrate >> > to >> > > hbase2; being up on hbase-1.0.0+ will be sufficient. >> > > + You'll have to update your hbase1 coprocessors to deploy them on >> > hbase2. >> > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. >> > > watching for region split on RegionServer no longer makes sense given >> > > Master runs all splits now. >> > > + An hbase1 client can run against an hbase2 cluster but it will only >> be >> > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do >> > admin >> > > ops using an hbase1 Admin client against an hbase2 cluster. We have >> some >> > > egregious API violations in branch-1; e.g. we have protobuf in our API >> > (See >> > > HBASE-15607). The notion is that we can't afford a deprecation cycle >> > > purging this stuff from our Admin API. >> > > >> > > What you all think? >> > > >> > > St.Ack >> > > >> > >> > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
This kinda helps, but these seem more like expectations. I was going more for things like HFile format changed, meta table structure changed, coprocessor implementations changed (these are just examples, I don't know if any of these actually changed). More technical differences between branch-1 and branch-2 which then can help us get the right expectations for compatibility. On Wed, Aug 2, 2017 at 6:34 PM, Stackwrote: > On Wed, Aug 2, 2017 at 5:25 PM, Zach York > wrote: > > > Do we know what the major pain points for migration are? Can we discuss > > that/get a list going? > > > > > Here's a few in outline: > > + There is issue of formats, of hbase-2.x being able to read hbase-1.x data > whether from HDFS or ZooKeeper or off the wire. > + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x > cluster; no holes in the API or unintelligible serializations. > + There is then the little dance that has us rolling restart from an > hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it will > assign regions to the new hbase-2.x regionservers as they come on line. > TBD. > > Is this what you mean sir? > > S > > > > I think without that knowledge it is hard (for me at least :) ) to > > determine where we should set our sights in terms of migration. > > > > Thanks, > > Zach > > > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > > > What are our expectations regards compatibility between hbase1 and > > hbase2? > > > > > > Lets have a chat about it. Here are some goal posts. > > > > > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. > No > > > migration from < hbase-1 (Is this too onerous? Should we support 0.98 > => > > > 2.0?). > > > + You do NOT have to upgrade to the latest release of hbase1 to migrate > > to > > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > > + You'll have to update your hbase1 coprocessors to deploy them on > > hbase2. > > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > > > watching for region split on RegionServer no longer makes sense given > > > Master runs all splits now. > > > + An hbase1 client can run against an hbase2 cluster but it will only > be > > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do > > admin > > > ops using an hbase1 Admin client against an hbase2 cluster. We have > some > > > egregious API violations in branch-1; e.g. we have protobuf in our API > > (See > > > HBASE-15607). The notion is that we can't afford a deprecation cycle > > > purging this stuff from our Admin API. > > > > > > What you all think? > > > > > > St.Ack > > > > > >
Re: [DISCUSS] hbase-2.0.0 compatibility expectations
On Wed, Aug 2, 2017 at 5:25 PM, Zach Yorkwrote: > Do we know what the major pain points for migration are? Can we discuss > that/get a list going? > > Here's a few in outline: + There is issue of formats, of hbase-2.x being able to read hbase-1.x data whether from HDFS or ZooKeeper or off the wire. + An hbase-1.x client should be able to Get/Put and Scan an hbase-2.x cluster; no holes in the API or unintelligible serializations. + There is then the little dance that has us rolling restart from an hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then it will assign regions to the new hbase-2.x regionservers as they come on line. TBD. Is this what you mean sir? S > I think without that knowledge it is hard (for me at least :) ) to > determine where we should set our sights in terms of migration. > > Thanks, > Zach > > On Wed, Aug 2, 2017 at 4:38 PM, Stack wrote: > > > What are our expectations regards compatibility between hbase1 and > hbase2? > > > > Lets have a chat about it. Here are some goal posts. > > > > + You have to upgrade to hbase-1.x before you can migrate to hbase-2. No > > migration from < hbase-1 (Is this too onerous? Should we support 0.98 => > > 2.0?). > > + You do NOT have to upgrade to the latest release of hbase1 to migrate > to > > hbase2; being up on hbase-1.0.0+ will be sufficient. > > + You'll have to update your hbase1 coprocessors to deploy them on > hbase2. > > A bunch of CP API has/will change by the time hbase2 comes out; e.g. > > watching for region split on RegionServer no longer makes sense given > > Master runs all splits now. > > + An hbase1 client can run against an hbase2 cluster but it will only be > > able to do DML (Get/Put/Scan, etc.). We do not allow being able to do > admin > > ops using an hbase1 Admin client against an hbase2 cluster. We have some > > egregious API violations in branch-1; e.g. we have protobuf in our API > (See > > HBASE-15607). The notion is that we can't afford a deprecation cycle > > purging this stuff from our Admin API. > > > > What you all think? > > > > St.Ack > > >