On Mon, Sep 4, 2017 at 6:55 PM, Francis Christopher Liu <
toffer....@gmail.com> wrote:
> Just some high-level thoughts on rolling upgradeability (I'm repeating a
> few things already said).
>
> 1. No service interruption for 1.x clients while a cluster is being
> ugpraded to 2.x.
- This includes support for apis commonly used in a running system:
> delegation tokens, getting region locations, etc.
>
Agreed. Having hbase-1 clients read the basic info they need to operate
seems doable w/ a little bit of work (having meta table edits mirrored in
zk). Let me add note to test delegation tokens continue to work.
> 2. The somewhat reverse of #1 is also necessary for Master DMLs to meta.
>
Come again. hbase1 won't be able to admin a hbase2 (basic read-only admin
function will work like checking if table is online or not, etc.)
> 3. No interruption for replication between 1.x and 2.x cluster
> 4. Restart master first before any other service sounds reasonable to me.
> Will we support RU for zkless mode?
>
I'll not be testing RU for ZKLESS mode. Any one up for this?
> 5. 2.x should be either able to understand formats written by 1.x or online
> migration is done as a preparation step.
>
Yep. Auto-migration preferred.
> 6. Data generated by 2.x daemons should be readable in 1.x (ie HFile,
> zookeeper, etc). Some deploys may have large amounts of data could be
> cumbersome/impractical (ie cost, time, etc) to copy the data.
>
>
Thanks Francis,
S
> Thanks,
> Francis
>
>
>
> On Fri, Sep 1, 2017 at 10:01 PM Chia-Ping Tsai <chia7...@apache.org>
> wrote:
>
> > The most of issues use hadoop flag, so the filter looks like this.
> > project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1, 2.0.0-alpha-2,
> > 2.0.0-alpha-3,
> > 3.0) AND (labels in (incompatibleChange, incompatible, incompatibility)
> OR
> > "Hadoop Flags" in ("Incompatible change"))
> >
> > On 2017-08-04 05:46, Ted Yu <yuzhih...@gmail.com> wrote:
> > > I expanded the condition in the filter like this:
> > >
> > > project = HBASE AND fixVersion in (2.0.0, 2.0.0-alpha-1,
> 2.0.0-alpha-2,
> > > 3.0) AND labels in (incompatibleChange, incompatible, incompatibility)
> > >
> > > Still there're only two showing up.
> > >
> > > On Thu, Aug 3, 2017 at 9:07 AM, Zach York <
> zyork.contribut...@gmail.com>
> > > wrote:
> > >
> > > > I tried to filter based on imcompatible labels and there were only
> two
> > > > JIRAs returned [1]. I have a hard time believing that there were only
> > two
> > > > breaking changes from 1.x to 2.x.
> > > >
> > > > [1]
> > > > https://issues.apache.org/jira/browse/HBASE-17957?jql=
> > > > project%20%3D%20HBASE%20AND%20fixVersion%20%3D%202.0.0%
> > > > 20AND%20labels%20in%20(incompatibleChange%2C%20incompatible%2C%
> > > > 20incompatibility)
> > > >
> > > > On Thu, Aug 3, 2017 at 8:54 AM, Zach York <
> > zyork.contribut...@gmail.com>
> > > > wrote:
> > > >
> > > > > This kinda helps, but these seem more like expectations. I was
> going
> > more
> > > > > for things like HFile format changed, meta table structure changed,
> > > > > coprocessor implementations changed (these are just examples, I
> don't
> > > > know
> > > > > if any of these actually changed).
> > > > >
> > > > > More technical differences between branch-1 and branch-2 which then
> > can
> > > > > help us get the right expectations for compatibility.
> > > > >
> > > > > On Wed, Aug 2, 2017 at 6:34 PM, Stack <st...@duboce.net> wrote:
> > > > >
> > > > >> On Wed, Aug 2, 2017 at 5:25 PM, Zach York <
> > zyork.contribut...@gmail.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Do we know what the major pain points for migration are? Can we
> > > > discuss
> > > > >> > that/get a list going?
> > > > >> >
> > > > >> >
> > > > >> Here's a few in outline:
> > > > >>
> > > > >> + There is issue of formats, of hbase-2.x being able to read
> > hbase-1.x
> > > > >> data
> > > > >> whether from HDFS or ZooKeeper or off the wire.
> > > > >> + An hbase-1.x client should be able to Get/Put and Scan an
> > hbase-2.x
> > > > >> cluster; no holes in the API or unintelligible serializations.
> > > > >> + There is then the little dance that has us rolling restart from
> an
> > > > >> hbase-1.x cluster to hbase-2.x; i.e. upgrade master first and then
> > it
> > > > will
> > > > >> assign regions to the new hbase-2.x regionservers as they come on
> > line.
> > > > >> TBD.
> > > > >>
> > > > >> Is this what you mean sir?
> > > > >>
> > > > >> S
> > > > >>
> > > > >>
> > > > >> > I think without that knowledge it is hard (for me at least :) )
> to
> > > > >> > determine where we should set our sights in terms of migration.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Zach
> > > > >> >
> > > > >> > On Wed, Aug 2, 2017 at 4:38 PM, Stack <st...@duboce.net> wrote:
> > > > >> >
> > > > >> > > What are our expectations regards compatibility between hbase1
> > and
> > > > >> > hbase2?
> > > > >> > >
> > > > >> > > Lets have a chat about it. Here are some goal posts.
> > > > >> > >
> > > > >> > > + You have to upgrade to hbase-1.x before you can migrate to
> > > > hbase-2.
> > > > >> No
> > > > >> > > migration from < hbase-1 (Is this too onerous? Should we
> support
> > > > 0.98
> > > > >> =>
> > > > >> > > 2.0?).
> > > > >> > > + You do NOT have to upgrade to the latest release of hbase1
> to
> > > > >> migrate
> > > > >> > to
> > > > >> > > hbase2; being up on hbase-1.0.0+ will be sufficient.
> > > > >> > > + You'll have to update your hbase1 coprocessors to deploy
> them
> > on
> > > > >> > hbase2.
> > > > >> > > A bunch of CP API has/will change by the time hbase2 comes
> out;
> > e.g.
> > > > >> > > watching for region split on RegionServer no longer makes
> sense
> > > > given
> > > > >> > > Master runs all splits now.
> > > > >> > > + An hbase1 client can run against an hbase2 cluster but it
> will
> > > > only
> > > > >> be
> > > > >> > > able to do DML (Get/Put/Scan, etc.). We do not allow being
> able
> > to
> > > > do
> > > > >> > admin
> > > > >> > > ops using an hbase1 Admin client against an hbase2 cluster. We
> > have
> > > > >> some
> > > > >> > > egregious API violations in branch-1; e.g. we have protobuf in
> > our
> > > > API
> > > > >> > (See
> > > > >> > > HBASE-15607). The notion is that we can't afford a deprecation
> > cycle
> > > > >> > > purging this stuff from our Admin API.
> > > > >> > >
> > > > >> > > What you all think?
> > > > >> > >
> > > > >> > > St.Ack
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>
