@Lars: agreed on your point of leaving the name for the 92 znode the same. On upgrading a 0.94.0 or 0.94.1 cluster out of sync, I think it would work. The 0.94.2 client should work against all 0.94.2, 0.94.1, and 0.94.0 versions (and all 0.92.x versions as well). We just need to apply HBASE-6268 to 0.94.2. So you would first upgrade the client, then the server(s). The reason this works is because the client, with HBASE-6268 applied, can handle the znode being in either the 0.92 or 0.94 format, it doesn't need to know which is which. I'll of course test all this :).
Greg On Fri, Aug 31, 2012 at 7:02 PM, lars hofhansl <[email protected]> wrote: > Also, if understand this correctly this would not help if somebody had > deployed an 0.94.0 or 0.94.1 cluster and want to upgrade the client and > server out of sync. > In one configuration the client would work against 0.94.0 and 0.94.1 but > not against 0.94.2. In the other configuration the client would work > against 0.94.2 but not against 0.94.0 or 0.94.1. > > > There is however and clean upgrade path to 0.92.2 and from there to 0.94.2 > if we just fix this in 0.92.2. > > > -- Lars > > > > ________________________________ > From: lars hofhansl <[email protected]> > To: Gregory Chanan <[email protected]>; "[email protected]" < > [email protected]> > Sent: Friday, August 31, 2012 6:34 PM > Subject: Re: 0.92/0.94 compatibility and HBASE-5206 > > Sounds complicated. But since you are the folks with customers that'll > upgrade to from 0.92 to 0.94, let's do this. > > The only input I'd have is that format we'll use going forward will not > have a version attached to it. > > So maybe the 92 version would still be called > "zookeeper.znode.tableEnableDisable" and the new node could have a > different name "zookeeper.znode.tableEnableDisableNew" (or something). > > -- Lars > > > > ________________________________ > From: Gregory Chanan <[email protected]> > To: [email protected]; lars hofhansl <[email protected]> > Sent: Friday, August 31, 2012 5:54 PM > Subject: Re: 0.92/0.94 compatibility and HBASE-5206 > > > Actually, I think we can make 0.94.2 compatible with both {0.94.0,0.94.1} > and {0.92.0,0.92.1}, although one of those sets will require configuration > changes. > > The basic problem is that there is a znode for each table > "zookeeper.znode.tableEnableDisable" that is handled differently. > > On 0.92.0 and 0.92.1 the states for this table are: > [ disabled, disabling, enabling ] or deleted if the table is enabled > > On 0.94.1 and 0.94.2 the states for this table are: > [ disabled, disabling, enabling, enabled ] > > What saves us is that the location of this znode is configurable. So the > basic idea is to have the 0.94.2 master write two different znodes, > "zookeeper.znode.tableEnableDisabled92" and > "zookeeper.znode.tableEnableDisabled94" where the 92 node is in 92 format, > the 94 node is in 94 format. And internally, the master would only use the > 94 format in order to solve the original bug HBASE-5155 solves. > We can of course make one of these the same default as exists now, so we > don't need to make config changes for one of 0.92 or 0.94 clients. I argue > that 0.92 clients shouldn't have to make config changes for the same reason > I argued above. But that is debatable. > > Then, I think the only question left is Stack's question of how to bring > along the {0.94.0, 0.94.1} crew. A {0.94.0, 0.94.1} client would work > against a 0.94.2 cluster by just > configuring "zookeeper.znode.tableEnableDisable" in the client to be > whatever "zookeeper.znode.tableEnableDisabled94" is in the cluster. A > 0.94.2 client would work against both a {0.94.0, 0.94.1} and {0.92.0, > 0.92.1} cluster if it had HBASE-6268 applied. About rolling upgrade from > {0.94.0, 0.94.1} to 0.94.2 -- I'd have to think about that. Do the > regionservers ever read the tableEnableDisabled znode? > > Greg > > On Fri, Aug 31, 2012 at 4:41 PM, lars hofhansl <[email protected]> > wrote: > > I guess you voiced your opinion in your initial email already (you prefer > to break compatibility between 0.94.0/0.94.1 with 0.94.2). > >If that is indeed the consensus, please file a jira against 0.94.2. > > > >-- Lars > > > > > > > > > >----- Original Message ----- > >From: lars hofhansl <[email protected]> > >To: "[email protected]" <[email protected]> > >Cc: > >Sent: Friday, August 31, 2012 4:24 PM > >Subject: Re: 0.92/0.94 compatibility and HBASE-5206 > > > >What do you think we should do? > > > >1. Breaking compatibility between minor versions is bad. (i.e. we should > fix this in 0.92.2 as currently proposed) > >2. At the same time 0.92 might be in wider distribution and that the > upgrade path 0.94 might be more important (and include the fix that you > propose). > > > >I'm +1 on #1 and +0 on #2. > > > > > >I agree we need better cross-version integration testing and be generally > more diligent about this. > > > >-- Lars > >________________________________ > >From: Gregory Chanan <[email protected]> > >To: [email protected] > >Cc: lars hofhansl <[email protected]> > >Sent: Friday, August 31, 2012 4:06 PM > >Subject: Re: 0.92/0.94 compatibility and HBASE-5206 > > > >@Lars: > >you are correct that this would break compatibility between {0.94.0, > >0.94.1} and 0.94.2. > > > >We clearly need better compatibility testing, these issues are hard to > find > >by just looking at patches. > > > >Greg > > > >On Fri, Aug 31, 2012 at 4:03 PM, Ted Yu <[email protected]> wrote: > > > >> I should also apologize for HBASE-5206 where I didn't maintain > >> compatibility in the first place. > >> > >> We just need to find the solution which minimizes impact of this issue. > >> > >> Cheers > >> > >> On Fri, Aug 31, 2012 at 3:55 PM, lars hofhansl <[email protected]> > >> wrote: > >> > >> > Won't we then break compatibility between 0.94.0 and 0.94.1 with > 0.94.2? > >> > I do not have a strong opinion about this. > >> > > >> > It was my fault that HBASE-5206 slipped into 0.94.0, I apologize for > >> that. > >> > > >> > I was going to spin the first 0.94.2 today. Is the general consensus > that > >> > I should wait? > >> > > >> > > >> > -- Lars > >> > > >> > > >> > > >> > ________________________________ > >> > From: Gregory Chanan <[email protected]> > >> > To: [email protected] > >> > Sent: Friday, August 31, 2012 3:13 PM > >> > Subject: 0.92/0.94 compatibility and HBASE-5206 > >> > > >> > There has been some discussion on the JIRA lately about what to do > about > >> a > >> > 0.92/0.94 compatibility issue. I wanted to bring this up to a larger > >> > audience in order to solicit additional opinions. > >> > > >> > HBASE-5206 (https://issues.apache.org/jira/browse/HBASE-5206), which > is > >> in > >> > 0.94.0 and 0.94.1, breaks compatibility with 0.92.0 and 0.92.1. The > >> issue > >> > is described in the release notes for HBASE-5155 (of which HBASE-5206 > is > >> a > >> > forward port). Excerpted here: > >> > > >> > This issue is an incompatible change. > >> > If an HBase client with the changes for HBASE-5155 and a server > >> (master) > >> > without the changes for HBASE-5155 is used, then the is_enabled (from > >> HBase > >> > Shell) or isTableEnabled() (from HBaseAdmin) will return false though > the > >> > table is already enabled as per the master. > >> > > >> > If the HBase client does have the changes for HBASE-5155 and the > server > >> > does not have the changes for HBASE-5155, then if we try to Enable a > >> table > >> > then the client will hang. > >> > > >> > The reason is because, > >> > Prior to HBASE-5155 once the table is enabled the znode in the > >> zookeeper > >> > created for the table is deleted. > >> > After HBASE-5155 once the table is enabled the znode in the > zookeeper > >> > created for the table is not deleted, whereas the same node is updated > >> with > >> > the status ENABLED. > >> > > >> > The client also expects the status of the znode in the zookeeper to > be > >> in > >> > the ENABLED state if the table has been enabled successfully. > >> > The above changes makes the client behaviour incompatible if the > client > >> > does not have this fix whereas the server has this fix. > >> > If both the client and the server does not have this fix, then the > >> > behaviour is as expected. > >> > > >> > So we have a situation where 0.92.0 and 0.92.1 clients are not > compatible > >> > with 0.94.0 and 0.94.1 servers and visa-versa. > >> > > >> > The issue is, what should we do about the 0.94.2 server? As far as I > can > >> > tell, we cannot make both {0.92.0,0.92.1} and {0.94.0,0.94.1} clients > >> > happy. I propose we make the 0.94.2 server compatible with the > {0.92.0, > >> > 0.92.1} client set. My thinking is that we should optimize for the > most > >> > common use case. Given that both CDH4 and HDP 1.0 ship 0.92.1, and > that > >> > 0.94 is relatively new, it seems more likely that users will upgrade > >> > servers from 0.92.1 to 0.94.2+ than from an earlier 0.94 version. I > >> think > >> > we can do this while actually fixing the issue that HBASE-5206 > purports > >> to > >> > fix by moving the ENABLED state to a different znode. I'll even > >> volunteer > >> > to do that :). I don't think there is an obvious or easy answer > here, so > >> > wanted to get other's opinions. > >> > > >> > As for the client, HBASE-6268 ( > >> > https://issues.apache.org/jira/browse/HBASE-6268) was introduced in > >> 0.92.2 > >> > to handle both cases of servers ({0.92.0,0.92.1} and {0.94.0,0.94.1}) > by > >> > accepting both possibilities for the table znode. I think that > approach > >> > works whether we leave the server as is or go with what I've proposed > -- > >> > but we should test. > >> > > >> > Greg > >> > > > > >
