I agree. Just marked 15265 and 10800 incompatible change.
> On Dec 7, 2016, at 2:38 AM, 张铎 <palomino...@gmail.com> wrote: > > I think the imcompatible change is HBASE-15265? HBASE-14949 aims to fix the > incompatible... Just like HBASE-16189 to HBASE-10800. > > I think we should mark HBASE-15265 and HBASE-10800 as Incompatible Change? > > 2016-12-07 18:29 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > >> Shouldn't HBASE-14949 be marked Incompatible Change ? >> >>> On Wed, Dec 7, 2016 at 2:17 AM, 张铎 <palomino...@gmail.com> wrote: >>> >>> See this one >>> >>> https://issues.apache.org/jira/browse/HBASE-14949 >>> >>> 2016-12-07 18:09 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: >>> >>>> Which other features ? >>>> Can you name them ? >>>> >>>> Thanks >>>> >>>>> On Wed, Dec 7, 2016 at 2:06 AM, 张铎 <palomino...@gmail.com> wrote: >>>>> >>>>> I believe there are other features that require upgrading to a >> specific >>>>> version in each 1.x release before rolling upgrading to 2.0? I think >>> this >>>>> is a typical trick to add incompatible changes... >>>>> >>>>> 2016-12-07 18:02 GMT+08:00 ramkrishna vasudevan < >>>>> ramkrishna.s.vasude...@gmail.com>: >>>>> >>>>>> If its going to be a pain then we need to support writing the older >>>> class >>>>>> name in the FFT (trailer) and then use our internal mapping. >>>>>> >>>>>>> On Wed, Dec 7, 2016 at 3:22 PM, Anoop John <anoop.hb...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I read the comments in the issue 16189.. This concern was raised >>>>>>> there already.. This is what I replied there. >>>>>>> "Ya we may need to make such a need. User on 1.x has to first >>> upgrade >>>>>>> (rolling upgrade supported) to latest version in 1.x and then to >>> 2.0. >>>>>>> This was discussed some other place also. I believe in the mail >>>> thread >>>>>>> where we discussed abt rolling upgrade support in 2.0" >>>>>>> >>>>>>> Not able to find the original mail thread which discussed abt >>> rolling >>>>>>> upgrade support in 2.0 Pls correct if took some thing wrong >>> way.. >>>>>>> Ya I agree that this way of indirection might be inconvenience >> for >>>>>>> users. >>>>>>> >>>>>>> >>>>>>> -Anoop- >>>>>>> >>>>>>> On Wed, Dec 7, 2016 at 3:13 PM, Ted Yu <yuzhih...@gmail.com> >>> wrote: >>>>>>>> bq. write old name only and within code we will have to map it >>> with >>>>> new >>>>>>>> name. >>>>>>>> >>>>>>>> This is a viable approach. >>>>>>>> Even if we document upgrading to 1.2.3+, some users may not >>> perform >>>>>> this >>>>>>>> step before upgrading to 2.0 >>>>>>>> >>>>>>>> On Tue, Dec 6, 2016 at 10:04 PM, Anoop John < >>> anoop.hb...@gmail.com >>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>>> Ya that bug raised by Enis was very valid.. So this means >> when >>>>>>>>> rolling upgrade happens, if some of the other RSs with version >>>>> <1.2.3 >>>>>>>>> , which is not having this fix, this issue might come up! >>>>>>>>> How to address then? Do we need to enforce a 1.2.3+ upgrade >>> 1st >>>>> and >>>>>>>>> then only 2.0 rolling upgrade? >>>>>>>>> >>>>>>>>> Or else we will need to fix in 2.0.. We write the new >>> Comparator >>>>>>>>> class name in FFT (trunk code) To fix, we might have to >> write >>>> old >>>>>>>>> name only and within code we will have to map it with new >> name. >>> It >>>>>>>>> will be ugly! It can be fixed only in 3.0 may be.. But that >> can >>>>> make >>>>>>>>> the rolling upgrade story easier for users.. Just saying the >>>>>>>>> possibilities.. >>>>>>>>> >>>>>>>>> Thanks Ted to bring it up again. >>>>>>>>> >>>>>>>>> -Anoop- >>>>>>>>> >>>>>>>>> On Wed, Dec 7, 2016 at 10:04 AM, ramkrishna vasudevan >>>>>>>>> <ramkrishna.s.vasude...@gmail.com> wrote: >>>>>>>>>> I think when Enis reported the issue ( >>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-16189), in >>> rolling >>>>>>> upgrade >>>>>>>>>> regions could move around. So a region created in 2.0 moved >>> to a >>>>>>> server >>>>>>>>>> with 1.x. >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> Ram >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Dec 7, 2016 at 1:27 AM, Stack <st...@duboce.net> >>> wrote: >>>>>>>>>> >>>>>>>>>>> On Tue, Dec 6, 2016 at 10:19 AM, Ted Yu < >> yuzhih...@gmail.com >>>> >>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Looking at http://hbase.apache.org/book. >>>>> html#executing.the.0.96. >>>>>>>>> upgrade >>>>>>>>>>> , >>>>>>>>>>>> there is step of running "bin/hbase upgrade -check" >>>>>>>>>>>> >>>>>>>>>>>> How about adding a sample hfile which contains >>>>>>>>>>>> CellComparator$MetaCellComparator >>>>>>>>>>>> so that the upgrade check can read and verify ? >>>>>>>>>>> >>>>>>>>>>> We don't support downgrade. Never have. >>>>>>>>>>> St.Ack >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Tue, Dec 6, 2016 at 8:50 AM, Ted Yu < >>> yuzhih...@gmail.com> >>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> The build I used yesterday didn't include HBASE-16189 >>>>>>>>>>>>> <https://issues.apache.org/jira/browse/HBASE-16189> >>>>>>>>>>>>> >>>>>>>>>>>>> Once it is included, the cluster can be downgraded >> fine. >>>>>>>>>>>>> >>>>>>>>>>>>> I wonder how users would know that their existing >>>> deployment >>>>>> has >>>>>>>>>>>>> HBASE-16189 <https://issues.apache.org/ >>>>> jira/browse/HBASE-16189 >>>>>>> >>>>>>>>> before >>>>>>>>>>>>> upgrading to 2.0 release. >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Dec 6, 2016 at 2:29 AM, ramkrishna vasudevan < >>>>>>>>>>>>> ramkrishna.s.vasude...@gmail.com> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> @Ted >>>>>>>>>>>>>> Does your version have this fix >>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-16189 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regards >>>>>>>>>>>>>> Ram >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Dec 6, 2016 at 3:56 PM, Ted Yu < >>>> yuzhih...@gmail.com >>>>>> >>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Is the assumption that hbase:meta would not split ? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In other thread, Francis Liu was proposing >> supporting >>>>>>> splittable >>>>>>>>>>>>>>> hbase:meta in 2.0 release. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cheers >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Dec 6, 2016, at 2:20 AM, 张铎 < >>> palomino...@gmail.com >>>>> >>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Could this be solved by hosting meta only on >> master? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> BTW, MetaCellComparator is introduced in >>> HBASE-10800. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2016-12-06 17:44 GMT+08:00 Ted Yu < >>>> yuzhih...@gmail.com >>>>>> : >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> When I restarted a cluster with 1.1 , I found >> that >>>>>>> hbase:meta >>>>>>>>>>>> region >>>>>>>>>>>>>>>>> (written to by the previously deployed 2.0) >>> couldn't >>>> be >>>>>>>>> opened: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Caused by: java.io.IOException: >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >>>>> hfile.CorruptHFileException: >>>>>>>>> Problem >>>>>>>>>>>>>> reading >>>>>>>>>>>>>>>>> HFile Trailer from file hdfs:// >>> yz1.xx.com:8020/apps/ >>>>>>>>>>>>>> hbase/data/data/ >>>>>>>>>>>>>>>>> hbase/meta/1588230740/info/ >>>>>> 599fc8a37311414e876803312009a9 >>>>>>> 86 >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HStore. >>>>>>> openStoreFiles( >>>>>>>>>>>>>> HStore.java: >>>>>>>>>>>>>>>>> 579) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HStore. >>>>>>> loadStoreFiles( >>>>>>>>>>>>>> HStore.java: >>>>>>>>>>>>>>>>> 534) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>> regionserver.HStore.<init>( >>>>>>>>>>>> HStore.java:275) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HRegion. >>>>>>> instantiateHSto >>>>>>>>>>>>>> re(HRegion. >>>>>>>>>>>>>>>>> java:5150) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>> regionserver.HRegion$1.call( >>>>>>> HRegion. >>>>>>>>>>>>>> java:912) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>> regionserver.HRegion$1.call( >>>>>>> HRegion. >>>>>>>>>>>>>> java:909) >>>>>>>>>>>>>>>>> at java.util.concurrent. >>>>>> FutureTask.run(FutureTask. >>>>>>>>>>> java:266) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> java.util.concurrent.Executors$RunnableAdapter. >>> call( >>>>>>>>>>>>>> Executors.java:511) >>>>>>>>>>>>>>>>> at java.util.concurrent. >>>>>> FutureTask.run(FutureTask. >>>>>>>>>>> java:266) >>>>>>>>>>>>>>>>> ... 3 more >>>>>>>>>>>>>>>>> Caused by: org.apache.hadoop.hbase.io. >>>>>>>>>>> hfile.CorruptHFileException: >>>>>>>>>>>>>>> Problem >>>>>>>>>>>>>>>>> reading HFile Trailer from file hdfs:// >>>>>>>>>>>>>>>>> yz1.xx.com:8020/apps/hbase/data/data/hbase/ >>>>>>> meta/1588230740/ >>>>>>>>>>>>>>>>> info/599fc8a37311414e876803312009a986 >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >>>>>> hfile.HFile.pickReaderVersion( >>>>>>>>>>>>>>> HFile.java:483) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >>> hfile.HFile.createReader( >>>>>>>>>>>> HFile.java:511) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>>> regionserver.StoreFile$Reader. >>>>>>>>>>>>>>>>> <init>(StoreFile.java:1128) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>> regionserver.StoreFileInfo. >>>>>>>>>>>>>>>>> open(StoreFileInfo.java:267) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>> regionserver.StoreFile.open( >>>>>>> StoreFil >>>>>>>>>>>>>> e.java:409) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.StoreFile. >>>>>>>>>>>>>>>>> createReader(StoreFile.java:517) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.regionserver.HStore. >>>>>>> createStoreFileA >>>>>>>>>>>>>> ndReader( >>>>>>>>>>>>>>>>> HStore.java:687) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>> regionserver.HStore.access$ >>>>> 000( >>>>>>>>>>>>>> HStore.java:130) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>> regionserver.HStore$1.call( >>>>>>>>>>>> HStore.java:554) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase. >>> regionserver.HStore$1.call( >>>>>>>>>>>> HStore.java:551) >>>>>>>>>>>>>>>>> ... 6 more >>>>>>>>>>>>>>>>> Caused by: java.io.IOException: java.lang. >>>>>>>>>>> ClassNotFoundException: >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.CellComparator$ >>>>>> MetaCellComparator >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >> hfile.FixedFileTrailer. >>>>>>> getCompara >>>>>>>>>>>>>> torClass( >>>>>>>>>>>>>>>>> FixedFileTrailer.java:581) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >> hfile.FixedFileTrailer. >>>>>>>>>>>> deserializeFromPB( >>>>>>>>>>>>>>>>> FixedFileTrailer.java:300) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >> hfile.FixedFileTrailer. >>>>>>>>>>>>>>>>> deserialize(FixedFileTrailer.java:242) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >> hfile.FixedFileTrailer. >>>>>>>>>>> readFromStream( >>>>>>>>>>>>>>>>> FixedFileTrailer.java:407) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >>>>>> hfile.HFile.pickReaderVersion( >>>>>>>>>>>>>>> HFile.java:468) >>>>>>>>>>>>>>>>> ... 15 more >>>>>>>>>>>>>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.CellComparator$ >>>>>> MetaCellComparator >>>>>>>>>>>>>>>>> at java.net.URLClassLoader. >>>>>>>>> findClass(URLClassLoader.java: >>>>>>>>>>>> 381) >>>>>>>>>>>>>>>>> at java.lang.ClassLoader. >>>>>>> loadClass(ClassLoader.java: >>>>>>>>> 424) >>>>>>>>>>>>>>>>> at sun.misc.Launcher$ >>>> AppClassLoader.loadClass( >>>>>>>>>>>> Launcher.java: >>>>>>>>>>>>>> 331) >>>>>>>>>>>>>>>>> at java.lang.ClassLoader. >>>>>>> loadClass(ClassLoader.java: >>>>>>>>> 357) >>>>>>>>>>>>>>>>> at java.lang.Class.forName0(Native >> Method) >>>>>>>>>>>>>>>>> at java.lang.Class.forName(Class. >> java:264) >>>>>>>>>>>>>>>>> at >>>>>>>>>>>>>>>>> org.apache.hadoop.hbase.io. >> hfile.FixedFileTrailer. >>>>>>> getCompara >>>>>>>>>>>>>> torClass( >>>>>>>>>>>>>>>>> FixedFileTrailer.java:579) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> When user does rolling upgrade from 1.1 to 2.0, >> the >>>>> above >>>>>>> may >>>>>>>>>>> cause >>>>>>>>>>>>>>> problem >>>>>>>>>>>>>>>>> if hbase:meta region is updated by server running >>> 2.0 >>>>> but >>>>>>>>> later >>>>>>>>>>>>>>> assigned to >>>>>>>>>>>>>>>>> a region server which still runs 1.1 (due to >> crash >>> of >>>>> the >>>>>>>>> server >>>>>>>>>>>>>> running >>>>>>>>>>>>>>>>> 2.0, e.g.) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I want to get community feedback on the severity >> of >>>>> this >>>>>>>>> issue. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks >>
