See this one https://issues.apache.org/jira/browse/HBASE-14949
2016-12-07 18:09 GMT+08:00 Ted Yu <yuzhih...@gmail.com>: > Which other features ? > Can you name them ? > > Thanks > > On Wed, Dec 7, 2016 at 2:06 AM, 张铎 <palomino...@gmail.com> wrote: > > > I believe there are other features that require upgrading to a specific > > version in each 1.x release before rolling upgrading to 2.0? I think this > > is a typical trick to add incompatible changes... > > > > 2016-12-07 18:02 GMT+08:00 ramkrishna vasudevan < > > ramkrishna.s.vasude...@gmail.com>: > > > > > If its going to be a pain then we need to support writing the older > class > > > name in the FFT (trailer) and then use our internal mapping. > > > > > > On Wed, Dec 7, 2016 at 3:22 PM, Anoop John <anoop.hb...@gmail.com> > > wrote: > > > > > > > I read the comments in the issue 16189.. This concern was raised > > > > there already.. This is what I replied there. > > > > "Ya we may need to make such a need. User on 1.x has to first upgrade > > > > (rolling upgrade supported) to latest version in 1.x and then to 2.0. > > > > This was discussed some other place also. I believe in the mail > thread > > > > where we discussed abt rolling upgrade support in 2.0" > > > > > > > > Not able to find the original mail thread which discussed abt rolling > > > > upgrade support in 2.0 Pls correct if took some thing wrong way.. > > > > Ya I agree that this way of indirection might be inconvenience for > > > > users. > > > > > > > > > > > > -Anoop- > > > > > > > > On Wed, Dec 7, 2016 at 3:13 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > bq. write old name only and within code we will have to map it with > > new > > > > > name. > > > > > > > > > > This is a viable approach. > > > > > Even if we document upgrading to 1.2.3+, some users may not perform > > > this > > > > > step before upgrading to 2.0 > > > > > > > > > > On Tue, Dec 6, 2016 at 10:04 PM, Anoop John <anoop.hb...@gmail.com > > > > > > wrote: > > > > > > > > > >> Ya that bug raised by Enis was very valid.. So this means when > > > > >> rolling upgrade happens, if some of the other RSs with version > > <1.2.3 > > > > >> , which is not having this fix, this issue might come up! > > > > >> How to address then? Do we need to enforce a 1.2.3+ upgrade 1st > > and > > > > >> then only 2.0 rolling upgrade? > > > > >> > > > > >> Or else we will need to fix in 2.0.. We write the new Comparator > > > > >> class name in FFT (trunk code) To fix, we might have to write > old > > > > >> name only and within code we will have to map it with new name. It > > > > >> will be ugly! It can be fixed only in 3.0 may be.. But that can > > make > > > > >> the rolling upgrade story easier for users.. Just saying the > > > > >> possibilities.. > > > > >> > > > > >> Thanks Ted to bring it up again. > > > > >> > > > > >> -Anoop- > > > > >> > > > > >> On Wed, Dec 7, 2016 at 10:04 AM, ramkrishna vasudevan > > > > >> <ramkrishna.s.vasude...@gmail.com> wrote: > > > > >> > I think when Enis reported the issue ( > > > > >> > https://issues.apache.org/jira/browse/HBASE-16189), in rolling > > > > upgrade > > > > >> > regions could move around. So a region created in 2.0 moved to a > > > > server > > > > >> > with 1.x. > > > > >> > > > > > >> > Regards > > > > >> > Ram > > > > >> > > > > > >> > > > > > >> > On Wed, Dec 7, 2016 at 1:27 AM, Stack <st...@duboce.net> wrote: > > > > >> > > > > > >> >> On Tue, Dec 6, 2016 at 10:19 AM, Ted Yu <yuzhih...@gmail.com> > > > wrote: > > > > >> >> > > > > >> >> > Looking at http://hbase.apache.org/book. > > html#executing.the.0.96. > > > > >> upgrade > > > > >> >> , > > > > >> >> > there is step of running "bin/hbase upgrade -check" > > > > >> >> > > > > > >> >> > How about adding a sample hfile which contains > > > > >> >> > CellComparator$MetaCellComparator > > > > >> >> > so that the upgrade check can read and verify ? > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > >> >> We don't support downgrade. Never have. > > > > >> >> St.Ack > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > On Tue, Dec 6, 2016 at 8:50 AM, Ted Yu <yuzhih...@gmail.com> > > > > wrote: > > > > >> >> > > > > > >> >> > > The build I used yesterday didn't include HBASE-16189 > > > > >> >> > > <https://issues.apache.org/jira/browse/HBASE-16189> > > > > >> >> > > > > > > >> >> > > Once it is included, the cluster can be downgraded fine. > > > > >> >> > > > > > > >> >> > > I wonder how users would know that their existing > deployment > > > has > > > > >> >> > > HBASE-16189 <https://issues.apache.org/ > > jira/browse/HBASE-16189 > > > > > > > > >> before > > > > >> >> > > upgrading to 2.0 release. > > > > >> >> > > > > > > >> >> > > On Tue, Dec 6, 2016 at 2:29 AM, ramkrishna vasudevan < > > > > >> >> > > ramkrishna.s.vasude...@gmail.com> wrote: > > > > >> >> > > > > > > >> >> > >> @Ted > > > > >> >> > >> Does your version have this fix > > > > >> >> > >> https://issues.apache.org/jira/browse/HBASE-16189 > > > > >> >> > >> > > > > >> >> > >> Regards > > > > >> >> > >> Ram > > > > >> >> > >> > > > > >> >> > >> On Tue, Dec 6, 2016 at 3:56 PM, Ted Yu < > yuzhih...@gmail.com > > > > > > > >> wrote: > > > > >> >> > >> > > > > >> >> > >> > Is the assumption that hbase:meta would not split ? > > > > >> >> > >> > > > > > >> >> > >> > In other thread, Francis Liu was proposing supporting > > > > splittable > > > > >> >> > >> > hbase:meta in 2.0 release. > > > > >> >> > >> > > > > > >> >> > >> > Cheers > > > > >> >> > >> > > > > > >> >> > >> > > On Dec 6, 2016, at 2:20 AM, 张铎 <palomino...@gmail.com > > > > > > wrote: > > > > >> >> > >> > > > > > > >> >> > >> > > Could this be solved by hosting meta only on master? > > > > >> >> > >> > > > > > > >> >> > >> > > BTW, MetaCellComparator is introduced in HBASE-10800. > > > > >> >> > >> > > > > > > >> >> > >> > > Thanks. > > > > >> >> > >> > > > > > > >> >> > >> > > 2016-12-06 17:44 GMT+08:00 Ted Yu < > yuzhih...@gmail.com > > >: > > > > >> >> > >> > > > > > > >> >> > >> > >> Hi, > > > > >> >> > >> > >> When I restarted a cluster with 1.1 , I found that > > > > hbase:meta > > > > >> >> > region > > > > >> >> > >> > >> (written to by the previously deployed 2.0) couldn't > be > > > > >> opened: > > > > >> >> > >> > >> > > > > >> >> > >> > >> Caused by: java.io.IOException: > > > > >> >> > >> > >> org.apache.hadoop.hbase.io. > > hfile.CorruptHFileException: > > > > >> Problem > > > > >> >> > >> reading > > > > >> >> > >> > >> HFile Trailer from file hdfs://yz1.xx.com:8020/apps/ > > > > >> >> > >> hbase/data/data/ > > > > >> >> > >> > >> hbase/meta/1588230740/info/ > > > 599fc8a37311414e876803312009a9 > > > > 86 > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore. > > > > openStoreFiles( > > > > >> >> > >> HStore.java: > > > > >> >> > >> > >> 579) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore. > > > > loadStoreFiles( > > > > >> >> > >> HStore.java: > > > > >> >> > >> > >> 534) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore.<init>( > > > > >> >> > HStore.java:275) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion. > > > > instantiateHSto > > > > >> >> > >> re(HRegion. > > > > >> >> > >> > >> java:5150) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$1.call( > > > > HRegion. > > > > >> >> > >> java:912) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HRegion$1.call( > > > > HRegion. > > > > >> >> > >> java:909) > > > > >> >> > >> > >> at java.util.concurrent. > > > FutureTask.run(FutureTask. > > > > >> >> java:266) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> java.util.concurrent.Executors$RunnableAdapter.call( > > > > >> >> > >> Executors.java:511) > > > > >> >> > >> > >> at java.util.concurrent. > > > FutureTask.run(FutureTask. > > > > >> >> java:266) > > > > >> >> > >> > >> ... 3 more > > > > >> >> > >> > >> Caused by: org.apache.hadoop.hbase.io. > > > > >> >> hfile.CorruptHFileException: > > > > >> >> > >> > Problem > > > > >> >> > >> > >> reading HFile Trailer from file hdfs:// > > > > >> >> > >> > >> yz1.xx.com:8020/apps/hbase/data/data/hbase/ > > > > meta/1588230740/ > > > > >> >> > >> > >> info/599fc8a37311414e876803312009a986 > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.io. > > > hfile.HFile.pickReaderVersion( > > > > >> >> > >> > HFile.java:483) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.HFile.createReader( > > > > >> >> > HFile.java:511) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase. > regionserver.StoreFile$Reader. > > > > >> >> > >> > >> <init>(StoreFile.java:1128) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreFileInfo. > > > > >> >> > >> > >> open(StoreFileInfo.java:267) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreFile.open( > > > > StoreFil > > > > >> >> > >> e.java:409) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.StoreFile. > > > > >> >> > >> > >> createReader(StoreFile.java:517) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore. > > > > createStoreFileA > > > > >> >> > >> ndReader( > > > > >> >> > >> > >> HStore.java:687) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore.access$ > > 000( > > > > >> >> > >> HStore.java:130) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore$1.call( > > > > >> >> > HStore.java:554) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.regionserver.HStore$1.call( > > > > >> >> > HStore.java:551) > > > > >> >> > >> > >> ... 6 more > > > > >> >> > >> > >> Caused by: java.io.IOException: java.lang. > > > > >> >> ClassNotFoundException: > > > > >> >> > >> > >> org.apache.hadoop.hbase.CellComparator$ > > > MetaCellComparator > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer. > > > > getCompara > > > > >> >> > >> torClass( > > > > >> >> > >> > >> FixedFileTrailer.java:581) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer. > > > > >> >> > deserializeFromPB( > > > > >> >> > >> > >> FixedFileTrailer.java:300) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer. > > > > >> >> > >> > >> deserialize(FixedFileTrailer.java:242) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer. > > > > >> >> readFromStream( > > > > >> >> > >> > >> FixedFileTrailer.java:407) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.io. > > > hfile.HFile.pickReaderVersion( > > > > >> >> > >> > HFile.java:468) > > > > >> >> > >> > >> ... 15 more > > > > >> >> > >> > >> Caused by: java.lang.ClassNotFoundException: > > > > >> >> > >> > >> org.apache.hadoop.hbase.CellComparator$ > > > MetaCellComparator > > > > >> >> > >> > >> at java.net.URLClassLoader. > > > > >> findClass(URLClassLoader.java: > > > > >> >> > 381) > > > > >> >> > >> > >> at java.lang.ClassLoader. > > > > loadClass(ClassLoader.java: > > > > >> 424) > > > > >> >> > >> > >> at sun.misc.Launcher$ > AppClassLoader.loadClass( > > > > >> >> > Launcher.java: > > > > >> >> > >> 331) > > > > >> >> > >> > >> at java.lang.ClassLoader. > > > > loadClass(ClassLoader.java: > > > > >> 357) > > > > >> >> > >> > >> at java.lang.Class.forName0(Native Method) > > > > >> >> > >> > >> at java.lang.Class.forName(Class.java:264) > > > > >> >> > >> > >> at > > > > >> >> > >> > >> org.apache.hadoop.hbase.io.hfile.FixedFileTrailer. > > > > getCompara > > > > >> >> > >> torClass( > > > > >> >> > >> > >> FixedFileTrailer.java:579) > > > > >> >> > >> > >> > > > > >> >> > >> > >> When user does rolling upgrade from 1.1 to 2.0, the > > above > > > > may > > > > >> >> cause > > > > >> >> > >> > problem > > > > >> >> > >> > >> if hbase:meta region is updated by server running 2.0 > > but > > > > >> later > > > > >> >> > >> > assigned to > > > > >> >> > >> > >> a region server which still runs 1.1 (due to crash of > > the > > > > >> server > > > > >> >> > >> running > > > > >> >> > >> > >> 2.0, e.g.) > > > > >> >> > >> > >> > > > > >> >> > >> > >> I want to get community feedback on the severity of > > this > > > > >> issue. > > > > >> >> > >> > >> > > > > >> >> > >> > >> Thanks > > > > >> >> > >> > >> > > > > >> >> > >> > > > > > >> >> > >> > > > > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > >> >> > > > > >> > > > > > > > > > >
