Hi Mich, I'm glad you're able to make progress. It sounds like your native libraries are not installed or activated correctly. It's possible they're not drop-in compatible between Hadoop2 and Hadoop3, in which case recompilation might be necessary. I'm not up-to-current on the state of those affairs (though potentially someone else reading here is...)
Have you looked through this section of our book? http://hbase.apache.org/book.html#hadoop.native.lib Bonne chance, Nick On Thu, Feb 20, 2020 at 11:45 AM Mich Talebzadeh <[email protected]> wrote: > Thanks Nick, > > Looking at this matrix <http://hbase.apache.org/book.html#hadoop>, I > upgraded Hadoop from 3.1 to 3.1.1. This should allow me to used Hbase 2.1.8 > > So I did both. One needs to change few hbase-site.xml setting etc to make > it work. Also the addition of htrace-core-3.1.0-incubating.jar to > $HBASE_HOME/lib. > > So both the HMaster and HRegionServer are running on master node > and HRegionServer on another node. > > The drawback is that one cannot use snappy compression in create table > command, something like below > > create 'trading:MARKETDATAHBASESPEED', {NAME=> 'PRICE_INFO', COMPRESSION > => 'SNAPPY'} > > This is the error I get on HRegionServer log > > org.apache.hadoop.hbase.DoNotRetryIOException: Compression algorithm > 'snappy' previously failed test. > > Eventually table is sort of created but mot complete. Also getting rid of > this table becomes problematic > > hbase(main):002:0> drop 'trading:MARKETDATAHBASESPEED' > ERROR: The procedure 31 is still running > For usage try 'help "drop"' > Took 668.3693 seconds > hbase(main):003:0> drop 'trading:MARKETDATAHBASESPEED' > ERROR: The procedure 32 is still running > For usage try 'help "drop"' > Took 668.2910 seconds > > > So the table cannot be dropped. > > All fun and games > > Cheers, > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > < > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > >* > > > > http://talebzadehmich.wordpress.com > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Thu, 20 Feb 2020 at 18:54, Nick Dimiduk <[email protected]> wrote: > > > changed subject, dropped dev > > > > Hello Mich, > > > > The HBase binary distribution includes all Hadoop client jars necessary > for > > HBase to function on top of HDFS. The version of those Hadoop jars is > that > > of Hadoop 2.8.5. Duo is saying that the Hadoop 2.8.5 client works > against a > > HDFS 3.1.x cluster. Thus, this binary release of HBase is expected to > work > > with HDFS 3.1. > > > > Alternatively, you can choose to recompile HBase using Hadoop 3.1.x as > the > > dependency. In that case, you'll need to retrieve the source, establish a > > build environment, and perform the build yourself -- the HBase project > does > > not distribute a binary artifact compiled against Hadoop 3. > > > > Thanks, > > Nick > > > > On Thu, Feb 20, 2020 at 3:20 AM Mich Talebzadeh < > [email protected] > > > > > wrote: > > > > > Thank you. > > > > > > In your points below > > > > > > If you want to run HBase on top of a 3.1.x HDFS cluster, you can just > use > > > the binaries, hadoop 2.8.5 client can communicate with 3.1.x server. > > > > > > If you want HBase to use hadoop 3.1.x, then you need to build the > > binaries > > > by yourself, use the hadoop 3 profile. > > > > > > Could you please clarify (since Hbase can work with Hadoop), which > > version > > > of new release has been tested against which version of Hadoop. > > > > > > Also it is not very clear which binaries you are referring to? "Hadoop > > > 2.8.5 client can communicate with "Hadoop" 3.1 server. So basically > > > download Hadoop 2.8.5 client for Hbase use? > > > > > > > > > Regards, > > > > > > > > > > > > Dr Mich Talebzadeh > > > > > > > > > > > > LinkedIn * > > > > > > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > > < > > > > > > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > > >* > > > > > > > > > > > > http://talebzadehmich.wordpress.com > > > > > > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for > any > > > loss, damage or destruction of data or any other property which may > arise > > > from relying on this email's technical content is explicitly > disclaimed. > > > The author will in no case be liable for any monetary damages arising > > from > > > such loss, damage or destruction. > > > > > > > > > > > > > > > On Thu, 20 Feb 2020 at 09:45, 张铎(Duo Zhang) <[email protected]> > > wrote: > > > > > > > If you want to run HBase on top of a 3.1.x HDFS cluster, you can just > > use > > > > the binaries, hadoop 2.8.5 client can communicate with 3.1.x server. > > > > > > > > If you want HBase to use hadoop 3.1.x, then you need to build the > > > binaries > > > > by yourself, use the hadoop 3 profile. > > > > > > > > Mich Talebzadeh <[email protected]> 于2020年2月20日周四 下午5:15写道： > > > > > > > > > Hi, > > > > > > > > > > Thanks. > > > > > > > > > > Does this version of Hbase work with Hadoop 3.1? I am still stuck > > with > > > > > Hbase 1.2.7 > > > > > > > > > > Hadoop 3.1.0 > > > > > Source code repository https://github.com/apache/hadoop -r > > > > > 16b70619a24cdcf5d3b0fcf4b58ca77238ccbe6d > > > > > Compiled by centos on 2018-03-30T00:00Z > > > > > Compiled with protoc 2.5.0 > > > > > > > > > > Regards, > > > > > > > > > > Dr Mich Talebzadeh > > > > > > > > > > > > > > > > > > > > LinkedIn * > > > > > > > > > > > > > > > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > > > > < > > > > > > > > > > > > > > > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > > > > >* > > > > > > > > > > > > > > > > > > > > http://talebzadehmich.wordpress.com > > > > > > > > > > > > > > > *Disclaimer:* Use it at your own risk. Any and all responsibility > for > > > any > > > > > loss, damage or destruction of data or any other property which may > > > arise > > > > > from relying on this email's technical content is explicitly > > > disclaimed. > > > > > The author will in no case be liable for any monetary damages > arising > > > > from > > > > > such loss, damage or destruction. > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, 20 Feb 2020 at 08:48, Duo Zhang <[email protected]> > wrote: > > > > > > > > > > > The HBase team is happy to announce the immediate availability of > > > > Apache > > > > > > HBase 2.1.9. > > > > > > > > > > > > Download from https://hbase.apache.org/downloads.html > > > > > > > > > > > > Apache HBase is an open-source, distributed, versioned, > > > non-relational > > > > > > database. Apache HBase gives you low latency random access to > > > billions > > > > of > > > > > > rows with millions of columns atop non-specialized hardware. To > > learn > > > > > more > > > > > > about HBase, see https://hbase.apache.org/. > > > > > > > > > > > > HBase 2.1.9 is the latest release of the HBase 2.1 line, > continuing > > > on > > > > > the > > > > > > theme of bringing a stable, reliable database to the Apache Big > > Data > > > > > > ecosystem and beyond. 2.1.9 includes ~62 bug and improvement > fixes > > > done > > > > > > since the 2.1.8. > > > > > > > > > > > > For instructions on verifying ASF release downloads, please see > > > > > > > > > > > > https://www.apache.org/dyn/closer.cgi#verify > > > > > > > > > > > > Project member signature keys can be found at > > > > > > > > > > > > https://www.apache.org/dist/hbase/KEYS > > > > > > > > > > > > Thanks to all the contributors who made this release possible! > > > > > > > > > > > > Best, > > > > > > The HBase Dev Team > > > > > > > > > > > > > > > > > > > > >
