Vladimir, Isn't it the point of "hbase.wal.dir" to have WAL in a different directory or even a different file system? [1] Also, both HBase WAL and IgniteHadoopFileSystem use 'org.apache.hadoop.fs.FileSystem'. I am just curious how it breaks HBase.
[1] https://issues.apache.org/jira/browse/HBASE-17437[2] https://github.com/apache/hbase/blob/c38bf12444aca77c7cb12637147c07dc711acbe9/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java -- Roman On Thursday, June 8, 2017 5:00 PM, Vladimir Ozerov <[email protected]> wrote: Hi Takashi, "igfs://" prefix should be used in your application code, in those places where data is accessed. It is illegal to change "hbase.wal.dir" property, as it breaks HBase internals. On Thu, Jun 8, 2017 at 6:30 AM, Takashi Sasaki <[email protected]> wrote: Hello, I used igfs:// instead of hdfs:// for hbase.wal.dir property, then HBase Master Server throwed Exception. 2017-06-08 02:51:56,745 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase. master.HMaster. at org.apache.hadoop.hbase. master.HMaster. constructMaster(HMaster.java: 2577) at org.apache.hadoop.hbase. master.HMasterCommandLine. startMaster( HMasterCommandLine.java:231) at org.apache.hadoop.hbase. master.HMasterCommandLine.run( HMasterCommandLine.java:137) at org.apache.hadoop.util. ToolRunner.run(ToolRunner. java:70) at org.apache.hadoop.hbase.util. ServerCommandLine.doMain( ServerCommandLine.java:126) at org.apache.hadoop.hbase. master.HMaster.main(HMaster. java:2587) Caused by: java.io.IOException: File system is already initialized: org.apache.ignite.internal. processors.hadoop.impl.igfs. HadoopIgfsWrapper@1dbd580 at org.apache.ignite.hadoop.fs. v1.IgniteHadoopFileSystem. initialize( IgniteHadoopFileSystem.java: 215) at org.apache.hadoop.hbase.fs. HFileSystem.<init>( HFileSystem.java:87) at org.apache.hadoop.hbase. regionserver.HRegionServer. initializeFileSystem( HRegionServer.java:634) at org.apache.hadoop.hbase. regionserver.HRegionServer.< init>(HRegionServer.java:576) at org.apache.hadoop.hbase. master.HMaster.<init>(HMaster. java:397) at sun.reflect. NativeConstructorAccessorImpl. newInstance0(Native Method) I checked Ignite source code. It seems to be called initialize method more than once, so the server throw the exception. I added properties to core-site.xml: <property> <name>fs.igfs.impl</name> <value>org.apache.ignite. hadoop.fs.v1. IgniteHadoopFileSystem</value> </property> <property> <name>fs.AbstractFileSystem. igfs.impl</name> <value>org.apache.ignite. hadoop.fs.v2. IgniteHadoopFileSystem</value> </property> I changed property hbase-site.xml: <property> <name>hbase.wal.dir</name> <value>igfs://igfs@/user/ hbase/WAL</value> </property> Hadoop version: 2.7.3 HBase version: 1.3.0 Ignite version: 2.0.0 How can I solve this problem? Give me advice if you like. Thanks, Takashi 2017-06-07 21:38 GMT+09:00 Takashi Sasaki <[email protected]>: > Hello, > > I'm newbie of Ignite, so have some question. > > When I use Secondary File System to write HBase WAL, should I use > igfs:// instead of hdfs:// ? > > hbase-site.xml(default) is hdfs://. > > ---------- > <property> > <name>hbase.wal.dir</name> > <value>hdfs://[dnsname]:[port] /user/hbase/WAL</value> > </property> > ---------- > > Does the secondary file system require some configuration changes to Hbase? > > Please give me advice. > > Thanks, > Takashi
