I see this in your first set of errors: 2011-09-12 10:21:39,606 INFO [main] wal.HLog(396): getNumCurrentReplicas--HDFS-826 not available;
That seems strange for a CDH -- IIRC, it should have the above. What is the context you are running the merge in? My guess is that we are finding other than hbase-site.xml with fs.default.name first when Configuration picks up config. files. Are you running for mapreduce context? (Or from a webcontainer context)? St.Ack On Mon, Oct 17, 2011 at 3:20 PM, Parmod Mehta <[email protected]> wrote: > Thanks J-D, the problem was resolved in our dev cluster but have reappeared > in our staging cluster. I added the fs.default.name in the hbase-site.xml as > I did it in the dev cluster with no luck (we running hbase-0.90.4) > > [exec] java.lang.IllegalArgumentException: Wrong FS: > file:/tmp/hbase-root/hbase/-ROOT-/70236052/.regioninfo, expected: hdfs:// > staging.server.com:54310 > [exec] at > org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410) > [exec] at > org.apache.hadoop.hdfs.DistributedFileSystem.checkPath(DistributedFileSystem.java:106) > [exec] at > org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:162) > [exec] at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:542) > [exec] at > org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748) > [exec] at > org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:413) > [exec] at > org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:342) > [exec] at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2564) > [exec] at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2550) > [exec] at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2519) > [exec] at > org.apache.hadoop.hbase.util.MetaUtils.openRootRegion(MetaUtils.java:269) > [exec] at > org.apache.hadoop.hbase.util.MetaUtils.scanRootRegion(MetaUtils.java:201) > [exec] at > org.apache.hadoop.hbase.util.Merge.mergeTwoRegions(Merge.java:204) > [exec] at org.apache.hadoop.hbase.util.Merge.run(Merge.java:110) > [exec] at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > [exec] at > com.telcontar.tools.data.util.MergeUtil.runMergeTool(MergeUtil.java:57) > [exec] at > com.telcontar.tools.data.util.MergeUtil.run(MergeUtil.java:120) > [exec] at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > [exec] at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) > [exec] at > com.telcontar.tools.data.util.MergeUtil.main(MergeUtil.java:100) > [exec] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > [exec] at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > [exec] at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > [exec] at java.lang.reflect.Method.invoke(Method.java:616) > [exec] at org.apache.hadoop.util.RunJar.main(RunJar.java:186) > [exec] 11/10/17 15:16:33 INFO wal.HLog: main.logSyncer exiting > > > > > On Mon, Sep 12, 2011 at 10:45 AM, Jean-Daniel Cryans > <[email protected]>wrote: > >> Usually it means you need to set fs.default.name in hbase-site.xml >> >> J-D >> >> On Mon, Sep 12, 2011 at 10:33 AM, Parmod Mehta <[email protected]> >> wrote: >> > Running the merge tool to merge regions on hbase-0.90.1-cdh3u0 run into >> this >> > exception trace. The first INFO level log message I guess is ok (checking >> if >> > the cluster is down). >> > >> > Not sure - why we running into this? bug? >> > >> > java.lang.IllegalArgumentException: Wrong FS: hdfs:// >> > internalhost.com:54310/hbase/-ROOT-/70236052/.regioninfo, expected: >> file:/// >> > >> > >> > 2011-09-12 10:21:38,364 INFO [main] util.Merge(82): Verifying that file >> > system is available... >> > 2011-09-12 10:21:38,370 INFO [main] util.Merge(91): Verifying that HBase >> is >> > not running... >> > 2011-09-12 10:21:38,576 INFO [main] >> > client.HConnectionManager$HConnectionImplementation(355): ZooKeeper >> > available but no active master location found >> > 2011-09-12 10:21:38,578 INFO [main] >> > client.HConnectionManager$HConnectionImplementation(372): getMaster >> attempt >> > 0 of 1 failed; no more retrying. >> > org.apache.hadoop.hbase.MasterNotRunningException >> > at >> > >> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:356) >> > at >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:90) >> > at >> > >> org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:1202) >> > at org.apache.hadoop.hbase.util.Merge.run(Merge.java:93) >> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >> > at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379) >> > 2011-09-12 10:21:38,913 INFO [main] util.Merge(199): Merging regions >> > TABLE_2,ERSx0973541,1305737576487.c0a59ee581f619456c962227073f9dd6. and >> > TABLE_2,LEx09136249,1305737576487.85f0adaeb78d362596ea8d178bda5629. in >> table >> > TABLE_2 >> > 2011-09-12 10:21:38,940 INFO [main] wal.HLog(364): HLog configuration: >> > blocksize=32 MB, rollsize=30.4 MB, enabled=true, flushlogentries=1, >> > optionallogflushinternal=1000ms >> > 2011-09-12 10:21:39,016 WARN [main] util.NativeCodeLoader(52): Unable to >> > load native-hadoop library for your platform... using builtin-java >> classes >> > where applicable >> > 2011-09-12 10:21:39,602 INFO [main] wal.SequenceFileLogWriter(126): >> Using >> > syncFs -- HDFS-200 >> > 2011-09-12 10:21:39,605 INFO [main] wal.HLog(510): New hlog >> > /home/pmehta/.logs_1315848098931/hlog.1315848098940 >> > 2011-09-12 10:21:39,606 INFO [main] wal.HLog(396): >> > getNumCurrentReplicas--HDFS-826 not available; >> > >> hdfs_out=org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer@3a289d2e >> , >> > >> exception=org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.getNumCurrentReplicas() >> > 2011-09-12 10:21:39,631 FATAL [main] util.Merge(114): Merge failed >> > java.lang.IllegalArgumentException: Wrong FS: hdfs:// >> > internalhost.com:54310/hbase/-ROOT-/70236052/.regioninfo, expected: >> file:/// >> > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410) >> > at >> > >> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56) >> > at >> > >> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:379) >> > at >> > >> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) >> > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748) >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:408) >> > at >> > org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:337) >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2546) >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2532) >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2501) >> > at >> > org.apache.hadoop.hbase.util.MetaUtils.openRootRegion(MetaUtils.java:269) >> > at >> > org.apache.hadoop.hbase.util.MetaUtils.scanRootRegion(MetaUtils.java:201) >> > at >> > org.apache.hadoop.hbase.util.Merge.mergeTwoRegions(Merge.java:204) >> > at org.apache.hadoop.hbase.util.Merge.run(Merge.java:110) >> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >> > at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379) >> > 2011-09-12 10:21:39,634 INFO [main.logSyncer] wal.HLog$LogSyncer(973): >> > main.logSyncer exiting >> > 2011-09-12 10:21:39,640 ERROR [main] util.Merge(381): exiting due to >> error >> > java.lang.IllegalArgumentException: Wrong FS: hdfs:// >> > internalhost.com:54310/hbase/.META./1028785192/.regioninfo, expected: >> > file:/// >> > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410) >> > at >> > >> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56) >> > at >> > >> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:379) >> > at >> > >> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) >> > at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:748) >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegion.checkRegioninfoOnFilesystem(HRegion.java:408) >> > at >> > org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:337) >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2546) >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2532) >> > at >> > >> org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:2501) >> > at >> > org.apache.hadoop.hbase.util.MetaUtils.openMetaRegion(MetaUtils.java:276) >> > at >> > org.apache.hadoop.hbase.util.MetaUtils.scanMetaRegion(MetaUtils.java:261) >> > at org.apache.hadoop.hbase.util.Merge.run(Merge.java:115) >> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >> > at org.apache.hadoop.hbase.util.Merge.main(Merge.java:379) >> > >> > >> > thanks >> > >> > > > > -- > > - Parmod Mehta >
