I've looked really quick and saw the repaired metas on hdfs so I'm fairly sure I did the rebuild on hdfs. If you have your HBASE_CONF_DIR and HADOOP_CONF_DIR set to point to what your hbase instance uses, you probably don't even need to specify the -base dir option.
Actually, I think the fs.defaultFS is basically dead code and a red herring -- I think an updated version of the tool would be "Tool-ified" so that you can pass in -Dfs.defaultname=hdfs:// or other hadoop/hbase xml settings in at the command line. Jon. On Thu, Jan 5, 2012 at 4:57 PM, Ted Yu <[email protected]> wrote: > Jonathan made a good point. > I tried 'bin/hbase classpath' on the cluster and didn't see hdfs-site.xml > in the output. > > I think we should relax the requirement that hdfs-site.xml is in the > classpath of hbck. > > Cheers > > On Thu, Jan 5, 2012 at 4:53 PM, Jonathan Hsieh <[email protected]> wrote: > > > Actually, fs.default.name should be pulled from your hdfs-site.xml file > -- > > if you do a hbase classpath, can you tell which hdfs-site.xml file is > being > > pulled in? > > > > Jon. > > > > On Thu, Jan 5, 2012 at 4:26 PM, Jonathan Hsieh <[email protected]> wrote: > > > > > Hey Vlad, > > > > > > I wrote the tool -- and I've used it to repair a fairly messed up META > > > table. I must of used on a local filesystem copy of META (just got all > > the > > > .regioninfo files in their directory paths), and then shipped the > > repaired > > > version of the .META. dir to the customer. > > > > > > This is definitely a bug. FIle the jira and I'll try to fix in the > next > > > few days. > > > > > > Jon. > > > > > > > > > On Thu, Jan 5, 2012 at 4:16 PM, Vladimir Rodionov < > > [email protected] > > > > wrote: > > > > > >> Ted, > > >> > > >> "fs.default.name" is a standard config property name which is > described > > >> here: > > >> http://hadoop.apache.org/common/docs/current/core-default.html > > >> > > >> It is not CDH -specific. If you are right than this tool has never > been > > >> tested. > > >> > > >> Best regards, > > >> Vladimir Rodionov > > >> Principal Platform Engineer > > >> Carrier IQ, www.carrieriq.com > > >> e-mail: [email protected] > > >> > > >> ________________________________________ > > >> From: Ted Yu [[email protected]] > > >> Sent: Thursday, January 05, 2012 4:06 PM > > >> To: [email protected] > > >> Subject: Re: OfflineMetaRepair? > > >> > > >> Vlad: > > >> In the future, please drop unrelated discussion from bottom of your > > email. > > >> > > >> I think what you saw was caused by FS default name not being set > > >> correctly. > > >> In hbck: > > >> conf.set("fs.defaultFS", conf.get(HConstants.HBASE_DIR)); > > >> But cdh3 uses: > > >> conf.set("fs.default.name", "hdfs://localhost:0"); > > >> ./src/test/org/apache/hadoop/hdfs/server/namenode/TestCheckpoint.java > > >> > > >> You can try adding the following line after line 77 of > > >> OfflineMetaRepair.java: > > >> conf.set("fs.default.name", path); > > >> and rebuilding hbase 0.90.6 (tip of 0.92 branch) > > >> > > >> If the above works, please file a JIRA. > > >> > > >> Thanks > > >> > > >> On Thu, Jan 5, 2012 at 3:30 PM, Vladimir Rodionov > > >> <[email protected]>wrote: > > >> > > >> > 0.90.5 > > >> > > > >> > I am trying to repair .META. table using this tool > > >> > > > >> > 1. HBase cluster was shutdown > > >> > > > >> > Then I ran: > > >> > > > >> > 2. [name01 bin]$ hbase > > >> org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair > > >> > -base hdfs://us01-ciqps1-name01.carrieriq.com:9000/hbase -details > > >> > > > >> > > > >> > This is waht I got: > > >> > > > >> > 12/01/05 23:23:15 INFO util.HBaseFsck: Loading HBase regioninfo from > > >> > HDFS... > > >> > 12/01/05 23:23:30 ERROR util.HBaseFsck: Bailed out due to: > > >> > java.lang.IllegalArgumentException: Wrong FS: hdfs:// > > >> > > > >> > > > us01-ciqps1-name01.carrieriq.com:9000/hbase/M2M-INTEGRATION-MM_TION-1325190318714/0003d2ede27668737e192d8430dbe5d0/.regioninfo > > >> , > > >> > expected: file:/// > > >> > at > > org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:352) > > >> > at > > >> > > > >> > > > org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47) > > >> > at > > >> > > > >> > > > org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:368) > > >> > at > > >> > > > >> > > > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251) > > >> > at > > >> > > > >> > > > org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:126) > > >> > at > > >> > > > >> > > org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:284) > > >> > at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:398) > > >> > at > > >> > > > org.apache.hadoop.hbase.util.HBaseFsck.loadMetaEntry(HBaseFsck.java:256) > > >> > at > > >> > > > org.apache.hadoop.hbase.util.HBaseFsck.loadTableInfo(HBaseFsck.java:284) > > >> > at > > >> > > org.apache.hadoop.hbase.util.HBaseFsck.rebuildMeta(HBaseFsck.java:402) > > >> > at > > >> > > > >> > > > org.apache.hadoop.hbase.util.hbck.OfflineMetaRepair.main(OfflineMetaRepair.java:90) > > >> > > > >> > > > >> > Q: What am I doing wrong? > > >> > > > >> > Best regards, > > >> > Vladimir Rodionov > > >> > Principal Platform Engineer > > >> > Carrier IQ, www.carrieriq.com > > >> > e-mail: [email protected] > > >> > > > >> > > > >> > > >> Confidentiality Notice: The information contained in this message, > > >> including any attachments hereto, may be confidential and is intended > > to be > > >> read only by the individual or entity to whom this message is > > addressed. If > > >> the reader of this message is not the intended recipient or an agent > or > > >> designee of the intended recipient, please note that any review, use, > > >> disclosure or distribution of this message or its attachments, in any > > form, > > >> is strictly prohibited. If you have received this message in error, > > please > > >> immediately notify the sender and/or [email protected] and > > >> delete or destroy any copy of this message and its attachments. > > >> > > > > > > > > > > > > -- > > > // Jonathan Hsieh (shay) > > > // Software Engineer, Cloudera > > > // [email protected] > > > > > > > > > > > > > > > -- > > // Jonathan Hsieh (shay) > > // Software Engineer, Cloudera > > // [email protected] > > > -- // Jonathan Hsieh (shay) // Software Engineer, Cloudera // [email protected]
