[
https://issues.apache.org/jira/browse/HBASE-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705119#action_12705119
]
stack commented on HBASE-1342:
------------------------------
Patch coming. Here is what it does meantime.
It adds a file on HRegion initialization. If already there, does nothing.
File is named .regioninfo. Presumption is that the families list is gotten
from schema, not by doing a listing of whats under the region directory (I've
added code to stop a user creating a family named .regioninfo).
File looks like this:
{code}
^...@^@^...@^@^...@^@^...@^@^...@^@^@
-ROOT-,,0...@^@^...@^@^...@^d^f-root-^a^a^@^...@^@^...@^@^...@^gis_root^@^...@^@^dtr...@^@^...@^gis_meta^@^...@^@^dtr...@^@^...@^a^g^dinfo^@^...@^@^...@^@^...@^kbloomfilter^@^...@^@^efal...@^@^...@^kcompression^@^...@^@^dno...@^@^...@^hversions^@^...@^@^b...@^@^...@^flength^@^...@^@
21474836...@^@^...@^cttl^@^...@^@^b...@^@^@ blocksi...@^@^...@^d8192^@^...@^@
in_memo...@^@^...@^efalse^@^...@^@
blockcac...@^@^...@^dtrue^@^...@^@^...@cخ^o
REGION => {NAME => '-ROOT-,,0', STARTKEY => '', ENDKEY => '', ENCODED =>
70236052, TABLE => {{NAME => '-ROOT-', IS_ROOT => 'true', IS_META => 'true',
FAMILIES => [{NAME => 'info', BLOOMFILTER => 'false', COMPRESSION => 'NONE',
VERSIONS => '10', LENGTH => '2147483647', TTL => '-1', BLOCKSIZE => '8192',
IN_MEMORY => 'false', BLOCKCACHE => 'true'}], INDEXES => []}}
{code}
Its the serialized HRI followed by text version. Two newlines separates them.
To read the binary part to reconstruct an HRI, just do something like this
(using shell):
{code}
durruti:trunk stack$ ./bin/hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Version: 0.20.0-dev, r770006, Fri May 1 10:52:11 PDT 2009
hbase(main):001:0> import org.apache.hadoop.fs.FileSystem
=> Java::OrgApacheHadoopFs::FileSystem
hbase(main):002:0> fs = FileSystem.get(@configuration)
=> #<Java::OrgApacheHadoopFs::LocalFileSystem:0x875de82
@java_object=#<Java::JavaObject:0x314dd379>>
hbase(main):003:0> import org.apache.hadoop.hbase.HRegionInfo
file:/Users/stack/checkouts/hbase/trunk/lib/jruby-complete-1.2.0.jar!/META-INF/jruby.home/lib/ruby/site_ruby/1.8/builtin/javasupport/core_ext/object.rb:34
warning: already initialized constant HRegionInfo
=> Java::OrgApacheHadoopHbase::HRegionInfo
hbase(main):004:0> hri = HRegionInfo.new
=> #<Java::OrgApacheHadoopHbase::HRegionInfo:0x12082fd2
@java_object=#<Java::JavaObject:0x45cfa4ed>>
hbase(main):006:0> import org.apache.hadoop.fs.Path
=> Java::OrgApacheHadoopFs::Path
hbase(main):007:0> p =
Path.new("/tmp/hbase-stack/hbase/-ROOT-/70236052/.regioninfo/")
=> #<Java::OrgApacheHadoopFs::Path:0x779dfb5a
@java_object=#<Java::JavaObject:0x70db5d33>>
hbase(main):009:0> fin = fs.open(p)
=> #<Java::OrgApacheHadoopFs::FSDataInputStream:0x3dbe5cb8
@java_object=#<Java::JavaObject:0x32d7970b>>
hbase(main):011:0> hri.readFields(fin)
hbase(main):012:0> fin.close()
hbase(main):014:0> p hri.toString()
"REGION => {NAME => '-ROOT-,,0', STARTKEY => '', ENDKEY => '', ENCODED =>
70236052, TABLE => {{NAME => '-ROOT-', IS_ROOT => 'true', IS_META => 'true',
FAMILIES => [{NAME => 'info', BLOOMFILTER => 'false', VERSIONS => '10',
COMPRESSION => 'NONE', LENGTH => '2147483647', TTL => '-1', BLOCKSIZE =>
'8192', IN_MEMORY => 'false', BLOCKCACHE => 'true'}], INDEXES => []}}"
{code}
That should be some help reconstructing .META.
Let me review the master logging now to make sure good stuff shows there at
info level.
> Add to filesystem info needed to rebuild .META.
> -----------------------------------------------
>
> Key: HBASE-1342
> URL: https://issues.apache.org/jira/browse/HBASE-1342
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Fix For: 0.20.0
>
>
> From Cosmin up on list:
> {code}
> Hi,
> Is there any way to rebuild the .META. Table with information in HDFS?
> Long story short, the namenode ran out of disk and crashed and we couldn't
> have it load the edits.new file at startup (the edits.new file seemed
> corrupted and it failed to read some Long value) so we removed the edits.new
> file. Apparently the edits file had 10 - 15 entries. Some edit data on
> -ROOT-, some edit data on .META., some data on a table compaction
> (comapction.dir) and some data on some hbase logs. Only last entry seem to
> be bogus, having almost a megabyte of trailing zeroes.
> After renaming edits.new HDFS started ok and HBase data is present. However
> when we started Hbase one table was missing entirely and one table was
> empty. The HBase master report shows Regions On FS: 94, but the report on
> region servers shows only 10 regions on all the region servers.
> Is there any way to cleanup hbase and rebuild the .META. table from HDFS?
> This would be ok even if it implies throwing up some data.
> If we would clean the edits.new file and restart HDFS or if we would
> manually apply the edits would that still make consistent changes to HBase
> data?
> Thanks,
> Cosmin
> {code}
> Lets log the info needed to rebuild your .META. or add things like
> HRegionInfo made creating a region as a file under each region directory.
> That don't seem hard and the info could be used rebuilding. What else would
> we need?
> Marking as blocker/critical for 0.20.0.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
