There are an awful lot of storefiles for ROOT and META, and not many
tables or regions at all.
I strongly discourage using HBase atop S3 directly. You could try
getting HDFS running on S3, and then put HBase on that. You might have
more luck, though I still don't recommend S3 at all w/ HBase.
Ananth T. Sarathy wrote:
Pastebinned...
http://pastebin.com/m53ccdbb7
Ananth T Sarathy
On Mon, Oct 26, 2009 at 1:58 PM, Ananth T. Sarathy <
[email protected]> wrote:
Here is the output...
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/-ROOT-
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/-ROOT-/70236052
-rwxrwxrwx 1 622 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/.regioninfo
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/-ROOT-/70236052/info
-rwxrwxrwx 1 841 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/1996912273062753577
-rwxrwxrwx 1 461 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/2193779228149112035
-rwxrwxrwx 1 462 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/2351329094215113700
-rwxrwxrwx 1 461 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/246199753790950739
-rwxrwxrwx 1 461 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/2525006410357531294
-rwxrwxrwx 1 462 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/2791558595254923243
-rwxrwxrwx 1 461 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/4021322725584207875
-rwxrwxrwx 1 461 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/4849111854392651007
-rwxrwxrwx 1 462 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/5363712926256617749
-rwxrwxrwx 1 461 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/5611302436720701967
-rwxrwxrwx 1 461 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/6831054531189082272
-rwxrwxrwx 1 462 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/68808045018098702
-rwxrwxrwx 1 462 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/7253796415415350972
-rwxrwxrwx 1 461 1969-12-31 19:00
/hbasedata/-ROOT-/70236052/info/7757549506442648999
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/.META.
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/.META./1028785192
-rwxrwxrwx 1 932 1969-12-31 19:00
/hbasedata/.META./1028785192/.regioninfo
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/.META./1028785192/historian
-rwxrwxrwx 1 911 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/1968569179310304585
-rwxrwxrwx 1 907 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/2158222582121734940
-rwxrwxrwx 1 907 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/2646558927756874624
-rwxrwxrwx 1 907 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/3553325538317585634
-rwxrwxrwx 1 2037 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/4092877814999091772
-rwxrwxrwx 1 1185 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/4380516119952135238
-rwxrwxrwx 1 2437 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/6276910425096618885
-rwxrwxrwx 1 911 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/7295248711127991783
-rwxrwxrwx 1 911 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/7481877460145101553
-rwxrwxrwx 1 911 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/752086821053742201
-rwxrwxrwx 1 907 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/7817013763537032897
-rwxrwxrwx 1 1173 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/8114421049620083173
-rwxrwxrwx 1 1563 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/8322527460340654381
-rwxrwxrwx 1 911 1969-12-31 19:00
/hbasedata/.META./1028785192/historian/8999997006964561665
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/.META./1028785192/info
-rwxrwxrwx 1 694 1969-12-31 19:00
/hbasedata/.META./1028785192/info/1512108120879032405
-rwxrwxrwx 1 3971 1969-12-31 19:00
/hbasedata/.META./1028785192/info/2775950188169392903
-rwxrwxrwx 1 646 1969-12-31 19:00
/hbasedata/.META./1028785192/info/3098482905740093325
-rwxrwxrwx 1 696 1969-12-31 19:00
/hbasedata/.META./1028785192/info/3098738884562255978
-rwxrwxrwx 1 1561 1969-12-31 19:00
/hbasedata/.META./1028785192/info/383466394636378400
-rwxrwxrwx 1 694 1969-12-31 19:00
/hbasedata/.META./1028785192/info/4974231550598739590
-rwxrwxrwx 1 694 1969-12-31 19:00
/hbasedata/.META./1028785192/info/5034205868382579974
-rwxrwxrwx 1 694 1969-12-31 19:00
/hbasedata/.META./1028785192/info/5137021646723327707
-rwxrwxrwx 1 696 1969-12-31 19:00
/hbasedata/.META./1028785192/info/5626349865156916725
-rwxrwxrwx 1 696 1969-12-31 19:00
/hbasedata/.META./1028785192/info/6707659272909568633
-rwxrwxrwx 1 696 1969-12-31 19:00
/hbasedata/.META./1028785192/info/7346625667936340286
-rwxrwxrwx 1 696 1969-12-31 19:00
/hbasedata/.META./1028785192/info/7438651887567350087
-rwxrwxrwx 1 5699 1969-12-31 19:00
/hbasedata/.META./1028785192/info/7671471165225263042
-rwxrwxrwx 1 1202 1969-12-31 19:00
/hbasedata/.META./1028785192/info/8586093328550564694
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/.META./compaction.dir
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/.logs
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/.logs/ip-10-242-71-191.ec2.internal,60020,1256571494907
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/.logs/ip-10-244-9-171.ec2.internal,60020,1255635341245
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/.logs/ip-10-244-9-171.ec2.internal,60020,1256571493935
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/.logs/ip-10-245-82-160.ec2.internal,58439,1255623467258
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/.logs/ip-10-245-82-160.ec2.internal,60020,1255977369779
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/.logs/ip-10-245-82-160.ec2.internal,60020,1256571494768
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/GS_Applications
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711
-rwxrwxrwx 1 2632 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/.regioninfo
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Application
-rwxrwxrwx 1 13303446 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Application/2124252833220218245
-rwxrwxrwx 1 26149 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Application/5509287308900349804
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/BinaryData
-rwxrwxrwx 1 267219 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/BinaryData/2009456488953384412
-rwxrwxrwx 1 66582 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/BinaryData/4704251150705174118
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/BinaryRetrieval
-rwxrwxrwx 1 1325402 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/BinaryRetrieval/1389090912532859839
-rwxrwxrwx 1 3093 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/BinaryRetrieval/9178892710156256966
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Files
-rwxrwxrwx 1 69548974 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Files/6175860477346559387
-rwxrwxrwx 1 157418897 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Files/7670651200769000729
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Info
-rwxrwxrwx 1 71844 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Info/1715357090325080119
-rwxrwxrwx 1 36052800 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Info/7204941744454360171
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Network
-rwxrwxrwx 1 42060118 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/Network/3039242992290063476
-rwxrwxrwx 1 148161186 1969-12-31 19:00
/hbasedata/GS_Applications/1264304711/oldlogfile.log
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1732076772
-rwxrwxrwx 1 2484 1969-12-31 19:00
/hbasedata/GS_Applications/1732076772/.regioninfo
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139
-rwxrwxrwx 1 2780 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/.regioninfo
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Application
-rwxrwxrwx 1 27428283 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Application/7122413424433440685
-rwxrwxrwx 1 47203 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Application/8284859005131013460
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/BinaryData
-rwxrwxrwx 1 311872 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/BinaryData/8301036036223997974
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/BinaryRetrieval
-rwxrwxrwx 1 6368 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/BinaryRetrieval/1610935246237259262
-rwxrwxrwx 1 2738315 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/BinaryRetrieval/8651784529191787005
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Files
-rwxrwxrwx 1 127201108 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Files/7247655344177400794
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Info
-rwxrwxrwx 1 74860411 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Info/514906809998228574
-rwxrwxrwx 1 162245 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Info/8300037259597514515
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Network
-rwxrwxrwx 1 228 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/Network/2748465749080635338
-rwxrwxrwx 1 32857683 1969-12-31 19:00
/hbasedata/GS_Applications/1888816139/oldlogfile.log
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_Applications/compaction.dir
drwxrwxrwx - 0 1969-12-31 19:00 /hbasedata/GS_EventLists
drwxrwxrwx - 0 1969-12-31 19:00
/hbasedata/GS_EventLists/453741364
-rwxrwxrwx 1 1191 1969-12-31 19:00
/hbasedata/GS_EventLists/453741364/.regioninfo
-rwxrwxrwx 1 3 1969-12-31 19:00 /hbasedata/hbase.version
Ananth T Sarathy
On Mon, Oct 26, 2009 at 1:56 PM, stack <[email protected]> wrote:
If you open the .regioninfo file inside in each region, you can see its
content? You might be able to make loadtable.rb work by making it go via
jets3s but would take some tinkering on your part.
As to why the regions were lost from .META. in first place, I'm not sure
why; was there no flush of .META.? Usually this is frequent so you
should
have some of your table info.
We're about hdfs here in this channel, usually, and speaking for myself,
I've played with hadoop on s3 but not hbase so I'm not much good to you
I'm
afraid.
St.Ack
On Mon, Oct 26, 2009 at 10:42 AM, Ananth T. Sarathy <
[email protected]> wrote:
yeah I see a folder with my table name. and inside that I see 3 other
folders named with a bunch of numbers (which i assume are the regions).
Ananth T Sarathy
On Mon, Oct 26, 2009 at 1:38 PM, stack <[email protected]> wrote:
If you list the hbase homedir in s3, do you see tables and then under
that
regions?
The loadtable.rb script won't work against a fs that is other than
hdfs
unfortunately.
St.Ack
On Mon, Oct 26, 2009 at 10:36 AM, Ananth T. Sarathy <
[email protected]> wrote:
well where i have hadoop installed (not running) since not using it
for
hdfs
bin/hadoop fs -lsr /hbase.rootdir
lsr: Cannot access /hbase.rootdir: No such file or directory.
Ananth T Sarathy
On Mon, Oct 26, 2009 at 1:24 PM, stack <[email protected]> wrote:
When you do bin/hadoop fs -lsr /hbase.rootdir what happens?
St.Ack
On Mon, Oct 26, 2009 at 10:05 AM, Ananth T. Sarathy <
[email protected]> wrote:
@ the command line? I get a command not found.
Ananth T Sarathy
On Mon, Oct 26, 2009 at 1:02 PM, stack <[email protected]>
wrote:
Its in head of the 0.20 branch (We should roll a 0.20.2 soon).
What happens if you do a lsr /hbase?
St.Ack
On Mon, Oct 26, 2009 at 9:52 AM, Ananth T. Sarathy <
[email protected]> wrote:
I don't have a loadtable.rb
is that in 0.20.0?
This is what i have in bin
Formatter.rb hbase-config.sh regionservers.sh
zookeepers.sh
HBase.rb hbase-daemon.sh rename_table.rb
copy_table.rb hbase-daemons.sh start-hbase.sh
hbase hirb.rb stop-hbase.sh
Ananth T Sarathy
On Mon, Oct 26, 2009 at 12:45 PM, stack <[email protected]>
wrote:
Your log would seem to say that there are no tables in
hbase:
2009-10-26 11:40:13,984 INFO
org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scan of 0 row(s) of meta region
{server:
10.245.82.160:60020, regionname: .META.,,1, startKey: <>}
complete
Do as Jon suggests. Do you see listing of regions? If
so,
it
would
seem
that edits to .META. table are not persisting on your s3
hdfs.
You
might
be able to add in all tables using the bin/loadtable.rb
script;
it
reads
the
.regioninfo files in all regions and per region adds to
.META.
an
entry.
St.Ack
On Mon, Oct 26, 2009 at 9:31 AM, Ananth T. Sarathy <
[email protected]> wrote:
I am confused , why would I need a hadoop home if I am
using
s3
and
the
jets3t package to write to s3?
Ananth T Sarathy
On Mon, Oct 26, 2009 at 12:25 PM, Jonathan Gray <
[email protected]
wrote:
Not S3, HDFS. Can you checkout the web ui or using
the
command-line
interface?
$HADOOP_HOME/bin/hadoop dfs -lsr /hbase
...would be a good start
Ananth T. Sarathy wrote:
i see all my blocks in my s3 bucket.
Ananth T Sarathy
On Mon, Oct 26, 2009 at 12:17 PM, Jonathan Gray <
[email protected]>
wrote:
Do you see the files/blocks in HDFS?
Ananth T. Sarathy wrote:
I just restarted Hbase and when I go into the shell
and
type
list,
none
of
my tables are listed, but I see all the data/blocks
in
s3.
here is the master log when it's restarted
http://pastebin.com/m1ebb7217
this happened once before, but we just started over
since
it
was
early
in
our process. This time we have a lot of data, and
need
to
keep
it.
Ananth T Sarathy
