Holger Stenzhorn wrote:
Hi,
Of course I yest another newbie but at least I have read the 10
minutes introduction... :-)
So I am running HBase on a local filesystem.
Attached you can find the (hopefully) necessary part of the
master-log-file. Does not look to bad, right?
Well, the -ROOT- is deployed but there is no mention of the other
necessary meta table, .META. Usually in the log you'll see a repeat of
the 'assigning region', etc. messages but instead for the first region
of the .META. table. See below for what it should look like:
...
2007-11-01 17:29:59,545 INFO org.apache.hadoop.hbase.HMaster: assigning
region -ROOT-,,0 to the only server 208.76.47.63:50814
2007-11-01 17:29:59,546 INFO org.apache.hadoop.hbase.HRegionServer:
MSG_REGION_OPEN : regionname: -ROOT-,,0, startKey: <>, tableDesc: {name:
-ROOT-, families: {info:={name: info, max versions: 1, compression:
NONE, in memory: false, max length: 2147483647, bloom filter: none}}}
2007-11-01 17:29:59,595 INFO org.apache.hadoop.hbase.HRegion: region
-ROOT-,,0 available
2007-11-01 17:30:02,548 INFO org.apache.hadoop.hbase.HMaster:
208.76.47.63:50814 serving -ROOT-,,0
2007-11-01 17:30:02,549 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 208.76.47.63:50814}
2007-11-01 17:30:02,583 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 208.76.47.63:50814} complete
2007-11-01 17:30:05,559 INFO org.apache.hadoop.hbase.HMaster: assigning
region .META.,,1 to the only server 208.76.47.63:50814
2007-11-01 17:30:05,562 INFO org.apache.hadoop.hbase.HRegionServer:
MSG_REGION_OPEN : regionname: .META.,,1, startKey: <>, tableDesc: {name:
.META., families: {info:={name: info, max versions: 1, compression:
NONE, in memory: false, max length: 2147483647, bloom filter: none}}}
2007-11-01 17:30:05,566 INFO org.apache.hadoop.hbase.HRegion: region
.META.,,1 available
2007-11-01 17:30:08,564 INFO org.apache.hadoop.hbase.HMaster:
208.76.47.63:50814 serving .META.,,1
2007-11-01 17:30:08,566 INFO org.apache.hadoop.hbase.HMaster:
regionname: .META.,,1, startKey: <>, tableDesc: {name: .META., families:
{info:={name: info, max versions: 1, compression: NONE, in memory:
false, max length: 2147483647, bloom filter: none}}} open on
208.76.47.63:50814
2007-11-01 17:30:08,566 INFO org.apache.hadoop.hbase.HMaster: updating
row .META.,,1 in table -ROOT-,,0 with startcode 8124486671143545733 and
server 208.76.47.63:50814
2007-11-01 17:30:08,571 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <.META.,,1>, server: 208.76.47.63:50814}
2007-11-01 17:30:08,574 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<.META.,,1>, server: 208.76.47.63:50814} complete
2007-11-01 17:31:02,584 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 208.76.47.63:50814}
2007-11-01 17:31:02,600 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 208.76.47.63:50814} complete
2007-11-01 17:31:08,575 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <.META.,,1>, server: 208.76.47.63:50814}
2007-11-01 17:31:08,579 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<.META.,,1>, server: 208.76.47.63:50814} complete
2007-11-01 17:31:08,579 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
....
Can you retry with DEBUG enabled to see if you see any mention of why
.META. is not being deployed? (One way to do this is to edit the
bin/hbase-daemon.sh and set HADOOP_ROOT_LOGGER to "DEBUG,DRFA" rather
than "INFO,DRFA").
BUT in the regionserver-log I get the warning:
2007-11-01 21:04:57,545 WARN org.apache.hadoop.hbase.HRegionServer:
Not starting a distinct region server because hbase.master is set to
'local' mode
Could that be the problem?
You are second person to be confused by this message. Let me fix it.
Its meant to be an informational message conveying why a distinct
regionserver running in its own jvm has not been started (It hasn't
started because the hbase.master property is set to 'local'. In this
case a single regionserver and master are put running all in a single
jvm instance).
St.Ack
Cheers
Holger
2007-11-01 21:04:54,601 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 60000: starting
2007-11-01 21:04:54,611 INFO org.apache.hadoop.hbase.HMaster: received
start message from: 192.168.18.7:4535
2007-11-01 21:04:54,631 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 60000: starting
2007-11-01 21:04:54,631 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 60000: starting
2007-11-01 21:04:54,631 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 60000: starting
2007-11-01 21:04:54,631 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 60000: starting
2007-11-01 21:04:54,631 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 60000: starting
2007-11-01 21:04:54,631 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 60000: starting
2007-11-01 21:04:54,631 INFO org.apache.hadoop.hbase.HRegionServer:
Root dir: /tmp/hadoop-holste/hbase
2007-11-01 21:04:54,731 WARN org.apache.hadoop.util.NativeCodeLoader:
Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
2007-11-01 21:04:54,731 INFO org.apache.hadoop.hbase.HLog: new log
writer created at
/tmp/hadoop-holste/hbase/log_192.168.18.7_-3681279163683101450_4535/hlog.dat.000
2007-11-01 21:04:54,741 INFO org.mortbay.http.HttpServer: Version
Jetty/5.1.4
2007-11-01 21:04:54,741 INFO org.mortbay.util.Container: Started
HttpContext[/static,/static]
2007-11-01 21:04:54,741 INFO org.mortbay.util.Container: Started
HttpContext[/logs,/logs]
2007-11-01 21:04:54,921 INFO org.mortbay.util.Container: Started
[EMAIL PROTECTED]
2007-11-01 21:04:54,921 INFO org.mortbay.util.Container: Started
WebApplicationContext[/,/]
2007-11-01 21:04:54,931 INFO org.mortbay.http.SocketListener: Started
SocketListener on 0.0.0.0:60030
2007-11-01 21:04:54,931 INFO org.mortbay.util.Container: Started
[EMAIL PROTECTED]
2007-11-01 21:04:54,931 INFO org.apache.hadoop.ipc.Server: IPC Server
listener on 4535: starting
2007-11-01 21:04:54,931 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 0 on 4535: starting
2007-11-01 21:04:54,931 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 1 on 4535: starting
2007-11-01 21:04:54,931 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 4535: starting
2007-11-01 21:04:54,931 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 4535: starting
2007-11-01 21:04:54,931 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 4535: starting
2007-11-01 21:04:54,931 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 4535: starting
2007-11-01 21:04:54,931 INFO org.apache.hadoop.hbase.HRegionServer:
HRegionServer started at: 192.168.18.7:4535
2007-11-01 21:04:54,931 INFO org.apache.hadoop.hbase.HMaster:
assigning region -ROOT-,,0 to the only server 192.168.18.7:4535
2007-11-01 21:04:54,941 INFO org.apache.hadoop.hbase.HRegionServer:
MSG_REGION_OPEN : regionname: -ROOT-,,0, startKey: <>, tableDesc:
{name: -ROOT-, families: {info:={name: info, max versions: 1,
compression: NONE, in memory: false, max length: 2147483647, bloom
filter: none}}}
2007-11-01 21:04:54,961 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 6 on 4535: starting
2007-11-01 21:04:54,961 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 4535: starting
2007-11-01 21:04:54,961 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 4535: starting
2007-11-01 21:04:54,961 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 4535: starting
2007-11-01 21:04:54,961 INFO org.apache.hadoop.hbase.HRegion: region
-ROOT-,,0 available
2007-11-01 21:04:57,945 INFO org.apache.hadoop.hbase.HMaster:
192.168.18.7:4535 serving -ROOT-,,0
2007-11-01 21:04:57,945 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 192.168.18.7:4535}
2007-11-01 21:04:57,975 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0,
startKey: <>, server: 192.168.18.7:4535} complete
2007-11-01 21:05:03,743 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-01 21:05:57,976 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 192.168.18.7:4535}
2007-11-01 21:05:57,976 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0,
startKey: <>, server: 192.168.18.7:4535} complete
2007-11-01 21:06:03,743 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-01 21:06:58,063 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 192.168.18.7:4535}
2007-11-01 21:06:58,063 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0,
startKey: <>, server: 192.168.18.7:4535} complete
2007-11-01 21:07:03,873 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
Michael Stack wrote:
Thanks for the detail Holger. Helps.
Reading it, it looks like the cluster hasn't started up properly; the
NoSuchElementException would seem to indicate that the basic startup
deploying the catalog meta tables hasn't happened or has gotten
mangled somehow. Whats in your hbase master log file? Do you see
successful deploy of the -ROOT- and .META. regions to regionserver?
Are you on local filesystem or HDFS? If latter, because you are on
windows, you need to set the hbase.root in hbase-site.xml. See
http://wiki.apache.org/lucene-hadoop/Hbase/10Minutes (If you haven't
already).
Yours,
St.Ack
Holger Stenzhorn wrote:
Hi,
I checked out Hadoop (including HBase) from its Subversion
repository today,
build it successfully (on Cygwin) and started HBase in "local" mode.
Then I took your little example program from the Wiki and it crashes
at the
last line:
HBaseConfiguration conf = new HBaseConfiguration();
HTableDescriptor desc = new HTableDescriptor("test");
desc.addFamily(new HColumnDescriptor("content:"));
HBaseAdmin admin = new HBaseAdmin(conf);
admin.createTable(desc);
...and I get the following stacktrace:
java.io.IOException: java.io.IOException:
java.util.NoSuchElementException
at java.util.TreeMap.key(TreeMap.java:1206)
at java.util.TreeMap$NavigableSubMap.lastKey(TreeMap.java:1435)
at
java.util.Collections$SynchronizedSortedMap.lastKey(Collections.java:2125)
at
org.apache.hadoop.hbase.HMaster.createTable(HMaster.java:2460)
at
org.apache.hadoop.hbase.HMaster.createTable(HMaster.java:2424)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:379)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:596)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
at
org.apache.hadoop.hbase.HBaseAdmin.createTableAsync(HBaseAdmin.java:150)
at
org.apache.hadoop.hbase.HBaseAdmin.createTable(HBaseAdmin.java:119)
at Test.main(Test.java:19)
After doing a little debugging I found the culprit line in
org.apache.hadoop.hbase.HMaster:
MetaRegion m =
(onlineMetaRegions.containsKey(newRegion.getRegionName())
?
onlineMetaRegions.get(newRegion.getRegionName()) :
onlineMetaRegions.get(onlineMetaRegions.headMap(
newRegion.getTableDesc().getName()).lastKey()));
Looking at that line and its Taking out the single statement
onlineMetaRegions.containsKey(newRegion.getRegionName()) --> false
onlineMetaRegions.get(newRegion.getRegionName()) --> null
newRegion.getTableDesc().getName() --> test
onlineMetaRegions.headMap(newRegion.getTableDesc().getName())
--> {}
onlineMetaRegions.headMap(newRegion.getTableDesc().getName()).lastKey()
--> *CRASH*
Does anybody else experience that? ...or is there even a solution
for that?
BTW: The same happens when I use the HBase shell... When I type "create
table test (test);" I get the message
"java.util.NoSuchElementException"
also.
Cheers,
Holger
