Since I am just a beginner at Hadoop and Hbase I cannot really tell
whether an exception is a "real" one or just a "hint" - but exceptions
always look scary... :-)
Anyways, when I did cut the allowed heap for the server and the test
class both down to 2GB. In this case I just get following (not too
optimistic) results...
Well, I post all the log I think that might be necessary since I cannot
say which exception is important and which one not.
...and again, thanks a lot for your help: It is highly appreciated!
Cheers,
Holger
test case log: (nothing (!) deleted between 00:04:14 and 00:09:01)
-------------
07/11/09 00:03:31 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:03:34 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:03:35 INFO mapred.JobClient: map 100% reduce 83%
07/11/09 00:03:37 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:03:59 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:04:02 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:04:05 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:04:08 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:04:11 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:04:14 INFO mapred.LocalJobRunner: reduce > reduce
07/11/09 00:09:01 WARN mapred.LocalJobRunner: job_local_1
java.net.SocketTimeoutException: timed out waiting for rpc response
at org.apache.hadoop.ipc.Client.call(Client.java:484)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:184)
at $Proxy1.batchUpdate(Unknown Source)
at org.apache.hadoop.hbase.HTable.commit(HTable.java:724)
at org.apache.hadoop.hbase.HTable.commit(HTable.java:701)
at
org.apache.hadoop.hbase.mapred.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:89)
at
org.apache.hadoop.hbase.mapred.TableOutputFormat$TableRecordWriter.write(TableOutputFormat.java:63)
at
org.apache.hadoop.mapred.ReduceTask$2.collect(ReduceTask.java:308)
at TriplesTest$TestReducer.reduce(TriplesTest.java:77)
at TriplesTest$TestReducer.reduce(TriplesTest.java:51)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:326)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:164)
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:831)
at TriplesTest.run(TriplesTest.java:156)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at TriplesTest.main(TriplesTest.java:162)
master log:
-----------
2007-11-09 00:04:10,948 INFO org.apache.hadoop.hbase.HMaster: updating
row
triples,http://dblp.l3s.de/d2r/resource/publications/conf/iccd/HeydariP01a,1194566644908
in table .META.,,1 with startcode -7685273460109660997 and server
127.0.1.1:50446
2007-11-09 00:04:13,898 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 50446, call
batchUpdate(triples,http://dblp.l3s.de/d2r/resource/publications/conf/dac/SharmaJ93,1194566613933,
9223372036854775807, [EMAIL PROTECTED])
from 127.0.1.1:45771: error:
org.apache.hadoop.hbase.NotServingRegionException:
triples,http://dblp.l3s.de/d2r/resource/publications/conf/dac/SharmaJ93,1194566613933
org.apache.hadoop.hbase.NotServingRegionException:
triples,http://dblp.l3s.de/d2r/resource/publications/conf/dac/SharmaJ93,1194566613933
at
org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1325)
at
org.apache.hadoop.hbase.HRegionServer.getRegion(HRegionServer.java:1297)
at
org.apache.hadoop.hbase.HRegionServer.startUpdate(HRegionServer.java:1238)
at
org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:1113)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:379)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:596)
2007-11-09 00:04:15,296 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:04:15,302 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:04:21,136 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:04:21,157 INFO org.apache.hadoop.hbase.HMaster: Deleting
region triples,,1194566426117 because daughter splits no longer hold
references
2007-11-09 00:04:21,234 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:04:21,234 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-09 00:05:15,297 INFO org.apache.hadoop.hbase.HRegion: Optional
flush called 10 times when data present without flushing. Forcing one.
2007-11-09 00:05:15,317 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:05:15,434 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:05:21,157 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:05:21,276 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:05:21,276 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-09 00:06:15,338 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:06:15,395 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:06:21,185 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:06:21,197 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:06:21,197 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-09 00:07:15,358 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:07:15,470 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:07:21,209 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:07:21,292 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:07:21,292 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-09 00:08:15,379 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:08:15,495 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:08:21,229 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:08:21,292 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:08:21,292 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-09 00:09:15,400 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:09:15,463 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:09:21,251 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:09:21,263 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:09:21,263 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-09 00:10:05,931 WARN org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 50446, call
batchUpdate(triples,http://dblp.l3s.de/d2r/resource/publications/conf/iccd/HeydariP01a,1194566644908,
9223372036854775807, [EMAIL PROTECTED])
from 127.0.1.1:45778: output error
java.nio.channels.ClosedChannelException
at
sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:126)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
at
org.apache.hadoop.ipc.SocketChannelOutputStream.flushBuffer(SocketChannelOutputStream.java:108)
at
org.apache.hadoop.ipc.SocketChannelOutputStream.write(SocketChannelOutputStream.java:89)
at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:615)
2007-11-09 00:10:15,436 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:10:15,441 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scan of meta region regionname: -ROOT-,,0, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:10:21,271 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:10:21,427 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scan of meta region regionname: .META.,,1, startKey:
<>, server: 127.0.1.1:50446} complete
2007-11-09 00:10:21,427 INFO org.apache.hadoop.hbase.HMaster: all meta
regions scanned
2007-11-09 00:10:21,445 INFO org.apache.hadoop.hbase.HRegion: starting
compaction on region
triples,http://dblp.l3s.de/d2r/resource/publications/conf/iccd/HeydariP01a,1194566644908
2007-11-09 00:10:38,731 INFO
org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker:
RegionServer:0.splitOrCompactChecker exiting
2007-11-09 00:10:38,731 FATAL org.apache.hadoop.hbase.HRegionServer: Set
stop flag in RegionServer:0.splitOrCompactChecker
java.lang.OutOfMemoryError: Java heap space
at
org.apache.hadoop.hbase.io.ImmutableBytesWritable.readFields(ImmutableBytesWritable.java:111)
at
org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1629)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1712)
at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:435)
at
org.apache.hadoop.hbase.HStore$MapFileCompactionReader.next(HStore.java:784)
at org.apache.hadoop.hbase.HStore.compact(HStore.java:847)
at org.apache.hadoop.hbase.HStore.compact(HStore.java:727)
at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:631)
at org.apache.hadoop.hbase.HStore.compactHelper(HStore.java:563)
at org.apache.hadoop.hbase.HStore.compact(HStore.java:558)
at org.apache.hadoop.hbase.HRegion.compactStores(HRegion.java:719)
at
org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.checkForSplitsOrCompactions(HRegionServer.java:198)
at
org.apache.hadoop.hbase.HRegionServer$SplitOrCompactChecker.chore(HRegionServer.java:188)
at org.apache.hadoop.hbase.Chore.run(Chore.java:58)
2007-11-09 00:10:38,732 INFO org.apache.hadoop.hbase.Leases:
RegionServer:0 closing leases
2007-11-09 00:10:38,732 INFO org.apache.hadoop.hbase.HRegionServer:
worker thread exiting
2007-11-09 00:10:38,732 INFO
org.apache.hadoop.hbase.HRegionServer$LogRoller:
RegionServer:0.logRoller exiting
2007-11-09 00:10:38,871 INFO
org.apache.hadoop.hbase.Leases$LeaseMonitor: RegionServer:0.leaseChecker
exiting
2007-11-09 00:10:38,872 INFO org.apache.hadoop.hbase.Leases:
RegionServer:0 closed leases
2007-11-09 00:10:38,872 INFO org.apache.hadoop.ipc.Server: Stopping
server on 50446
2007-11-09 00:10:38,872 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 50446: exiting
2007-11-09 00:10:38,872 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 3 on 50446: exiting
2007-11-09 00:10:38,872 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 4 on 50446: exiting
2007-11-09 00:10:38,872 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 5 on 50446: exiting
2007-11-09 00:10:38,872 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 7 on 50446: exiting
2007-11-09 00:10:38,872 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 8 on 50446: exiting
2007-11-09 00:10:38,873 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 9 on 50446: exiting
2007-11-09 00:10:38,873 INFO org.apache.hadoop.ipc.Server: Stopping IPC
Server listener on 50446
2007-11-09 00:10:39,431 INFO org.apache.hadoop.hbase.HRegionServer:
Stopping infoServer
2007-11-09 00:10:39,432 INFO org.mortbay.util.ThreadedServer: Stopping
Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]
2007-11-09 00:10:39,731 INFO org.mortbay.http.SocketListener: Stopped
SocketListener on 0.0.0.0:60030
2007-11-09 00:10:40,004 INFO org.mortbay.util.Container: Stopped
HttpContext[/static,/static]
2007-11-09 00:10:43,216 INFO org.mortbay.util.Container: Stopped
HttpContext[/logs,/logs]
2007-11-09 00:10:43,217 INFO org.mortbay.util.Container: Stopped
[EMAIL PROTECTED]
2007-11-09 00:10:43,653 INFO org.mortbay.util.Container: Stopped
WebApplicationContext[/,/]
2007-11-09 00:10:43,653 INFO org.mortbay.util.Container: Stopped
[EMAIL PROTECTED]
2007-11-09 00:11:08,772 INFO org.apache.hadoop.hbase.Leases:
HMaster.leaseChecker lease expired 1376903167/1376903167
2007-11-09 00:11:08,772 INFO org.apache.hadoop.hbase.HMaster:
127.0.1.1:50446 lease expired
2007-11-09 00:11:15,452 INFO org.apache.hadoop.hbase.HMaster:
HMaster.rootScanner scanning meta region regionname: -ROOT-,,0,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:11:15,453 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 1 time(s).
2007-11-09 00:11:16,472 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 2 time(s).
2007-11-09 00:11:17,492 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 3 time(s).
2007-11-09 00:11:18,512 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 4 time(s).
2007-11-09 00:11:19,532 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 5 time(s).
2007-11-09 00:11:20,552 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 6 time(s).
2007-11-09 00:11:21,292 INFO org.apache.hadoop.hbase.HMaster:
HMaster.metaScanner scanning meta region regionname: .META.,,1,
startKey: <>, server: 127.0.1.1:50446}
2007-11-09 00:11:21,572 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 7 time(s).
2007-11-09 00:11:22,592 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 8 time(s).
2007-11-09 00:11:23,612 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 9 time(s).
2007-11-09 00:11:24,632 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 10 time(s).
2007-11-09 00:11:25,678 WARN org.apache.hadoop.hbase.HMaster: Scan ROOT
region
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:152)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:542)
at org.apache.hadoop.ipc.Client.call(Client.java:471)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:184)
at $Proxy1.openScanner(Unknown Source)
at
org.apache.hadoop.hbase.HMaster$BaseScanner.scanRegion(HMaster.java:218)
at
org.apache.hadoop.hbase.HMaster$RootScanner.scanRoot(HMaster.java:531)
at
org.apache.hadoop.hbase.HMaster$RootScanner.maintenanceScan(HMaster.java:574)
at org.apache.hadoop.hbase.HMaster$BaseScanner.chore(HMaster.java:199)
at org.apache.hadoop.hbase.Chore.run(Chore.java:58)
2007-11-09 00:11:25,678 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 1 time(s).
2007-11-09 00:11:26,702 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 2 time(s).
2007-11-09 00:11:27,722 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 3 time(s).
2007-11-09 00:11:28,742 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 4 time(s).
2007-11-09 00:11:28,842 INFO org.apache.hadoop.hbase.HMaster: process
shutdown of server 127.0.1.1:50446: logSplit: false, rootChecked: false,
rootRescanned: false, numberOfMetaRegions: 1, onlineMetaRegions.size(): 1
2007-11-09 00:11:28,843 INFO org.apache.hadoop.hbase.HLog: splitting 3
log(s) in /home/holger/tmp/hbase/log_127.0.1.1_-7685273460109660997_50446
2007-11-09 00:11:29,762 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 5 time(s).
2007-11-09 00:11:30,782 INFO org.apache.hadoop.ipc.Client: Retrying
connect to server: /127.0.1.1:50446. Already tried 6 time(s).
2007-11-09 00:11:30,823 INFO org.apache.hadoop.fs.FSInputChecker: Found
checksum error: org.apache.hadoop.fs.ChecksumException: Checksum error:
file:/home/holger/tmp/hbase/log_127.0.1.1_-7685273460109660997_50446/hlog.dat.010
at 18873344
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:210)
at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:234)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:157)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
at org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1763)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1663)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1709)
at org.apache.hadoop.hbase.HLog.splitLog(HLog.java:157)
at
org.apache.hadoop.hbase.HMaster$PendingServerShutdown.process(HMaster.java:2018)
at org.apache.hadoop.hbase.HMaster.run(HMaster.java:1086)
2007-11-09 00:11:30,824 WARN org.apache.hadoop.hbase.HMaster: Processing
pending operations: PendingServerShutdown of 127.0.1.1:50446
org.apache.hadoop.fs.ChecksumException: Checksum error:
file:/home/holger/tmp/hbase/log_127.0.1.1_-7685273460109660997_50446/hlog.dat.010
at 18873344
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:210)
at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:234)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:157)
at java.io.DataInputStream.readFully(DataInputStream.java:178)
at
org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
at org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1763)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1663)
at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1709)
at org.apache.hadoop.hbase.HLog.splitLog(HLog.java:157)
at
org.apache.hadoop.hbase.HMaster$PendingServerShutdown.process(HMaster.java:2018)
at org.apache.hadoop.hbase.HMaster.run(HMaster.java:1086)
2007-11-09 00:11:30,824 INFO org.apache.hadoop.hbase.HMaster: process
shutdown of server 127.0.1.1:50446: logSplit: false, rootChecked: false,
rootRescanned: false, numberOfMetaRegions: 1, onlineMetaRegions.size(): 1
stack wrote:
Do you think things are not working because you see a
NotServingRegionException in the log? These exceptions in the log are
'normal' (See HADOOP-1724 for discussion on throwing exceptions as
part of 'normal' operation). Otherwise, your log looks fine to me.
10G for a heap in my experience is overly large. What happens if you
knock it down to a 1G? Do things still work?
St.Ack
