I think that explains why memstore wasn't flushed.
See snippet from region server log:
2010-09-24 11:21:18,603 DEBUG
org.apache.hadoop.hbase.io.hfile.LruBlockCache: Cache Stats: Sizes:
Total=6.5512085MB (6869440), Free=791.7863MB (830248128), Max=798.3375MB
(837117568), Counts: Blocks=1, Access=384, Hit=383, Miss=1, Evictions=0,
Evicted=0, Ratios: Hit Ratio=99.73958134651184%, Miss
Ratio=0.2604166744276881%, Evicted/Run=NaN
2010-09-24 11:21:20,459 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x12b44afa1f50001 to sun.nio.ch.selectionkeyi...@3d3cdaa
java.io.IOException: TIMED OUT
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:906)
2010-09-24 11:21:21,512 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event,
state: Disconnected, type: None, path: null
2010-09-24 11:21:22,849 INFO org.apache.zookeeper.ClientCnxn: Attempting
connection to server sjc9-flash-grid01.carrieriq.com/10.32.56.155:2181
2010-09-24 11:21:22,849 INFO org.apache.zookeeper.ClientCnxn: Priming
connection to java.nio.channels.SocketChannel[connected local=/
10.32.56.156:45455 remote=sjc9-flash-grid01.carrieriq.com/10.32.56.155:2181]
2010-09-24 11:21:22,873 INFO org.apache.zookeeper.ClientCnxn: Server
connection successful
2010-09-24 11:21:23,065 WARN org.apache.zookeeper.ClientCnxn: Exception
closing session 0x12b44afa1f50001 to sun.nio.ch.selectionkeyi...@383de71c
java.io.IOException: Session Expired
at
org.apache.zookeeper.ClientCnxn$SendThread.readConnectResult(ClientCnxn.java:589)
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:709)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
2010-09-24 11:21:23,065 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event,
state: Expired, type: None, path: null
2010-09-24 11:21:23,065 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: ZooKeeper session
expired
2010-09-24 11:21:23,101 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics:
request=0.0, regions=45, stores=119, storefiles=5, storefileIndexSize=0,
memstoreSize=7, compactionQueueSize=0, usedHeap=61, maxHeap=3991,
blockCacheSize=6869440, blockCacheFree=830248128, blockCacheCount=1,
blockCacheHitRatio=99, fsReadLatency=0, fsWriteLatency=0, fsSyncLatency=0
2010-09-24 11:21:24,457 INFO org.apache.hadoop.ipc.HBaseServer: Stopping
server on 60020
Obviously this method in HRegionServer triggered abort():
public void process(WatchedEvent event) {
In this situation, why should internalFlushcache() be governed by abort ?
Thanks
On Fri, Sep 24, 2010 at 4:54 PM, Jean-Daniel Cryans <[email protected]>wrote:
> Like that:
>
> http://hbase.apache.org/docs/r0.89.20100726/xref/org/apache/hadoop/hbase/regionserver/HRegion.html#519
>
> (this is a link to the 0.89 code, but 0.20 works exactly the same way)
>
> J-D
>
> On Fri, Sep 24, 2010 at 4:49 PM, Ted Yu <[email protected]> wrote:
> > When HBase cluster shuts down, how is memstore flushing triggered ?
> >
> > Thanks
> >
> > On Fri, Sep 24, 2010 at 4:01 PM, Jean-Daniel Cryans <[email protected]
> >wrote:
> >
> >> > StripedHBaseTable is just a wrapper around HTable.
> >>
> >> Ok.
> >>
> >> > QA installed another application which queries HBase. When QA didn't
> see
> >> > meaningful data from the application.
> >> > Why didn't HBase serve data from Memstore ?
> >>
> >> I cannot answer this question, as far as I know 0.20.6 doesn't have a
> >> bug that prevents you from writing data (and many people use it in
> >> production environments). My guess is, the data never got into HBase
> >> or it was written with wrong timestamps and is hidden by delete
> >> markers or something "time-travelly" like that. If they do play with
> >> timestamps, ask them not to or verify they do the right thing.
> >>
> >> > QA used stop-hbase.sh to stop HBase. Would Memstore contents be
> flushed
> >> to
> >> > Storefile at that moment ?
> >>
> >> Yep, if you don't see any store file then the data never made it into
> >> hbase.
> >>
> >> >
> >> > QA servers were running JVM version 1.6.0_18
> >>
> >> From
> >>
> http://hbase.apache.org/docs/r0.20.6/api/overview-summary.html#requirements
> >>
> >> "Java 1.6.x, preferably from Sun. Use the latest version available
> >> except u18 (u19 is fine)."
> >>
> >> J-D
> >>
> >
>
