[
https://issues.apache.org/jira/browse/HBASE-1060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657728#action_12657728
]
schubert zhang commented on HBASE-1060:
---------------------------------------
Sometimes, when I do mapreduce job (eg. sorting) on a large dataset (eg. 30mil
records), the lease may exprie too.
Now, I change the size of Region smaller (from 256MB to 64MB) and change the
number of map task smaller (from 8 per node to 3, there are 4codes pre node).
The failure not happen.
> RegionServer lease expire due to LogRoller take long time and the
> RegionServer cannot recovery
> -------------------------------------------------------------------------------------------------
>
> Key: HBASE-1060
> URL: https://issues.apache.org/jira/browse/HBASE-1060
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.18.1
> Environment: I have one master and 4 region servers.
> The test use batchUpdate() to insert many record to achieve big data sets.
> Reporter: schubert zhang
>
> Usually, after tens GB of data, following exception occurs, it seems
> LogRoller take long time and the RegionServer cannot report to master, and
> the lease expire. But it cannot recovery.
> ---------
> 2008-12-13 11:13:39,347 INFO org.apache.hadoop.hbase.regionserver.LogRoller:
> Rolling hlog. Number of entries: 30015
> 2008-12-13 11:13:39,354 INFO org.apache.hadoop.hbase.regionserver.HLog: New
> log writer created at
> /hbase/log_10.24.1.14_1229053900800_60020/hlog.dat.1229138019352
> 2008-12-13 11:13:47,991 INFO org.apache.hadoop.hbase.regionserver.LogRoller:
> Rolling hlog. Number of entries: 30015
> 2008-12-13 11:13:49,138 INFO org.apache.hadoop.hbase.regionserver.HLog: New
> log writer created at
> /hbase/log_10.24.1.14_1229053900800_60020/hlog.dat.1229138029136
> 2008-12-13 11:15:49,867 WARN org.apache.hadoop.hbase.util.Sleeper: We slept
> 120318ms, ten times longer than scheduled: 3000
> 2008-12-13 11:15:50,175 WARN
> org.apache.hadoop.hbase.regionserver.HRegionServer: unable to report to
> master for 120343 milliseconds - retrying
> 2008-12-13 11:15:50,271 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_CALL_SERVER_STARTUP
> 2008-12-13 11:15:50,419 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 6 on 60020, call get([...@4b2a82b3, [...@53eadc52, [...@6026b688, -1, -1)
> from 10.24.1.18:58581: error:
> org.apache.hadoop.hbase.NotServingRegionException: TableDes,,1228799531753
> org.apache.hadoop.hbase.NotServingRegionException: TableDes,,1228799531753
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:1859)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1321)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> 2008-12-13 11:15:50,434 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 3 on 60020, call get([...@26ee4c87, [...@66e24708, [...@11a592fd, -1, -1)
> from 10.24.1.12:46262: error:
> org.apache.hadoop.hbase.NotServingRegionException: TableDes,,1228799531753
> org.apache.hadoop.hbase.NotServingRegionException: TableDes,,1228799531753
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:1859)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1321)
> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
> 2008-12-13 11:15:50,431 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 2 on 60020, call get([...@519182c2, [...@281fb864, [...@3a82c1d0, -1, -1)
> from 10.24.1.16:34950: error:
> org.apache.hadoop.hbase.NotServingRegionException: TableDes,,1228799531753
> org.apache.hadoop.hbase.NotServingRegionException: TableDes,,122879953175
> .................many exceptions.......................
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
