[
https://issues.apache.org/jira/browse/HBASE-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13924403#comment-13924403
]
Biju Nair commented on HBASE-10643:
-----------------------------------
Also the failures were intermittent and not consistent i.e one or two of the RS
fails and others were successful. There may be EC2 factors in play. But it
would be good to allocate the memory before registering with ZK. I am assuming
this will also help in handling issues like user configuring more memory than
physically available.
> Failure in RS when using large size bucketcache
> -----------------------------------------------
>
> Key: HBASE-10643
> URL: https://issues.apache.org/jira/browse/HBASE-10643
> Project: HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.98.0, 0.96.0
> Reporter: Biju Nair
> Labels: bucketCache, regionserver
>
> When RS is brought up with XX:MaxDirectMemorySize of 22GB or higher, RS fails
> after a successful start. From the RS logs it looks like the bucketCache
> memory allocation is taking more time makes the RS considered dead by ZK. One
> option to fix the problem would be to allocate the bucketCache before
> registering with ZK.
> 2014-02-28 18:54:42,967 WARN [regionserver60020.compactionChecker]
> util.Sleeper: We slept 33496ms instead of 10000ms, this is likely due to a
> long garbage collecting pause and it's usually bad, see
> http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
> 2014-02-28 18:54:42,967 WARN [regionserver60020.periodicFlusher]
> util.Sleeper: We slept 33496ms instead of 10000ms, this is likely due to a
> long garbage collecting pause and it's usually bad, see
> http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
> 2014-02-28 18:54:42,967 WARN [JvmPauseMonitor] util.JvmPauseMonitor:
> Detected pause in JVM or host machine (eg GC): pause of approximately 23988ms
> GC pool 'ParNew' had collection(s): count=1 time=24432ms
> 2014-02-28 18:54:43,006 FATAL [regionserver60020] regionserver.HRegionServer:
> ABORTING region server bbg-master2.bbg-test.hdp,60020,1393628951236:
> org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected;
> currently processing bbg-master2.bbg-test.hdp,60020,1393628951236 as dead
> server
> at
> org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:341)
> at
> org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:254)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
