[
https://issues.apache.org/jira/browse/HBASE-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12679439#action_12679439
]
Jean-Daniel Cryans commented on HBASE-1238:
-------------------------------------------
Turns out the patch works well :
{code}
2009-03-05 00:37:54,348 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://camanomade.wordpress.com/,1236183697332 because global
memcache limit of 676.7m exceeded; currently 646.4m and flushing till 422.9m
2009-03-05 00:38:01,738 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://astore.amazon.com/blogcarnival23-20/detail/0761104844/178-5338054-2231726,1235269774078
because global memcache limit of 676.7m exceeded; currently 622.2m and
flushing till 422.9m
2009-03-05 00:38:02,556 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://www.fsprostab.com/english/index.htm,1235943861247 because
global memcache limit of 676.7m exceeded; currently 600.9m and flushing till
422.9m
2009-03-05 00:38:48,789 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://blackbamboohawaii.com/infinity/index.html,1236182065945
because global memcache limit of 676.7m exceeded; currently 580.9m and flushing
till 422.9m
2009-03-05 00:38:49,271 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://ask.metafilter.com/60882/MidSouth-vacation,1235821461371
because global memcache limit of 676.7m exceeded; currently 563.1m and flushing
till 422.9m
2009-03-05 00:38:49,865 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://www.galapagosisland.net/travel_insurance.html,1236106016987
because global memcache limit of 676.7m exceeded; currently 545.2m and flushing
till 422.9m
2009-03-05 00:38:50,449 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://backcountryblog.blogspot.com/2006/10/hucking-cliffs-is-soooo-last-week.html,1236167469547
because global memcache limit of 676.7m exceeded; currently 528.2m and
flushing till 422.9m
2009-03-05 00:38:50,985 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://www.fletcherlake.com/index.htm,1236166974226 because global
memcache limit of 676.7m exceeded; currently 511.3m and flushing till 422.9m
2009-03-05 00:38:51,803 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://www.eurailblog.com/archives/cat_wales.shtml,1235786159128
because global memcache limit of 676.7m exceeded; currently 495.4m and flushing
till 422.9m
2009-03-05 00:38:52,242 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://www.island.net/~sointula/accom.html,1236183944276 because
global memcache limit of 676.7m exceeded; currently 479.8m and flushing till
422.9m
2009-03-05 00:38:52,774 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://www.manukapoint.com/Contact__20Us/__;window.mm_menu_b25ee76db7ed98b94e5945a386e8b470_2.addMenuItem__,1235779612713
because global memcache limit of 676.7m exceeded; currently 465.8m and
flushing till 422.9m
2009-03-05 00:38:53,252 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://wellsbeachme.com/wc/index.htm,1236106056942 because global
memcache limit of 676.7m exceeded; currently 452.4m and flushing till 422.9m
2009-03-05 00:38:55,239 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://www.greeka.com/saronic/agistri/hotels/agistri-rosys-little-village/index.html,1236104694918
because global memcache limit of 676.7m exceeded; currently 439.3m and
flushing till 422.9m
2009-03-05 00:38:55,525 INFO
org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced flushing of
web_pages,http://www.manukapoint.com/Contact__20Us/__;window.mm_menu_09fa2b8abdc5d7b0d6a4fd2cb2877ef1_2.addMenuItem__,1236219657625
because global memcache limit of 676.7m exceeded; currently 426.4m and
flushing till 422.9m
2009-03-05 00:39:04,436 INFO org.apache.hadoop.hbase.regionserver.HRegion:
starting compaction on region
web_pages,http://camanomade.wordpress.com/,1236183697332
2009-03-05 00:39:08,996 INFO org.apache.hadoop.hbase.regionserver.HRegion:
compaction completed on region
web_pages,http://camanomade.wordpress.com/,1236183697332 in 4sec
2009-03-05 00:39:08,997 INFO org.apache.hadoop.hbase.regionserver.HRegion:
starting compaction on region
web_pages,http://astore.amazon.com/blogcarnival23-20/detail/0761104844/178-5338054-2231726,1235269774078
2009-03-05 00:39:13,673 INFO org.apache.hadoop.hbase.regionserver.HRegion:
compaction completed on region
web_pages,http://astore.amazon.com/blogcarnival23-20/detail/0761104844/178-5338054-2231726,1235269774078
in 4sec
...
{code}
I suggest we commit this for 0.19.1, maybe even for 0.20.0 until a better
solution is found.
> Under upload, region servers are unable to compact when loaded with hundreds
> of regions
> ---------------------------------------------------------------------------------------
>
> Key: HBASE-1238
> URL: https://issues.apache.org/jira/browse/HBASE-1238
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.19.0
> Reporter: Jean-Daniel Cryans
> Priority: Blocker
> Fix For: 0.19.1, 0.20.0
>
> Attachments: hbase-1238.patch
>
>
> We have a situation where each region server is loaded with 100+ regions,
> most of them in the same table. During a long upload of webpages, each
> memcache gets filled near equally fast so that the global memcache limit is
> usually triggered before the max memcache size. Since that emergency flush
> does not trigger compactions, the number of store files just keeps growing
> until it fails on all kinds of errors.
> We need a better story for this as this is a "normal" situation.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
