The log you cited should come from PeriodicMemstoreFlusher.
public static final String MEMSTORE_PERIODIC_FLUSH_INTERVAL =
"hbase.regionserver.optionalcacheflushinterval";
/** Default interval for the memstore flush */
public static final int DEFAULT_CACHE_FLUSH_INTERVAL = 3600000;
Have you find direct correlation between the timestamp of
RegionTooBusyException
and the periodic flush logs ?
Thanks
On Thu, Sep 15, 2016 at 11:39 AM, marjana <[email protected]> wrote:
> So I have tried many different settings on my target cluster. But I still
> get:
> RegionTooBusyException: Above memstore limit
> regionName=3079_01,e4000000,1473944476785.35b4d8395a61af0eefc99f3d7f931c
> d2.,
> server=hb01dn15,16020,1473800758347, memstoreSize=806131464,
> blockingMemStoreSize=805306368
>
> If I look at hb01dn15 for that time, I see a lot of INFO messages,
> "MemStoreFlushercore requesting flush because e has an old edit so flush to
> free WALs after random delay of 89811ms"
> I think these flushes are the cause of regionserver being too busy.
> Our used heap (max 1700m) stays well below maxHeap (6332m).
>
> My nondefault settings:
> <property>
> <name>hbase.replication</name>
> <value>true</value>
> </property>
> <property>
> <name>hbase.client.keyvalue.maxsize</name>
> <value>1073741824</value>
> </property>
> <property>
> <name>hbase.rpc.timeout</name>
> <value>180000</value>
> <description>
> </description>
> </property>
> <property>
> <name>hbase.client.scanner.timeout.period</name>
> <value>180000</value>
> <description>
> </description>
> </property>
> <property>
> <name>hbase.hregion.memstore.block.multiplier</name>
> <value>6</value>
> </property>
> <property>
> <name>hbase.hstore.blockingStoreFiles</name>
> <value>100</value>
> </property>
>
>
> Any suggestions on how to slow down the flushes to free WALs?
> Thanks
>
>
>
>
> --
> View this message in context: http://apache-hbase.679495.n3.
> nabble.com/copyTable-mapreduce-job-failing-with-ScannerTimeoutException-
> tp4082544p4082682.html
> Sent from the HBase User mailing list archive at Nabble.com.
>
