@hbase.apache.org
Cc:
Sent: Thursday, May 16, 2013 2:16 AM
Subject: GET performance degrades over time
Hi,
My setup is as follows:
24 regionservers (7GB RAM, 8-core CPU, 5GB heap space)
hbase 0.94.4
5-7 regions per regionserver
I am doing an avg of 4k-5k random gets per regionserver per second
Thanks for all the help in advance!
Answers inline..
Hi Viral,
some questions:
Are you adding new data or deleting data over time?
Yes I am continuously adding new data. The puts have not slowed down but
that could also be an after effect of deferred log flush.
Do you have bloom
Look at how much Hard Disk utilization you have (IOPS / Svctm). You may
just be under scaled for the QPS you desire for both read + write load. If
you are performing random gets, you could expect around the low to mid
100's IOPS/sec per HDD. Use bonnie++ / IOZone / IOPing to verify.
Also you
Yes bloom filters have been enabled: ROWCOL
Can u try with ROW bloom?
-Anoop-
On Fri, May 17, 2013 at 12:20 PM, Viral Bajaria viral.baja...@gmail.comwrote:
Thanks for all the help in advance!
Answers inline..
Hi Viral,
some questions:
Are you adding new data or deleting data
On Fri, May 17, 2013 at 8:23 AM, Jeremy Carroll phobos...@gmail.com wrote:
Look at how much Hard Disk utilization you have (IOPS / Svctm). You may
just be under scaled for the QPS you desire for both read + write load. If
you are performing random gets, you could expect around the low to mid
Hi,
My setup is as follows:
24 regionservers (7GB RAM, 8-core CPU, 5GB heap space)
hbase 0.94.4
5-7 regions per regionserver
I am doing an avg of 4k-5k random gets per regionserver per second and the
performance is acceptable in the beginning. I have also done ~10K gets for
a single regionserver
This generally happens when the same block is accessed for the HFile. Are
you seeing any contention on the HDFS side?
Regards
Ram
On Thu, May 16, 2013 at 4:19 PM, Bing Jiang jiangbinglo...@gmail.comwrote:
Have you checked your HBase environment? I think it perhaps come from:
1) System uses
Have you checked your HBase environment? I think it perhaps come from:
1) System uses more swap frequently when your continue to execute Gets
operation?
I have set swap to 0. AFAIK, that's a recommended practice. Let me know if
that should not be followed for nodes running HBase.
2) check
This generally happens when the same block is accessed for the HFile. Are
you seeing any contention on the HDFS side?
When you say contention what should I be looking for ? slow operations to
respond to data block requests ? or some specific metric in ganglia ?
-Viral
Michael is correct.
More information available about swap value on wikipedia:
http://en.wikipedia.org/wiki/Swappiness
2013/5/16 Michael Segel michael_se...@hotmail.com
Going from memory, the swap value setting to 0 is a suggestion. You may
still actually swap, but I think its a 'last resort'
Going from memory, the swap value setting to 0 is a suggestion. You may
still actually swap, but I think its a 'last resort' type of thing.
When you look at top, at the top of the page, how much swap do you see?
When I look at top it says: 0K total, 0K used, 0K free (as expected). I can
try
Going from memory, the swap value setting to 0 is a suggestion. You may still
actually swap, but I think its a 'last resort' type of thing.
When you look at top, at the top of the page, how much swap do you see?
On May 16, 2013, at 1:43 PM, Viral Bajaria viral.baja...@gmail.com wrote:
Have
If you're not swapping then don't worry about it.
My comment was that even though you set the swap to 0, and I'm going from
memory, its possible for some swap to occur.
(But I could be wrong. )
You really don't have a lot of memory, and you have a 5GB heap... MSLABS on?
Could you be facing
If you're not swapping then don't worry about it.
My comment was that even though you set the swap to 0, and I'm going from
memory, its possible for some swap to occur.
(But I could be wrong. )
Thanks for sharing this info. Will remember for future debugging too.
Checked the vm.swappiness
14 matches
Mail list logo
