[
https://issues.apache.org/jira/browse/HBASE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481996#comment-13481996
]
Lars Hofhansl edited comment on HBASE-7008 at 10/23/12 6:08 PM:
----------------------------------------------------------------
Did some quick unscientific testing (with Nagles disabled!):
rows with 1 column (100 bytes):
||Caching||ms/row||
|1|0.17|
|10|0.03|
|100|0.006|
|1000|0.003|
|2000|0.003|
|5000|0.002|
|10000|0.002|
rows with 10 columns (100 bytes each) in a single column family:
||Caching||ms/row||
|1|0.3|
|10|0.2 **|
|100|0.05|
|1000|0.03|
|2000|0.02|
|5000|0.02|
|10000|0.02|
(**) Must have crossed a packet size boundary (localhost packet size is 65k)
rows with 10 columns (1000 bytes each) in a single column family:
||Caching||ms/row||
|1|1|
|10|0.5|
|100|0.2|
|1000|0.09|
|2000|0.08|
|5000|0.07|
|10000|0.1|
rows with 100 columns (1000 bytes each) in a single column family:
||Caching||ms/row||
|1|3|
|10|1.6|
|100|0.8|
|1000|0.7|
|2000|**|
(**) Crashed the RegionServer in the default setting (OOM?)
rows with 100 columns (10000 bytes each - so 1mb rows) in a single column
family:
||Caching||ms/row||
|1|18|
|10|8|
|100|8|
|1000|**|
(**) Anything over 150 crashed the RegionServer in the default setting (OOM?)
This was all done with client and server on the same machine, so worst case for
a larger cache setting. If the client is further away these effects will be
much more pronounced.
So it seems a 100 would be a good default value to start with.
Edit: Remeasured some of the numbers.
was (Author: lhofhansl):
Did some quick unscientific testing (with Nagles disabled!):
rows with 1 column (100 bytes):
||Caching||ms/row||
|1|0.17|
|10|0.03|
|100|0.006|
|1000|0.003|
|2000|0.003|
|5000|0.002|
|10000|0.002|
rows with 10 columns (100 bytes each) in a single column family:
||Caching||ms/row||
|1|0.3|
|10|0.2 **|
|100|0.05|
|1000|0.03|
|2000|0.02|
|5000|0.02|
|10000|0.02|
(**) Must have crossed a packet size boundary (localhost packet size is 65k)
rows with 10 columns (1000 bytes each) in a single column family:
||Caching||ms/row||
|1|1|
|10|0.5|
|100|0.2|
|1000|0.09|
|2000|0.08|
|5000|0.07|
|10000|0.1|
rows with 100 columns (1000 bytes each) in a single column family:
||Caching||ms/row||
|1|16|
|10|1.6|
|100|0.8|
|1000|0.7|
|2000|**|
(**) Crashed the RegionServer in the default setting (OOM?)
rows with 100 columns (10000 bytes each - so 1mb rows) in a single column
family:
||Caching||ms/row||
|1|28|
|10|8|
|100|8|
|1000|**|
(**) Anything over 150 crashed the RegionServer in the default setting (OOM?)
This was all done with client and server on the same machine, so worst case for
a larger cache setting. If the client is further away these effects will be
much more pronounced.
So it seems a 100 would be a good default value to start with.
> Set scanner caching to a better default
> ---------------------------------------
>
> Key: HBASE-7008
> URL: https://issues.apache.org/jira/browse/HBASE-7008
> Project: HBase
> Issue Type: Bug
> Components: Client
> Reporter: liang xie
> Assignee: liang xie
> Fix For: 0.94.3, 0.96.0
>
> Attachments: 7008-0.94.txt, 7008-0.94-v2.txt, 7008-v3.txt,
> 7008-v4.txt, HBASE-7008.patch, HBASE-7008-v2.patch
>
>
> per
> http://search-hadoop.com/m/qaRu9iM2f02/Set+scanner+caching+to+a+better+default%253F&subj=Set+scanner+caching+to+a+better+default+
> let's set to 100 by default
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
