[
https://issues.apache.org/jira/browse/HBASE-3?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565597#action_12565597
]
Michael Bieniosek commented on HBASE-3:
---------------------------------------
Bryan,
I am experimenting with different topologies. The distribution of our random
reads is extremely skewed, so I set up a single proxy cache (varnishd) for all
our traffic. This works much better than past experiments with smaller
separate caches on individual work nodes. The single proxy cache has scaled to
our needs, despite living in a single process, so the traffic is already
aggregated by the time the cache misses go to the rest server. So, I'd like to
have a single rest server serving as the backend. If this isn't possible, I'm
willing to expand the number of rest servers, but in that case it is not the
traffic aggregation that is limiting our scaling.
> rest server: configure number of threads for jetty
> --------------------------------------------------
>
> Key: HBASE-3
> URL: https://issues.apache.org/jira/browse/HBASE-3
> Project: Hadoop HBase
> Issue Type: Bug
> Components: rest
> Reporter: Michael Bieniosek
> Attachments: hbase-4.patch
>
>
> I am running a mapreduce job (~400 simultaneous map tasks) that makes random
> reads from hbase. I have put a varnishd reverse proxy cache in front of a
> single rest server. The single rest server's jetty appears to be running out
> of threads:
> 2008-02-01 23:17:16,971 INFO org.mortbay.http.SocketListener: LOW ON THREADS
> ((256-256+1)<2) on [EMAIL PROTECTED]:61234
> 2008-02-01 23:17:17,116 WARN org.mortbay.http.SocketListener: OUT OF THREADS:
> [EMAIL PROTECTED]:61234
> 2008-02-01 23:17:19,255 INFO org.mortbay.http.SocketListener: LOW ON THREADS
> ((256-256+1)<2) on [EMAIL PROTECTED]:61234
> The default for jetty is to use a thread pool of 256 threads. But I'd like
> to be able to specify (preferably in hadoop-site.xml) how large the thread
> pool should be -- in this case it needs to have as many threads as I have
> simultaneous map tasks.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
