[
https://issues.apache.org/jira/browse/HBASE-9587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13772308#comment-13772308
]
Jean-Daniel Cryans commented on HBASE-9587:
-------------------------------------------
More info about my testing:
- 10k regions
- 5 region servers
- PE 16 clients with 1 HTable each but sharing the same connection and
executor pool.
If I do a randomWrite of 4GB, my client is using 16+(16*200)=3.2k threads (plus
a few strays) and OOMEs with a default heap (but 1.5GB is enough). Runtime is 8
minutes.
On a sequentialWrite of 4GB, my client uses about 100 threads and succeeds with
a default heap. Runtime is 1 minute 20 seconds.
A big part of the difference in runtime seems to come when this happens:
bq. 2013-09-19 20:39:53,056 INFO [TestClient-15] client.AsyncProcess: :
Waiting for the global number of running tasks to be equals or less than 0,
tasksSent=1035, tasksDone=898, currentTasksDone=898, tableName=TestTable
Interesting bit here, it says "global" but it's per AsyncProcess and I have 16
of them.
> AsyncProcess max outstanding cap should be per connection
> ---------------------------------------------------------
>
> Key: HBASE-9587
> URL: https://issues.apache.org/jira/browse/HBASE-9587
> Project: HBase
> Issue Type: Bug
> Reporter: Elliott Clark
> Assignee: Elliott Clark
>
> JD noticed that when running PE there were ~4k threads running. This is
> waaaaay too many.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
