[
https://issues.apache.org/jira/browse/PHOENIX-2570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086036#comment-15086036
]
Samarth Jain edited comment on PHOENIX-2570 at 1/6/16 7:06 PM:
---------------------------------------------------------------
- Transaction manager is now started only for test suites that need
transactions enabled. Currently we are starting txManager even for those test
suites that don't need it. Tests that need transactions already set
QueryServices.TRANSACTIONS_ENABLED to true in their @BeforeClass test suite
methods. I made the change to use that variable to conditionally start the
txManager.
- Decrease the count for index and metadata RPC handlers. We were using the
default of 30 for both. I noticed in the jstacks that we had almost all 120
threads sitting idle per JVM. By reducing the handlers to 5, with 5 forked
JVMs, this change brought down the thread count by 500.
- Reduce the size of thread pool that is created on establishing an
HConnection. By default, we were creating a thread pool of size 256 on
establishing every HConnection. By using the value of 10, reduced the thread
count by ~240.
I like the idea of making the changes in QueryServicesTestImpl. We would need
to add corresponding setter methods in QueryServicesOptions.
Will make the change to use the DEFAULT_TRANSACTIONS_ENABLED flag instead.
+1 on making the default to 7. Alternatively, we can leave the default as it is
and override the value for our jenkins builds.
was (Author: samarthjain):
- Transaction manager is now started only for tests that need transactions
enabled. The transaction tests already set QueryServices.TRANSACTIONS_ENABLED
to true in their @BeforeClass methods. I made the change to use that variable
to conditionally start the txManager.
- Decrease the count for index and metadata RPC handlers. We were using the
default of 30 for both. I noticed in the jstacks that we had almost all 120
threads sitting idle per JVM. By reducing the handlers to 5, with 5 forked
JVMs, this change brought down the thread count by 500.
- Reduce the size of thread pool that is created on establishing an
HConnection. By default, we were creating a thread pool of size 256 on
establishing every HConnection. By using the value of 10, reduced the thread
count by ~240.
I like the idea of making the changes in QueryServicesTestImpl. We would need
to add corresponding setter methods in QueryServicesOptions.
Will make the change to use the DEFAULT_TRANSACTIONS_ENABLED flag instead.
+1 on making the default to 7. Alternatively, we can leave the default as it is
and override the value for our jenkins builds.
> Start transaction manager only when needed and reduce number of threads
> spawned for tests
> -----------------------------------------------------------------------------------------
>
> Key: PHOENIX-2570
> URL: https://issues.apache.org/jira/browse/PHOENIX-2570
> Project: Phoenix
> Issue Type: Bug
> Reporter: Samarth Jain
> Assignee: Samarth Jain
> Attachments: PHOENIX-2570_4.x-HBase-0.98.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
