[
https://issues.apache.org/jira/browse/HBASE-12736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jurriaan Mous updated HBASE-12736:
----------------------------------
Attachment: HBASE-12736-v4.patch
You are right. It uses way less Threads in general. I was too focussed on the
graph.
I was worried about the more active threads and more memory usage. But it gets
all cleaned up and they end up the same.
Looking into what was causing it it was pretty simple. Since all MetaScanners
use the same connection, they use the same thread pool. And now each scanner
does not close the connection and thus they don't clean up their threads along
the way. And the thread pool limit is not reached while scanning so the peaks
are higher.
> Let MetaScanner recycle a given Connection
> ------------------------------------------
>
> Key: HBASE-12736
> URL: https://issues.apache.org/jira/browse/HBASE-12736
> Project: HBase
> Issue Type: Improvement
> Components: Client
> Reporter: Jurriaan Mous
> Assignee: Jurriaan Mous
> Attachments: HBASE-12736-v1.patch, HBASE-12736-v2.patch,
> HBASE-12736-v3.patch, HBASE-12736-v4.patch, HBASE-12736.patch,
> profile_TestMetaScanner_after_patch.png,
> profile_TestMetaScanner_before_patch.png
>
>
> It is very heavy to create a Connection on each meta scan. Connections create
> RpcClients, RpcClients create RPC connections and all cannot be recycled.
> Tests with a lot of metascans are very heavy with the async client.
> This issue is to make anything that uses metaScans reuse the same connection.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
