[
https://issues.apache.org/jira/browse/HCATALOG-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421010#comment-13421010
]
Rohini Palaniswamy commented on HCATALOG-370:
---------------------------------------------
More than the time optimization we need this patch to enable running multiple
clients that query hcat in parallel. Users who launched 6 pig jobs in parallel
and used HCatLoader/HCatStorer ran into connection refused errors from server
with "Kerberos replay attack" errors. Cutting down the number of connection
calls (which reduces the number of authentications) allows running more pig
jobs in parallel.
> Create a HiveMetaStoreClient cache in hcatalog
> ----------------------------------------------
>
> Key: HCATALOG-370
> URL: https://issues.apache.org/jira/browse/HCATALOG-370
> Project: HCatalog
> Issue Type: Improvement
> Components: client
> Affects Versions: 0.4
> Reporter: Rohini Palaniswamy
> Priority: Minor
> Attachments: J370.patch
>
>
> Hcat makes multiple calls to hivemetastore client and every time the
> connection is setup and destroyed which is costly. It would be nice to have a
> centralized cache, that has a connection per unique identifier (ugi,hcat
> server uri and anything else mandatory). Instead of creating clients all over
> the code, it can be retrieved from this cache. Connections in the cache can
> be closed using WeakReference and ReferenceQueue or based on some time
> interval.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
