[ 
https://issues.apache.org/jira/browse/HAWQ-1622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shivram Mani resolved HAWQ-1622.
--------------------------------
       Resolution: Fixed
    Fix Version/s: 2.4.0.0-incubating

The PR has been merged to master.

Based on performance tests we see 17x speedup and 50x reduction in threads=

Closing JIRA...

> Cache PXF proxy UGI so that cleanup of FileSystem cache doesn't have to be 
> done on each request
> -----------------------------------------------------------------------------------------------
>
>                 Key: HAWQ-1622
>                 URL: https://issues.apache.org/jira/browse/HAWQ-1622
>             Project: Apache HAWQ
>          Issue Type: Improvement
>          Components: PXF
>            Reporter: Alexander Denissov
>            Assignee: Lav Jain
>            Priority: Major
>             Fix For: 2.4.0.0-incubating
>
>
> Closing PXF proxy UGIs on each request (implemented in HAWQ-1621) slows down 
> PXF request response time significantly when several threads work 
> concurrently as it locks FileSystem cache and holds the lock while the 
> cleanup of DFSClients is completed.
> This can be avoided by caching the proxy UGI for a given proxy user between 
> requests. Care must be taken to remove the cached entry after some 
> pre-defined TTL if and only if there are no current threads using any 
> FileSystem entries held by the cache. A combination of TTL-based cache with 
> ref-counting might be utilized to achieve this.
>  
> For some example of this, see: 
> https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/service/UserGroupInformationService.java
> Caching UGIs might be tricky when Kerberos support is implemented later, see: 
> https://issues.apache.org/jira/browse/HIVE-3098?focusedCommentId=13398979&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-13398979



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to