[jira] [Commented] (ATLAS-179) Atlas hook causes mem leak and hive server 2 crashes

[Shwetha G S (JIRA)]

    [ 
https://issues.apache.org/jira/browse/ATLAS-179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14910094#comment-14910094
 ] 

Shwetha G S commented on ATLAS-179:
-----------------------------------

{quote}
why do we need a thread pool?
{quote}
For async execution

{quote}
Lets use the factory method in Executors.newSingleThreadExecutor()
{quote}
Single thread won't scale. Executors.newSingleThreadExecutor() in turn calls 
the same constructor that we are using
{code}
    public static ExecutorService newSingleThreadExecutor() {
        return new FinalizableDelegatedExecutorService
            (new ThreadPoolExecutor(1, 1,
                                    0L, TimeUnit.MILLISECONDS,
                                    new LinkedBlockingQueue<Runnable>()));
    }
{code}

{quote}
HiveConf should be gotten from HookContext
{quote}
Thread pool is static, hook context is available only in run(). So, we can't 
use HookContext.hiveConf. Moreover, I have refactored so that these configs are 
read from atlas application.properties instead so that we don't need to set any 
configs in hive-site.xml

> Atlas hook causes mem leak and hive server 2 crashes
> ----------------------------------------------------
>
>                 Key: ATLAS-179
>                 URL: https://issues.apache.org/jira/browse/ATLAS-179
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Shwetha G S
>            Assignee: Shwetha G S
>            Priority: Critical
>
> Two issues:
> 1. While queueing for ThreadPoolExecutor, we add hive query plan. Looks like 
> query plan contains the fetch tasks and their job confs. Job confs contain 
> java.util.Properties and can be huge
> 2. We have unbounded queue in ThreadPoolExecutor



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)