Mustafa Iman created HIVE-23744:

             Summary: Reduce query startup latency
                 Key: HIVE-23744
             Project: Hive
          Issue Type: Task
          Components: llap
    Affects Versions: 4.0.0
            Reporter: Mustafa Iman
            Assignee: Mustafa Iman
         Attachments: am_schedule_and_transmit.png, task_start.png

When I run queries with large number of tasks for a single vertex, I see a 
significant delay before all tasks start execution in llap daemons. 

Although llap daemons have the free capacity to run the tasks, it takes a 
significant time to schedule all the tasks in AM and actually transmit them to 

"am_schedule_and_transmit" shows scheduling of tasks of tpcds query 55. It 
shows only the tasks scheduled for one of 10 llap daemons. The scheduler works 
in a single thread, scheduling tasks one by one. A delay in scheduling of one 
task, delays all the tasks.



Another issue is that it takes long time to fill all the execution slots in 
llap daemons even though they are all empty initially. This is caused by 
LlapTaskCommunicator using a fixed number of threads (10 by default) to send 
the tasks to daemons. Also this communication is synchronized so these threads 
block communication staying idle. "task_start.png" shows running tasks on an 
llap daemon that has 12 execution slots. By the time 12th task starts running, 
more than 100ms already passes. That slot stays idle all this time. 


This message was sent by Atlassian Jira

Reply via email to