[
https://issues.apache.org/jira/browse/TAJO-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14205903#comment-14205903
]
Jinho Kim commented on TAJO-540:
--------------------------------
Hi [~hyunsik]
Actually, I have no detailed plan yet.
I did implement trunk-based fair-scheduler and It does not scale out on big
cluster
In my opinion, we must change the ResourceManager.
Here is my simple plan:
1. Add ResourceManager in TajoWorker (consider the physical memory and cpu)
2. Add WorkerMonitor and TaskSchedulers in TajoWorker (sparrow like)
3. Add TaskExecutor in TajoWorker
4. Add TaskLoadBalancer in QueryMaster (probe the workers and throttling the
task)
5. Add FrontendQueryScheduler in TajoMaster
I will attach the sequence diagram.
If you have any idea, please feel free to give your opinion.
> (Umbrella) Implement Tajo Query Scheduler
> -----------------------------------------
>
> Key: TAJO-540
> URL: https://issues.apache.org/jira/browse/TAJO-540
> Project: Tajo
> Issue Type: New Feature
> Reporter: Hyunsik Choi
>
> Currently, there is no Tajo query scheduler. So, all queries launched
> simultaneously compete cluster resource which is managed by
> TajoResourceManager.
> In this issue, we will investigate, design, and implement a Tajo query
> scheduler. This is an umbrella issue for that. We will create subtasks for
> them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
