[
https://issues.apache.org/jira/browse/TAJO-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359478#comment-14359478
]
Hyunsik Choi commented on TAJO-540:
-----------------------------------
In general, query (or job) scheduler aims at the maximum resource utilization.
For multi-tenancy, we also need to consider the fairness for multiple users (or
queries). BTW, the maximum resource utilization and fairness are usually
conflict to each other in many cases. To mitigate this problem, many scheduler
seems to use preemption approach.
In this point, our resource and scheduler system has the following problems:
* A query exclusively uses allocated resources at the first time until the
query is completed or failed.
* There is no mechanism to deallocate resources during query processing.
* Preempt is also not allowed.
To achieve the multi tenancy, we should change our resource circulation.
Especially, resource allocation must be fine grained instead of per query.
So, I'll create a jira issue to change the resource circulation. We have to do
this issue firstly in my opinion.
> (Umbrella) Implement Tajo Query Scheduler
> -----------------------------------------
>
> Key: TAJO-540
> URL: https://issues.apache.org/jira/browse/TAJO-540
> Project: Tajo
> Issue Type: New Feature
> Reporter: Hyunsik Choi
>
> Currently, there is no Tajo query scheduler. So, all queries launched
> simultaneously compete cluster resource which is managed by
> TajoResourceManager.
> In this issue, we will investigate, design, and implement a Tajo query
> scheduler. This is an umbrella issue for that. We will create subtasks for
> them.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
