[
https://issues.apache.org/jira/browse/TAJO-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966953#comment-13966953
]
Hyunsik Choi commented on TAJO-540:
-----------------------------------
Hi [~coderplay],
Thank you for sharing nice articles. I'm very sorry for late response due to
preparing the 0.8.0 release work and the work as an employee. Also, for one
month, I've investigated the background of this area that you listed.
I'd like to discuss some questions and suggestions that you threw. First of
all, I agree with your suggestion for porting sparrow to Tajo. Now, we need the
query scheduler to well support multiple users and multiple running queries. As
you mentioned, sparrow is proper to these requirements while it is low latency.
Also, you concerned with the way we use Yarn, and you propose two ways. One of
them is that we use yarn as higher layer resource management and sparrow-like
scheduler as a lower layer scheduler. I'd like to throw +1 for this suggestion.
One of problems you concerned was that this approach will cause the radical
change. Especially, there are two-types schedulers. The many dependent modules
make this work very hard.
So, I suggest to comment out all TajoYarnResourceManager and the scheduler code
related to Yarn. Then, we only focus on the sparrow-like scheduler for
standalone scheduler. Later, we can recover the yarn scheduler. I think that
this way makes our work more faster.
What do you think about my proposal? After we get some agreement, we can
discuss several stages for this work. I'm looking forward to your response.
> (Umbrella) Implement Tajo Query Scheduler
> -----------------------------------------
>
> Key: TAJO-540
> URL: https://issues.apache.org/jira/browse/TAJO-540
> Project: Tajo
> Issue Type: New Feature
> Reporter: Hyunsik Choi
> Assignee: Min Zhou
> Fix For: 1.0-incubating
>
>
> Currently, there is no Tajo query scheduler. So, all queries launched
> simultaneously compete cluster resource which is managed by
> TajoResourceManager.
> In this issue, we will investigate, design, and implement a Tajo query
> scheduler. This is an umbrella issue for that. We will create subtasks for
> them.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
