[
https://issues.apache.org/jira/browse/TAJO-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14631293#comment-14631293
]
ASF GitHub Bot commented on TAJO-1397:
--------------------------------------
Github user hyunsik commented on a diff in the pull request:
https://github.com/apache/tajo/pull/608#discussion_r34887013
--- Diff:
tajo-core/src/main/java/org/apache/tajo/engine/query/TaskRequest.java ---
@@ -21,32 +21,31 @@
*/
package org.apache.tajo.engine.query;
+import org.apache.tajo.ResourceProtos.TaskRequestProto;
import org.apache.tajo.TaskAttemptId;
import org.apache.tajo.catalog.proto.CatalogProtos;
import org.apache.tajo.common.ProtoObject;
import org.apache.tajo.conf.TajoConf;
import org.apache.tajo.engine.planner.enforce.Enforcer;
import org.apache.tajo.engine.planner.global.DataChannel;
-import org.apache.tajo.ipc.TajoWorkerProtocol;
import org.apache.tajo.plan.serder.PlanProto;
import org.apache.tajo.worker.FetchImpl;
import java.util.List;
-public interface TaskRequest extends
ProtoObject<TajoWorkerProtocol.TaskRequestProto> {
+public interface TaskRequest extends ProtoObject<TaskRequestProto> {
- public TaskAttemptId getId();
- public List<CatalogProtos.FragmentProto> getFragments();
- public String getOutputTableId();
- public boolean isClusteredOutput();
- public PlanProto.LogicalNodeTree getPlan();
- public boolean isInterQuery();
- public void setInterQuery();
- public void addFetch(String name, FetchImpl fetch);
- public List<FetchImpl> getFetches();
- public boolean shouldDie();
- public void setShouldDie();
- public QueryContext getQueryContext(TajoConf conf);
- public DataChannel getDataChannel();
- public Enforcer getEnforcer();
+ String getQueryMasterHostAndPort();
+ TaskAttemptId getId();
+ List<CatalogProtos.FragmentProto> getFragments();
+ String getOutputTableId();
--- End diff --
unused method
> Resource allocation should be fine grained.
> -------------------------------------------
>
> Key: TAJO-1397
> URL: https://issues.apache.org/jira/browse/TAJO-1397
> Project: Tajo
> Issue Type: Improvement
> Components: QueryMaster, resource manager, Worker
> Reporter: Hyunsik Choi
> Assignee: Jinho Kim
> Fix For: 0.11.0
>
> Attachments: ResoruceSequence.jpg, TAJO-1397.patch,
> TAJO-1397_2.patch, TAJO-1397_3.patch, TAJO-1397_4.patch,
> old_resource_circuit.png, resource_circuit.png
>
>
> See the comment:
> https://issues.apache.org/jira/browse/TAJO-540?focusedCommentId=14359478&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14359478
> From the discussion in TAJO-540
> {quote}
> In general, query (or job) scheduler aims at the maximum resource
> utilization. For multi-tenancy, we also need to consider the fairness for
> multiple users (or queries). BTW, the maximum resource utilization and
> fairness are usually conflict to each other in many cases. To mitigate this
> problem, many scheduler seems to use preemption approach.
> In this point, our resource and scheduler system has the following problems:
> * A query exclusively uses allocated resources at the first time until the
> query is completed or failed.
> * There is no mechanism to deallocate resources during query processing.
> * Preempt is also not allowed.
> To achieve the multi tenancy, we should change our resource circulation.
> Especially, resource allocation must be fine grained instead of per query.
> So, I'll create a jira issue to change the resource circulation. We have to
> do this issue firstly in my opinion. If we achieve this, implementing
> multi-tenant scheduler would be much easier than now. It would be a good
> starting point of this issue.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
