[
https://issues.apache.org/jira/browse/TEZ-4662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated TEZ-4662:
------------------------------
Description:
The primary goal is to decouple Tez from YARN by implementing or integrating
any functionality that YARN currently provides to Tez.
*First Approach*: Containers scheduled by an external framework
In this model, Tez has no control over container scheduling.
Mock ContainerLauncher:
Assume containers are started independently and later discovered by the
ApplicationMaster (AM).
Container Discovery:
If task containers are launched independently, a discovery mechanism must be
implemented — for example, using Kubernetes or ZooKeeper.
Resource Management:
(To be defined — assumes limited or no direct control by Tez over resource
allocation.)
*Advanced Approach*: Pluggable Resource Management Framework
Develop a pluggable resource management layer that can integrate with
Kubernetes-based systems (or others) to perform the same responsibilities that
YARN currently handles for Tez.
was:
The main goal here is to decouple from Yarn, so implement or integrate anything
that Yarn currently gives to Tez.
1. Mock ContainerLauncher: first approach assumes that containers are started
independently and discovered by AM.
2. Container discovery: if task containers are launched independently,
container discovery should be implemented (Kubernetes, Zookeeper).
3. Resource management: assuming that
> Cloud-native Tez
> ----------------
>
> Key: TEZ-4662
> URL: https://issues.apache.org/jira/browse/TEZ-4662
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: László Bodor
> Priority: Major
> Labels: Kubernetes
>
> The primary goal is to decouple Tez from YARN by implementing or integrating
> any functionality that YARN currently provides to Tez.
> *First Approach*: Containers scheduled by an external framework
> In this model, Tez has no control over container scheduling.
> Mock ContainerLauncher:
> Assume containers are started independently and later discovered by the
> ApplicationMaster (AM).
> Container Discovery:
> If task containers are launched independently, a discovery mechanism must be
> implemented — for example, using Kubernetes or ZooKeeper.
> Resource Management:
> (To be defined — assumes limited or no direct control by Tez over resource
> allocation.)
> *Advanced Approach*: Pluggable Resource Management Framework
> Develop a pluggable resource management layer that can integrate with
> Kubernetes-based systems (or others) to perform the same responsibilities
> that YARN currently handles for Tez.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
