GitHub user yrenat edited a comment on the discussion: Proposal: Automatically terminate idle Kubernetes Computing Units
@chenlica I have done some researches and my current understanding is that other systems have related concepts, but not exactly the same idea. In Texera, a CU is kept with information such as owner `uid`, `cuid`, name, type, URI, resource limits, and creation/termination state. Workflow executions reference a CU through `workflow_executions.cuid`. A CU also has access control and sharing through `computing_unit_user_access`. It is selected by the user from the frontend, and the access-control service routes requests to the CU's recorded URI. Therefore, the CU combines several concepts: - A user-facing execution environment - A resource profile - A runtime endpoint - ACL and shareability - Workflow-execution affinity - Lifecycle state Other systems usually split these responsibilities across lower-level concepts: | System | Differences| | --- | --- | | Apache Spark | Spark applications run as a driver plus executors acquired from a cluster manager such as Standalone, YARN, or Kubernetes. Each application gets its own executors, but these are not usually reusable, user-selected, shareable workspace-like resources. See [Spark Cluster Overview](https://spark.apache.org/docs/latest/cluster-overview.html). | | Apache Flink | Flink has Application Mode and Session Mode. Application Mode creates a cluster per submitted application, while Session Mode shares one cluster across multiple applications or jobs. This is close in terms of resource isolation, but users submit jobs to a cluster or session rather than selecting a persistent Texera-style CU object. See [Flink Deployment Overview](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/overview/). | | Ray | Ray clusters consist of worker nodes connected to a head node and may autoscale based on application resource requests. The main resource is the Ray cluster rather than a Texera-style per-user CU with DB identity, ACL, workflow binding, and gateway routing. See [Ray Cluster Overview](https://docs.ray.io/en/latest/cluster/getting-started.html). | | Dask | `dask.distributed` uses a central scheduler coordinating workers and clients. Users connect a Python session to the scheduler and submit work. This is similar to an interactive distributed execution environment, but the abstraction is scheduler/worker/client rather than a first-class shareable workflow execution unit. See [Dask Distributed](https://distributed.dask.org/en/stable/). | | JupyterHub | I think this shares the most similarity with our Texera platform. JupyterHub authenticates users, spawns a single-user server, configures proxy routing, and can cull idle servers. However, it is notebook/session-oriented rather than workflow-engine-oriented. See [JupyterHub Concepts](https://jupyterhub.readthedocs.io/en/stable/explanation/concepts.html). | So I would say that technically Texera's CU is not completely unique. However, it does sound unique because it is integrated directly into the workflow system and treated as a user-visible, shareable, routable, and persistent resource. Pros: 1. Users can reuse a warm execution environment, reducing startup latency. 2. Resource limits and runtime settings are attached to a named, user-facing object. 3. Texera can define idleness using workflow semantics rather than only CPU or memory metrics (like the PR that I am working on: https://github.com/apache/texera/pull/6046). 4. CU lifecycle management can keep DB state, pod state, access control, and gateway routing consistent. 5. Sharing and permissions can be handled at the Texera resource level. Cons: 1. Generic Kubernetes autoscaling or TTL mechanisms do not understand CU semantics. Which means that we have to build things on top of Kubernetes, increasing latency and redundancy. 2. Texera has to maintain lifecycle reconciliation logic between DB records and pods. 3. Idle cleanup is product-specific and must carefully handle workflow results and users who leave and return later. Again, that PR https://github.com/apache/texera/pull/6046 4. The abstraction adds complexity because UI selection, workflow execution, access control, routing, and pod management all depend on `cuid`. Therefore, for this idle-cleanup feature, I think cleanup should happen through Texera's CU manager rather than directly through Kubernetes. Only Texera knows whether a CU is idle according to workflow state and can update both the runtime resource and Texera's DB state consistently (see the first Con listed above). GitHub link: https://github.com/apache/texera/discussions/6264#discussioncomment-17635671 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
