GitHub user kunwp1 created a discussion: Admin dashboard: monitoring and
management of computing units
### Background
I manage a Kubernetes-based Texera deployment on a server with a limited amount
of hardware resources. Users create computing units (CUs) with their own specs
(CPU cores, memory, GPU), and the cluster capacity is shared across everyone.
### The problem
I regularly get into this situation: a user wants to create a CU with a certain
spec, but the cluster does not have enough free resources, so the creation
fails. To unblock them, I need to find CUs that are not being used and
terminate one, but the admin dashboard gives me no way to do any of this.
What I have to do today, entirely outside Texera:
1. SSH into the server and use `kubectl` to list the CU pods and inspect their
specs (cores, memory, GPU).
2. Query the `workflow_computing_unit` table in the database to figure out
which user owns each CU (the `uid` column).
3. Only then do I have enough information to decide which CU to terminate.
Termination is also awkward for an admin. The existing terminate endpoint
(`DELETE /api/computing-unit/{cuid}/terminate`) does handle cleanup properly:
it deletes the pod *and* sets `terminate_time` in the database, but it is
strictly owner-only. So as an admin, my only option is deleting the pod via
`kubectl`, which leaves the database record marked as active until the lazy
reconciliation in the list endpoint runs and that only happens the next time a
user with access to that CU lists their units. There is currently no background
job reconciling the database with the cluster state.
We have an ongoing discussion
(https://github.com/apache/texera/discussions/6264) and PR
(https://github.com/apache/texera/pull/6046) to automatically terminate the CUs
if the CUs are idle, but I believe there should also be a direct path for the
admin to terminate CUs easily.
### Current state of the codebase
- The admin dashboard has four views (Users, Executions, Gmail, Settings). None
cover computing units.
- There is no admin API for listing CUs across users; `GET /api/computing-unit`
only returns the caller's own and shared units.
- Termination is owner-only with no admin override.
- DB↔cluster reconciliation is lazy (piggybacks on the list endpoint), so
direct pod deletions leave stale records.
### Proposal
Add an **Admin Computing Units** view to the admin dashboard, alongside the
existing Users/Executions views:
1. **Monitoring**: a table of all CUs across all users: name, owner, type
(kubernetes/local), status, creation time, and resource specs (CPU, memory,
GPU, JVM memory, shm size). All of this already exists in
`workflow_computing_unit`; it just needs an admin-scoped endpoint and UI.
Ideally it would also show current resource usage, like the user-facing CU list
already does. Additionally, we can add one more view that shows what's the
total available resources in the deployment and what percentage and numbers are
currently being used by users, what's the remaining resources available, etc.
2. **Management**: an admin terminate action that reuses the existing
termination path (pod deletion + `terminate_time` update), enabled by an admin
override in the authorization check.
3. **Deployment-level resource overview**: a view showing the total hardware
capacity of the deployment, how much of it is currently allocated to users' CUs
(in both absolute numbers and percentages), and how much remains available, so
an admin can immediately see whether a requested spec can fit and which CUs are
the best candidates to reclaim.
Implementation-wise this could follow the existing pattern of
`AdminUserResource` / `AdminExecutionResource` on the backend, plus a new admin
component on the frontend mirroring the admin execution table.
Happy to work on this if the direction sounds reasonable to the maintainers.
### Open questions
- Should admin-initiated termination notify the CU owner, and/or require a
reason that gets logged?
*Generated-by: Claude Code (reviewed and edited by the author)*
GitHub link: https://github.com/apache/texera/discussions/6322
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
