villebro opened a new pull request, #43659:
URL: https://github.com/apache/superset/pull/43659
### SUMMARY
Two related Global Task Framework (GTF) improvements, against the
`gaq-to-gtf` feature branch.
**1. Task List filters**
- The **Type** filter changes from a free-text `contains` match to a
**select dropdown** populated from the distinct task types actually present.
This exposes the `BaseSupersetModelRestApi` `/distinct/<column>` endpoint on
`TaskRestApi` (`allowed_distinct_fields = {"task_type"}`); it applies
`base_filters`, so the dropdown is scoped per-user exactly like the list itself.
- A new **Name** free-text (`contains`) filter is added (`task_name` was
already a `search_column`).
**2. Worker self-fencing on lost metastore contact**
The reaper (shipped in #43627) handles a *dead* worker, but a worker that is
**alive yet cut off from the metastore** (network partition, metastore outage)
would keep running a query the reaper has already — or will shortly — mark
`FAILURE`. This adds the worker-side complement:
- While a worker holds a task it heartbeats `tasks.last_heartbeat`. If those
writes keep failing for longer than `GTF_ORPHAN_TASK_TIMEOUT` — the *same*
window the reaper uses to declare a task orphaned — the worker **fails the task
from the inside**, cancelling any in-flight warehouse query via the existing
abort path (a fresh connection to the warehouse, reachable even when the
metastore is not).
- **A single failed write is tolerated**; only a sustained outage spanning
the orphan window fences, so a transient blip never kills a healthy task.
- There is **no handover** to another worker — the task simply fails (which
is why this keeps the "heartbeat/orphan" framing rather than calling it a
"lease").
- **No schema change**: builds on the existing `last_heartbeat` column and
`GTF_ORPHAN_TASK_TIMEOUT` config.
Implementation notes:
- `TaskContext` gains `trigger_self_fence()` + a `fence_triggered` flag, and
the timeout timer's abort logic is extracted into a shared `_abort_locally()`
helper (so both the timeout path and the fence path go through one code path).
A fence is finalized as `FAILURE` — distinct from a timeout's `TIMED_OUT` and a
user abort's `ABORTED`. `trigger_self_fence` no-ops if an abort is already
underway or the task work already completed.
- `task_heartbeat` yields a `HeartbeatController`; the executor registers
the fence callback once the context is built.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
_Task List "Type" filter is now a dropdown of the distinct task types, plus
a new free-text "Name" filter. (Screenshots to add.)_
### TESTING INSTRUCTIONS
- **Filters:** open the Task List with `GLOBAL_TASK_FRAMEWORK` enabled; the
**Type** filter is a dropdown listing distinct task types (scoped to tasks you
can see), and the **Name** filter does a substring match on the task name.
- **Self-fencing (manual):** with a Celery worker + a cancellable engine
(e.g. Postgres) + `DISTRIBUTED_COORDINATION_CONFIG`, start an async chart
query, then sever the worker's connection to the metastore for longer than
`GTF_ORPHAN_TASK_TIMEOUT`. The worker fails the task from the inside and the
backend query disappears from `pg_stat_activity`; a brief blip shorter than the
window does not fence.
- **Automated:** `pytest tests/unit_tests/tasks/` (heartbeat self-fence,
context `trigger_self_fence`, executor fence-finalization).
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags: `GLOBAL_TASK_FRAMEWORK`
- [x] Changes UI
- [ ] Includes DB Migration
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
