rebenitez1802 opened a new pull request, #45859:
URL: https://github.com/apache/superset/pull/45859
### SUMMARY
On Impala, pressing Stop while SQL Lab was reading a query's results marked
the query `stopped`, but Impala kept executing it and held its admission slot
until it finished. `UPDATING.md` listed this as a known gap of #45232.
This is the common case for a long `SELECT`, not an edge case. Impala
reports a streaming `SELECT` as `FINISHED` as soon as its rows can be fetched:
about 0.1 s after submission on Impala 4.5.2, with or without result spooling.
`handle_cursor` then stops polling, and the rest of the query runs while the
worker is inside `fetch_data`. Aggregations and CTAS stay `RUNNING` while they
work, so polling already covered them.
**Changes**
- **Stop checks during the fetch.** While it reads the results of a SQL Lab
query that `handle_cursor` polled, `fetch_data` checks for a committed stop
between result RPCs, at most once per `DB_POLL_INTERVAL_SECONDS`.
- The stop is applied on the worker's own cursor and thread with
`cancel_operation()`, as #45232 does while polling. There is no HTTP call to
the coordinator and no second connection, so it also works where the web UI is
unreachable or the internal-hosts opt-out is off.
- Impala returns each result RPC within `FETCH_ROWS_TIMEOUT_MS` (10 s by
default) even when no rows are ready. A stop therefore takes effect within
about `FETCH_ROWS_TIMEOUT_MS` + `DB_POLL_INTERVAL_SECONDS`.
- **One more check when polling ends.** `handle_cursor` checks again as it
finishes. This catches a stop committed during the last poll interval, or
before an operation that is already `FINISHED` at its first poll.
- **Failed checks don't fail the query.** If a stop check itself fails, for
example on a metadata database error, the fetch continues: the check rolls back
its transaction and the query is not failed. A time limit that fires during a
check is still raised.
- **The async executor reports these stops as stops.** When a fetch fails
because the query was stopped, the async SQLExecutor task
(`Database.execute_async`) reports the stop. It no longer logs an unhandled
ERROR with a traceback. Genuine failures keep the SQL of the statement that
failed.
- **Unchanged:** charts, MCP and any other fetch that `handle_cursor` did
not poll.
**Known limit.** Impala checks for cancellation between row batches. A
single batch whose rows are slow to compute, such as an expensive function over
one batch, runs to the end of that batch first. This is documented.
**Overlap with draft #45845.** It changes the same Impala engine spec and
tests, and replaces the `SoftTimeLimitExceeded` import with
`TIME_LIMIT_EXCEPTIONS`. Whichever PR lands second needs to adapt to the other.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Not applicable; there are no UI changes. Here is the backend behaviour on
Impala 4.5.2, for a Stop sent 5 s into the fetch of `SET BATCH_SIZE=1; SELECT
sleep(1000) AS s, x FROM <200-row table>`:
| | Before | After |
| --- | --- | --- |
| Superset outcome | `stopped` | `stopped`, no rows, no error |
| Impala | keeps executing the query and holds its admission slot until it
finishes | stops executing the query and releases the slot |
### TESTING INSTRUCTIONS
**Unit tests.** They use a fake impyla cursor and need no driver.
```
pytest tests/unit_tests/db_engine_specs/test_impala.py \
tests/unit_tests/db_engine_specs/test_impala_pending.py \
tests/unit_tests/sql/execution/test_celery_task.py
```
**Real cluster.** These results come from the public Apache Impala 4.5.2
quickstart (`docker/quickstart.yml` at tag `4.5.2` of apache/impala), with
PostgreSQL metadata, a separate Celery worker,
`IMPALA_CANCEL_QUERY_ALLOW_INTERNAL_HOSTS = False`, and
`DB_POLL_INTERVAL_SECONDS = {"impala": 0.5}`. At the default interval of 5 s,
release takes up to about 5 s longer.
| Path | Fetch-phase Stop (3 runs each) | Released after Stop |
| --- | --- | --- |
| SQL Lab sync | `stopped`, 0 rows, no error | 5.4–5.6 s |
| SQL Lab async (separate worker) | `stopped` | 5.4–5.6 s |
| `Database.execute_async` + `cancel()` | `STOPPED`, no worker ERROR log |
5.4–5.6 s |
Each Stop was sent 5 s into the fetch, and Impala released the query at the
end of the 10 s result RPC in flight. The following were unchanged:
- normal success on all three paths;
- a Stop while `RUNNING` (released within 0.4–0.7 s);
- a Stop while held in admission;
- a Stop after the query completed, which keeps its `success` result.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
