fitzee opened a new pull request, #42624:
URL: https://github.com/apache/superset/pull/42624
### SUMMARY
Fixes scheduled-dashboard report reliability for DOM-heavy dashboards such
as OpenTable dashboard 805.
The root cause was a vacuous readiness predicate: when React had mounted
zero production chart holders, the browser path warned and continued, allowing
blank/spinner screenshots to be delivered. Independent screenshot, Celery, and
`working_timeout` values also left executions without one authoritative
deadline, while a lost worker could leave `WORKING` blocking later schedules.
This change:
- requires at least one production-safe chart holder before either standard
or tiled dashboard capture; zero holders are polled until timeout, and
readiness failure propagates without a raw screenshot fallback;
- keeps viewport-scoped holder readiness for virtualization and records the
backend chart count as a diagnostic/tiling hint rather than an equality gate;
- introduces one configurable 900-second monotonic report deadline with
capture (60s), delivery (120s), cleanup (30s), and Celery hard-timeout grace
(30s) settings;
- threads that deadline from report execution through navigation, mounting,
readiness, capture/PDF/data generation, and delivery while preserving the
existing task-derived thumbnail budget;
- makes soft-timeout and Celery failure cleanup idempotently promote the
owned `WORKING` log to `ERROR`, bounds stale report recovery to the report
budget, permits a distinct next schedule to proceed, and refuses to replay the
same execution ID to avoid duplicate delivery;
- emits key/value logs containing capture kind,
execution/report/dashboard/chart IDs, URL, expected/mounted/ready holder
counts, elapsed/remaining time, attempt, and terminal reason;
- reuses the production-safe selectors and task-budget helpers from
#42253/#42427 and follows the safe no-fallback direction in #42273. #42118
informed the total-budget design; this implementation centralizes the report
deadline instead of adding another independent constant.
No browser retry was added: Playwright already creates a fresh browser
context per capture, and retrying the same execution ID can duplicate a
delivery when a worker is lost after the provider accepted it.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Not applicable; this is background report orchestration and browser-capture
behavior.
### TESTING INSTRUCTIONS
Automated validation completed:
```bash
pytest -q \
tests/unit_tests/utils/test_report_execution.py \
tests/unit_tests/utils/test_screenshot_utils.py \
tests/unit_tests/utils/webdriver_test.py \
tests/unit_tests/commands/report/execute_test.py \
tests/unit_tests/commands/report/test_execute_now.py \
tests/integration_tests/reports/scheduler_tests.py
# 202 passed
pre-commit run --files <all 16 changed files>
# passed (mypy, Ruff format/check, pylint, and applicable hooks)
```
`pre-commit run --all-files` was also executed. It reached the full
repository and exposed unrelated base/environment failures: 14 existing mypy
errors in version-restore tests, missing frontend `node_modules`/read-only
default npm cache, existing repo-wide PT004/E402 lint findings, and cache/tool
setup failures. No unrelated auto-fixes are included in this branch; the
changed-file pre-commit suite and mypy delta pass.
Staged validation plan on the environment reproducing commit `c5c287ca`:
1. Retain the `c5c287ca` OpenTable dashboard 805 run as the control, then
build this branch in staging without production deployment. Configure the
default 900/60/120/30/30-second values explicitly.
2. Run dashboard 805 (52 charts: 33 `pivot_table_v2`, 19 `table`) as PDF.
Confirm `expected_holders=52`, holder counts progress from zero to a positive
mounted set, every viewport-visible mounted holder reaches a terminal state,
capture occurs only afterward, and the execution reaches `SUCCESS` with one
delivery inside 15 minutes.
3. Reproduce UI saturation/slow mounting. While mounted holders remain zero,
confirm repeated readiness polling, no screenshot/PDF construction, and no
report delivery. Let mounting recover before the readiness sub-budget expires
and confirm the same attempt completes using a fresh browser context.
4. Hold readiness beyond the derived readiness budget (total minus
capture/delivery/cleanup reserves). Confirm timeout propagation, `ERROR`
terminal state/reason, remaining cleanup capacity, and no incomplete capture or
report delivery.
5. Terminate a worker after the execution enters `WORKING`. Confirm the
Celery failure hook marks the owned row `ERROR` when delivered; otherwise
confirm stale recovery at the 15-minute bound. A distinct later execution ID
must proceed in that schedule, while replaying the stale execution ID must
terminate without a duplicate send.
6. Run a dashboard thumbnail and a chart report to verify thumbnails retain
their task-derived 300-second behavior and chart reports use the shared report
deadline.
7. Inspect structured logs for `capture_kind`, `execution_id`,
`report_schedule_id`, `dashboard_id`, `chart_id`, `url`, `expected_holders`,
`mounted_holders`, `ready_holders`, `elapsed_seconds`, `remaining_seconds`,
`attempt`, and `terminal_reason` across success and timeout paths.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
This PR is intentionally draft pending the staged `c5c287ca` plan. It must
not be merged or deployed until that plan is complete.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
