rusackas opened a new pull request, #40084:
URL: https://github.com/apache/superset/pull/40084
### SUMMARY
While developing local Superset extensions inside Docker on macOS, the
hot-reload loop was unreliable:
1. **Spurious reloads.** Docker's VirtioFS / osxfs surfaces inotify events
on every file *read*, not just writes. The watcher's `on_any_event` reacted to
all of them, so every Python import that touched a file under the extension
directory triggered a server restart.
2. **Restart storms during builds.** A single `webpack --watch` rebuild that
wrote N files produced N restarts in quick succession — Flask would begin
restarting before the build finished, and the dev loop would stall.
3. **Recursive triggers.** The watcher touched `superset/__init__.py` to
signal Flask's reloader. Any subsequent read of `__init__.py` looked like
another change → another restart → another read → loop.
4. **Nested extension assets 404'd.** `FRONTEND_REGEX` and the
`/api/v1/extensions/<publisher>/<name>/<file>` route both used single-segment
matchers, so extensions that needed to serve worker / WASM / chunk files in
subdirectories (e.g. DuckDB WASM, Pyodide) couldn't.
### Changes
**`superset/extensions/local_extensions_watcher.py`**
- Only react to `FileCreated`, `FileModified`, and `FileMoved` events (skip
access/open/close).
- Verify the file content actually changed via SHA-256 — first-seen events
are recorded as baseline and don't trigger a restart.
- Debounce: at most one restart per second across all files.
- Switch the trigger from `superset/__init__.py` to a dedicated sentinel
`superset/extensions/.reload_trigger` that Python never imports. Watcher
creates the sentinel on startup.
**`docker/docker-bootstrap.sh`**
- Add `--extra-files /app/superset/extensions/.reload_trigger` so Flask
watches the sentinel.
- Exclude `superset/__init__.py` from the Flask reloader so the previous
trigger path can't reintroduce the loop.
**`docker-compose.yml`**
- Bind-mount `./local_extensions:/app/local_extensions` to match
`LOCAL_EXTENSIONS` in the dev `superset_config_docker.py`.
**`superset/extensions/api.py` + `superset/extensions/utils.py`**
- `FRONTEND_REGEX` accepts nested paths inside `frontend/dist`.
- `/api/v1/extensions/<publisher>/<name>/<path:file>` URL converter accepts
nested paths so extensions can serve worker / WASM / chunk subfolders.
### TESTING INSTRUCTIONS
1. Run `docker compose up`.
2. Place a local extension in `./local_extensions/` and register it via
`LOCAL_EXTENSIONS` in `docker/pythonpath_dev/superset_config_docker.py`.
3. Run `npm run watch` inside the extension's frontend dir.
4. Confirm:
- First load of the extension does **not** trigger a restart (was:
instant restart from baseline inotify events).
- Editing a source file produces exactly **one** restart, ~1s after
webpack finishes writing.
- Reading Python code (e.g. autoreload after editing a non-extension
file) does not trigger an extension-reload cascade.
- Nested extension assets
(`/api/v1/extensions/{publisher}/{name}/subdir/file.wasm`) are served, not 404.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
