eschutho opened a new pull request, #41935:
URL: https://github.com/apache/superset/pull/41935
## What warning
Datadog showed the third-party `UserWarning: pkg_resources is deprecated as
an API` (raised from `sqlalchemy-redshift`'s own `__init__.py`, which still
does `from pkg_resources import ...`) firing at high volume (~76/day) in
production logs, split across the main web/celery process (py3.10) and
`superset/mcp_service` (a separate py3.11 process).
There's already an attempted suppression for this in
`superset/db_engine_specs/redshift.py` (`warnings.filterwarnings("ignore",
message=r"pkg_resources is deprecated as an API")`), with a comment reasoning
that `Database.db_engine_spec` is always accessed (which imports this module)
before `create_engine()` ever triggers SQLAlchemy's lazy dialect import of
`sqlalchemy_redshift`. Despite that, the warning is still firing in production,
so the existing fix has a gap somewhere in the ordering.
## What changed
- `superset/mcp_service/__init__.py` + `superset/mcp_service/server.py`:
added the same filter, mirroring the *exact* existing pattern already used for
the `authlib.jose` deprecation warning in these same two files (mcp_service is
a separate process/import graph from the main app and had zero suppression for
this specific warning).
- `superset/utils/logging_configurator.py`: registered the same filter
unconditionally in `DefaultLoggingConfigurator.configure_logging()`, which runs
first thing in `SupersetAppInitializer.init_app()` (both the web app and celery
workers go through `create_app()` → `init_app()`). This is a more robust,
order-independent suppression point that doesn't depend on
`Database.db_engine_spec` having been accessed first.
- `superset/db_engine_specs/redshift.py`: left the original filter in place
(harmless, idempotent) and updated its comment to point at the new primary
suppression location.
## No behavior change
Purely additive `warnings.filterwarnings("ignore", ...)` calls — no change
to any request/response behavior, just suppressing log noise from a warning
that carries no actionable signal (the underlying `sqlalchemy-redshift` pin
can't move past `<0.9` to the `pkg_resources`-free `1.0.0` release without a
SQLAlchemy 2.0 migration, per the existing comment referencing #39750).
## Test plan
- `python3 -m py_compile` on all 4 changed files — clean.
- `ruff check` — clean.
- Verified live in a Python shell that registering the exact filter added
here suppresses the literal warning text as raised by the installed
`pkg_resources`/setuptools 80.x package.
- Confirmed via code trace that celery's worker entrypoint
(`superset/tasks/celery_app.py`) calls `create_app()`, so it goes through the
same `init_app()` → `configure_logging()` path as the web process.
- Grepped the repo for other `sqlalchemy_redshift`/`create_engine(`
touchpoints — no additional code paths need their own filter.
- Existing unit tests unaffected (no assertions cover this specific filter
today, consistent with existing coverage of the `authlib.jose` filter it
mirrors).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
