mikebridge opened a new pull request, #45544:
URL: https://github.com/apache/superset/pull/45544
### SUMMARY
TL;DR: Web and worker processes use the same semantic-layer cache keys when
they reach one metadata database through different addresses. **Expect a
one-time cache miss for semantic-layer results after deploy when using the
default namespace.** Existing explicit namespaces keep their key scheme.
Previously, host aliases, proxy ports or database aliases could make a
worker write a result that the web process could not find, causing another
async submission. The fallback uses the installation identity already stored in
the active metadata database, rather than the connection address. Explicit
string/callable namespace overrides still win. The lookup uses the caller's
existing connection, is memoized within the metadata session and does not
commit its pending work. Only a missing identity requires an independent
bootstrap transaction.
Update web, worker and MCP roles together. Full database clones copy the
installation identity too: configure distinct explicit namespaces for isolated
workspaces. A metadata session must not span workspaces. Backup restoration
still requires cache clearing or namespace rotation.
Unlike the old address fallback, a full database clone no longer gets a
distinct default cache generation merely from its different database name; the
explicit namespace separation above is a behavior change to account for when
cloning.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Not applicable; backend cache identity only. No live deployment was
exercised.
### TESTING INSTRUCTIONS
Run `tests/unit_tests/semantic_layers/metadata_namespace_test.py` alongside
`simple_cache_clear_test.py`, `execution_context_test.py`, and
`models_test.py`. The regression uses independent metadata engines/sessions,
real durable identities and DATA-cache writes/reads; the API path returns a
cached response without another task submission across address variants.
Additional cases cover a one-connection pool, NullPool connection counts and
missing-row bootstrap without committing caller work.
Local result: 268 tests passed including the native MCP inventory case.
Changed-file hooks and POT drift pass. Independent PostgreSQL probes covered
missing-row bootstrap at READ COMMITTED, REPEATABLE READ and SERIALIZABLE, with
and without pending caller writes. MySQL, live proxy deployments and actual
Celery delivery were not exercised. With MySQL SERIALIZABLE, a missing-row
lookup can take a gap lock and delay independent bootstrap; initialize the
durable identity before entering such a transaction. SQLite fails promptly if
bootstrap would wait on the caller's active transaction.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [x] Required feature flags: `SEMANTIC_LAYERS`
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in SIP-59)
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
