sadpandajoe commented on code in PR #44851:
URL: https://github.com/apache/superset/pull/44851#discussion_r4158896408
##########
superset-frontend/src/explore/actions/exploreActions.ts:
##########
@@ -272,6 +284,63 @@ export function syncDatasourceMetadata(datasource:
Dataset) {
return { type: SYNC_DATASOURCE_METADATA, datasource };
}
+/** Refresh only the active view's derived metadata; chart selections stay
intact. */
+export function refreshSemanticMetadata(
+ viewId: number,
+ sessionIsCurrent: () => boolean,
+) {
+ return async (
+ dispatch: Dispatch,
+ getState: () => Pick<ExplorePageState, 'explore'>,
+ ) => {
+ const isCurrent = () => {
+ const { datasource } = getState().explore;
+ return (
+ sessionIsCurrent() &&
+ Number(datasource.id) === viewId &&
+ String(datasource.type) === 'semantic_view'
+ );
+ };
+ if (!isCurrent()) return;
+ // A pre-sync compatibility response cannot replace a post-sync answer.
+ compatibilityRequestSeq += 1;
+ const { json } = await SupersetClient.get({
+ endpoint:
`/fetch_datasource_metadata?datasourceKey=${viewId}__semantic_view`,
+ });
+ if (!isCurrent()) return;
+ const formData = getFormDataFromControls(getState().explore.controls);
+ dispatch(syncDatasourceMetadata(json as Dataset));
+ // Rebuild the controls against fresh fields using their existing values
and
+ // normal removed-member validation, without rewriting form_data or
querying.
+ dispatch(setExploreControls(formData));
Review Comment:
Syncing metadata on an otherwise unchanged chart now adds an unsaved “Undid
or redid a change” entry: versionSessionLogMiddleware records every
SET_EXPLORE_CONTROLS action, and useVersionActions blocks restore while that
log is nonempty. Could this metadata-only rebuild avoid the user-edit action,
with a real-store regression test asserting unchanged chart selections and an
empty session log after sync?
##########
docs/developer_docs/semantic-metadata-store.md:
##########
@@ -0,0 +1,167 @@
+---
+title: Shared semantic metadata storage
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Enablement and scope
+
+This host implementation supports the optional [SDK metadata
contract](./semantic-metadata-contract.md).
+It provides storage and cache identity; it does not add refresh endpoints or
UI.
+Both `SEMANTIC_LAYERS` and `SEMANTIC_LAYER_METADATA_REFRESH_ENABLED` remain off
+by default. A provider must explicitly declare support and supply the adapter
+and captured view token. Legacy providers retain their existing behavior.
+
+Before enabling, configure `DISTRIBUTED_COORDINATION_CONFIG` with Redis or
Redis
+Sentinel, and `SEMANTIC_LAYER_METADATA_NAMESPACE` with a trusted, nonempty
+string or zero-argument callable returning the deployment and tenant namespace.
+Never derive that namespace from unvalidated request fields. The host combines
+it with the stored connection UUID, provider type, configuration and
credentials
+under an HMAC using `SECRET_KEY`; keys do not contain raw credentials.
+
+Enable only on a homogeneous compatible host/provider fleet. Redis rollback or
+restore can resurrect retired entries: change the namespace before re-enabling
+a recovered fleet. This cache does not provide durable cross-failover ordering.
+
+## Publication and separate invalidation
+
+Each successful catalog publication receives a fresh opaque token, even if the
+discovery JSON is unchanged. A discovery digest is not a complete upstream
+semantic-model revision. Hits retain the token and expiry. Failures retain the
+previous observation's original expiry, when it still exists.
+
+A single lease admits one writer; publication atomically compares its owner,
+installs the observation and releases the lease. Catalog invalidation
atomically
+deletes both the observation and the old writer's authority. An expired or
+invalidated writer cannot publish over a newer observation.
+
+Compatibility answers and query results include the token captured with their
+view's metadata and the view configuration. Compatibility also has an
independent
+random generation: clearing compatibility retires all its selection variants
+without fetching metadata or invalidating query results. Late fills retain
their
+old captured key. Existing query/RLS identity and selected-query force refresh
+remain in the query-cache path. No global key scan or upstream cache purge
occurs.
+
+Bounds are a 30-second metadata I/O budget, a non-renewing lease capped by the
+owner’s remaining budget (and at most 60 seconds), a
+300-second catalog lifetime measured from acquisition start, and a 10-MiB
+serialized envelope limit. Cold readers wait and re-read within the same
budget.
+A busy explicit refresh returns `in_progress`; an unknown write outcome returns
+`indeterminate`, not success or a blind retry.
+
+## Operation lifetime and transport
+
+HTTP requests establish the absolute monotonic deadline before authentication
+hooks. Celery tasks establish it before task execution; eager/nested work
shares
+the active operation. Other synchronous host callers must enter
+`metadata_operation()` before access checks. A later store call never
replenishes
+the budget; explicit worker budgets are capped at 30 seconds. The host passes
+that deadline explicitly to `adapter.bind(store, deadline=...)`. The adapter
+passes it to `store.read(fetch, deadline=...)` for discovery and to
+`store.refresh(fetch, deadline=...)` for an explicit refresh. An earlier caller
+deadline narrows both store work and Redis transport; a deadline beyond the
+operation ceiling is rejected. A call never mutates the operation or another
+call's budget. Invalid/exhausted call deadlines fail before even cache-hit
I/O. Access to an
+already-captured layer or view remains valid after that budget expires, so a
+long-running chart query does not lose its observation. Further metadata I/O
+still fails at the original deadline. Provider instances and views are scoped
to that operation, so reusing
+a SQLAlchemy model in a later request cannot reuse an old provider observation.
+
+Private Redis clients use the installed redis-py asyncio transport and one
+cancellation timeout per command, bounded by the operation's remaining time.
+This covers connection setup, Sentinel discovery and response parsing; retries
+are disabled. The synchronous bridge owns and closes each event loop/client,
+without changing shared coordinator pools. An uncancellable system DNS lookup
+may finish in its resolver thread after timeout; the cancelled command cannot
+connect or publish when that lookup finishes. Calling it inside an
already-running
+asyncio loop fails explicitly; async host integrations need a synchronous
worker.
+Provider fetches receive the same absolute deadline and must enforce it in
their
+own transport. Monotonic values are never serialized or used to order
publications.
+
+## Enablement limits
+
+Read authorization remains with the canonical caller policy, using its full
+chart, dashboard, guest-token or datasource context. Model construction must
not
+replace those policies with a narrower datasource permission check.
+
+**Production enablement is blocked on chart-data authorization ordering.** The
Review Comment:
This still tells operators production enablement is blocked until a future
PR adds the authorization ordering and maintenance commands, but this stack now
includes both and semantic-metadata-operations.md describes the guard as in
place. Could this section distinguish the remaining deployment checks from the
prerequisites already implemented, so the two enablement guides agree?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
