mikebridge commented on code in PR #44851:
URL: https://github.com/apache/superset/pull/44851#discussion_r4159905635
##########
superset/common/query_context_factory.py:
##########
@@ -114,6 +124,12 @@ def create( # pylint: disable=too-many-arguments
custom_cache_timeout=custom_cache_timeout,
cache_values=cache_values,
)
+ if defer_discovery:
+ security_manager.raise_for_access(query_context=context)
+ query: QueryObject
+ for query in queries_:
+ self._process_query_object(datasource_model_instance,
form_data, query)
Review Comment:
The duplicate tooltip claim is covered by
ac2497f6b92619a707677c571f8c6e83d9c003fe: the final chart-data validation still
rejects the completed guest query when tooltip processing adds an unshared
column. The regression exercises the real factory and comparator with refresh
enabled and disabled, before query execution. Persisted semantic guest-chart
support is not claimed.
##########
superset-frontend/src/explore/actions/exploreActions.ts:
##########
@@ -272,6 +284,63 @@ export function syncDatasourceMetadata(datasource:
Dataset) {
return { type: SYNC_DATASOURCE_METADATA, datasource };
}
+/** Refresh only the active view's derived metadata; chart selections stay
intact. */
+export function refreshSemanticMetadata(
+ viewId: number,
+ sessionIsCurrent: () => boolean,
+) {
+ return async (
+ dispatch: Dispatch,
+ getState: () => Pick<ExplorePageState, 'explore'>,
+ ) => {
+ const isCurrent = () => {
+ const { datasource } = getState().explore;
+ return (
+ sessionIsCurrent() &&
+ Number(datasource.id) === viewId &&
+ String(datasource.type) === 'semantic_view'
+ );
+ };
+ if (!isCurrent()) return;
+ // A pre-sync compatibility response cannot replace a post-sync answer.
+ compatibilityRequestSeq += 1;
+ const { json } = await SupersetClient.get({
+ endpoint:
`/fetch_datasource_metadata?datasourceKey=${viewId}__semantic_view`,
+ });
+ if (!isCurrent()) return;
+ const formData = getFormDataFromControls(getState().explore.controls);
+ dispatch(syncDatasourceMetadata(json as Dataset));
+ // Rebuild the controls against fresh fields using their existing values
and
+ // normal removed-member validation, without rewriting form_data or
querying.
+ dispatch(setExploreControls(formData));
Review Comment:
Addressed in d4dd8809a47e0653f6473af871161341fe5ee0a8 with a metadata-only
datasource/control action. A real-store regression first reproduced the phantom
undo/redo entry; unchanged selections now keep an empty session log. A second
red-first case verifies that normal validation removing a selected value
records exactly the actual control change. Saved form_data stays untouched, and
ordinary undo/redo still logs. Final UI validation: 114 tests passed and the
strict source-resolved type check passed.
##########
docs/developer_docs/semantic-metadata-store.md:
##########
@@ -0,0 +1,167 @@
+---
+title: Shared semantic metadata storage
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+## Enablement and scope
+
+This host implementation supports the optional [SDK metadata
contract](./semantic-metadata-contract.md).
+It provides storage and cache identity; it does not add refresh endpoints or
UI.
+Both `SEMANTIC_LAYERS` and `SEMANTIC_LAYER_METADATA_REFRESH_ENABLED` remain off
+by default. A provider must explicitly declare support and supply the adapter
+and captured view token. Legacy providers retain their existing behavior.
+
+Before enabling, configure `DISTRIBUTED_COORDINATION_CONFIG` with Redis or
Redis
+Sentinel, and `SEMANTIC_LAYER_METADATA_NAMESPACE` with a trusted, nonempty
+string or zero-argument callable returning the deployment and tenant namespace.
+Never derive that namespace from unvalidated request fields. The host combines
+it with the stored connection UUID, provider type, configuration and
credentials
+under an HMAC using `SECRET_KEY`; keys do not contain raw credentials.
+
+Enable only on a homogeneous compatible host/provider fleet. Redis rollback or
+restore can resurrect retired entries: change the namespace before re-enabling
+a recovered fleet. This cache does not provide durable cross-failover ordering.
+
+## Publication and separate invalidation
+
+Each successful catalog publication receives a fresh opaque token, even if the
+discovery JSON is unchanged. A discovery digest is not a complete upstream
+semantic-model revision. Hits retain the token and expiry. Failures retain the
+previous observation's original expiry, when it still exists.
+
+A single lease admits one writer; publication atomically compares its owner,
+installs the observation and releases the lease. Catalog invalidation
atomically
+deletes both the observation and the old writer's authority. An expired or
+invalidated writer cannot publish over a newer observation.
+
+Compatibility answers and query results include the token captured with their
+view's metadata and the view configuration. Compatibility also has an
independent
+random generation: clearing compatibility retires all its selection variants
+without fetching metadata or invalidating query results. Late fills retain
their
+old captured key. Existing query/RLS identity and selected-query force refresh
+remain in the query-cache path. No global key scan or upstream cache purge
occurs.
+
+Bounds are a 30-second metadata I/O budget, a non-renewing lease capped by the
+owner’s remaining budget (and at most 60 seconds), a
+300-second catalog lifetime measured from acquisition start, and a 10-MiB
+serialized envelope limit. Cold readers wait and re-read within the same
budget.
+A busy explicit refresh returns `in_progress`; an unknown write outcome returns
+`indeterminate`, not success or a blind retry.
+
+## Operation lifetime and transport
+
+HTTP requests establish the absolute monotonic deadline before authentication
+hooks. Celery tasks establish it before task execution; eager/nested work
shares
+the active operation. Other synchronous host callers must enter
+`metadata_operation()` before access checks. A later store call never
replenishes
+the budget; explicit worker budgets are capped at 30 seconds. The host passes
+that deadline explicitly to `adapter.bind(store, deadline=...)`. The adapter
+passes it to `store.read(fetch, deadline=...)` for discovery and to
+`store.refresh(fetch, deadline=...)` for an explicit refresh. An earlier caller
+deadline narrows both store work and Redis transport; a deadline beyond the
+operation ceiling is rejected. A call never mutates the operation or another
+call's budget. Invalid/exhausted call deadlines fail before even cache-hit
I/O. Access to an
+already-captured layer or view remains valid after that budget expires, so a
+long-running chart query does not lose its observation. Further metadata I/O
+still fails at the original deadline. Provider instances and views are scoped
to that operation, so reusing
+a SQLAlchemy model in a later request cannot reuse an old provider observation.
+
+Private Redis clients use the installed redis-py asyncio transport and one
+cancellation timeout per command, bounded by the operation's remaining time.
+This covers connection setup, Sentinel discovery and response parsing; retries
+are disabled. The synchronous bridge owns and closes each event loop/client,
+without changing shared coordinator pools. An uncancellable system DNS lookup
+may finish in its resolver thread after timeout; the cancelled command cannot
+connect or publish when that lookup finishes. Calling it inside an
already-running
+asyncio loop fails explicitly; async host integrations need a synchronous
worker.
+Provider fetches receive the same absolute deadline and must enforce it in
their
+own transport. Monotonic values are never serialized or used to order
publications.
+
+## Enablement limits
+
+Read authorization remains with the canonical caller policy, using its full
+chart, dashboard, guest-token or datasource context. Model construction must
not
+replace those policies with a narrower datasource permission check.
+
+**Production enablement is blocked on chart-data authorization ordering.** The
Review Comment:
Updated in ac2497f6b92619a707677c571f8c6e83d9c003fe, the layer where the old
prerequisite statement becomes false, and carried into the UI layer. The docs
distinguish implemented authorization ordering and maintenance commands from
remaining deployment configuration, transport/timeouts, topology/load/failover,
async/CLI adaptation and UI/live-provider acceptance checks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
