This is an automated email from the ASF dual-hosted git repository.
aminghadersohi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/superset.git
The following commit(s) were added to refs/heads/master by this push:
new 6d97cde59a6 fix(semantic-layers): export/import semantic-view charts
by typed reference (#44396)
6d97cde59a6 is described below
commit 6d97cde59a6ab74cc84db168b2ace0c86feee20a
Author: Mike Bridge <[email protected]>
AuthorDate: Thu Oct 1 23:09:51 2026 -0600
fix(semantic-layers): export/import semantic-view charts by typed reference
(#44396)
---
UPDATING.md | 20 +
.../importing-exporting-datasources.mdx | 32 +
docs/static/resources/openapi.json | 5 +-
superset/charts/api.py | 3 +
superset/charts/schemas.py | 15 +-
superset/commands/chart/export.py | 6 +-
superset/commands/chart/importers/v1/__init__.py | 29 +-
superset/commands/dashboard/export.py | 3 +
superset/commands/dashboard/export_example.py | 31 +
.../commands/dashboard/importers/v1/__init__.py | 26 +-
superset/commands/importers/v1/assets.py | 32 +-
superset/commands/importers/v1/examples.py | 19 +
superset/dashboards/api.py | 13 +-
superset/semantic_layers/bundle_schemas.py | 36 +
superset/semantic_layers/import_export.py | 240 ++++++
superset/translations/messages.pot | 3 +
.../commands/chart/semantic_export_test.py | 127 +++
.../dashboard/export_example_semantic_test.py | 175 ++++
.../semantic_layers/import_export_test.py | 915 +++++++++++++++++++++
19 files changed, 1703 insertions(+), 27 deletions(-)
diff --git a/UPDATING.md b/UPDATING.md
index cecf631618b..fe6c9669c8f 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -24,6 +24,26 @@ assists people when migrating to a new version.
## Next
+- Example export (`/export_as_example/`) rejects dashboards whose charts or
+ native-filter targets use semantic views; use the ordinary chart/dashboard
+ bundle export instead.
+
+- Semantic-view chart and dashboard bundles use typed UUID references and
require
+ a destination with support for this format and an already provisioned,
accessible
+ view with the same UUID. They do not provision semantic layers/views or
export
+ their configuration or credentials. Older readers cannot import these
semantic
+ bundles. Older dashboard bundles with semantic filter targets but no typed
+ `datasourceRef` are also rejected; re-export them with this version rather
than
+ binding their local IDs to unrelated destination tables. Imports fail
explicitly
+ if a semantic dependency is missing, inaccessible, feature-disabled or
+ provider-unregistered. Chart and dashboard exports omit references for
missing
+ views so unrelated assets can still be archived; orphaned charts and
dashboard
+ targets cannot be imported without repairing their datasources. Existing-view
+ access, feature and provider checks remain enforced, including in full asset
+ exports. Ordinary table bundles retain their existing format. The examples
+ loader rejects semantic bundles;
+ use the chart, dashboard or assets importer instead.
+
### Apache Doris connection form and `DBS_AVAILABLE_DENYLIST`
`DBS_AVAILABLE_DENYLIST` is matched against an engine spec's `default_driver`.
diff --git a/docs/admin_docs/configuration/importing-exporting-datasources.mdx
b/docs/admin_docs/configuration/importing-exporting-datasources.mdx
index 28dfe285a9b..21c224680a2 100644
--- a/docs/admin_docs/configuration/importing-exporting-datasources.mdx
+++ b/docs/admin_docs/configuration/importing-exporting-datasources.mdx
@@ -112,6 +112,38 @@ When importing a dashboard ZIP with the **overwrite**
option enabled, any existi
If you import without the overwrite flag, existing charts with conflicting
UUIDs are left unchanged and the import skips those objects. Use overwrite when
you want to push a fully updated dashboard (including chart definitions) from a
development or staging environment to production.
+## Charts and dashboards using semantic views
+
+Semantic charts and dashboard filter/display-control targets can be restored
against
+an **existing semantic view with the same UUID**. The destination must enable
+`SEMANTIC_LAYERS`, register the view's provider, and grant the importing user
access
+to the view or its parent semantic layer, in addition to the usual asset import
+permissions. Missing, inaccessible or disabled dependencies fail the import
rather
+than producing a partially imported bundle.
+
+Chart bundles identify these sources with `datasource_ref: {type:
semantic_view,
+uuid: <view-uuid>}`. Dashboard targets use `datasourceRef` with the same shape.
+Import resolves the UUID and rewrites local datasource IDs; it never
substitutes a
+table with the same numeric ID. Ordinary table bundles retain `dataset_uuid`.
+Older Superset readers that do not support semantic references cannot import
these
+semantic bundles.
+
+Older dashboard bundles whose semantic filter targets lack `datasourceRef` are
+rejected rather than binding their local IDs to unrelated tables. Re-export
those
+dashboards with a version that supports typed references. If a chart or
dashboard
+filter target references a deleted semantic view, export omits that reference
so
+other assets in the archive can still be exported. Repair the orphaned
datasource
+before exporting it for import elsewhere. Checks for inaccessible, disabled,
and
+unregistered existing semantic dependencies remain enforced, including in full
+asset exports. Disabling `SEMANTIC_LAYERS` or removing a provider registration
+therefore still blocks exports containing references to existing semantic
views.
+
+This is **reference restoration, not semantic-layer provisioning**. Bundles do
not
+include semantic-layer/view configuration or credentials, and importing does
not
+create provider connections or views, or contact providers. Provision the
matching
+view and access grants separately before importing. Full cross-workspace
semantic
+provisioning is outside this workflow.
+
## UUIDs in API Responses
The REST API POST endpoints for **datasets**, **charts**, and **dashboards**
include the auto-generated `uuid` field in the response body:
diff --git a/docs/static/resources/openapi.json
b/docs/static/resources/openapi.json
index b6e69b83a9e..ba10dc75580 100644
--- a/docs/static/resources/openapi.json
+++ b/docs/static/resources/openapi.json
@@ -19732,7 +19732,7 @@
},
"/api/v1/dashboard/{pk}/export_as_example/": {
"get": {
- "description": "Exports a dashboard with its charts and datasets in
the example format used by the Superset example loading system. The export
includes Parquet data files and YAML configuration files.",
+ "description": "Exports a dashboard with its charts and datasets in
the example format used by the Superset example loading system. The export
includes Parquet data files and YAML configuration files. Charts and
native-filter targets that use semantic views are not supported.",
"parameters": [
{
"description": "The dashboard id",
@@ -19782,6 +19782,9 @@
"404": {
"$ref": "#/components/responses/404"
},
+ "422": {
+ "description": "The dashboard contains unsupported semantic-view
assets"
+ },
"500": {
"$ref": "#/components/responses/500"
}
diff --git a/superset/charts/api.py b/superset/charts/api.py
index 113c477bad9..266d5126773 100644
--- a/superset/charts/api.py
+++ b/superset/charts/api.py
@@ -115,6 +115,7 @@ from superset.security.manager import (
get_extra_editor_subject_ids,
get_extra_editors_by_pk,
)
+from superset.semantic_layers.import_export import SemanticReferenceError
from superset.subjects.filters import (
FilterRelatedSubjects,
subject_type_filter,
@@ -1432,6 +1433,8 @@ class ChartRestApi(SoftDeleteApiMixin,
BaseSupersetModelRestApi):
)
except ChartNotFoundError:
return self.response_404()
+ except SemanticReferenceError as ex:
+ return self.response(ex.status, message=ex.message)
buf.seek(0)
return send_export_zip(buf, filename)
diff --git a/superset/charts/schemas.py b/superset/charts/schemas.py
index ca10e5718cf..7f9bb4156a8 100644
--- a/superset/charts/schemas.py
+++ b/superset/charts/schemas.py
@@ -29,6 +29,7 @@ from marshmallow import (
Schema,
validate,
validates,
+ validates_schema,
ValidationError,
)
from marshmallow.validate import Length, NoneOf, Range
@@ -37,6 +38,7 @@ from marshmallow_union import Union
from superset.common.chart_data import ChartDataResultFormat,
ChartDataResultType
from superset.common.chart_data_timing import CHART_DATA_TIMING_VERSION
from superset.db_engine_specs.base import builtin_time_grains
+from superset.semantic_layers.bundle_schemas import SemanticViewReferenceSchema
from superset.subjects.schemas import SubjectResponseSchema
from superset.tags.models import TagType
from superset.utils import pandas_postprocessing, schema as utils
@@ -2020,12 +2022,23 @@ class ImportV1ChartSchema(Schema):
cache_timeout = fields.Integer(allow_none=True)
uuid = fields.UUID(required=True)
version = fields.String(required=True)
- dataset_uuid = fields.UUID(required=True)
+ dataset_uuid: fields.UUID = fields.UUID()
+ datasource_ref: fields.Nested = fields.Nested(SemanticViewReferenceSchema)
is_managed_externally = fields.Boolean(allow_none=True, dump_default=False)
external_url = fields.String(allow_none=True,
validate=utils.validate_external_url)
tags = fields.List(fields.String(), allow_none=True)
extra = fields.Dict(allow_none=True, load_only=True)
+ @validates_schema
+ def validate_datasource_reference(
+ self, data: dict[str, Any], **kwargs: Any
+ ) -> None:
+ """Require exactly one legacy table or typed semantic reference."""
+ if ("dataset_uuid" in data) == ("datasource_ref" in data):
+ raise ValidationError(
+ "Specify exactly one of dataset_uuid and datasource_ref."
+ )
+
class ChartCacheWarmUpRequestSchema(Schema):
chart_id = fields.Integer(
diff --git a/superset/commands/chart/export.py
b/superset/commands/chart/export.py
index 83ab5ddf7ee..8de3b96b70f 100644
--- a/superset/commands/chart/export.py
+++ b/superset/commands/chart/export.py
@@ -31,6 +31,7 @@ from superset.commands.export.models import (
)
from superset.commands.tag.export import ExportTagsCommand
from superset.models.slice import Slice
+from superset.semantic_layers.import_export import export_view_reference
from superset.tags.models import TagType
from superset.utils.dict_import_export import EXPORT_VERSION
from superset.utils.file import get_filename
@@ -74,7 +75,10 @@ class ExportChartsCommand(ExportModelsCommand):
logger.info("Unable to decode `params` field: %s",
payload["params"])
payload["version"] = EXPORT_VERSION
- if model.table:
+ if model.datasource_type == "semantic_view":
+ if model.semantic_view:
+ payload["datasource_ref"] =
export_view_reference(model.semantic_view)
+ elif model.table:
payload["dataset_uuid"] = str(model.table.uuid)
# Fetch tags from the database if TAGGING_SYSTEM is enabled
diff --git a/superset/commands/chart/importers/v1/__init__.py
b/superset/commands/chart/importers/v1/__init__.py
index 6340c713f4e..297413907da 100644
--- a/superset/commands/chart/importers/v1/__init__.py
+++ b/superset/commands/chart/importers/v1/__init__.py
@@ -35,6 +35,10 @@ from superset.daos.chart import ChartDAO
from superset.databases.schemas import ImportV1DatabaseSchema
from superset.datasets.schemas import ImportV1DatasetSchema
from superset.extensions import feature_flag_manager
+from superset.semantic_layers.import_export import (
+ consume_chart_semantic_reference,
+ resolve_bundle_references,
+)
from superset.subjects.utils import get_default_viewers_for_current_user
@@ -59,10 +63,11 @@ class ImportChartsCommand(ImportModelsCommand):
contents: dict[str, Any] | None = None,
) -> None:
contents = {} if contents is None else contents
+ semantic_info: dict[str, dict[str, Any]] =
resolve_bundle_references(configs)
# discover datasets associated with charts
dataset_uuids: set[str] = set()
for file_name, config in configs.items():
- if file_name.startswith("charts/"):
+ if file_name.startswith("charts/") and "dataset_uuid" in config:
dataset_uuids.add(config["dataset_uuid"])
# discover databases associated with datasets
@@ -86,7 +91,7 @@ class ImportChartsCommand(ImportModelsCommand):
and config["database_uuid"] in database_ids
):
config["database_id"] = database_ids[config["database_uuid"]]
- dataset = import_dataset(config, overwrite=False)
+ dataset: SqlaTable = import_dataset(config, overwrite=False)
# Key on the bundle's own uuid, which is what the bundle's
# charts reference. An import that resolves onto an existing
# dataset by physical identity returns a row whose uuid
@@ -99,18 +104,24 @@ class ImportChartsCommand(ImportModelsCommand):
# import charts with the correct parent ref
for file_name, config in configs.items():
- if file_name.startswith("charts/") and config["dataset_uuid"] in
datasets:
+ if file_name.startswith("charts/") and (
+ "datasource_ref" in config or config.get("dataset_uuid") in
datasets
+ ):
# Ignore obsolete filter-box charts.
if config["viz_type"] == "filter_box":
continue
# update datasource id, type, and name
- dataset = datasets[config["dataset_uuid"]]
- dataset_dict = {
- "datasource_id": dataset.id,
- "datasource_type": "table",
- "datasource_name": dataset.table_name,
- }
+ dataset_dict: dict[str, Any] | None =
consume_chart_semantic_reference(
+ config, semantic_info
+ )
+ if dataset_dict is None:
+ dataset = datasets[config["dataset_uuid"]]
+ dataset_dict = {
+ "datasource_id": dataset.id,
+ "datasource_type": "table",
+ "datasource_name": dataset.table_name,
+ }
config = update_chart_config_dataset(config, dataset_dict)
chart = import_chart(
config, overwrite=overwrite,
default_viewers=default_viewers
diff --git a/superset/commands/dashboard/export.py
b/superset/commands/dashboard/export.py
index 3d831dc20fe..19138a9e61d 100644
--- a/superset/commands/dashboard/export.py
+++ b/superset/commands/dashboard/export.py
@@ -36,6 +36,7 @@ from superset.commands.dataset.export import
ExportDatasetsCommand
from superset.daos.dataset import DatasetDAO
from superset.models.dashboard import Dashboard
from superset.models.slice import Slice
+from superset.semantic_layers.import_export import export_dashboard_references
from superset.tags.models import TagType
from superset.utils.dict_import_export import EXPORT_VERSION
from superset.utils.file import get_filename
@@ -331,6 +332,7 @@ class ExportDashboardsCommand(ExportModelsCommand):
payload[new_name] = {}
metadata = payload.get("metadata") or {}
+ export_dashboard_references(metadata)
referenced_dataset_ids = {
dataset_id
@@ -470,6 +472,7 @@ class ExportDashboardsCommand(ExportModelsCommand):
if export_related:
metadata = payload.get("metadata") or {}
+ export_dashboard_references(metadata)
# Extract all native filter datasets and export referenced datasets
referenced_dataset_ids: set[int] = set()
diff --git a/superset/commands/dashboard/export_example.py
b/superset/commands/dashboard/export_example.py
index 477e6a70ebf..a6260c5f867 100644
--- a/superset/commands/dashboard/export_example.py
+++ b/superset/commands/dashboard/export_example.py
@@ -28,12 +28,15 @@ from io import BytesIO
from typing import Any, Callable, TYPE_CHECKING
import yaml
+from flask_babel import gettext as _
from superset.commands.base import BaseCommand
from superset.commands.dashboard.exceptions import DashboardNotFoundError
+from superset.commands.exceptions import CommandInvalidError
from superset.common.db_query_status import QueryStatus
from superset.daos.dashboard import DashboardDAO
from superset.exceptions import SupersetSecurityException
+from superset.utils import json
if TYPE_CHECKING:
from superset.connectors.sqla.models import SqlaTable
@@ -500,6 +503,16 @@ def _make_bytes_generator(data: bytes) -> Callable[[],
bytes]:
return lambda: data
+class ExportExampleSemanticViewError(CommandInvalidError):
+ """Example bundles cannot represent semantic-view dependencies."""
+
+ def __init__(self) -> None:
+ """Return a client error without exporting a misleading dataset
mapping."""
+ super().__init__(
+ _("Example export does not support charts or filters using
semantic views.")
+ )
+
+
class ExportExampleCommand(BaseCommand):
"""Export dashboard as an example bundle with Parquet data and YAML
configs.
@@ -532,6 +545,24 @@ class ExportExampleCommand(BaseCommand):
if not self._dashboard:
raise DashboardNotFoundError()
+ chart: Slice
+ for chart in self._dashboard.slices:
+ if chart.datasource_type == "semantic_view":
+ raise ExportExampleSemanticViewError()
+
+ metadata: dict[str, Any]
+ try:
+ metadata = json.loads(self._dashboard.json_metadata or "{}")
+ except ValueError:
+ # Match export_dashboard_yaml's handling of invalid JSON metadata.
+ return
+ native_filter: dict[str, Any]
+ target: dict[str, Any]
+ for native_filter in metadata.get("native_filter_configuration", []):
+ for target in native_filter.get("targets", []):
+ if target.get("datasourceType") == "semantic_view":
+ raise ExportExampleSemanticViewError()
+
def run(self) -> Iterator[tuple[str, Callable[[], bytes]]]: # noqa: C901
"""Yield (filename, content_generator) tuples for ZIP packaging.
diff --git a/superset/commands/dashboard/importers/v1/__init__.py
b/superset/commands/dashboard/importers/v1/__init__.py
index 22084581f73..a511878f81c 100644
--- a/superset/commands/dashboard/importers/v1/__init__.py
+++ b/superset/commands/dashboard/importers/v1/__init__.py
@@ -49,6 +49,11 @@ from superset.extensions import feature_flag_manager
from superset.migrations.shared.native_filters import migrate_dashboard
from superset.models.dashboard import Dashboard, dashboard_slices
from superset.models.slice import Slice
+from superset.semantic_layers.import_export import (
+ consume_chart_semantic_reference,
+ resolve_bundle_references,
+ restore_dashboard_references,
+)
from superset.subjects.utils import get_default_viewers_for_current_user
from superset.themes.schemas import ImportV1ThemeSchema
from superset.utils.decorators import transaction
@@ -105,6 +110,7 @@ class ImportDashboardsCommand(ImportModelsCommand):
**kwargs: Any,
) -> None:
contents = {} if contents is None else contents
+ semantic_info: dict[str, dict[str, Any]] =
resolve_bundle_references(configs)
# discover charts, datasets, and themes associated with dashboards
chart_uuids: set[str] = set()
dataset_uuids: set[str] = set()
@@ -121,7 +127,11 @@ class ImportDashboardsCommand(ImportModelsCommand):
# discover datasets associated with charts
for file_name, config in configs.items():
- if file_name.startswith("charts/") and config["uuid"] in
chart_uuids:
+ if (
+ file_name.startswith("charts/")
+ and config["uuid"] in chart_uuids
+ and "dataset_uuid" in config
+ ):
dataset_uuids.add(config["dataset_uuid"])
# discover databases associated with datasets
@@ -176,12 +186,15 @@ class ImportDashboardsCommand(ImportModelsCommand):
charts = []
chart_ids: dict[str, int] = {}
for file_name, config in configs.items():
- if (
- file_name.startswith("charts/")
- and config["dataset_uuid"] in dataset_info
+ if file_name.startswith("charts/") and (
+ "datasource_ref" in config or config.get("dataset_uuid") in
dataset_info
):
# update datasource id, type, and name
- dataset_dict = dataset_info[config["dataset_uuid"]]
+ dataset_dict: dict[str, Any] | None =
consume_chart_semantic_reference(
+ config, semantic_info
+ )
+ if dataset_dict is None:
+ dataset_dict = dataset_info[config["dataset_uuid"]]
config = update_chart_config_dataset(config, dataset_dict)
chart = import_chart(
@@ -225,6 +238,9 @@ class ImportDashboardsCommand(ImportModelsCommand):
dashboards: list[Dashboard] = []
for file_name, config in configs.items():
if file_name.startswith("dashboards/"):
+ restore_dashboard_references(
+ config.get("metadata") or {}, semantic_info
+ )
config = update_id_refs(config, chart_ids, dataset_info)
# Handle theme UUID to ID mapping
if "theme_uuid" in config and config["theme_uuid"] in
theme_ids:
diff --git a/superset/commands/importers/v1/assets.py
b/superset/commands/importers/v1/assets.py
index c0e92a85d2f..e64e6262790 100644
--- a/superset/commands/importers/v1/assets.py
+++ b/superset/commands/importers/v1/assets.py
@@ -14,7 +14,6 @@
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-from functools import partial
from typing import Any, Optional
from marshmallow import Schema
@@ -52,10 +51,23 @@ from superset.models.dashboard import Dashboard
from superset.models.slice import Slice
from superset.models.sql_lab import SavedQuery
from superset.queries.saved_queries.schemas import ImportV1SavedQuerySchema
+from superset.semantic_layers.import_export import (
+ consume_chart_semantic_reference,
+ resolve_bundle_references,
+ restore_dashboard_references,
+ SemanticReferenceError,
+)
from superset.subjects.utils import get_default_viewers_for_current_user
from superset.utils.decorators import on_error, transaction
+def _on_import_error(ex: Exception) -> None:
+ """Keep dependency validation actionable after the transaction rolls
back."""
+ if isinstance(ex, SemanticReferenceError):
+ raise ex
+ on_error(ex, catches=(Exception,), reraise=ImportFailedError)
+
+
class ImportAssetsCommand(BaseCommand):
"""
Command for importing databases, datasets, charts, dashboards and saved
queries.
@@ -103,6 +115,7 @@ class ImportAssetsCommand(BaseCommand):
overwrite: bool = True,
) -> None:
contents = {} if contents is None else contents
+ semantic_info: dict[str, dict[str, Any]] =
resolve_bundle_references(configs)
# import databases first
database_ids: dict[str, int] = {}
dataset_info: dict[str, dict[str, Any]] = {}
@@ -153,7 +166,11 @@ class ImportAssetsCommand(BaseCommand):
charts = []
for file_name, config in configs.items():
if file_name.startswith("charts/"):
- dataset_dict = dataset_info[config["dataset_uuid"]]
+ dataset_dict: dict[str, Any] | None =
consume_chart_semantic_reference(
+ config, semantic_info
+ )
+ if dataset_dict is None:
+ dataset_dict = dataset_info[config["dataset_uuid"]]
config = update_chart_config_dataset(config, dataset_dict)
chart = import_chart(
config, overwrite=overwrite,
default_viewers=default_viewers
@@ -171,6 +188,9 @@ class ImportAssetsCommand(BaseCommand):
# import dashboards
for file_name, config in configs.items():
if file_name.startswith("dashboards/"):
+ restore_dashboard_references(
+ config.get("metadata") or {}, semantic_info
+ )
config = update_id_refs(config, chart_ids, dataset_info)
dashboard = import_dashboard(
config, overwrite=overwrite,
default_viewers=default_viewers
@@ -229,13 +249,7 @@ class ImportAssetsCommand(BaseCommand):
if chart.viz_type == "filter_box":
db.session.delete(chart)
- @transaction(
- on_error=partial(
- on_error,
- catches=(Exception,),
- reraise=ImportFailedError,
- )
- )
+ @transaction(on_error=_on_import_error)
def run(self) -> None:
self.validate()
self._import(self._configs, self.sparse, self.contents, self.overwrite)
diff --git a/superset/commands/importers/v1/examples.py
b/superset/commands/importers/v1/examples.py
index 71f851ca2bf..56a2b62226b 100644
--- a/superset/commands/importers/v1/examples.py
+++ b/superset/commands/importers/v1/examples.py
@@ -45,6 +45,10 @@ from superset.databases.schemas import ImportV1DatabaseSchema
from superset.datasets.schemas import ImportV1DatasetSchema
from superset.exceptions import QueryClauseValidationException
from superset.models.core import Database
+from superset.semantic_layers.import_export import (
+ dashboard_targets,
+ SemanticReferenceError,
+)
from superset.sql.parse import transpile_to_dialect
from superset.subjects.utils import get_default_viewers_for_current_user
from superset.utils.core import get_example_default_schema
@@ -128,6 +132,8 @@ class ImportExamplesCommand(ImportModelsCommand):
self.overwrite,
self.force_data,
)
+ except SemanticReferenceError:
+ raise
except Exception as ex:
raise self.import_error() from ex
@@ -148,6 +154,19 @@ class ImportExamplesCommand(ImportModelsCommand):
contents: Optional[dict[str, Any]] = None,
force_data: bool = False,
) -> None:
+ for path, config in configs.items():
+ if (path.startswith("charts/") and "datasource_ref" in config) or (
+ path.startswith("dashboards/")
+ and any(
+ "datasourceRef" in target
+ or target.get("datasourceType") == "semantic_view"
+ for target in dashboard_targets(config.get("metadata") or
{})
+ )
+ ):
+ raise SemanticReferenceError(
+ "Semantic references are not supported by the examples
loader; "
+ "use the chart, dashboard or assets importer."
+ )
# import databases
database_ids: dict[str, int] = {}
for file_name, config in configs.items():
diff --git a/superset/dashboards/api.py b/superset/dashboards/api.py
index 5b3e2c9133d..35d5959addc 100644
--- a/superset/dashboards/api.py
+++ b/superset/dashboards/api.py
@@ -83,7 +83,10 @@ from superset.commands.dashboard.exceptions import (
DashboardUpdateFailedError,
)
from superset.commands.dashboard.export import ExportDashboardsCommand
-from superset.commands.dashboard.export_example import ExportExampleCommand
+from superset.commands.dashboard.export_example import (
+ ExportExampleCommand,
+ ExportExampleSemanticViewError,
+)
from superset.commands.dashboard.fave import AddFavoriteDashboardCommand
from superset.commands.dashboard.importers.dispatcher import
ImportDashboardsCommand
from superset.commands.dashboard.permalink.create import
CreateDashboardPermalinkCommand
@@ -180,6 +183,7 @@ from superset.security.manager import (
get_extra_editor_subject_ids,
get_extra_editors_by_pk,
)
+from superset.semantic_layers.import_export import SemanticReferenceError
from superset.semantic_layers.models import SemanticView
from superset.subjects.filters import (
FilterRelatedSubjects,
@@ -1730,6 +1734,8 @@ class DashboardRestApi(
)
except DashboardNotFoundError:
return self.response_404()
+ except SemanticReferenceError as ex:
+ return self.response(ex.status, message=ex.message)
buf.seek(0)
return send_export_zip(buf, filename)
@@ -1754,6 +1760,7 @@ class DashboardRestApi(
Exports a dashboard with its charts and datasets in the example
format used by the Superset example loading system. The export
includes Parquet data files and YAML configuration files.
+ Charts and native-filter targets that use semantic views are not
supported.
parameters:
- in: path
schema:
@@ -1785,6 +1792,8 @@ class DashboardRestApi(
$ref: '#/components/responses/403'
404:
$ref: '#/components/responses/404'
+ 422:
+ description: The dashboard contains unsupported semantic-view
assets
500:
$ref: '#/components/responses/500'
"""
@@ -1806,6 +1815,8 @@ class DashboardRestApi(
bundle.writestr(filename, content_fn())
except DashboardNotFoundError:
return self.response_404()
+ except ExportExampleSemanticViewError as ex:
+ return self.response_422(message=str(ex))
buf.seek(0)
diff --git a/superset/semantic_layers/bundle_schemas.py
b/superset/semantic_layers/bundle_schemas.py
new file mode 100644
index 00000000000..2f0eeeb9f60
--- /dev/null
+++ b/superset/semantic_layers/bundle_schemas.py
@@ -0,0 +1,36 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""ORM-free serialized references for semantic-view bundles."""
+
+from typing import Any
+
+from marshmallow import fields, Schema, validate
+
+
+class SemanticViewReferenceSchema(Schema):
+ """A semantic reference never carries configuration or an
environment-local ID."""
+
+ type: fields.String = fields.String(
+ required=True, validate=validate.Equal("semantic_view")
+ )
+ uuid: fields.UUID = fields.UUID(required=True)
+
+
+def reference_uuid(reference: Any) -> str:
+ """Validate the closed reference shape and canonicalize its UUID."""
+ parsed: dict[str, Any] = SemanticViewReferenceSchema().load(reference)
+ return str(parsed["uuid"])
diff --git a/superset/semantic_layers/import_export.py
b/superset/semantic_layers/import_export.py
new file mode 100644
index 00000000000..c1b64e75b5c
--- /dev/null
+++ b/superset/semantic_layers/import_export.py
@@ -0,0 +1,240 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Reference-only bundle support for already provisioned semantic views."""
+
+from collections.abc import Iterator
+from typing import Any
+
+from marshmallow import ValidationError
+from sqlalchemy.orm import joinedload, load_only
+
+from superset import db
+from superset.commands.exceptions import CommandInvalidError
+from superset.exceptions import SupersetSecurityException
+from superset.extensions import feature_flag_manager
+from superset.semantic_layers.bundle_schemas import reference_uuid
+from superset.semantic_layers.models import SemanticLayer, SemanticView
+from superset.semantic_layers.registry import registry
+
+
+class SemanticReferenceError(CommandInvalidError):
+ """A bundle dependency cannot be resolved without changing destination
policy."""
+
+ def __init__(self, message: str) -> None:
+ """Expose an actionable validation error without protected view
metadata."""
+ super().__init__(
+ message, [ValidationError(message,
field_name="semantic_references")]
+ )
+
+
+def dashboard_targets(metadata: dict[str, Any]) -> Iterator[dict[str, Any]]:
+ """Visit native-filter and display-control datasource targets."""
+ for key in ("native_filter_configuration", "chart_customization_config"):
+ controls: Any = metadata.get(key) or []
+ if not isinstance(controls, list):
+ raise SemanticReferenceError("Dashboard controls must be a list.")
+ for control in controls:
+ if not isinstance(control, dict):
+ raise SemanticReferenceError("Dashboard control must be an
object.")
+ targets: Any = control.get("targets") or []
+ if not isinstance(targets, list) or any(
+ not isinstance(target, dict) for target in targets
+ ):
+ raise SemanticReferenceError(
+ "Dashboard targets must be a list of objects."
+ )
+ yield from targets
+
+
+def _check_enabled() -> None:
+ """Reject semantic references when their host feature is disabled."""
+ if not feature_flag_manager.is_feature_enabled("SEMANTIC_LAYERS"):
+ raise SemanticReferenceError(
+ "Semantic view references require SEMANTIC_LAYERS."
+ )
+
+
+def _check_view(view: SemanticView | None) -> SemanticView:
+ """Authorize before disclosing metadata, without instantiating a
provider."""
+ if view is None:
+ raise SemanticReferenceError(
+ "Semantic view reference is missing or inaccessible."
+ )
+ try:
+ view.raise_for_access()
+ except SupersetSecurityException as ex:
+ raise SemanticReferenceError(
+ "Semantic view reference is missing or inaccessible."
+ ) from ex
+ if not view.semantic_layer or view.semantic_layer.type not in registry:
+ raise SemanticReferenceError("Semantic view provider is not
registered.")
+ return view
+
+
+def export_view_reference(view: SemanticView | None) -> dict[str, str]:
+ """Export only a UUID and type for an accessible existing view."""
+ _check_enabled()
+ resolved: SemanticView = _check_view(view)
+ return {"type": "semantic_view", "uuid": str(resolved.uuid)}
+
+
+def export_dashboard_references(metadata: dict[str, Any]) -> None:
+ """Replace semantic target IDs without ever resolving them as tables."""
+ targets: list[dict[str, Any]] = []
+ ids: set[int] = set()
+ for target in dashboard_targets(metadata):
+ source_type: Any = target.get("datasourceType")
+ if source_type is None:
+ source_type = "table"
+ if source_type == "table":
+ continue
+ if source_type != "semantic_view":
+ raise SemanticReferenceError("Unsupported dashboard datasource
type.")
+ raw_id: Any = target.get("datasetId")
+ if isinstance(raw_id, bool) or not (
+ isinstance(raw_id, int) or (isinstance(raw_id, str) and
raw_id.isdecimal())
+ ):
+ raise SemanticReferenceError(
+ "Semantic dashboard target requires a datasetId."
+ )
+ ids.add(int(raw_id))
+ targets.append(target)
+ if not targets:
+ return
+ views: dict[int, SemanticView] = {
+ view.id: view
+ for view in db.session.query(SemanticView)
+ .options(
+ load_only(
+ SemanticView.id,
+ SemanticView.uuid,
+ SemanticView.name,
+ SemanticView.perm,
+ SemanticView.semantic_layer_uuid,
+ ),
+ joinedload(SemanticView.semantic_layer).load_only(
+ SemanticLayer.type, SemanticLayer.perm
+ ),
+ )
+ .filter(SemanticView.id.in_(ids))
+ .all()
+ }
+ references: dict[int, dict[str, str]] = {
+ id_: export_view_reference(view) for id_, view in views.items()
+ }
+ for target in targets:
+ reference: dict[str, str] | None =
references.get(int(target["datasetId"]))
+ if reference is not None:
+ target["datasourceRef"] = reference.copy()
+ else:
+ target.pop("datasourceRef", None)
+ target.pop("datasetId", None)
+ target.pop("datasetUuid", None)
+
+
+def _target_uuid(target: dict[str, Any]) -> str | None:
+ """Reject ambiguous semantic targets instead of trusting archived local
IDs."""
+ if "datasourceRef" in target:
+ if (
+ target.get("datasourceType", "semantic_view") != "semantic_view"
+ or "datasetUuid" in target
+ or "datasetId" in target
+ ):
+ raise ValidationError("Conflicting semantic dashboard reference.")
+ return reference_uuid(target["datasourceRef"])
+ if target.get("datasourceType") not in (None, "table"):
+ raise ValidationError("Semantic dashboard target requires
datasourceRef.")
+ return None
+
+
+def _bundle_reference_uuids(configs: dict[str, Any]) -> set[str]:
+ """Validate the entire reference graph before consulting the metastore."""
+ uuids: set[str] = set()
+ for path, config in configs.items():
+ if path.startswith("charts/") and "datasource_ref" in config:
+ if "dataset_uuid" in config:
+ raise ValidationError("Specify only one chart datasource
reference.")
+ uuids.add(reference_uuid(config["datasource_ref"]))
+ elif path.startswith("dashboards/"):
+ for target in dashboard_targets(config.get("metadata") or {}):
+ uuid: str | None = _target_uuid(target)
+ if uuid is not None:
+ uuids.add(uuid)
+ return uuids
+
+
+def resolve_bundle_references(
+ configs: dict[str, Any],
+) -> dict[str, dict[str, Any]]:
+ """Preflight every semantic dependency before the caller writes any
asset."""
+ try:
+ uuids: set[str] = _bundle_reference_uuids(configs)
+ except ValidationError as ex:
+ raise SemanticReferenceError(
+ f"Invalid semantic datasource reference: {ex}"
+ ) from ex
+ if not uuids:
+ return {}
+ _check_enabled()
+ views: dict[str, SemanticView] = {
+ str(view.uuid): view
+ for view in db.session.query(SemanticView)
+ .options(
+ load_only(
+ SemanticView.id,
+ SemanticView.uuid,
+ SemanticView.name,
+ SemanticView.perm,
+ SemanticView.semantic_layer_uuid,
+ ),
+ joinedload(SemanticView.semantic_layer).load_only(
+ SemanticLayer.type, SemanticLayer.perm
+ ),
+ )
+ .filter(SemanticView.uuid.in_(uuids))
+ .all()
+ }
+ result: dict[str, dict[str, Any]] = {}
+ for uuid in sorted(uuids):
+ view: SemanticView = _check_view(views.get(uuid))
+ result[uuid] = {
+ "datasource_id": view.id,
+ "datasource_type": "semantic_view",
+ "datasource_name": view.name,
+ }
+ return result
+
+
+def consume_chart_semantic_reference(
+ config: dict[str, Any], semantic_info: dict[str, dict[str, Any]]
+) -> dict[str, Any] | None:
+ """Remove a prevalidated semantic reference and return its destination
info."""
+ if "datasource_ref" not in config:
+ return None
+ return semantic_info[reference_uuid(config.pop("datasource_ref"))]
+
+
+def restore_dashboard_references(
+ metadata: dict[str, Any], semantic_info: dict[str, dict[str, Any]]
+) -> None:
+ """Rebind validated semantic targets using destination IDs and types."""
+ for target in dashboard_targets(metadata):
+ uuid: str | None = _target_uuid(target)
+ if uuid is not None:
+ target.pop("datasourceRef")
+ target["datasetId"] = semantic_info[uuid]["datasource_id"]
+ target["datasourceType"] = "semantic_view"
diff --git a/superset/translations/messages.pot
b/superset/translations/messages.pot
index 896be08253b..7c9bcd5cebe 100644
--- a/superset/translations/messages.pot
+++ b/superset/translations/messages.pot
@@ -6137,6 +6137,9 @@ msgstr ""
msgid "Example"
msgstr ""
+msgid "Example export does not support charts or filters using semantic views."
+msgstr ""
+
msgid "Examples"
msgstr ""
diff --git a/tests/unit_tests/commands/chart/semantic_export_test.py
b/tests/unit_tests/commands/chart/semantic_export_test.py
new file mode 100644
index 00000000000..d219cc2479c
--- /dev/null
+++ b/tests/unit_tests/commands/chart/semantic_export_test.py
@@ -0,0 +1,127 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Chart export must produce a document its import schema accepts."""
+
+from unittest.mock import Mock
+from uuid import uuid4
+
+import pytest
+import yaml
+from marshmallow import ValidationError
+
+from superset.charts.schemas import ImportV1ChartSchema
+from superset.commands.chart.export import ExportChartsCommand
+from superset.connectors.sqla.models import SqlaTable
+from superset.daos.chart import ChartDAO
+from superset.models.slice import Slice
+from superset.semantic_layers.models import SemanticLayer, SemanticView
+from superset.semantic_layers.registry import registry
+
+
[email protected]("source_type", ["table", "semantic_view"])
+def test_exported_chart_preserves_datasource_for_import_schema(
+ app_context: None, source_type: str, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """Use real transient models and the real exporter/schema, without a
warehouse."""
+ table: SqlaTable = SqlaTable(id=7, table_name="physical", uuid=uuid4())
+ view: SemanticView = SemanticView(
+ id=7,
+ name="semantic",
+ uuid=uuid4(),
+ semantic_layer=SemanticLayer(type="test-provider"),
+ )
+ monkeypatch.setattr(
+ "superset.extensions.feature_flag_manager.is_feature_enabled",
+ lambda flag: flag == "SEMANTIC_LAYERS",
+ )
+ monkeypatch.setattr(SemanticView, "raise_for_access", Mock())
+ monkeypatch.setitem(registry, "test-provider", Mock())
+ chart: Slice = Slice(
+ id=11,
+ uuid=uuid4(),
+ slice_name="Roundtrip source",
+ viz_type="table",
+ datasource_type=source_type,
+ datasource_id=7,
+ params='{"datasource":"7__' + source_type + '"}',
+ table=table if source_type == "table" else None,
+ semantic_view=view if source_type == "semantic_view" else None,
+ )
+
+ content: str = ExportChartsCommand._file_content(chart)
+ ImportV1ChartSchema().load(yaml.safe_load(content))
+ # Distinct UUIDs despite equal numeric IDs: exporting a semantic source
+ # must neither lose its identity nor substitute the same-id table's UUID.
+ if source_type == "semantic_view":
+ assert str(view.uuid) in content
+ assert str(table.uuid) not in content
+ else:
+ assert yaml.safe_load(content)["dataset_uuid"] == str(table.uuid)
+
+
[email protected]("semantic_enabled", [True, False])
+def test_orphaned_semantic_chart_does_not_abort_mixed_export(
+ app_context: None,
+ monkeypatch: pytest.MonkeyPatch,
+ semantic_enabled: bool,
+) -> None:
+ """A missing view must not prevent unrelated charts from being archived."""
+ table: SqlaTable = SqlaTable(id=7, table_name="physical", uuid=uuid4())
+ plain: Slice = Slice(
+ id=10,
+ uuid=uuid4(),
+ slice_name="plain",
+ viz_type="table",
+ datasource_type="table",
+ datasource_id=7,
+ table=table,
+ params="{}",
+ )
+ orphan: Slice = Slice(
+ id=11,
+ uuid=uuid4(),
+ slice_name="orphan",
+ viz_type="table",
+ datasource_type="semantic_view",
+ datasource_id=81,
+ semantic_view=None,
+ params='{"datasource":"81__semantic_view"}',
+ )
+ monkeypatch.setattr(ChartDAO, "find_by_ids", Mock(return_value=[plain,
orphan]))
+ monkeypatch.setattr(
+ "superset.extensions.feature_flag_manager.is_feature_enabled",
+ lambda flag: semantic_enabled and flag == "SEMANTIC_LAYERS",
+ )
+
+ contents: dict[str, str] = {
+ name: content()
+ for name, content in ExportChartsCommand(
+ [plain.id, orphan.id], export_related=False
+ ).run()
+ }
+
+ assert "metadata.yaml" in contents
+ assert yaml.safe_load(contents["charts/plain_10.yaml"])["dataset_uuid"] ==
str(
+ table.uuid
+ )
+ orphan_content: dict[str, object] = yaml.safe_load(
+ contents["charts/orphan_11.yaml"]
+ )
+ assert "datasource_ref" not in orphan_content
+ assert "dataset_uuid" not in orphan_content
+ with pytest.raises(ValidationError):
+ ImportV1ChartSchema().load(orphan_content)
diff --git
a/tests/unit_tests/commands/dashboard/export_example_semantic_test.py
b/tests/unit_tests/commands/dashboard/export_example_semantic_test.py
new file mode 100644
index 00000000000..faebd8492b4
--- /dev/null
+++ b/tests/unit_tests/commands/dashboard/export_example_semantic_test.py
@@ -0,0 +1,175 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Example bundles must not confuse semantic-view IDs with table IDs."""
+
+from collections.abc import Callable
+from typing import Any
+from unittest.mock import Mock
+from uuid import UUID
+
+import pytest
+import yaml
+from flask.testing import FlaskClient
+from pytest_mock import MockerFixture
+from werkzeug.test import TestResponse
+
+from superset.commands.dashboard import export_example
+from superset.commands.dashboard.export_example import (
+ ExportExampleCommand,
+ ExportExampleSemanticViewError,
+)
+from superset.connectors.sqla.models import SqlaTable
+from superset.models.core import Database
+from superset.models.dashboard import Dashboard
+from superset.models.slice import Slice
+from superset.semantic_layers.models import SemanticView
+from superset.utils import json
+
+
[email protected]
+def example_dashboard(mocker: MockerFixture) -> Dashboard:
+ """Build real model objects; only the command's DAO lookup is replaced."""
+ table: SqlaTable = SqlaTable(
+ id=7,
+ uuid=UUID("00000000-0000-0000-0000-000000000007"),
+ table_name="ordinary_table",
+ database=Database(sqlalchemy_uri="sqlite://"),
+ )
+ chart: Slice = Slice(
+ id=1,
+ uuid=UUID("00000000-0000-0000-0000-000000000001"),
+ slice_name="Table chart",
+ datasource_type="table",
+ datasource_id=7,
+ table=table,
+ params="{}",
+ viz_type="table",
+ )
+ dashboard: Dashboard = Dashboard(
+ id=1,
+ uuid=UUID("00000000-0000-0000-0000-000000000010"),
+ dashboard_title="Example",
+ slug="example",
+ position_json="{}",
+ json_metadata=json.dumps(
+ {
+ "native_filter_configuration": [
+ {"targets": [{"datasetId": 7, "column": {"name":
"country"}}]}
+ ]
+ }
+ ),
+ slices=[chart],
+ )
+ mocker.patch(
+ "superset.commands.dashboard.export_example.DashboardDAO.find_by_id",
+ return_value=dashboard,
+ )
+ return dashboard
+
+
+def add_semantic_assets(dashboard: Dashboard, asset: str) -> None:
+ """Use the same numeric ID in two datasource namespaces."""
+ view: SemanticView = SemanticView(
+ id=7,
+ uuid=UUID("00000000-0000-0000-0000-000000000077"),
+ name="semantic",
+ )
+ if asset in {"collision", "chart", "orphan_chart"}:
+ chart: Slice = Slice(
+ id=2,
+ uuid=UUID("00000000-0000-0000-0000-000000000002"),
+ slice_name="Semantic chart",
+ datasource_type="semantic_view",
+ datasource_id=7,
+ semantic_view=None if asset == "orphan_chart" else view,
+ params="{}",
+ viz_type="table",
+ )
+ assert chart.datasource is None
+ dashboard.slices.insert(0, chart)
+ if asset in {"collision", "filter"}:
+ dashboard.json_metadata = json.dumps(
+ {
+ "native_filter_configuration": [
+ {
+ "targets": [
+ {
+ "datasetId": 7,
+ "datasourceType": "semantic_view",
+ "column": {"name": "country"},
+ }
+ ]
+ }
+ ]
+ }
+ )
+
+
[email protected]("asset", ["collision", "chart", "filter",
"orphan_chart"])
+def test_example_export_rejects_semantic_assets(
+ example_dashboard: Dashboard,
+ asset: str,
+ mocker: MockerFixture,
+) -> None:
+ """Reject before producing the first bundle entry, including orphan
charts."""
+ add_semantic_assets(example_dashboard, asset)
+ dataset_builder: Mock = mocker.spy(export_example, "export_dataset_yaml")
+ chart_builder: Mock = mocker.spy(export_example, "export_chart")
+ dashboard_builder: Mock = mocker.spy(export_example,
"export_dashboard_yaml")
+ with pytest.raises(ExportExampleSemanticViewError, match="semantic views"):
+ next(ExportExampleCommand(1, export_data=False).run())
+ dataset_builder.assert_not_called()
+ chart_builder.assert_not_called()
+ dashboard_builder.assert_not_called()
+
+
+def test_table_only_example_bundle_is_unchanged(example_dashboard: Dashboard)
-> None:
+ """Keep the ordinary dataset/chart/filter UUID mapping and bundle
layout."""
+ files: dict[str, Callable[[], bytes]] = dict(
+ ExportExampleCommand(1, export_data=False).run()
+ )
+ assert set(files) == {"dataset.yaml", "charts/Table_chart.yaml",
"dashboard.yaml"}
+ dataset: dict[str, Any] = yaml.safe_load(files["dataset.yaml"]())
+ chart: dict[str, Any] = yaml.safe_load(files["charts/Table_chart.yaml"]())
+ dashboard: dict[str, Any] = yaml.safe_load(files["dashboard.yaml"]())
+ assert (
+ dataset["uuid"]
+ == chart["dataset_uuid"]
+ == str(example_dashboard.slices[0].table.uuid)
+ )
+ assert dashboard["metadata"]["native_filter_configuration"][0]["targets"]
== [
+ {"datasetUuid": dataset["uuid"], "column": {"name": "country"}}
+ ]
+
+
[email protected]("asset", ["collision", "chart", "filter",
"orphan_chart"])
+def test_example_route_rejects_semantic_assets(
+ example_dashboard: Dashboard,
+ client: FlaskClient,
+ full_api_access: None,
+ mocker: MockerFixture,
+ asset: str,
+) -> None:
+ """The real command's rejection reaches the client as 422 rather than a
ZIP."""
+ add_semantic_assets(example_dashboard, asset)
+ mocker.patch("superset.dashboards.api.get_user_id", return_value=1)
+ response: TestResponse = client.get(
+ "/api/v1/dashboard/1/export_as_example/?export_data=false"
+ )
+ assert response.status_code == 422
+ assert response.is_json
+ assert "semantic views" in response.json["message"]
diff --git a/tests/unit_tests/semantic_layers/import_export_test.py
b/tests/unit_tests/semantic_layers/import_export_test.py
new file mode 100644
index 00000000000..4c279a6d53b
--- /dev/null
+++ b/tests/unit_tests/semantic_layers/import_export_test.py
@@ -0,0 +1,915 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Typed bundle identities must not become table IDs or grant datasource
access."""
+
+import copy
+import importlib
+import inspect
+from types import ModuleType
+from typing import Any
+from unittest.mock import Mock
+from uuid import UUID
+
+import pytest
+import yaml
+from flask import current_app, Response
+from flask_appbuilder.api import safe
+from marshmallow import ValidationError
+from sqlalchemy.orm import Session
+
+from superset import security_manager
+from superset.charts.schemas import ImportV1ChartSchema
+from superset.commands.chart.export import ExportChartsCommand
+from superset.commands.exceptions import CommandInvalidError, ImportFailedError
+from superset.connectors.sqla.models import SqlaTable
+from superset.models.core import Database
+from superset.models.dashboard import Dashboard
+from superset.models.slice import Slice
+from superset.semantic_layers import import_export as refs
+from superset.semantic_layers.models import SemanticLayer, SemanticView
+from superset.utils import json
+
+VIEW_UUID: str = "efafe588-5c67-4d1c-a9bb-ab1839f8cf59"
+CHART_UUID: str = "d766261b-5cb0-4ea1-b62c-0da4860116c1"
+
+
[email protected]
+def view(app_context: None, monkeypatch: pytest.MonkeyPatch) -> SemanticView:
+ """A real model and access predicate, with no provider instantiation."""
+ model: SemanticView = SemanticView(
+ id=81,
+ uuid=UUID(VIEW_UUID),
+ name="existing semantic view",
+ perm="view-grant",
+ semantic_layer=SemanticLayer(type="test-provider", perm="layer-grant"),
+ )
+ monkeypatch.setattr(
+ refs.feature_flag_manager,
+ "is_feature_enabled",
+ lambda flag: flag == "SEMANTIC_LAYERS",
+ )
+ monkeypatch.setattr(security_manager, "can_access_all_datasources",
lambda: False)
+ monkeypatch.setattr(
+ security_manager,
+ "can_access",
+ lambda permission, resource: resource == "view-grant",
+ )
+ monkeypatch.setitem(
+ refs.registry,
+ "test-provider",
+ Mock(spec=[], side_effect=AssertionError("provider must not be
instantiated")),
+ )
+ query: Mock = Mock()
+ query.options.return_value = query
+ query.filter.return_value.all.return_value = [model]
+ monkeypatch.setattr(refs.db.session, "query", Mock(return_value=query))
+ return model
+
+
+def chart_config() -> dict[str, Any]:
+ """Use deliberately different archived and destination IDs."""
+ return {
+ "uuid": CHART_UUID,
+ "version": "1.0.0",
+ "slice_name": "semantic chart",
+ "viz_type": "table",
+ "datasource_ref": {"type": "semantic_view", "uuid": VIEW_UUID},
+ "params": {"datasource": "7__semantic_view", "metrics": ["revenue"]},
+ "query_context": json.dumps(
+ {
+ "datasource": {"id": 7, "type": "semantic_view"},
+ "form_data": {"datasource": "7__semantic_view"},
+ "queries": [
+ {
+ "datasource": {"id": 7, "type": "semantic_view"},
+ "metrics": ["revenue"],
+ }
+ ],
+ }
+ ),
+ }
+
+
[email protected](
+ "reference",
+ [
+ None,
+ {},
+ {"type": "table", "uuid": VIEW_UUID},
+ {"type": "semantic_view", "uuid": "bad"},
+ {
+ "type": "semantic_view",
+ "uuid": VIEW_UUID,
+ "configuration": {"token": "not-allowed"},
+ },
+ ],
+)
+def test_chart_schema_rejects_invalid_reference(reference: Any) -> None:
+ """A missing dataset_uuid is permitted only with a valid semantic
reference."""
+ config: dict[str, Any] = chart_config()
+ config["datasource_ref"] = reference
+ with pytest.raises(ValidationError):
+ ImportV1ChartSchema().load(config)
+
+
+def test_chart_schema_rejects_ambiguous_and_missing_reference() -> None:
+ """Never pick one of two contradictory source representations."""
+ config: dict[str, Any] = chart_config()
+ config["dataset_uuid"] = VIEW_UUID
+ with pytest.raises(ValidationError):
+ ImportV1ChartSchema().load(config)
+ config.pop("datasource_ref")
+ ImportV1ChartSchema().load(config)
+ config.pop("dataset_uuid")
+ with pytest.raises(ValidationError):
+ ImportV1ChartSchema().load(config)
+
+
[email protected](
+ "module_name,command_name",
+ [
+ ("superset.commands.chart.importers.v1", "ImportChartsCommand"),
+ ("superset.commands.dashboard.importers.v1",
"ImportDashboardsCommand"),
+ ("superset.commands.importers.v1.assets", "ImportAssetsCommand"),
+ ],
+)
+def test_each_importer_rebinds_semantic_chart(
+ view: SemanticView,
+ monkeypatch: pytest.MonkeyPatch,
+ module_name: str,
+ command_name: str,
+) -> None:
+ """Exercise real entry-point orchestration/remapping, stubbing only
writers."""
+ module: ModuleType = importlib.import_module(module_name)
+ importer: Any = getattr(module, command_name)
+ writer: Mock = Mock(
+ return_value=Mock(id=91, uuid=UUID(CHART_UUID), viz_type="table")
+ )
+ monkeypatch.setattr(module, "import_chart", writer)
+ monkeypatch.setattr(module, "get_default_viewers_for_current_user",
lambda: [])
+ configs: dict[str, Any] = {"charts/chart.yaml": chart_config()}
+ importer._import(configs, overwrite=True)
+ writer.assert_called_once()
+ actual: dict[str, Any] = writer.call_args.args[0]
+ assert actual["datasource_id"] == 81
+ assert actual["datasource_type"] == "semantic_view"
+ assert "datasource_ref" not in actual
+ assert "dataset_uuid" not in actual
+ assert actual["params"] == {
+ "datasource": "81__semantic_view",
+ "metrics": ["revenue"],
+ }
+ context: dict[str, Any] = json.loads(actual["query_context"])
+ assert context["datasource"] == {"id": 81, "type": "semantic_view"}
+ assert context["form_data"]["datasource"] == "81__semantic_view"
+ assert context["queries"][0] == {
+ "datasource": {"id": 81, "type": "semantic_view"},
+ "metrics": ["revenue"],
+ }
+
+
[email protected]("failure", ["missing", "denied", "disabled",
"provider"])
[email protected](
+ "module_name,command_name",
+ [
+ ("superset.commands.chart.importers.v1", "ImportChartsCommand"),
+ ("superset.commands.dashboard.importers.v1",
"ImportDashboardsCommand"),
+ ("superset.commands.importers.v1.assets", "ImportAssetsCommand"),
+ ],
+)
+def test_dependency_failure_precedes_any_bundle_write(
+ view: SemanticView,
+ monkeypatch: pytest.MonkeyPatch,
+ failure: str,
+ module_name: str,
+ command_name: str,
+) -> None:
+ """Even unrelated database/dataset assets must not write before
preflight."""
+ module: ModuleType = importlib.import_module(module_name)
+ importer: Any = getattr(module, command_name)
+ writes: list[Mock] = []
+ for name in ("import_database", "import_dataset", "import_chart"):
+ writer: Mock = Mock(side_effect=AssertionError("write before
preflight"))
+ monkeypatch.setattr(module, name, writer)
+ writes.append(writer)
+ if failure == "missing":
+
refs.db.session.query.return_value.filter.return_value.all.return_value = []
+ elif failure == "denied":
+ monkeypatch.setattr(security_manager, "can_access", lambda *args:
False)
+ elif failure == "disabled":
+ monkeypatch.setattr(
+ refs.feature_flag_manager, "is_feature_enabled", lambda flag: False
+ )
+ else:
+ monkeypatch.delitem(refs.registry, "test-provider")
+ configs: dict[str, Any] = {
+ "databases/database.yaml": {"uuid": "db"},
+ "datasets/dataset.yaml": {"uuid": "table", "database_uuid": "db"},
+ "charts/chart.yaml": chart_config(),
+ }
+ with pytest.raises(refs.SemanticReferenceError):
+ importer._import(configs, overwrite=True)
+ for writer in writes:
+ writer.assert_not_called()
+
+
[email protected](
+ "control", ["native_filter_configuration", "chart_customization_config"]
+)
+def test_dashboard_semantic_target_roundtrip(view: SemanticView, control: str)
-> None:
+ """Typed target serialization preserves the semantic view UUID and type."""
+ metadata: dict[str, Any] = {
+ control: [
+ {
+ "targets": [
+ {
+ "datasetId": 81,
+ "datasourceType": "semantic_view",
+ "column": {"name": "country"},
+ }
+ ]
+ }
+ ]
+ }
+ refs.export_dashboard_references(metadata)
+ target: dict[str, Any] = metadata[control][0]["targets"][0]
+ assert "datasetId" not in target
+ assert "datasetUuid" not in target
+ assert target["datasourceRef"] == {"type": "semantic_view", "uuid":
VIEW_UUID}
+ info: dict[str, dict[str, Any]] = refs.resolve_bundle_references(
+ {"dashboards/d.yaml": {"metadata": copy.deepcopy(metadata)}}
+ )
+ refs.restore_dashboard_references(metadata, info)
+ assert target == {
+ "datasetId": 81,
+ "datasourceType": "semantic_view",
+ "column": {"name": "country"},
+ }
+
+
+def test_layer_grant_authorizes_reference_without_view_grant(
+ view: SemanticView, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """Use the model's actual parent-layer grant rule, not a new permission
rule."""
+ monkeypatch.setattr(
+ security_manager,
+ "can_access",
+ lambda permission, resource: resource == "layer-grant",
+ )
+ assert refs.export_view_reference(view) == {
+ "type": "semantic_view",
+ "uuid": VIEW_UUID,
+ }
+
+
+def test_table_only_bundle_does_not_need_semantic_feature(
+ app_context: None, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """Do not query semantic models or flags for a legacy bundle."""
+ forbidden: Mock = Mock(side_effect=AssertionError("semantic lookup for
table"))
+ monkeypatch.setattr(refs.db.session, "query", forbidden)
+ monkeypatch.setattr(refs.feature_flag_manager, "is_feature_enabled",
forbidden)
+ assert (
+ refs.resolve_bundle_references({"charts/a.yaml": {"dataset_uuid":
VIEW_UUID}})
+ == {}
+ )
+
+
[email protected]("dataset_id", [None, True, False, "", "invalid", "²",
1.5])
+def test_export_rejects_invalid_semantic_target_id_before_lookup(
+ app_context: None, monkeypatch: pytest.MonkeyPatch, dataset_id: Any
+) -> None:
+ """Invalid local IDs cannot reach a semantic lookup or mutate the
target."""
+ query: Mock = Mock(side_effect=AssertionError("invalid ID reached lookup"))
+ monkeypatch.setattr(refs.db.session, "query", query)
+ target: dict[str, Any] = {
+ "datasourceType": "semantic_view",
+ "datasetId": dataset_id,
+ }
+ metadata: dict[str, Any] = {"native_filter_configuration": [{"targets":
[target]}]}
+ before: dict[str, Any] = copy.deepcopy(metadata)
+ with pytest.raises(refs.SemanticReferenceError, match="requires a
datasetId"):
+ refs.export_dashboard_references(metadata)
+ query.assert_not_called()
+ assert metadata == before
+
+
+def test_bundle_preflight_rejects_ambiguous_chart_before_lookup(
+ app_context: None, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """Preflight itself rejects both identities, even without schema
loading."""
+ query: Mock = Mock(side_effect=AssertionError("ambiguous chart reached
lookup"))
+ monkeypatch.setattr(refs.db.session, "query", query)
+ config: dict[str, Any] = chart_config()
+ config["dataset_uuid"] = VIEW_UUID
+ excinfo: pytest.ExceptionInfo[refs.SemanticReferenceError]
+ with pytest.raises(
+ refs.SemanticReferenceError, match="Specify only one chart datasource
reference"
+ ) as excinfo:
+ refs.resolve_bundle_references({"charts/chart.yaml": config})
+ assert isinstance(excinfo.value.__cause__, ValidationError)
+ query.assert_not_called()
+ assert config["dataset_uuid"] == config["datasource_ref"]["uuid"] ==
VIEW_UUID
+
+
+def test_consume_chart_semantic_reference_preserves_table_reference() -> None:
+ """A table UUID must not resolve through the colliding semantic UUID
map."""
+ config: dict[str, Any] = {"dataset_uuid": VIEW_UUID}
+ semantic_info: dict[str, dict[str, Any]] = {
+ VIEW_UUID: {"datasource_id": 81, "datasource_type": "semantic_view"}
+ }
+ assert refs.consume_chart_semantic_reference(config, semantic_info) is None
+ assert config == {"dataset_uuid": VIEW_UUID}
+
+
+def test_consume_chart_semantic_reference_preserves_unrelated_config() -> None:
+ """Consume only the archive identity, leaving chart fields for the
writer."""
+ config: dict[str, Any] = chart_config()
+ expected: dict[str, Any] = copy.deepcopy(config)
+ expected.pop("datasource_ref")
+ destination: dict[str, Any] = {
+ "datasource_id": 81,
+ "datasource_type": "semantic_view",
+ }
+ assert (
+ refs.consume_chart_semantic_reference(config, {VIEW_UUID: destination})
+ == destination
+ )
+ assert config == expected
+
+
[email protected](
+ "control", ["native_filter_configuration", "chart_customization_config"]
+)
+def test_restore_mixed_targets_preserves_table_identity(control: str) -> None:
+ """Rebind a semantic target without changing a preceding same-UUID
table."""
+ table: dict[str, Any] = {"datasetUuid": VIEW_UUID, "datasourceType":
"table"}
+ semantic: dict[str, Any] = {
+ "datasourceRef": {"type": "semantic_view", "uuid": VIEW_UUID},
+ "column": {"name": "country"},
+ }
+ metadata: dict[str, Any] = {control: [{"targets": [table, semantic]}]}
+ refs.restore_dashboard_references(metadata, {VIEW_UUID: {"datasource_id":
81}})
+ assert table == {"datasetUuid": VIEW_UUID, "datasourceType": "table"}
+ assert semantic == {
+ "datasetId": 81,
+ "datasourceType": "semantic_view",
+ "column": {"name": "country"},
+ }
+ assert metadata[control][0]["targets"] == [table, semantic]
+
+
[email protected]
+def persisted_view(
+ app_context: None, session: Session, monkeypatch: pytest.MonkeyPatch
+) -> SemanticView:
+ """Use only an in-memory metastore, with colliding table/view
identities."""
+ Slice.metadata.create_all(session.get_bind())
+ monkeypatch.setattr(
+ refs.feature_flag_manager,
+ "is_feature_enabled",
+ lambda flag: flag == "SEMANTIC_LAYERS",
+ )
+ monkeypatch.setattr(
+ security_manager,
+ "can_access",
+ lambda permission, resource: (permission, resource)
+ in {
+ ("datasource_access", "view-grant"),
+ ("can_write", "Chart"),
+ ("can_write", "Dashboard"),
+ },
+ )
+ monkeypatch.setattr(security_manager, "can_access_all_datasources",
lambda: False)
+ monkeypatch.setattr(security_manager, "semantic_view_after_insert", Mock())
+ monkeypatch.setattr(security_manager, "semantic_layer_after_insert",
Mock())
+ monkeypatch.setitem(
+ refs.registry,
+ "test-provider",
+ Mock(spec=[], side_effect=AssertionError("provider call")),
+ )
+ layer: SemanticLayer = SemanticLayer(
+ name="local layer",
+ type="test-provider",
+ configuration='{"token":"layer-secret"}',
+ perm="layer-grant",
+ )
+ model: SemanticView = SemanticView(
+ id=81,
+ uuid=UUID(VIEW_UUID),
+ name="existing view",
+ semantic_layer=layer,
+ configuration='{"token":"view-secret"}',
+ perm="view-grant",
+ )
+ database: Database = Database(database_name="local",
sqlalchemy_uri="sqlite://")
+ table: SqlaTable = SqlaTable(
+ id=81, uuid=UUID(VIEW_UUID), table_name="collision", database=database
+ )
+ session.add_all([model, table])
+ session.commit()
+ return model
+
+
+def test_persisted_view_grant_does_not_authorize_another_view(
+ persisted_view: SemanticView,
+) -> None:
+ """The persisted fixture allows A but rejects B and its parent layer."""
+ assert refs.export_view_reference(persisted_view)["uuid"] == VIEW_UUID
+ denied: SemanticView = SemanticView(
+ name="denied view",
+ uuid=UUID("bff213f2-40c4-4da1-95c2-cea73731b713"),
+ perm="denied-view-grant",
+ semantic_layer=persisted_view.semantic_layer,
+ )
+ with pytest.raises(refs.SemanticReferenceError, match="missing or
inaccessible"):
+ refs.export_view_reference(denied)
+ assert not security_manager.can_access("wrong_permission", "view-grant")
+
+
[email protected](
+ "module_name,command_name",
+ [
+ ("superset.commands.chart.importers.v1", "ImportChartsCommand"),
+ ("superset.commands.dashboard.importers.v1",
"ImportDashboardsCommand"),
+ ("superset.commands.importers.v1.assets", "ImportAssetsCommand"),
+ ],
+)
+def test_import_persists_semantic_identity_not_same_id_table(
+ persisted_view: SemanticView,
+ session: Session,
+ monkeypatch: pytest.MonkeyPatch,
+ module_name: str,
+ command_name: str,
+) -> None:
+ """Real chart writer, ORM relationship, exporter and schema roundtrip."""
+ module: ModuleType = importlib.import_module(module_name)
+ importer: Any = getattr(module, command_name)
+ monkeypatch.setattr(module, "get_default_viewers_for_current_user",
lambda: [])
+ importer._import({"charts/chart.yaml": chart_config()}, overwrite=True)
+ session.flush()
+ session.expire_all()
+ chart: Slice = session.query(Slice).filter(Slice.uuid ==
UUID(CHART_UUID)).one()
+ assert chart.datasource_id == 81
+ assert chart.datasource_type == "semantic_view"
+ assert chart.table is None
+ assert chart.semantic_view.uuid == UUID(VIEW_UUID)
+ exported: str = ExportChartsCommand._file_content(chart)
+ assert "dataset_uuid" not in exported
+ assert "layer-secret" not in exported
+ assert "view-secret" not in exported
+ ImportV1ChartSchema().load(yaml.safe_load(exported))
+
+
[email protected](
+ "module_name,command_name",
+ [
+ ("superset.commands.dashboard.importers.v1",
"ImportDashboardsCommand"),
+ ("superset.commands.importers.v1.assets", "ImportAssetsCommand"),
+ ],
+)
+def test_dashboard_export_import_rebinds_both_target_kinds(
+ persisted_view: SemanticView,
+ session: Session,
+ monkeypatch: pytest.MonkeyPatch,
+ module_name: str,
+ command_name: str,
+) -> None:
+ """Real dashboard serialization and writes preserve typed filter
identities."""
+ from superset.commands.dashboard.export import ExportDashboardsCommand
+
+ source: Dashboard = Dashboard(
+ id=17,
+ uuid=UUID("d80091a6-c364-4cee-b2fa-479c67e46bcb"),
+ dashboard_title="semantic targets",
+ slices=[],
+ json_metadata=json.dumps(
+ {
+ key: [
+ {
+ "id": key,
+ "targets": [
+ {
+ "datasetId": 81,
+ "datasourceType": "semantic_view",
+ "column": {"name": "country"},
+ }
+ ],
+ }
+ ]
+ for key in ("native_filter_configuration",
"chart_customization_config")
+ }
+ ),
+ )
+ exported: str = ExportDashboardsCommand._file_content(source)
+ assert "datasetUuid" not in exported
+ assert "datasetId" not in exported
+ # Re-ID the destination to prove the archive is resolved by UUID, not 81.
+ session.query(SemanticView).filter(SemanticView.id == 81).update({"id":
82})
+ session.flush()
+ session.expire_all()
+ module: ModuleType = importlib.import_module(module_name)
+ importer: Any = getattr(module, command_name)
+ monkeypatch.setattr(module, "get_default_viewers_for_current_user",
lambda: [])
+ importer._import({"dashboards/d.yaml": yaml.safe_load(exported)},
overwrite=True)
+ session.flush()
+ session.expire_all()
+ dashboard: Dashboard = (
+ session.query(Dashboard).filter(Dashboard.uuid == source.uuid).one()
+ )
+ metadata: dict[str, Any] = json.loads(dashboard.json_metadata)
+ for key in ("native_filter_configuration", "chart_customization_config"):
+ assert metadata[key][0]["targets"] == [
+ {
+ "datasetId": 82,
+ "datasourceType": "semantic_view",
+ "column": {"name": "country"},
+ }
+ ]
+
+
+def test_dashboard_related_export_never_exports_same_id_table(
+ persisted_view: SemanticView, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """A semantic native target must not cause an unrelated table bundle
export."""
+ from superset.commands.dashboard import export as dashboard_export
+
+ source: Dashboard = Dashboard(
+ id=17,
+ uuid=UUID("d80091a6-c364-4cee-b2fa-479c67e46bcb"),
+ dashboard_title="semantic targets",
+ slices=[],
+ json_metadata=json.dumps(
+ {
+ "native_filter_configuration": [
+ {"targets": [{"datasetId": 81, "datasourceType":
"semantic_view"}]}
+ ]
+ }
+ ),
+ )
+ monkeypatch.setattr(
+ dashboard_export.ExportChartsCommand, "run", lambda *args, **kwargs:
iter(())
+ )
+ unexpected: Mock = Mock(
+ side_effect=AssertionError("semantic target exported a table")
+ )
+ monkeypatch.setattr(dashboard_export, "ExportDatasetsCommand", unexpected)
+ outputs: list[Any] =
list(dashboard_export.ExportDashboardsCommand._export(source))
+ assert len(outputs) == 1
+ unexpected.assert_not_called()
+
+
[email protected](
+ "target",
+ [
+ {"datasetId": 81, "datasourceType": "semantic_view"},
+ {
+ "datasourceRef": {"type": "semantic_view", "uuid": VIEW_UUID},
+ "datasetUuid": VIEW_UUID,
+ },
+ {
+ "datasourceRef": {"type": "semantic_view", "uuid": VIEW_UUID},
+ "datasourceType": "table",
+ },
+ {"datasourceRef": {"type": "table", "uuid": VIEW_UUID}},
+ ],
+)
+def test_ambiguous_dashboard_target_rejected_before_lookup(
+ app_context: None, monkeypatch: pytest.MonkeyPatch, target: dict[str, Any]
+) -> None:
+ """Raw IDs and contradictory discriminators cannot become imported
references."""
+ query: Mock = Mock(side_effect=AssertionError("invalid target reached
lookup"))
+ monkeypatch.setattr(refs.db.session, "query", query)
+ with pytest.raises(
+ refs.SemanticReferenceError, match="Invalid semantic datasource
reference"
+ ):
+ refs.resolve_bundle_references(
+ {
+ "dashboards/d.yaml": {
+ "metadata": {"native_filter_configuration": [{"targets":
[target]}]}
+ }
+ }
+ )
+
+
[email protected](
+ "module_name,command_name",
+ [
+ ("superset.commands.chart.importers.v1", "ImportChartsCommand"),
+ ("superset.commands.dashboard.importers.v1",
"ImportDashboardsCommand"),
+ ("superset.commands.importers.v1.assets", "ImportAssetsCommand"),
+ ],
+)
+def test_import_transaction_rolls_back_late_failure(
+ persisted_view: SemanticView,
+ session: Session,
+ monkeypatch: pytest.MonkeyPatch,
+ module_name: str,
+ command_name: str,
+) -> None:
+ """Exercise each public command transaction after a real chart flush."""
+ from superset.commands.chart.importers.v1.utils import import_chart
+
+ module: ModuleType = importlib.import_module(module_name)
+ command: Any = getattr(module, command_name)({})
+ command._configs = {"charts/chart.yaml": chart_config()}
+ monkeypatch.setattr(command, "validate", lambda: None)
+ monkeypatch.setattr(module, "get_default_viewers_for_current_user",
lambda: [])
+
+ def write_then_fail(config: dict[str, Any], **kwargs: Any) -> Slice:
+ """Flush a real imported chart before injecting a later command
failure."""
+ chart: Slice = import_chart(config, **kwargs)
+ session.flush()
+ assert chart.id is not None
+ assert session.query(Slice).filter(Slice.uuid ==
UUID(CHART_UUID)).count() == 1
+ raise ImportFailedError("injected after chart write")
+
+ monkeypatch.setattr(module, "import_chart", write_then_fail)
+ with pytest.raises(ImportFailedError):
+ command.run()
+ assert session.query(Slice).filter(Slice.uuid == UUID(CHART_UUID)).count()
== 0
+ assert (
+ session.query(SemanticView).filter(SemanticView.uuid ==
UUID(VIEW_UUID)).count()
+ == 1
+ )
+
+
[email protected](
+ "module_name,command_name",
+ [
+ ("superset.commands.chart.importers.v1", "ImportChartsCommand"),
+ ("superset.commands.dashboard.importers.v1",
"ImportDashboardsCommand"),
+ ("superset.commands.importers.v1.assets", "ImportAssetsCommand"),
+ ],
+)
+def test_public_command_preserves_clear_dependency_error(
+ persisted_view: SemanticView,
+ session: Session,
+ monkeypatch: pytest.MonkeyPatch,
+ module_name: str,
+ command_name: str,
+) -> None:
+ """Dependency failures must not turn into a generic unknown-error HTTP
500."""
+ module: ModuleType = importlib.import_module(module_name)
+ command: Any = getattr(module, command_name)({})
+ command._configs = {"charts/chart.yaml": chart_config()}
+ monkeypatch.setattr(command, "validate", lambda: None)
+ monkeypatch.setattr(security_manager, "can_access", lambda *args: False)
+ error: pytest.ExceptionInfo[CommandInvalidError]
+ with pytest.raises(CommandInvalidError, match="missing or inaccessible")
as error:
+ command.run()
+ assert error.value.status == 422
+ assert session.query(Slice).count() == 0
+
+
[email protected]("failure", ["denied", "disabled", "provider"])
[email protected]("kind", ["chart", "dashboard"])
+def test_export_rejects_unavailable_dependencies(
+ view: SemanticView, monkeypatch: pytest.MonkeyPatch, failure: str, kind:
str
+) -> None:
+ """A reference export must not bypass the view grant or feature/provider
gate."""
+ chart: Slice = Slice(
+ id=91,
+ uuid=UUID(CHART_UUID),
+ slice_name="semantic",
+ viz_type="table",
+ datasource_id=81,
+ datasource_type="semantic_view",
+ params="{}",
+ semantic_view=view,
+ )
+ if failure == "denied":
+ monkeypatch.setattr(security_manager, "can_access", lambda *args:
False)
+ elif failure == "disabled":
+ monkeypatch.setattr(
+ refs.feature_flag_manager, "is_feature_enabled", lambda flag: False
+ )
+ else:
+ monkeypatch.delitem(refs.registry, "test-provider")
+ metadata: dict[str, Any] = {
+ "native_filter_configuration": [
+ {"targets": [{"datasetId": 81, "datasourceType": "semantic_view"}]}
+ ]
+ }
+ if kind == "chart":
+ with pytest.raises(refs.SemanticReferenceError):
+ ExportChartsCommand._file_content(chart)
+ else:
+ with pytest.raises(refs.SemanticReferenceError):
+ refs.export_dashboard_references(metadata)
+ assert (
+ "datasourceRef"
+ not in metadata["native_filter_configuration"][0]["targets"][0]
+ )
+
+
[email protected](
+ "module_name,api_name,export_name",
+ [
+ ("superset.charts.api", "ChartRestApi", "ExportChartsCommand"),
+ ("superset.dashboards.api", "DashboardRestApi",
"ExportDashboardsCommand"),
+ ],
+)
+def test_export_api_preserves_dependency_error_under_safe(
+ app_context: None,
+ monkeypatch: pytest.MonkeyPatch,
+ module_name: str,
+ api_name: str,
+ export_name: str,
+) -> None:
+ """Keep FAB's exception wrapper: unknown errors would otherwise
become500."""
+ module: ModuleType = importlib.import_module(module_name)
+ api: Any = getattr(module, api_name)()
+ if api_name == "DashboardRestApi":
+ monkeypatch.setattr(module, "get_user_id", lambda: 1)
+
+ def failed_content() -> str:
+ """Model lazy archive serialization failing on a dependency."""
+ raise refs.SemanticReferenceError(
+ "Semantic view reference is missing or inaccessible."
+ )
+
+ exporter: Mock = Mock()
+ exporter.return_value.run.return_value = iter([("chart.yaml",
failed_content)])
+ monkeypatch.setattr(module, export_name, exporter)
+ with current_app.test_request_context():
+ response: Response = safe(inspect.unwrap(api.export))(api, rison=[1])
+ assert response.status_code == 422
+ assert (
+ response.get_json()["message"]
+ == "Semantic view reference is missing or inaccessible."
+ )
+
+
+def test_examples_rejects_semantic_bundle_before_writes(
+ app_context: None, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ """The fourth schema reader must reject rather than skip semantic data."""
+ from superset.commands.importers.v1 import examples
+
+ writer: Mock = Mock(side_effect=AssertionError("examples wrote before
rejection"))
+ monkeypatch.setattr(examples, "import_database", writer)
+ with pytest.raises(refs.SemanticReferenceError, match="examples"):
+ examples.ImportExamplesCommand._import(
+ {"databases/d.yaml": {"uuid": "db"}, "charts/c.yaml":
chart_config()}
+ )
+ writer.assert_not_called()
+
+
+def test_null_target_type_preserves_legacy_table_reference() -> None:
+ """Only a missing/null legacy type defaults to table, not arbitrary
unknowns."""
+ target: dict[str, Any] = {"datasourceType": None, "datasetUuid": VIEW_UUID}
+ metadata: dict[str, Any] = {"native_filter_configuration": [{"targets":
[target]}]}
+ assert (
+ refs.resolve_bundle_references({"dashboards/d.yaml": {"metadata":
metadata}})
+ == {}
+ )
+ refs.export_dashboard_references(metadata)
+ assert target == {"datasourceType": None, "datasetUuid": VIEW_UUID}
+
+
[email protected](
+ "controls", ["bad", [None], [{"targets": ["bad"]}], [{"targets": "bad"}]]
+)
+def test_malformed_dashboard_controls_fail_clearly(controls: Any) -> None:
+ """The new preflight/export traversal must not surface shape errors
as500."""
+ metadata: dict[str, Any] = {"native_filter_configuration": controls}
+ with pytest.raises(refs.SemanticReferenceError):
+ refs.resolve_bundle_references({"dashboards/d.yaml": {"metadata":
metadata}})
+ with pytest.raises(refs.SemanticReferenceError):
+ refs.export_dashboard_references(metadata)
+
+
[email protected]("source_type", ["unknown", "", False, 0])
+def test_unknown_target_type_is_not_reinterpreted_as_table(source_type: Any)
-> None:
+ """Reject unrecognized source identities rather than falling back on
truthiness."""
+ metadata: dict[str, Any] = {
+ "native_filter_configuration": [
+ {"targets": [{"datasourceType": source_type, "datasetId": 81}]}
+ ]
+ }
+ with pytest.raises(refs.SemanticReferenceError):
+ refs.resolve_bundle_references({"dashboards/d.yaml": {"metadata":
metadata}})
+ with pytest.raises(refs.SemanticReferenceError):
+ refs.export_dashboard_references(metadata)
+
+
[email protected](
+ "module_name,command_name,bundle_type",
+ [
+ ("superset.commands.chart.importers.v1", "ImportChartsCommand",
"Slice"),
+ (
+ "superset.commands.dashboard.importers.v1",
+ "ImportDashboardsCommand",
+ "Dashboard",
+ ),
+ ("superset.commands.importers.v1.assets", "ImportAssetsCommand",
"assets"),
+ ],
+)
+def test_public_import_parses_semantic_yaml_and_commits_chart(
+ persisted_view: SemanticView,
+ session: Session,
+ monkeypatch: pytest.MonkeyPatch,
+ module_name: str,
+ command_name: str,
+ bundle_type: str,
+) -> None:
+ """Include actual metadata/config parsing and schemas, not a validate
stub."""
+ module: ModuleType = importlib.import_module(module_name)
+ contents: dict[str, str] = {
+ "metadata.yaml": yaml.safe_dump({"version": "1.0.0", "type":
bundle_type}),
+ "charts/c.yaml": yaml.safe_dump(chart_config()),
+ }
+ monkeypatch.setattr(module, "get_default_viewers_for_current_user",
lambda: [])
+ command: Any = getattr(module, command_name)(contents, overwrite=True)
+ command.run()
+ chart: Slice = session.query(Slice).filter(Slice.uuid ==
UUID(CHART_UUID)).one()
+ assert chart.datasource_id == 81
+ assert chart.datasource_type == "semantic_view"
+ assert chart.table is None
+ assert chart.semantic_view.uuid == UUID(VIEW_UUID)
+
+
[email protected]("enabled", [True, False])
[email protected](
+ "control", ["native_filter_configuration", "chart_customization_config"]
+)
+def test_orphaned_dashboard_target_does_not_abort_mixed_export(
+ view: SemanticView, monkeypatch: pytest.MonkeyPatch, enabled: bool,
control: str
+) -> None:
+ """Archive unrelated dashboards without making orphan targets
importable."""
+ from superset.commands.dashboard import export as dashboard_export
+
+ refs.db.session.query.return_value.filter.return_value.all.return_value =
[]
+ monkeypatch.setattr(
+ refs.feature_flag_manager,
+ "is_feature_enabled",
+ lambda flag: enabled and flag == "SEMANTIC_LAYERS",
+ )
+ table: SqlaTable = SqlaTable(id=81, uuid=UUID(VIEW_UUID),
table_name="unrelated")
+ source: Dashboard = Dashboard(
+ id=17,
+ uuid=UUID(CHART_UUID),
+ dashboard_title="orphan target",
+ slices=[],
+ json_metadata=json.dumps(
+ {
+ control: [
+ {
+ "targets": [
+ {"datasetId": 81, "datasourceType":
"semantic_view"},
+ {"datasetId": 81, "datasourceType": "table"},
+ ]
+ }
+ ]
+ }
+ ),
+ )
+ plain: Dashboard = Dashboard(
+ id=18,
+ uuid=UUID("d80091a6-c364-4cee-b2fa-479c67e46bcb"),
+ dashboard_title="ordinary",
+ slices=[],
+ json_metadata="{}",
+ )
+ monkeypatch.setattr(
+ dashboard_export.DashboardDAO, "find_by_ids", lambda ids: [source,
plain]
+ )
+ monkeypatch.setattr(
+ dashboard_export.DatasetDAO, "find_by_ids", lambda ids: [table] if ids
else []
+ )
+ monkeypatch.setattr(
+ dashboard_export.ExportChartsCommand, "run", lambda *args, **kwargs:
iter(())
+ )
+ dataset_export: Mock = Mock()
+ dataset_export.return_value.run.return_value = iter(())
+ monkeypatch.setattr(dashboard_export, "ExportDatasetsCommand",
dataset_export)
+ outputs: dict[str, Any] = {
+ path: yaml.safe_load(content())
+ for path, content in dashboard_export.ExportDashboardsCommand([17,
18]).run()
+ }
+ assert "metadata.yaml" in outputs
+ assert dashboard_export.ExportDashboardsCommand._file_name(plain) in
outputs
+ exported: dict[str, Any] = outputs[
+ dashboard_export.ExportDashboardsCommand._file_name(source)
+ ]
+ targets: list[dict[str, Any]] = exported["metadata"][control][0]["targets"]
+ assert targets[0] == {"datasourceType": "semantic_view"}
+ assert targets[1]["datasetUuid"] == VIEW_UUID
+ dataset_export.assert_called_once_with([81])
+ with pytest.raises(refs.SemanticReferenceError, match="requires
datasourceRef"):
+ refs.resolve_bundle_references({"dashboards/orphan.yaml": exported})
