villebro opened a new issue, #44875:
URL: https://github.com/apache/superset/issues/44875
## [SIP] Proposal for a Widget Framework
> **Authors:** @villebro (co-authors: @michael-s-molina, @EnxDev, @rusackas,
@msyavuz)
>
> **Status:** Draft for discussion. A proof of concept on the `dashboard-v2`
feature branch (#43262, #43400, #43588) evaluated the approach end to end. None
of its code is assumed to carry over; this SIP specifies the design.
### The vision
Widgets are the building blocks of **Canvas** (the next generation of
Superset dashboards): charts, tables, metric tiles, filters, markdown blocks
and containers. This SIP proposes the framework behind them. Its main use case
is the chart, and it changes what charts can do:
- **Build and refine charts by talking to an agent.** "Show monthly revenue
by region, and make EMEA red." The agent reads only the parts of the control
panel it needs, sends small edits, and previews the result before anything is
saved.
- **Agents can't break a chart.** Every edit is validated against the
widget's schema. A bad edit is rejected with an error that says exactly how to
fix it.
- **Control panels of any depth, with no hand-written UI.** A widget
describes its properties once, as a schema. Superset generates the property
panel from it, and agents read the same schema.
- **Any visualization, delivered as an extension.** A widget is a React
component plus Python logic, so it can use ECharts, D3, deck.gl, WebGL or plain
HTML. The extensions framework already loads frontend and backend code, so a
new chart type ships as one package and registers exactly like the built-ins.
- **Charts that work without a browser.** Queries are built on the server,
so alerts, reports, thumbnails, cache warm-up and charts created through the
API all work without running JavaScript.
- **Fast, compact data.** Results travel as Apache Arrow, and small results
arrive in a single request.
- **Reusable or self-contained widgets.** Save a widget instance once and
use it on its own, or place it on as many canvases as you like. Alternatively,
keep it inline in a single Canvas. The same chart can even appear twice on one
Canvas, with differently scoped filters.
- **Immutable snapshots.** Export a Canvas as one self-contained document
that later edits can't change.
- **Data access that follows the chart.** The query is defined by the stored
widget instance, never by the caller. This lays the groundwork for sharing an
aggregate without sharing the underlying table.
- **Cross-filtering and drill-down for every widget,** built-in or
extension, through public hooks.
- **Upgrades that never lose work.** Schemas are versioned and migrated
automatically, and anything the server can't load is kept intact until it can.
v1 charts, Explore and existing dashboards keep working alongside the new
framework. v1 charts are deprecated in phases, each gated on a milestone rather
than a date.
### Motivation
Superset's chart system was designed for a human working in a browser, one
chart at a time. AI agents, server-side features and finer-grained sharing all
strain that design:
1. **AI agents can't operate charts.** A v1 control panel
(`controlPanel.tsx`) is imperative TypeScript, and its behavior is expressed as
functions (`visibility`, `mapStateToProps`, `shouldMapStateToProps`). None of
that can be serialized and handed to an agent. As a result, every agent-facing
surface describes chart types again by hand. For example, the MCP
`generate_chart` tool keeps its own per-type models (`XYChartConfig`,
`PieChartConfig`, `TableChartConfig` and others in
`superset/mcp_service/chart/schemas.py`), and the chat extension keeps its own
tool schemas. Each new chart type multiplies this work, and the copies drift
apart. Being serializable is also not enough on its own: a rich control panel
is far too large to put into an agent's context on every turn.
2. **Queries are built in the browser.** In v1, a chart's frontend plugin
builds its query (`buildQuery`), and the result is stored as `query_context`.
Several features need a chart's query without a browser: alerts and reports,
thumbnails, cache warm-up, and charts created through the API. They must either
rely on a previously saved `query_context` or run JavaScript. SIP-211 proposes
a Node.js sidecar to close this gap. We think the lasting fix is to move query
building to the backend.
3. **Viewing a chart effectively requires dataset access.**
`/api/v1/chart/data` accepts a complete `query_context` from the caller.
Because the caller shapes the query (metrics, group-bys, columns), the endpoint
has to authorize against the dataset. So anyone who can view a chart can also
query its dataset in other ways. There is no way to share an aggregate without
sharing the underlying table. This SIP lays the architectural groundwork to fix
that (section 8), but its first milestone deliberately keeps today's data
entitlement.
4. **The v1 rendering and interactivity pipeline is heavy.**
- `transformProps` is an imperative step that reshapes props before
rendering, and it fits poorly with React hooks.
- Native filters and cross-filters go through a central Redux `dataMask`
store. The dashboard computes which charts each filter applies to and pushes
merged form data down to each chart (`getFormDataWithExtraFilters`).
- A chart can't take part in interactivity on its own terms, because the
dashboard orchestrates everything.
### Proposed Change
#### 1. Concepts and overview
- **Widget:** a kind of thing that can be placed on a Canvas, such as the
Timeseries Bar chart, the markdown block or the filter bar. A widget is a
Python class (a subclass of `Widget`) plus a React renderer, provided by core
or by an extension.
- **Widget instance:** one configured occurrence of a widget, meaning props
that are valid against the widget's schema. An instance is either
**persisted**, a saved entity with its own UUID, or **inline**, stored inside a
Canvas document.
- **Placement:** an instance's spot on a Canvas, with its position, size and
filter scope.
- Persisted instances and canvases have a many-to-many relationship
through placements. A persisted instance can exist on its own with no placement
at all, as many charts in large deployments do today, or be placed any number
of times on one or several canvases.
- An inline instance always has exactly one placement.
- **Chart:** a widget that queries data and visualizes it, by mixing in
`QueryMixin`. Charts are the main use case and get most of the attention below,
but nothing in the base contract is specific to charts.
- **Canvas:** the next generation of Superset dashboards, made of structure
and placements. A companion Canvas SIP specifies the Canvas document.
People will call widget instances "widgets" in everyday speech; this
document says "widget instance" wherever the difference matters.
This gives a clean split. The Canvas grammar is small and fixed: a tree of
placements. Each instance's content is open-ended, but it is validated against
its widget's schema, wherever it is stored.
**How the pieces fit.** The sections below follow the life of a chart:
1. A widget registers its contract with the backend (section 2), including a
schema that describes its properties (section 3).
2. A user or an agent creates a widget instance. The property panel renders
the schema, while an agent reads it step by step (section 4).
3. The instance is stored either on its own or inside a Canvas (section 5).
A stored-on-its-own instance can be placed on any number of canvases, or none.
Every edit happens in an isolated, validated draft (section 6).
4. When the instance is shown, the backend builds, checks and runs its
queries (section 7) and returns the data through an endpoint keyed by the
instance (section 8).
5. Canvas filters and cross-filters reach each placement as runtime filters
(section 9), and the widget's React renderer draws the result (section 10).
6. Widgets evolve without breaking saved instances (section 11).
Agents reach every step through MCP (section 12).
<details>
<summary><b>2. Defining a widget: the widget contract</b></summary>
Widgets register into a backend registry in `superset-core`, exactly as
built-ins do. The following sketch is illustrative, and its names are not final:
```python
from superset_core.widgets import QueryMixin, Widget
from superset_core.widgets.decorators import widget
@widget(
id="timeseries-bar",
name="Timeseries Bar",
description="Bars over time. Use for period-over-period comparisons;
each series is a value of the grouping dimension.",
)
class TimeseriesBar(QueryMixin, Widget):
controls_class = TimeseriesBarControls # Pydantic model; its JSON
Schema is the property panel
schema_version = 1
behavior = WidgetBehavior(emits_filters=True, drill=True)
ui = WidgetUi(default_size=(6, 4))
# From QueryMixin: only widgets that query data implement these
@classmethod
def build_queries(cls, view: SemanticView, props: TimeseriesBarControls)
-> list[WidgetQuery]: ...
@classmethod
def transform_data(cls, view: SemanticView, props:
TimeseriesBarControls, results: list[SemanticResult]) -> WidgetPayload: ...
```
The decorator gives the widget its identity and registers it with the host,
which applies extension namespacing; the class carries its behavior. Semantic
layers (`@semantic_layer`) and MCP tools (`@tool`) register the same way.
Every widget declares:
- **Description:** what the widget is for, when to use it, and how to read
it. Humans see it in the palette, and agents use it to choose a widget. Every
widget instance also carries its own `description` field: what that instance
shows, its business meaning and any caveats.
- **Properties schema:** a Pydantic model whose JSON Schema is the property
panel (section 3), together with its `schema_version` (section 11).
- **Behavior:**
- For containers: which child widgets they accept, and how many.
- For charts: which interactivity they support, such as emitting filters
or drilling down. These replace v1's `Behavior` flags.
- **UI metadata:** label, default size and header controls.
A widget that queries data mixes in **`QueryMixin`**, which adds two methods:
- **`build_queries(view, props) -> list[WidgetQuery]`** replaces the
frontend `buildQuery`.
- It returns query *descriptions*, and the framework validates and
executes them (section 7).
- Each `WidgetQuery` has a name, a `SemanticQuery`, and a declared
runtime-filter policy.
- It returns a list from day one, so charts that need several queries (for
example a comparison period, or overlaid series) won't require a breaking
change later. Most charts return one query.
- **`transform_data(view, props, results) -> WidgetPayload`** replaces
`transformProps` for reshaping data on the server, such as pivoting, deriving
series, or joining the results of several queries.
- It returns named datasets plus a `context` object, for example which
dimensions were pivoted, that the renderer can use.
- It defaults to passing results through, and it is a pure function with
bounded, typed output (section 7).
**The backend is the single authority for widgets.**
- The palette, the Inspector and agents all read widget metadata from the
backend. The frontend only registers a renderer for a given widget id.
- Extension widgets are namespaced as `extensions.<publisher>.<name>.<widget
id>`.
- Registering an id that is already taken fails loudly on both backend and
frontend.
- A widget is checked thoroughly when it registers (Appendix B), and the
frontend and backend confirm they agree through a handshake (section 11).
As a result, every widget has a backend part, including purely
presentational ones. That is what lets an agent work with any widget without a
browser in the loop.
</details>
<details>
<summary><b>3. Describing properties: the control schema</b></summary>
Property panels are JSON Schema generated from Pydantic models, plus a small
set of `x-*` extension keywords, building on the Semantic Layer's progressive
JSON Schema pattern (`x-dynamic`, `x-dependsOn`, `x-enumNames`). This SIP makes
the vocabulary normative, so the backend, the frontend and agents can't drift
apart.
- **A supported subset of JSON Schema.** Schemas stay within a documented
subset of JSON Schema 2020-12 (Appendix A), so the property panel, the
disclosure engine, validators and MCP all interpret them the same way.
Constructs outside the subset are rejected at registration, and a shared
conformance suite checks every consumer.
- **A closed, documented list of keywords.** Every `x-*` keyword has a
defined meaning and at least one consumer. Unknown keywords are rejected when a
widget registers.
- **Renderers are chosen only by `x-control`, never by field name.**
Composite controls (dataset, metric, dimension, ad-hoc filter, time range,
color, code, and so on) live in a registry that extensions can add to, on both
the backend and the frontend.
- **Dynamic fields:**
- `x-dynamic` marks a field whose options the backend fills in from the
current props.
- An enricher declares the fields it reads separately from the rules that
show or hide a field.
- Enrichers run in dependency order, and cycles are detected.
- Options derived from data, such as the distinct series values behind
per-series styling, are computed on the backend from cached queries. The client
does not supply them.
- **Paths are JSON Pointers (RFC 6901).** This is the same addressing JSON
Patch uses, so keys derived from data, such as series names containing `.` or
`/`, can always be addressed.
- **Strict props, enforced by the framework.** Every props model inherits a
framework base model that forbids undeclared fields and enables Pydantic's
strict mode, checked recursively at registration (Appendix B). Undeclared props
are rejected rather than silently dropped, which is Pydantic's default, and
values are never coerced: `"5"` is not an integer and `"true"` is not a
boolean. Validation follows JSON semantics, so UUIDs, dates and timestamps are
accepted only in their canonical string format.
- **Explicit values only.** Stored props contain only the values that were
set explicitly; defaults come from the schema (section 11).
- **Translation.** Labels and descriptions are translated on the backend,
because that is where the schema lives.
- **Arbitrarily nested.** JSON Schema is recursive, so a customization
object can contain arrays of objects to any depth, and the same tree drives
progressive disclosure (section 4). Rendering any depth is a requirement the
conformance suite proves, not a given.
**How enrichment runs.**
- **Inputs.** Enrichers run on the backend from the instance's current
props. For an instance being edited, those are the props stored in its draft,
at its current revision (section 6). Runtime filters never affect the schema.
- **Data-derived options** come from cached queries. They are capped and
sorted, and flagged when truncated. Maps keyed by data values still accept keys
that aren't listed, so a value beyond the cap can still be styled.
- **Canonical keys.** NULL and non-string values have a documented canonical
key form, so they can be addressed by JSON Pointer.
- **Authorization.** Option lookups need the same datasource access as data
queries, and row-level security applies. Superset has no column-level
permissions, so dataset access covers every column.
- **Limits.** Enrichment queries are cached and time-limited.
- **Status per field.** Every dynamic field reports a status: ready, waiting
on inputs, empty, forbidden, or failed (with a message). The UI and agents can
tell "no options" apart from "not loaded yet" or "not allowed".
The first milestone's composite controls are:
- the four that mirror the `SemanticQuery` vocabulary: metrics, dimensions,
filters and time range;
- the presentational composites needed by the first widgets: color, code and
per-series overrides.
</details>
<details>
<summary><b>4. Reading the schema: progressive disclosure</b></summary>
Control panels must be able to scale to arbitrary depth. A chart's settings
nest naturally: per-series overrides, axes, legend, tooltip and conditional
formatting, each with its own sub-options. An AI agent, however, works within a
limited context window. Sending a widget's full schema on every turn would use
up a large share of it, and the cost would grow with every option a widget adds.
The framework therefore never makes an agent load a whole schema. It serves
a minimal root, and the agent expands only the branches the task needs. The
cost depends on what the agent touches, not on how deep the panel is.
**The minimum viable object.** When a schema is requested without paths, the
backend returns a pruned view built by fixed rules:
- **Required fields** are expanded down to their leaves, including inside
required nested objects. The root alone is enough to build a valid widget
instance.
- **Optional cheap fields** (scalars, bounded enums, arrays of scalars) are
shown inline. They are small and there is nothing to drill into.
- **Optional nested objects and arrays of objects** are replaced by a
drill-in marker. The marker holds the field's type, its short description (so
the agent can judge whether the branch is relevant), and the JSON Pointer to
expand. It never includes the child fields.
- **Partial objects are flagged.** An object that withheld children is
marked partial, and the root is marked as a minimal view.
- **Descriptions** of the widget and of the instance are always included.
They let an agent decide what a widget is for, or what an instance shows,
without reading its full configuration.
**Expanding branches.**
- **Several branches per call.** An agent expands one or more markers in a
single call by passing their paths.
- **Same rules at every level.** Each expanded branch is disclosed by the
same rules: its own required fields expanded, its deeper optional branches
collapsed again. Expanding a top-level branch of a deep panel therefore doesn't
return everything beneath it. A consumer can still ask for a complete branch
when it knows the branch is small.
- **Shared addressing.** Paths are the same JSON Pointers that JSON Patch
uses. An agent can expand `/customization/series` and then patch
`/customization/series/EMEA/color` using the same addresses. Validation errors
also point at paths the agent can expand directly.
- **Data-keyed maps.** Maps keyed by data values, such as per-series
settings, can be expanded by key.
**Dynamic branches.**
- **Flagged when data-dependent.** A marker is flagged `x-dynamic` when the
branch's shape depends on live data, for example a map keyed by a dimension's
distinct values. Branches without the flag never change, so an agent can expand
them once and reuse the result.
- **Works against a draft.** The agent passes its draft token (section 6),
and the backend enriches the branch from the draft's stored props. The agent
doesn't have to resend the props on every call.
**Large option lists.** Bounded enums are inlined. Option lists that can
grow without limit (datasets, columns, distinct values) are never inlined into
a schema view; the field points instead to a searchable, paginated options
lookup.
**One mechanism for every consumer.**
- **REST and MCP match.** Both serve the same minimal view and the same
format for expanded branches, and the property panel can use the same rules to
render collapsed sections lazily.
- **Views are deterministic for given inputs:** the schema and contract
version, the draft revision, the user's locale and permissions, and the
datasource's metadata. The same inputs give the same view, so views can be
cached and tested. Data-derived options can change with the data; dynamic
markers flag where.
</details>
<details>
<summary><b>5. Storing instances: persisted, inline and placed</b></summary>
Once created, a widget instance has to live somewhere. The two storage modes
differ only in where the props live and how access is checked:
| | Persisted instance | Inline instance |
|---|---|---|
| Stored in | A new `WidgetInstance` table | The Canvas document, on its
placement |
| Identity | Its own UUID | The Canvas UUID plus the placement id |
| Permissions | Its own editors and viewers | Taken from the Canvas |
| Placements | Zero or more, on any number of canvases | Exactly one |
| Addressed by | Agents, reports, links and canvases, directly | Through its
Canvas |
| Typical use | Shared charts, instances that need their own access rules |
Text, layout containers, one-off charts, exported snapshots |
**Persisted instances** live in a **new model**, not in the `Slice` table:
| Field | Purpose |
|---|---|
| `uuid` (primary key) | Stable public identifier |
| `widget_id` | Namespaced id of the widget this is an instance of |
| `schema_version` | Version of the widget's schema that the props conform
to |
| `props` | JSON, valid against the widget's schema; explicitly set values
only |
| `revision` | Incremented on every change, exposed as an ETag; updates must
state the revision they're based on |
| datasource reference(s) | For chart instances; used for access checks and
lineage |
| editors / viewers | Subject-based access, as charts and dashboards have |
| audit fields | Created and changed by, created and changed on |
**Inline instances** store the same `widget_id`, `schema_version` and
`props` on their placement in the Canvas document. Datasource references are
derived from the props for access checks and lineage, exactly as for persisted
instances. The Canvas document's own revision protects inline instances against
concurrent changes.
**Placements.** A Canvas shows widget instances through placements. A
placement holds the instance's position, size and filter scope, all specified
in the companion Canvas SIP. It either stores an inline instance or references
a persisted instance by UUID.
- **Persisted instances and canvases are many-to-many.**
- A persisted instance stands on its own: it can exist without any
placement, for example a chart that is only shared by link or used in a report.
- It can also be placed any number of times, on the same Canvas or on
several. For example, the same chart can appear twice with differently scoped
filters (section 9).
- Removing its last placement doesn't delete it.
- **An inline instance has exactly one placement.** To show it again,
promote it.
**Converting between modes.**
- An inline instance can be **promoted** to a persisted instance, to place
it again, reuse it elsewhere or give it its own permissions.
- A persisted instance can be **inlined** as a copy, which detaches it from
the shared original.
- Both are operations on the Canvas document, specified in the companion
Canvas SIP.
**Export snapshot.** Inline storage is what makes an "export snapshot"
feature straightforward:
- Exporting a snapshot produces one self-contained, immutable document: the
Canvas with every instance inline.
- Persisted instances the Canvas references are copied in at export time.
Later edits to a shared instance therefore don't change the snapshot.
- The snapshot can be archived, shared, compared, or imported again as a new
Canvas.
The snapshot format and lifecycle, including whether query results are
included, belong to the companion Canvas SIP.
**Common to both modes:**
- **Permissions:** a new `Widget` permission resource (`can_read`,
`can_write`) governs the widget and widget instance APIs.
- **Feature flag:** the whole surface sits behind a flag while it incubates.
With the flag off, the API routes and MCP tools aren't registered at all,
rather than only being hidden in the UI.
- **Import/export:** covers persisted instances alongside canvases. Inline
instances travel inside their Canvas.
- **`Slice` migration:** existing `Slice` rows are expected to move into
persisted widget instances in a later phase (see the migration plan).
</details>
<details>
<summary><b>6. Editing instances: drafts and validation</b></summary>
Wherever an instance is stored, it is never edited in place. Edits are made
to **drafts stored on the server**. Drafts reuse the key-value store that
already holds Explore's temporary state, and like that state they expire after
a period of inactivity.
**Every edit happens in an isolated draft, whatever the storage mode.**
- **Where a draft can start:** from a persisted instance, from an inline
instance (Canvas UUID plus placement id), or from a widget for a brand-new
instance.
- **What it holds:** only that instance's state, under its own token.
- **Why isolation matters:**
- An agent can iterate on the instance for as many turns as it needs:
patching it, validating it, expanding its schema and previewing its data.
- The agent never has to load or modify the whole Canvas document.
- Other users don't see the intermediate states.
- **Only an explicit commit writes the result back.**
This matters most for inline instances. Their state still has to live
somewhere on its own while it is being worked on, and the draft is that place.
**Token rules.** These apply to draft tokens and to the data chunk tokens of
section 8:
- **Unguessable, expiring, and never logged.** Draft tokens are also
invalidated on commit or delete.
- **A locator, never an authorization.** Access is re-checked on every use.
- **Bound to the authenticated principal and its authorization context, not
to a browser session.** The principal may be a user, an embedded guest token,
or a service identity acting for a user. A user's browser tab and an agent
acting as that user are the same principal, so they can work on the same draft.
**Versioning and concurrency.** A draft records the `schema_version` its
props conform to and a `revision` incremented on every change. Every write
states the revision it is based on, as an `If-Match` ETag or in the body. On a
mismatch the write is rejected with a conflict that includes the current
revision, so a concurrent edit from another tab or an agent is never silently
overwritten. At commit, a draft of a persisted instance checks the instance's
revision, and a draft of an inline instance checks the revision of the Canvas
it started from.
**How changes are sent.** The UI replaces the whole props object on each
edit. Agents send **JSON Patch (RFC 6902)** operations, which are compact and
express inserting, removing and reordering array items precisely.
**Patch mechanics.**
- All six RFC 6902 operations are supported. `move` and `copy` are validated
like an `add` at the destination, and a `move` also counts its source as a
written path. A failing `test` rejects the whole patch.
- Patches are atomic: they apply completely or not at all.
- Configurable limits cap document size, patch size and operations per patch.
**Validation is strict about what a write touches, and tolerant of what
isn't finished yet.**
- **A write can't introduce an error.** A write is rejected, leaving the
draft unchanged, if it would leave an error at any path it writes: a wrong
type, an unknown key, a value outside an enum, a reference that doesn't exist
(such as a metric the dataset doesn't have), or a malformed path. For a
whole-props replacement, the written paths are the ones whose values changed.
- **A draft may be unfinished.** Missing required fields never block a
write, so an instance can be built up one field at a time. References made
stale by a change elsewhere (switching the dataset makes the selected metrics
stale) are kept and reported, not silently cleared. An agent can also change
dependent fields together in one atomic patch.
- **Two gates.** Preview requires the fields that affect the query to be
valid. Commit requires full validation: the complete schema, datasource access,
existence of every referenced column and metric, migrations, and validation of
the queries the instance would build (section 7).
- **Visible state.** The whole draft is revalidated after every accepted
write, and exposes whether it can be previewed, whether it can be committed,
and its open errors.
**Errors are written for an agent to act on without guessing.** Each carries
a JSON Pointer path, a stable code, a plain-language message, what was expected
and received, permission-filtered suggestions (such as valid values or the
metrics available on the dataset), a next step (such as a path to expand or an
options lookup to call), and what it blocks. The format is in Appendix C.
**Committing.** A draft writes its props back to where the instance lives:
- **Persisted instance:** to the `WidgetInstance` table.
- **Inline instance:** to the Canvas document, through the Canvas's own
update operation (companion SIP).
- **New instance:** created in either mode.
**There is one write path.** The property panel, a raw JSON editor, an
extension calling the public Canvas API, and an MCP agent all go through the
same validation, whichever storage mode the instance uses.
</details>
<details>
<summary><b>7. Querying: <code>QueryMixin</code> and
<code>SemanticQuery</code></b></summary>
A chart turns an instance's props into queries on the backend.
`SemanticQuery`, introduced by the Semantic Layer work (SIP-182), is the only
query format widget authors see. It is a small, typed model: metrics,
dimensions, filters, order, limit and group limit. The four core composite
controls map onto it one-to-one, which keeps most `build_queries`
implementations simple.
**Datasets get an adapter.**
- Today a `SemanticQuery` can only be executed against an external semantic
view.
- This SIP adds a **`SemanticView` adapter for regular datasets
(`SqlaTable`)**. It runs a `SemanticQuery` through the existing query engine by
translating it into a `QueryObject`. That is the reverse of the mapping that
already exists for semantic views (`superset/semantic_layers/mapper.py`).
- Widget authors then write a single query path that works for both datasets
and semantic views.
- Charts backed by datasets keep every engine behavior they have today:
row-level security, Jinja templating, caching and time grains.
**Query boundaries.** Widgets are written by core and by extensions.
Extensions are trusted by the operator (see `SECURITY.md`), so the concern
isn't malicious code. The concern is a widget accidentally producing a query
that a Gamma user, an embedded guest or a custom role isn't entitled to run.
The framework therefore enforces the boundaries itself instead of relying on
each widget:
- **Widgets describe, the framework executes.** `build_queries` returns
query descriptions, and the framework binds them to the instance's datasource
and runs them. A widget never executes a query or opens a connection itself.
- **Every query is checked.**
- Metrics and dimensions are re-resolved by id against the datasource's
catalog.
- This matters because metrics and dimensions in a `SemanticQuery` carry
SQL `definition` strings. Any expression not found in the catalog is treated as
user-authored ad-hoc SQL and goes through the same validation as ad-hoc SQL in
v1.
- Filter, ordering and grouping fields are checked the same way.
- **Only declared datasources.** An instance queries the datasources its
props declare, after authorization. A query can't add an undeclared secondary
datasource.
- **Runtime filters**, meaning Canvas filters and cross-filters (section 9),
**are applied centrally** after `build_queries`. Widgets never add or remove
filters in code.
- **Declared policy per query.** Each `WidgetQuery` declares how runtime
filters apply to it:
- `apply`: the default;
- `ignore`: for example the denominator of a share-of-total;
- possibly a constrained subset later.
- **Visible to users.** The policy is reported in the data manifest
(section 8) and shown in the Inspector, so it's clear why a result didn't react
to a filter.
- **Not a security boundary.** Runtime filters only narrow data the user
is already entitled to, so the policy is a consistency rule.
- **Server limits win.** Row limits, query counts, timeouts and result sizes
configured on the server clamp whatever a widget requests.
- **Row-level security and user context are applied at execution**, in the
existing engine through the `SqlaTable` adapter, including cache keys that
depend on the user.
These rules uphold the Gamma ("only granted datasets"), embedded guest token
and custom-role rows of the role and capability matrix in `SECURITY.md`.
**Transforms.** `transform_data` is a pure function of the props and the
query results:
- **Behavior:**
- it issues no additional queries and does no I/O;
- it is deterministic, so its output can be cached and reports are
reproducible;
- it runs in the same execution context as its queries: synchronously in
M1, and later asynchronously through the Global Task Framework (SIP-227).
- **Output is bounded and typed:**
- `context` uses JSON types only (strings, numbers, booleans, null, lists
and string-keyed objects) and has a size cap;
- datasets are uniquely named, their Arrow types are normalized to a
documented supported set, and their size is capped.
- **Errors and warnings** are structured and returned in the manifest.
- **Multi-query failure:** if any of an instance's queries fails, the
instance fails. This is the M1 behavior; partial results are an open question.
**Caching.**
- Raw query results from `build_queries` and from schema enrichment are
cached through the existing cache layer and shared across instances.
- Transformed output is cached under a key made of:
- the identities of every raw result it was computed from. Those are the
cache keys of the underlying queries, which already capture every query input:
runtime filters, row-level security and other user context, and the
datasource's revision;
- the instance's revision;
- the implementation version (section 11);
- the locale, when the output depends on it.
Two executions of the same instance with different filters or different
user context therefore never share transformed output, and deploying new
transform code invalidates it.
- The server can warm caches, and alerts, reports and thumbnails can get a
chart's data, without running any JavaScript.
</details>
<details>
<summary><b>8. Delivering data: access by widget instance</b></summary>
Those queries run when a chart is displayed, and their results are fetched
by widget instance, not by query:
```
POST /api/v1/widget_instance/{uuid}/data (persisted
instance)
POST /api/v1/canvas/{uuid}/placement/{placement_id}/data (inline
instance)
body: {"runtime_filters": [...]}
→ 200 JSON manifest: {context, warnings, datasets: [{name, arrow_schema,
row_count, inline_data | chunks}]}
GET /api/v1/widget_instance/data/{token}/{dataset}/{chunk} (large results
only)
→ 200 application/vnd.apache.arrow.stream
```
- **The query definition always comes from storage.**
- The server loads the instance's props and datasource, from the
`WidgetInstance` table or from the Canvas document.
- It then validates and runs the instance's queries (section 7), runs
`transform_data`, and returns the result.
- The caller's only input is `runtime_filters`, meaning Canvas filters and
cross-filters. These are validated to be strictly additive constraints on
columns of the instance's own datasource.
- A caller cannot add metrics, remove group-bys, or select raw columns.
- **Unsaved instances** (drafts, see section 6) are previewed through their
draft token, using the same pipeline.
- **The transport is a JSON manifest plus Arrow IPC.**
- Query results are already Arrow tables on the backend, and the browser
decodes them with `apache-arrow`.
- **Small results** come back inline in the manifest response, so the
common case needs a single request per placement.
- **Large results** are split into Arrow IPC chunks fetched over plain
HTTP. This avoids page loads that need one manifest request plus many chunk
requests for every placement.
- The manifest itself stays JSON, which makes it easy to log, cache and
hand to an agent.
- Arrow Flight does not work in browsers and is not part of this proposal.
- **Chunk tokens** follow the token rules in section 6, and are also bound
to the instance revision and the resolved query, including its runtime filters.
Chunk responses carry `Cache-Control: private`, chunk size and count are
limited, and if a chunk fetch fails the renderer requests the manifest again.
- **Shared caches are safe.** Results are shared across users only when the
cache key captures everything that affects them: row-level security,
impersonation and templating that depends on the user. One Gamma user or
embedded guest can therefore never receive another principal's result.
- **Access checks:**
- **Persisted instance:** reading its data requires access to the instance
*and* to its datasource.
- **Inline instance:** reading its data requires access to the Canvas
*and* to the instance's datasource.
- **Either way, today's data entitlement is unchanged.**
- **Later:** because the stored definition owns the query's shape, the
design makes it possible to grant "read this instance's output" without
granting the dataset, giving out an aggregate without its granular rows. That
would add a row to the role and capability matrix in `SECURITY.md`, so it is
left for a follow-up proposal rather than enabled here.
</details>
<details>
<summary><b>9. Reacting to filters: the interactivity contract</b></summary>
Filters, cross-filters and drill-down move off the central Redux `dataMask`
store. This SIP defines only what a widget needs in order to take part. The
companion Canvas SIP defines the listener bus that connects placements on a
Canvas: where it lives, how filter scope is configured, how filter state is
held and serialized into URLs and permalinks, and how cascading filters work.
**Filter scope belongs to placements, not instances.** Each placement
subscribes to the filters in its own scope and sends them as `runtime_filters`.
Two placements of the same instance can therefore show different results.
Runtime filters are part of the cache key, so the two never share cached
results.
What a widget needs:
- **Runtime filters as input.** The data endpoints accept `runtime_filters`
in the same vocabulary as the chart's own filters: `{column, operator, value,
datasource?}`. That gives one filter concept throughout. The framework applies
them to the instance's queries (section 7).
- **Hooks to subscribe and emit.**
- `useRuntimeFilters()` returns the filters currently applying to the
placement, and refetching follows automatically.
- `emitFilter()` publishes a filter: clicking a bar emits `region = EMEA`.
Drill-down works the same way.
- The hooks are public API in `@apache-superset/core`, so built-in and
extension widgets take part in exactly the same way.
- **Declared interactivity.** The widget's behavior declaration says whether
it emits filters or supports drill-down, so a Canvas can offer the right
affordances.
For v2 widgets, the Redux `dataMask` / `getFormDataWithExtraFilters`
pipeline is replaced, not wrapped.
</details>
<details>
<summary><b>10. Rendering: the frontend contract</b></summary>
A widget's frontend is **a React component and nothing else**: no
`buildQuery`, no `transformProps`, and no control-panel code. Inside that
component the renderer can use any library, such as ECharts, D3, deck.gl, SVG,
the HTML canvas element, WebGL or plain HTML/CSS, or any mix of them. The
renderer doesn't need to know whether its instance is persisted or inline, or
how many times it is placed.
Authors compose ordinary hooks from `@apache-superset/core`:
`useWidgetData()` fetches the instance's data and refetches when runtime
filters change, and `useRuntimeFilters()` and `emitFilter()` take part in
interactivity (section 9).
The property panel is rendered generically from the served schema with
JSONForms, which Superset already uses for Semantic Layer configuration, plus
the composite renderer registry from section 3. Widget renderers are loaded
lazily, and each declares the schema versions it supports (section 11).
</details>
<details>
<summary><b>11. Evolving: versioning, compatibility and
lifecycle</b></summary>
Widgets change over time, through new releases and as extensions are
installed, upgraded or removed. Saved instances have to survive all of that.
**Schema versions and stored values.** Each widget's schema has an integer
`schema_version` that only increases. Only explicitly set values are stored,
which keeps props small for agents and diffs. Any change to what stored props
mean requires a version bump and a migrator, including changing a default: the
migrator writes the old default into existing instances, so they never change
behavior by accident.
**Migrations** are applied in memory whenever an instance is read, and the
upgraded props are saved on the next commit, so read paths never write. An
optional offline command upgrades every stored instance in bulk. Migrators must
form an unbroken chain to the current version, and there are no downgrades.
**When an instance can't be loaded.** An instance becomes a durable
placeholder in three cases:
- its migration fails;
- it was saved by a newer version of its widget than the server has;
- its widget is unavailable, for example because the extension that provides
it is disabled or removed.
The placeholder keeps the raw props, widget id, schema version and placement
exactly as stored, and never normalizes or overwrites them, so re-enabling the
extension or upgrading the server restores the instance. The rest of the Canvas
keeps working, and the instance's data endpoint returns a structured "widget
unavailable" error.
**Implementation version.** A widget's implementation version is the version
of the package that provides it: the Superset release for built-in widgets, and
the extension's manifest version for extension widgets. It is the same across
identical deployments (unlike a build hash) and changes with every release, so
any change to query or transform semantics is covered. That is conservative for
caching, but never stale. Transformed-output caches and exports use it.
**Exports and snapshots.** Exports record the widget id, the
`schema_version` and the implementation version. Old renderer and backend code
isn't kept around for snapshots, so a snapshot that contains an unavailable
widget shows placeholders.
**Frontend and backend handshake.** The backend decides which widgets exist,
but the browser also needs a matching renderer, and during rolling deploys or
extension updates the two can briefly disagree. Both sides declare a widget
contract version, each renderer declares the schema versions it can render, and
the frontend reports mismatches with the backend's widget list in diagnostics.
- **Only the renderer is missing or incompatible** (missing, failed to load,
or not supporting the instance's schema version): the visualization shows a
placeholder with a retry option. The property panel keeps working, because the
schema comes from the backend, as long as the backend understands the stored
schema version and the client supports the contract version and every control
the schema requires.
- **Otherwise, nothing can be edited.** An instance saved by a newer version
than the server knows, or a schema whose contract version the client doesn't
support, becomes a read-only placeholder for both rendering and editing.
- Schema responses carry an ETag derived from the contract version, schema
version, locale and draft revision, so caches are invalidated when any of them
changes.
</details>
<details>
<summary><b>12. Agent access: the MCP surface</b></summary>
Every step above is exposed to agents as MCP tools, through the existing MCP
service and its permission-aware `@tool` decorator. The tools wrap the same
commands the UI uses, so they share validation, permissions and behavior:
| Tool | Purpose |
|---|---|
| `list_widgets` | Discover registered widgets (id, name, description) |
| `get_widget_info` | A widget's metadata, including its behavior |
| `get_widget_control_schema` | The minimal view, or expanded branches by
path; takes a draft token so the draft's props drive enrichment (section 4) |
| `get_widget_field_options` | Search and paginate a field's options, such
as datasets, columns or distinct values |
| `create_widget_instance_draft` / `get_widget_instance_draft` | Start a
draft from a persisted instance, an inline instance or a widget; read a draft
and its validation state |
| `patch_widget_instance_draft` | Apply a JSON Patch under the rules of
section 6 |
| `preview_widget_instance_draft` | Run the draft's queries, optionally with
runtime filters; return a manifest or a small sample |
| `commit_widget_instance_draft` | Fully validate the draft and write it
back, or create a new instance |
</details>
### New or Changed Public Interfaces
<details>
<summary>Full list</summary>
**Backend (`superset-core`)**
- `superset_core.widgets`:
- the `Widget` base class, `QueryMixin`, and the `@widget` decorator (in
`superset_core.widgets.decorators`);
- the base props model, which forbids undeclared fields;
- behavior and UI metadata types;
- `WidgetQuery` (a named query with its runtime-filter policy) and
`WidgetPayload`;
- the enrichment API;
- the migrator API;
- the composite-control registry;
- the widget contract version.
- A `SemanticView` adapter for `SqlaTable` datasets.
**REST: widgets (`/api/v1/widget`)**
- `GET /`, `GET /{id}`;
- `POST /{id}/control-schema` (minimal or full view, optional `paths`,
optional draft token);
- `POST /{id}/field-options` (search and paginate a field's options);
- `POST /{id}/validate`.
**REST: widget instances (`/api/v1/widget_instance`)**
- **Persisted instances:** CRUD on `/{uuid}` with revision checks, plus
import and export.
- **Drafts:** `POST /draft` (from a persisted instance, an inline instance
or a widget), `GET|PUT|PATCH|DELETE /draft/{token}` (responses include the
validation state), `POST /draft/{token}/data`, `POST /draft/{token}/commit`.
- **Data:** `POST /{uuid}/data`, `GET /data/{token}/{dataset}/{chunk}`.
**REST: Canvas resource:** `POST
/api/v1/canvas/{uuid}/placement/{placement_id}/data` for inline instances. The
rest of the Canvas resource, including placements, promoting and inlining
instances, and exporting snapshots, is defined in the companion Canvas SIP.
**MCP:** the tools in section 12.
**Frontend (`@apache-superset/core`)**
- registration of widget renderers, keyed by namespaced widget id, with the
schema-version range each one supports;
- the data and interactivity hooks (`useWidgetData`, `useRuntimeFilters`,
`emitFilter`);
- runtime-filter types;
- the composite renderer registry;
- the widget contract version.
**Security:** a new `Widget` permission resource (`can_read`, `can_write`),
plus the data access rules (section 8), query boundaries (section 7) and token
rules (section 6).
**Configuration:** a feature flag gating the API routes, the MCP tools and
the Canvas builder (section 5).
**Models:** a `WidgetInstance` table for persisted instances, plus its
editors/viewers association tables. Inline instances live in the Canvas
document (companion SIP).
**Unchanged:** v1 chart plugins, Explore, existing dashboards,
`/api/v1/chart/data` and `query_context`.
</details>
### New dependencies
- **`apache-arrow` (npm):** decodes Arrow IPC in the browser. Apache License
2.0, maintained by the Apache Arrow project. `pyarrow` is already a backend
dependency.
- **A JSON Patch implementation for Python,** for example `jsonpatch`
(BSD-3-Clause). It is small and stable. Given the limited set of operations, it
could also be implemented in-house.
- **No new frontend form library:** `@jsonforms/*` is already a dependency.
### Migration Plan and Compatibility
Nothing changes for existing deployments until they opt in. v1 charts,
Explore and existing dashboards are untouched, and the new surface sits behind
a feature flag. Phases are gated on milestones rather than release dates; each
phase's release number is set when its milestone is reached.
1. **M1: Framework.** Ships behind the feature flag:
- the widget contract, the schema vocabulary and its conformance suite;
- the `SqlaTable` adapter and framework-enforced query boundaries;
- persisted instances (the table and its migration), inline instances and
placements;
- drafts, the data endpoints and the MCP tools;
- the interactivity hooks;
- the core widgets Canvas needs.
2. **M2: Parity.** The most-used visualization types are available as v2
widgets, with an authoring experience equivalent to Explore. The feature flag
turns on by default.
3. **M3: Convergence.** A migration converts saved `Slice` rows into
persisted widget instances wherever a v2 widget exists for them. Charts that
aren't on any dashboard become persisted instances without placements. v1 chart
plugins and the v1 chart-data path are formally deprecated, and the deprecation
is recorded in `UPDATING.md`.
4. **M4: Removal.** v1 chart plugins and their supporting code are removed
in the next major release.
**Database migrations:** M1 adds the widget instance tables, and M3 adds the
data migration from `Slice`. Upgrades of props between schema versions are
handled by each widget's own migrators (section 11), not by database migrations.
**Stored URLs:** v1 chart and dashboard URLs keep working through M3. Canvas
permalinks are defined in the companion Canvas SIP.
### Rejected Alternatives
<details>
<summary>Alternatives considered</summary>
- **A Node.js sidecar that builds queries on the server (SIP-211).** It
takes the browser out of the loop, but keeps query logic in JavaScript and adds
a runtime to deploy and operate. Query building in Python needs no extra
service, and is inspectable and testable on the backend.
- **Letting widgets execute their own queries.** Every widget, including
extension-provided ones, would have to get authorization, row-level security,
limits and caching right on its own.
- **Schemas owned by the frontend, or a serializable subset of
`controlPanel.tsx`.** An agent would need a browser, or a copy of the frontend
registry, to know what a widget accepts, and backend validation would need a
second copy of the rules.
- **Two independent registries (frontend and backend) joined by an id.** The
palette, the Inspector and agents could then disagree about which widgets exist.
- **Registering widgets by subclassing alone.** Abstract base classes and
mixins would register by accident, and there would be no natural point for the
host to apply extension namespacing.
- **Reusing the `Slice` table.** It would inherit ownership, tags and
dashboard relationships for free, but every v1 code path assumes `params` and
`query_context`, and the new table holds instances of every kind of widget, not
just charts. `Slice` rows can migrate into it later.
- **A single storage mode.** Persisted-only would make every text block and
container a separate row with its own permissions, and snapshots would have no
natural self-contained form. Inline-only would leave no stable id for reuse,
multiple placements, instances that live outside any Canvas, separate
permissions, or direct addressing by agents, reports and links.
- **Requiring every draft revision to be fully valid.** Building a new
instance, or switching its dataset, would need one large atomic replacement for
every change.
- **Shallow merges or JSON Merge Patch for writes.** Neither can precisely
express inserting, removing or reordering array items, which nested controls
need.
- **Storing defaults in props.** Every stored instance would freeze every
default, adding noise for agents and diffs.
- **Arrow Flight to the browser.** Browsers can't speak gRPC-based Flight
without gRPC-web and a proxy. Arrow IPC over HTTP gives the same columnar
benefits, and Flight can still be adopted later between services.
- **Wrapping the Redux `dataMask` pipeline for v2.** This would keep the
Canvas as the central orchestrator, and extensions could not take part on equal
terms.
</details>
### Prototype
A proof of concept on the `dashboard-v2` feature branch evaluated the
approach end to end. It confirmed that:
- core and extensions can register widgets through one contract in
`superset_core.widgets` (#43262);
- schemas generated from Pydantic models, with dynamic fields enriched in
dependency order, can drive data-aware controls for dataset, column, metric
(including custom SQL) and per-series styling (#43588);
- an agent can build and restyle charts through progressive disclosure,
without loading whole schemas (#43262);
- specific validation errors let an agent correct its own requests (#43262);
- an event bus lets filters and cross-filters reach widgets, scoped to the
same datasource by default (#43400);
- chat extensions can register client-side tools that drive the same widgets.
### Related proposals
- **SIP-182, Semantic Layer support:** the source of `SemanticQuery`,
`SemanticView`, and the progressive JSON Schema pattern this SIP builds on.
- **SIP-151, plugins architecture:** the extensions framework that delivers
widgets.
- **SIP-211, server-side query context generation:** the Node.js sidecar
alternative (see Rejected Alternatives).
- **SIP-219, a uniform model for built-in extensions:** built-in and
extension widgets share one registration model.
- **SIP-207, replacing Redux with Zustand and TanStack Query:**
complementary; v2 interactivity doesn't use Redux.
- **SIP-227, async and realtime by default:** the natural home for running
widget queries and transforms asynchronously.
- **Canvas SIP (companion, forthcoming):** the Canvas document, placements
and their filter scope, promoting and inlining instances, the snapshot export,
and the listener bus (filter state, URL serialization, cascading filters).
### Open questions
- **Default storage mode.** When a widget instance is created on a Canvas,
should it be inline or persisted by default? Should the default depend on the
widget (for example inline for text and containers, persisted for charts)?
- **Access through a Canvas.** Can Canvas viewers read a persisted instance
they reach through the Canvas without explicit access to the instance? This is
the same question `VIEWER_PROMISCUOUS_MODE` answers for dashboards and charts
today.
- **Inline instances beyond snapshots.** Are inline instances a general
storage mode on live canvases, or only the form instances take inside an
exported snapshot? Isolated drafts support either choice.
- **Snapshot data.** Does an exported snapshot include only instance
definitions, or also their query results? Arrow results make bundling the data
feasible.
- **Standard `context` keys.** Which parts of the result `context` (pivot
metadata, column roles, series descriptors) should be standardized first?
- **Descriptions.** Should widget descriptions be free text or lightly
structured (purpose, when to use, how to read), and should agents propose an
instance's description when creating it?
- **Running several queries.** Parallel or sequential execution, per-query
caching and `force`, partial results instead of failing the whole instance, and
reusable join helpers.
- **Batch data requests.** Should one request fetch data for several
placements, beyond inlining small results?
- **Remaining composite controls.** Time grain, conditional formatting,
annotation layers and the other v1 composites beyond the first milestone.
- **Pruning heuristics.** Should branches collapse by kind (objects
collapse, scalars stay inline, as in section 4) or by size (inline any branch
below a token budget)? And should expanding a branch accept a depth or budget
hint?
- **Recursive schemas.** Should self-referencing models, such as nested
filter groups, be supported in a later version of the schema subset?
- **Embedded canvases.** How guest tokens authorize reading data, for both
inline and persisted instances.
- **Reports and thumbnails.** How alerts, reports and thumbnails adopt
`build_queries` for v2 charts.
### Appendices
<details>
<summary><b>Appendix A: Supported JSON Schema subset and conformance
suite</b></summary>
**Dialect.** JSON Schema draft 2020-12, as emitted by Pydantic v2.
**Supported constructs:**
- **Types:** `object`, `array`, `string`, `number`, `integer`, `boolean`,
`null`.
- **Objects:**
- `properties` and `required`;
- `additionalProperties` set to a schema, for typed maps such as
per-series settings;
- `additionalProperties: false`, which is implied by strict props.
- **Arrays:** `items` with a single schema (homogeneous arrays), `minItems`,
`maxItems`, `uniqueItems`.
- **Values:** `enum`, `const`, `default`.
- **Scalar constraints:** `minimum`, `maximum`, `exclusiveMinimum`,
`exclusiveMaximum`, `multipleOf`, `minLength`, `maxLength`, `pattern`, `format`.
- **Annotations:** `title`, `description`, and the documented `x-*` keywords.
- **References:** local `$defs` and `$ref`. A `$ref` may also be wrapped in
a single-entry `allOf` to attach annotations.
**Canonical encodings:**
- **Nullable field:** `anyOf: [<schema>, {"type": "null"}]`.
- **Discriminated union:** `oneOf` plus `discriminator` (`propertyName` and
`mapping`). Each branch is an object whose discriminator property is a `const`.
- **Fixed set of choices:** `enum`, with labels in `x-enumNames`.
**Not supported, and rejected at registration:**
- tuples (`prefixItems`);
- conditional schemas (`if`/`then`/`else`, `dependentRequired`,
`dependentSchemas`);
- `patternProperties`;
- `not`;
- untagged `anyOf`/`oneOf` (other than the nullable encoding);
- `allOf` beyond the single-`$ref` wrapper;
- remote references;
- recursive references (deferred; see open questions).
**Pydantic constructs widget authors may use:**
- **Models:** subclasses of the framework's base props model.
- **Scalars:** `str`, `int`, `float`, `bool`, `Literal`, `Enum`, `UUID`,
`date`, `datetime`. These are validated strictly in JSON mode (section 3).
- **Containers:** `list[T]`, `set[T]` (as `uniqueItems`), `dict[str, T]`.
- **Optional and union types:** `Optional[T]`, and discriminated unions
(`Annotated[Union[...], Field(discriminator=...)]`).
- **Constraints:** declared with `Field`.
Anything that emits an unsupported construct fails registration with a
message naming the field and the construct.
**Unknown `x-control` values.**
- **Backend:** registration fails if a schema uses an `x-control` that isn't
registered.
- **Frontend:** a control it doesn't know, for example during version skew,
falls back to a generic renderer chosen by JSON type. If there is none, it
shows a read-only JSON view of the value. Unknown controls never silently drop
data.
**Conformance suite.** A shared set of fixtures, each made of:
- a schema;
- props;
- the expected minimal view and expansions;
- the expected validation errors, in the format of Appendix C;
- the expected rendering outcome.
Fixtures cover strict validation of booleans, integers, numbers, UUIDs,
dates, enums and nested models. Backend CI runs the suite for disclosure and
validation, frontend CI checks that every field renders and values round-trip
unchanged, and the MCP tests run it too. Widgets can contribute fixtures, and
extension authors can run the suite against their own widgets.
</details>
<details>
<summary><b>Appendix B: Registration checks</b></summary>
When a widget registers, the framework checks that:
1. **Strict props.** Its props model, and every nested model (including
inside lists, maps and unions), inherits the framework base model that forbids
undeclared fields and enables strict mode.
2. **Supported schema.** The generated schema stays within the supported
subset (Appendix A).
3. **Known keywords.** Every `x-*` keyword is in the documented list, and
every `x-control` names a registered composite control.
4. **Enrichers wired.** Every `x-dynamic` field has an enricher, and every
enricher input is a valid JSON Pointer into the schema.
5. **No cycles.** The enrichment dependency graph has no cycles.
6. **Valid defaults.** Every default value validates against its own field.
7. **Complete minimal view.** The minimal view (section 4) contains every
required path, so the root alone is enough to build a valid instance.
8. **Unbroken migrations.** Migrators form an unbroken chain from version 1
to the current `schema_version`.
9. **Consistent metadata.** Behavior declarations and UI metadata are
consistent, for example a container declares which child widgets it accepts.
10. **Compatible contract.** The widget declares a widget contract version
that the server supports.
If a built-in widget fails, startup stops, which CI catches. If an extension
widget fails, it isn't registered, the failure shows up in diagnostics, and
existing instances of that widget show placeholders (section 11).
Whether a matching frontend renderer exists can't be checked here, because
the backend can't see frontend bundles. That check is part of the handshake
(section 11).
</details>
<details>
<summary><b>Appendix C: Validation error format</b></summary>
Every validation error, whether returned for a rejected write, listed in a
draft's validation state, or returned at commit, has this shape:
```json
{
"path": "/dataBinding/metrics/0",
"code": "unknown_metric",
"message": "Metric \"sum__sales\" doesn't exist on dataset \"orders\".",
"expected": "a metric defined on dataset 42",
"received": "sum__sales",
"suggestions": ["sum__amount", "count"],
"next_step": {"options_lookup": "/dataBinding/metrics"},
"blocks": "write"
}
```
- **`path`:** a JSON Pointer to the offending value, usable directly for
disclosure and patching.
- **`code`:** a stable, documented identifier, for example `type_mismatch`,
`unknown_key`, `invalid_enum_value`, `unknown_metric`, `missing_required`,
`stale_reference`, `forbidden_datasource`, `test_failed`, `revision_conflict`
or `limit_exceeded`.
- **`message`:** plain language that names the field and the problem,
without internal jargon.
- **`expected` and `received`:** what the schema accepts, and what was sent.
- **`suggestions`:**
- **For enums and other bounded choices:** the valid values, nearest
matches first.
- **For unbounded references:** the closest matches, capped.
- **Always permission-filtered:** suggestions never include datasets,
columns, metrics or values the principal can't access.
- **`next_step`:** where to go next. That is a path to expand, an options
lookup to search, or another field to set first.
- **`blocks`:** what the error prevents:
- `write`: the write was rejected and the draft is unchanged;
- `preview`;
- `commit`.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
