msyavuz opened a new issue, #44989:
URL: https://github.com/apache/superset/issues/44989
## [SIP] Proposal for Canvas, the document that places widgets
Canvas is the next generation of Superset dashboards: a tree of placements
of SIP-231 widget instances, edited live through small, validated operations by
people and agents alike. It is the "companion Canvas SIP" that SIP-231 (#44875)
defers to for the Canvas document, placements and their filter scope, promoting
and inlining instances, the snapshot export, the listener bus (filter state,
URL serialization, cascading filters) and Canvas permalinks.
- **Agents edit a canvas reliably and cheaply.** "Put the revenue trend next
to the KPIs and make the region filter apply to the whole page." The agent
reads a compact document with readable ids and sends a few named operations;
the server validates every one.
- **A canvas can't be left broken.** Every write is checked against the
tree, each widget's nesting and layout rules, and its props schema. A bad write
is rejected with a JSON-pointer path and changes nothing.
- **Many editors at once.** Edits apply against a base revision; edits that
don't overlap merge, overlapping ones are rejected with the placements in
conflict. Open pages update live.
- **Any widget, from core or an extension.** Containers (tabs, groups, a
filter bar) are widgets too; the canvas grammar stays small and fixed.
- **Filters that follow the layout.** A filter drives the widgets in its tab
by default; a filter in the filter bar drives the whole canvas. Overrides are
explicit and per placement.
- **Upgrades that never lose work.** Placements whose widget is missing or
newer than the server stay as placeholders, exactly as stored.
Existing dashboards keep working alongside Canvas. Conversion and retirement
are phased and gated on milestones.
**Scope.** This SIP covers everything *between* placements: the document,
layout and nesting, filter scope and the listener bus, editing and concurrency,
snapshots, permalinks, storage and access for canvases. Everything *inside* a
placement (what props mean, their schema and validation, queries, data,
renderers and widget versions) is SIP-231's, and this SIP only references it;
the ops that write inline props are this SIP's.
> **Open questions for reviewers** (details at the end)
> 1. Changes this SIP needs from SIP-231: runtime choices for
customizations, and one shared flag.
> 2. Unfinished instances: can a placement hold a widget that isn't fully
configured yet?
> 3. Previewing a change before every viewer sees it.
> 4. Live layout edits: placing a widget can push others down for every
viewer at once. Is that acceptable without an edit mode?
> 5. The op log lets viewers read changes made before they had access.
> 6. History and undo: the op log could power both, but canvases aren't
versioned yet.
> 7. Filters across datasources: matching by column name, or an explicit
mapping on the scope override.
> 8. Conversion from dashboards: which unconverted features block a
dashboard from converting, and which are only reported.
### Motivation
1. **Dashboards are hard for agents to edit.** A dashboard is
`position_json` plus `json_metadata`, saved whole. Every layout change rewrites
the full tree, including `parents` arrays and wrapper `ROW`/`COLUMN`
components, and the API accepts any JSON. In our evaluation (see Evaluation
below), a small model left 4 of 27 hard-tier dashboards unrenderable without
being told, and every model paid 2.7–3.8× more per edit than on the canvas
schemas.
2. **The server doesn't know the dashboard grammar.** Nesting rules live in
the frontend (`isValidChild.ts`); on save the server only repairs unreachable
components. Ids are client-made and never checked.
3. **Last write wins.** A whole-document `PUT` silently overwrites
concurrent edits, which matters more once agents edit alongside people.
4. **Filter scope is stored as layout paths plus caches.** `rootPath` and
`excluded` with derived `chartsInScope`/`tabsInScope` caches; cross-filters and
customizations each have their own config shape.
5. **SIP-231 needs a home for placements.** Widget instances are persisted
or inline; both need a document that places them and scopes their filters, and
inline instances need a way to change.
#### Evaluation: which schema works best for agents
We evaluated candidate schemas by having Claude models make real edits
through MCP tools against a scored copy of a canvas: 26 tasks (16 single-intent
edits on a 15-placement canvas, 10 harder ones on a 51-placement canvas), 3
samples each, with Haiku 4.5 and Sonnet 5.5 on seven canvas schemas and today's
dashboards (which skip one hard task that has no dashboard equivalent), plus
the factorial screening below: 1,722 runs. A run is **clean** when the
requested end state holds and nothing else changed. Known-correct solutions
score 100% clean on every schema, and no task passes on an unchanged canvas.
| Hard tier, clean % / cost per run | Today's dashboards | Canvas schemas |
| --- | --- | --- |
| Haiku 4.5 | 74% / $0.142 | 93% / $0.053 |
| Sonnet 5.5 | 93% / $0.295 | 97% / $0.078 |
A fractional factorial over six schema dimensions (16 schemas, 240 runs per
level) found no reliability effect from referenced vs inline config,
tree-derived vs explicit scope, sparse vs normalized props, or flat vs nested
structure: every level landed within two points. Two dimensions changed cost by
about a third with no reliability loss: **readable ids** instead of UUIDs, and
**leaving resolved placements and scopes out** of the default read. Both are
adopted below (§3, §9). Failures clustered by task (bulk resizing, vague
references) rather than by schema.
### Proposed Change
#### 1. Concepts
Terms follow SIP-231.
- **Widget:** a kind of thing that can be placed, from core or an extension
(SIP-231 §2). Containers are widgets.
- **Widget instance:** one configured occurrence of a widget, **persisted**
(its own UUID, in SIP-231's `WidgetInstance` table) or **inline** (stored on
its placement as SIP-231's `widget_id`, `schema_version` and `props`, spelled
`widget`, `schemaVersion` and `props` in the document's camelCase).
- **Placement:** an instance's spot on a canvas (SIP-231 §5): its id,
layout, children and filter scope. Scope overrides are stored under
`interactions`, keyed by placement id, so every kind's overrides sit together;
they belong to the placement and go when it goes.
- **Canvas:** a document of placements, interactions and settings, plus row
metadata (title, slug, theme, CSS, certification, editors, viewers).
| Owned by this SIP | Owned by SIP-231 |
| --- | --- |
| The document: placements, tree, layout, interactions, settings | Widgets,
instances and their props |
| Containers' layout and nesting rules, scope roles (fields added to
`WidgetBehavior`) | Props schemas, validation, errors, progressive disclosure |
| Filter scope, the listener bus, cascading, permalinks | Runtime filter
vocabulary and its hooks; applying filters to queries |
| Operations (including edits to inline props), concurrency, live updates |
Edits to persisted instances |
| Canvas storage, slugs, access, embedding | `WidgetInstance` storage and
access; instance and placement data routes |
| Document versions (Alembic) | Props versions (migrators) |
| Snapshot format and access rules | What an instance contributes to a
snapshot |
| Canvas MCP tools | Widget MCP tools |
<details>
<summary><b>2. The canvas document</b></summary>
```json
{
"version": 1,
"root": {"layout": {"columns": 24, "gap": 16, "rowUnit": 40}, "children":
["kpis", "overview"]},
"nodes": {
"kpis": {"widget": "group", "schemaVersion": 1, "props": {}, "layout":
{"colSpan": 24, "rowSpan": 4},
"children": ["revenue", "orders"]},
"revenue": {"instance": "6f1c2b7e-2d4a-4c1e-9a53-0f3b8d2e7a10",
"layout": {"colSpan": 6}},
"orders": {"widget": "metric-tile", "schemaVersion": 1,
"props": {"dataBinding": {"datasetId": 7, "metrics":
["count"]}}, "layout": {"colSpan": 6}},
"overview": {"widget": "tabs", "schemaVersion": 1, "props": {},
"layout": {"colSpan": 24, "rowSpan": 12},
"children": ["emea"]},
"emea": {"widget": "tab", "schemaVersion": 1, "props": {"title":
"EMEA"}, "layout": {},
"children": ["region", "trend"]},
"region": {"widget": "filter.select", "schemaVersion": 1,
"props": {"datasetId": 7, "column": "region"}, "layout":
{"colSpan": 6, "rowSpan": 2}},
"trend": {"widget": "echarts", "schemaVersion": 1, "props": {"…": "…"},
"layout": {"colSpan": 18, "rowSpan": 8}}
},
"interactions": {
"filters": {"region": {"mode": "global", "exclude": ["revenue"]}},
"crossFilters": {},
"customizations": {}
},
"settings": {
"refresh": {"interval": 300, "stagger": 5000, "exempt": ["revenue"]},
"colors": {"scheme": "supersetColors", "labelColors": {"EMEA":
"#1f77b4"}},
"display": {"showTimestamps": true},
"crossFilters": {"enabled": true}
}
}
```
| Part | Rule |
| --- | --- |
| `version` | Document format version, mirrored in
`canvases.definition_version` |
| `root` | Not a placement; always exists, addressed as `"root"`. A grid: 24
columns, 16px gap, 40px rows by default |
| `nodes` | Placements, a flat map keyed by placement id. The tree is stored
only in `children` lists, whose order is reading order |
| Persisted placement | `instance`: the instance's UUID. Props, schema
version and access live with the instance |
| Inline placement | `widget`, `schemaVersion`, `props`: explicitly set
values only, valid against the widget's schema at that version |
| `layout` | Where the placement sits in its parent, validated against the
parent's rules |
| Grid layout | 1-based `col`, `row`, `colSpan`, `rowSpan`. `col` and `row`
are set together or omitted to auto-place in reading order. Spans default to
the widget's default size on `add`, else the full grid width and one row.
Colliding explicit positions are pushed down in reading order |
| `interactions` | Scope overrides per kind, keyed by placement (§5) |
| `settings` | Refresh, shared colors, timestamps, cross-filters on or off |
An instance can be placed more than once, even on one canvas; each placement
has its own layout, scope and filter values (SIP-231 §5, §9).
**Inline by default.** A widget added on a canvas is an inline instance,
whatever the widget: it lives and is shared with the canvas, travels with it,
and has no permissions of its own to manage. It becomes persisted when someone
promotes it (§6), to place it elsewhere, use it outside the canvas, or give it
its own access. Existing persisted instances are placed by reference.
</details>
<details>
<summary><b>3. Placement ids</b></summary>
Placement ids are readable slugs, unique within a canvas: `revenue`,
`revenue-trend`, `revenue-trend-2` (`^[a-z0-9]+(-[a-z0-9]+)*$`, at most 64
characters; `root` and `settings` are reserved). The server derives one from an
inline instance's `title` prop, else the widget's name; callers may choose
their own on `add`, so later operations in the same request can reference it, a
UI can draw the placement before the response, and a retried `add` fails
instead of duplicating.
An inline instance's identity is the canvas UUID plus its placement id
(SIP-231 §5). Persisted instances keep UUIDs, which are global.
Readable ids are an agent-facing decision backed by evidence: across 480
runs, readable ids cut agent cost by about a third against UUIDs with no loss
in reliability (see Evaluation).
</details>
<details>
<summary><b>4. Containers and layout</b></summary>
The canvas ships no layout primitives. Containers are widgets, and widgets
declare how they take part in a canvas in SIP-231's `WidgetBehavior` (agreed
with SIP-231, beside its `emits_filters` and `drill`); widget ids in them are
the registered ids, namespaced for extension widgets
(`extensions.<publisher>.<name>.<id>`):
- `container`, `accepted_children` (SIP-231's "which child widgets they
accept"), `allowed_parents`;
- `grid_columns` (children on a grid, collisions resolved by the server), a
`child_layout_model` (any schema, arranged by the container's renderer), or
neither (children take no layout, such as tab panes);
- `bounds_filter_scope` (default true);
- `filter` (sets runtime filters), `customization` (sets runtime choices,
§5) and `filterable` (receives them).
`WidgetUi` carries `default_size` (applied on `add` when a grid layout
leaves spans unset) and min and max spans. Core registers `tabs` (accepts only
`tab`), `tab` (24-column grid, only in `tabs`), `group` (12-column grid) and
`filter.bar` (24-column grid of `filter.select` that doesn't bound scope);
`tabs`, `tab` and `group` bound scope. Of the other built-ins, `echarts` and
`ag-grid-table` are filterable and cross-filter, `metric-tile` and `balloons`
are filterable, `filter.select` is a filter and filterable (so filters cascade,
§5) and `markdown` is a leaf.
The server resolves auto-placement and push-down and returns resolved
positions, so clients never reimplement placement.
</details>
<details>
<summary><b>5. Interactivity: filter scope and the listener bus</b></summary>
**Scope.** Filter scope belongs to placements (SIP-231 §9). Filters,
cross-filter sources and customizations (such as a dynamic group-by) share one
model, declared by widget behavior (`filter`, SIP-231's `emits_filters`,
`customization`) and received by `filterable` widgets:
- **`auto`** (default): drives every filterable placement under the nearest
container that bounds scope, or the whole canvas, never itself, minus
`exclude`. A filter in a tab drives the tab; a filter in the filter bar drives
the canvas. An unresolved container keeps its filters contained.
- **`global`**: every filterable placement, minus `exclude`.
- **`custom`**: exactly `targets`.
Resolved scopes are computed on read, never stored, and returned as
`filterScopes`, `crossFilterScopes` and `customizationScopes`. Cross-filter
scopes resolve empty while cross-filters are off.
**The listener bus** connects placements on an open canvas:
- **Where it lives.** One bus per open canvas page, holding the current
value each filter, cross-filter source and customization has published, keyed
by placement id and kind. Values are view state and never enter the document.
- **How widgets take part.** Through SIP-231's hooks: `emitFilter()`
publishes for the emitting placement, and `useRuntimeFilters()` returns the
values published by placements whose resolved scope includes this one, in
SIP-231's runtime-filter vocabulary (`{column, operator, value, datasource?}`).
A placement sends what reaches it as `runtime_filters` when it requests data
(SIP-231 §8).
- **Customizations.** A customization, such as the dynamic group-by of
SIP-185 (#35158), publishes a choice rather than a filter, with
`emitRuntimeProps({path: value})`. The receiving widget marks the props a
viewer may change at runtime with `x-runtime`, listing the allowed values (for
a group-by, the dimensions the instance offers); `useRuntimeProps()` returns
the choices reaching its placement, and `useWidgetData()` sends them as
`runtime_props`, validated against those values like a write. A viewer picks
among what the stored instance offers and can't add dimensions, metrics or
columns, so the stored definition still owns the query. These are amendments to
SIP-231 (see "Changes this SIP asks of SIP-231").
- **Which placements a filter reaches.** Scope decides the candidates;
datasource decides applicability. Every filter and cross-filter on the bus
carries its `datasource`, and reaches only in-scope placements that query that
datasource, which the server reports per placement with the resolved scopes.
Applying a filter across datasources, by column name or an explicit mapping, is
a later, additive field on scope overrides.
- **Cascading filters.** A filter widget that is also `filterable` receives
the filters whose scope includes it and narrows its options with them, so a
city filter in the scope of a country filter cascades with no separate
dependency setting. Published values are never rewritten by options, so mutual
scopes can't loop.
- **URLs and permalinks.** The bus state serializes to `{placement id:
{kind: value}}`. `POST /api/v1/canvas/{uuid}/permalink` stores it, with the
canvas revision, in the key-value store and returns a key; `/canvas/p/<key>/`
opens the canvas with that state. Values for placements that no longer exist,
or whose scope no longer reaches a target, are dropped on load.
</details>
<details>
<summary><b>6. Editing: operations</b></summary>
`POST /api/v1/canvas/` can seed a new canvas with a whole definition,
validated in full. After that, every change goes through one route, atomically,
against a base revision:
```
PATCH /api/v1/canvas/{id_or_uuid}/definition
{"base_revision": 7, "ops": [
{"op": "add", "id": "kpis", "widget": "group", "layout": {"colSpan": 24,
"rowSpan": 4}},
{"op": "add", "id": "revenue", "parent": "kpis", "instance": "6f1c2b7e-…",
"layout": {"colSpan": 6}},
{"op": "set_scope", "kind": "filter", "id": "region", "scope": {"mode":
"global"}}
]}
→ 200 {"revision": 8, "ops": [{…, "id": "kpis"}, {…, "id": "revenue"}, {…}],
"filterScopes": {…}, "placements": {…}}
```
| Op | Effect |
| --- | --- |
| `add` | Places a persisted (`instance`) or new inline (`widget`, `props`,
optional `schemaVersion`) instance under `parent` (default `root`) at `index`
(default last); inline props are migrated to and stored at the widget's current
schema version |
| `set_props` | Replaces an inline instance's props, with an optional
`schemaVersion` as on `add` |
| `patch_props` | Applies a JSON Patch (RFC 6902) to an inline instance's
props, so an edit to one nested prop doesn't resend the rest |
| `place` | Changes position and size within the parent (drag, resize) |
| `move` | Reparents or reorders, optionally with a new `layout`;
reparenting without one resets the layout to auto-place |
| `remove` | Removes a placement and its subtree, pruning them from scopes
and refresh exemptions |
| `set_scope` | Sets or clears a placement's scope override for one kind |
| `set_settings` | Replaces one settings section; `null` resets it |
| `promote` | Saves an inline instance as a persisted one: creates a
`WidgetInstance` from its widget, schema version and props, with the caller as
editor, and the placement references it. Needs SIP-231 `can_write` on `Widget` |
| `inline` | Copies a persisted instance's widget, schema version and props
onto its placement, detached from the original. Needs read access to the
instance |
- **Editing inline instances.** Inline props change only through canvas ops:
`add`, `set_props` and `patch_props`. Each write is validated in full on the
instance it writes (schema, datasource access, referenced columns and metrics,
and the queries it would build), and leaves other placements alone.
`schemaVersion` on a write lets props saved at an older version be migrated
rather than mislabelled. `promote` and `inline` touch the placement's `props`.
Persisted instances change through SIP-231's instance API with its own revision
check.
- **Atomic and live.** Ops apply in order to a copy; all land or none do. An
accepted request is the stored canvas; there is no edit mode or Save.
- **Concurrency.** Each op touches `(placement, field group)` pairs: `tree`,
`layout`, `props`, one group per scope kind, one per settings section. An `add`
touches only the new placement. A write based on an older revision merges
unless it overlaps a change logged since; then it gets 409 `{message, revision,
conflicts, stale: false}`. `tree` overlaps every group. A base revision ahead
of the canvas, or older than the retained log (or any base before the first
write after creation), gets 409 with `stale: true`: reload. A merged write
whose op no longer applies, such as an `add` into a parent someone removed,
gets 422.
- **Live updates.** Each write publishes `entity.changed` with
`{entity_type: "canvas", id}` only, and open pages refetch the definition
through the authorized API. Without a realtime transport, pages poll
`…/definition/changes?since=N` every 5 seconds and refetch when it reports a
newer revision; that route also returns the logged ops, grouped by revision,
and answers 409 `stale` when `since` is older than the retained log, upon which
the page reloads.
- **Errors** carry JSON-pointer paths, in the SIP-231 Appendix C spirit: 400
`{message: [{path, message}]}` for a malformed request (paths into the body),
and 422 `{message, errors: [{path, message}], operation?}` when an op can't
apply (`/ops/<index>`) or the result is invalid (paths into the document).
</details>
<details>
<summary><b>7. Validation and evolution</b></summary>
- **Strict where a write touches, tolerant elsewhere.** Tree integrity is
always checked. Widget behavior (nesting, spans, child layout, scope roles) is
enforced on the placements a write touches, and inline props are validated
where a write wrote them. A widget that tightens a rule or its schema never
blocks edits elsewhere on the canvas.
- **Placeholders.** A placement whose instance is missing, whose widget
isn't registered (say, an extension was removed), or whose props were saved by
a newer widget version or fail to migrate stays exactly as stored, with its
children. The rest of the canvas renders and stays editable; re-enabling the
widget restores it.
- **Props versions** follow SIP-231 §12: inline props are migrated in memory
on read by the widget's migrators and stored upgraded on the next write.
- **Document versions.** A change to the canvas document format bumps
`version` and ships an Alembic migration with a downgrade, which also clears
the op log so open pages reload; additive changes ship expand/contract across
two releases. A document at any other version, such as one from a newer server,
is refused with 422 rather than read with the wrong schema. Importing an older
snapshot (§11) first converts it with the same transformations its migrations
apply.
</details>
<details>
<summary><b>8. Storage, permissions and data</b></summary>
- **Tables:** `canvases` (integer id plus UUID, audit columns, `title`,
`description`, `definition`, `definition_version`, `revision`, `slug`, `css`,
`theme_id`, certification and external-management columns), `canvas_editors` /
`canvas_viewers` (subject-based, like dashboards), and `canvas_ops` (the op
log, last 500 revisions per canvas).
- **Slug:** optional, unique across canvases, letters, digits, `_` and `-`,
at most 255 characters, never all digits and never `list`. The canvas page is
`/canvas/<slug>/` when set, `/canvas/<id>/` otherwise.
- **Access:** admins, editors and viewers see a canvas, and
`EXTRA_ACCESS_QUERY_FILTERS["canvases"]` can widen that; embedded guests see
none. Writes need editorship (admins always have it); the creator is the
default editor. Seeing a canvas grants no data access by itself: a viewer also
needs access to each persisted instance placed on it and to its datasource
(SIP-231 §8), and placements they can't read render as no-access placeholders.
Granting placed instances through the canvas would add a row to the role and
capability matrix in `SECURITY.md` and is left for a later proposal.
Permissions are `can_read` and `can_write` on `Canvas`; widget APIs use
SIP-231's `Widget` resource.
- **Data:** SIP-231 §8 defines how inline instances are read, through `POST
/api/v1/canvas/{uuid}/placement/{placement_id}/data`. This SIP adds only the
canvas side: the caller needs read access to the canvas, and the placement must
resolve.
- **Feature flag:** one flag, `CANVAS` (off by default), shared with SIP-231
since neither ships without the other. It gates registration of the canvas and
widget REST APIs and MCP tools, the SPA routes and the menu, which 404 and hide
while it is off. Canvases are kept when it is turned off.
</details>
<details>
<summary><b>9. Agent access</b></summary>
MCP tools wrap the same commands as the REST API:
| Tool | Purpose |
| --- | --- |
| `get_canvas` | The document and its revision. Resolved positions and
scopes only on request (`include_resolved`) |
| `apply_canvas_ops` | Apply operations atomically against a base revision;
returns the new revision and the applied ops with their ids, or the errors |
| SIP-231 §13 tools | Discover widgets and read their schemas progressively;
edit persisted instances |
The default read is lean because resolved positions and scopes cost context
and didn't improve reliability in the evaluation. The REST `GET` and `PATCH`
responses always include them, since the page needs them to render.
</details>
<details>
<summary><b>10. Rendering</b></summary>
The page renders the resolved grid. Widget renderers are SIP-231 renderers,
registered once in its renderer registry by namespaced widget id with the
schema versions they support (SIP-231 §10, §12); core registers the containers
and `markdown`. Data and interactivity reach a renderer through SIP-231's
hooks, which the canvas page backs with the placement's instance and the
listener bus, so a renderer doesn't know whether its instance is inline or
persisted or how often it is placed.
On a canvas, renderers also get canvas context: the placement id, shared
colors, whether to show timestamps, and a refresh key that bumps on each
automatic refresh. Containers get their rendered children (with ids, widget ids
and layouts) or a ready-made grid. Each renderer runs inside an error boundary;
placements that don't resolve, or whose widget has no renderer for its schema
version, render a placeholder.
</details>
<details>
<summary><b>11. Snapshot export</b></summary>
`GET /api/v1/canvas/{uuid}/snapshot` returns one self-contained, immutable
document (SIP-231 §5):
- the canvas metadata (title, description, CSS, theme by UUID) and its
definition, with every persisted placement replaced by an inline copy of the
instance as it is at export time;
- for each widget used, its id, `schema_version` and implementation version
(SIP-231 §12).
Copying an instance inline puts it under whoever can open the snapshot, so
the exporter needs read access to every persisted instance; placements they
can't read are exported as unresolved placeholders without props. Snapshots
hold definitions only, never query results: data is read when the snapshot is
opened, under the reader's own access and row-level security. Snapshots aren't
stored by the server. Importing one through `POST /api/v1/canvas/` creates a
new canvas; widgets that are missing or newer than the server stay placeholders
(§7).
</details>
<details>
<summary><b>12. Embedding, reports and export (M2)</b></summary>
- **Embedding.** A canvas gets an embedded configuration (allowed domains),
as dashboards do. Guest tokens gain a `{"type": "canvas", "id": "<uuid>"}`
resource that grants reading that canvas's definition and the data of its
placements, inline or persisted, with the token's row-level security, and
nothing else; the access check is on the placement, so an instance isn't
readable through a canvas it isn't placed on. This adds a guest-token row to
the role and capability matrix in `SECURITY.md`, as dashboard guest tokens
have; grants through a canvas for regular viewers stay out of scope (§8).
- **Reports and thumbnails.** A canvas screenshot report or thumbnail
captures the layout, so it renders the canvas page headlessly, as dashboard
ones do. Data-only reports on a placement read the placement's data route and
need no browser.
- **Export and import.** A canvas bundle holds the canvas and the persisted
instances it places, by UUID, so importing updates them in place, alongside
today's asset bundles. A snapshot (§11) is the self-contained alternative.
</details>
### New or Changed Public Interfaces
<details>
<summary>Full list</summary>
- **REST** (`/api/v1/canvas/`): CRUD with editors and viewers (create may
seed a definition or import a snapshot); `GET|PATCH /{id_or_uuid}/definition`,
`GET /{id_or_uuid}/definition/changes?since=N`; `GET /schema` (JSON Schemas for
the definition, operations and grid placement, plus every registered widget's
behavior and UI metadata); `POST /{uuid}/placement/{placement_id}/data`; `GET
/{uuid}/snapshot`; `POST /{uuid}/permalink`, opened at `/canvas/p/<key>/`.
- **Operations:** `add`, `set_props`, `patch_props`, `place`, `move`,
`remove`, `set_scope`, `set_settings`, `promote`, `inline`.
- **MCP:** `get_canvas`, `apply_canvas_ops`, alongside SIP-231's §13 tools.
- **`superset-core`:** `superset_core.canvas.GridPlacement` and
`InstanceResolver` (looks up persisted instances); the container and scope
fields this SIP adds to SIP-231's `WidgetBehavior` and `WidgetUi`.
- **`@apache-superset/core`:** canvas context for renderers (placement id,
colors, timestamps, refresh key, children and grid for containers);
`getActiveCanvas` / `onDidChangeActiveCanvas`. Renderers register in SIP-231's
registry, and the canvas page implements SIP-231's `useWidgetData`,
`useRuntimeFilters` and `emitFilter` for its placements.
- **Extensions to SIP-231** (see "Changes this SIP asks of SIP-231"): the
`x-runtime` keyword, `runtime_props` on data requests, and the
`emitRuntimeProps` / `useRuntimeProps` hooks (§5).
- **Realtime:** `entity.changed` with `entity_type: "canvas"`.
- **Security:** `Canvas` permission resource (`can_read`, `can_write`); a
`canvas` guest-token resource type (M2).
- **Configuration:** `CANVAS` feature flag,
`SUBJECTS_RELATED_TYPES_CANVASES`.
- **Models:** `canvases`, `canvas_ops`, `canvas_editors`, `canvas_viewers`;
permalinks use the existing key-value store.
- **Unchanged:** dashboards, `position_json`, `json_metadata`, charts and
their APIs.
</details>
### Changes this SIP asks of SIP-231
These need SIP-231's agreement; the rest of this SIP builds on SIP-231 as
written.
- **Runtime choices.** `x-runtime` in the documented keyword list, and
`runtime_props` beside `runtime_filters` on the instance and placement data
routes, with `emitRuntimeProps` and `useRuntimeProps` beside `emitFilter` and
`useRuntimeFilters` (§5).
- **One flag.** Both SIPs share `CANVAS` (§8), and it turns on by default
only when both reach parity, rather than at SIP-231's M2 alone.
### New dependencies
None.
### Migration Plan and Compatibility
Nothing existing changes until deployments opt in; all new tables, routes
and permissions are additive.
1. **M1: Canvas.** Behind the `CANVAS` flag, shared with SIP-231: the
document, operations including `promote` and `inline`, concurrency, scopes, the
listener bus with SIP-231's hooks, permalinks, snapshot export, rendering, the
MCP tools, and core containers. Ships alongside SIP-231 M1.
2. **M2: Parity.** Widgets for the chart and filter types dashboards use;
export and import (a canvas bundle with its persisted instances); embedding;
canvas reports and thumbnails (§12); direct editing on the page. Reached
together with SIP-231's M2, after which the shared flag turns on by default.
3. **M3: Opt-in conversion.** "Convert to canvas" copies a dashboard into a
canvas and leaves the original; a conversion report lists anything not carried
over. Layout maps one to one (`ROW` widths ×2 to `colSpan`, `COLUMN` to
`group`, `TABS`/`TAB` to `tabs`/`tab`), native filters become filter widgets
with their scope as `auto` or an override, and `json_metadata` settings map to
`settings`.
4. **M4: Canvas by default.** New dashboards are canvases; bulk conversion
for the rest.
5. **M5: Retirement.** Dashboard URLs redirect; the old format and renderer
are removed in a major release, recorded in `UPDATING.md`.
Each phase is gated on its milestone, not a date.
### Rejected Alternatives
<details>
<summary>Alternatives considered</summary>
- **Building on `dashboards.position_json`.** It keeps whole-document
writes, client-side grammar and a dashboard-specific component set; conversion
is a one-way copy instead.
- **UUID placement ids.** They cost agents a third more per edit for no
reliability gain, and nothing needs them to be global: an inline instance's
identity is already the canvas plus its placement.
- **Server-generated ids only.** No references between ops in one request,
and retried adds duplicate.
- **A nested tree, or `parents` arrays.** A flat map with `children` gives
each placement one address and one source of structure.
- **Whole-document `PUT`, JSON Patch over the document, or key-value drafts
of the canvas.** Named ops validate intent and make overlap detection precise;
JSON Patch is kept for what it suits, a single instance's props (`patch_props`).
- **Whole-document revision checks, per-placement locks, or CRDT/OT.** A
revision plus a touch log merges independent edits with a small, server-side
model.
- **Filter scope as explicit target lists, or owned by the filter widget's
props.** Scope belongs to placements (SIP-231 §9), and tree-derived defaults
follow the layout without upkeep; explicit lists remain available as `custom`.
- **Upgrading the document on read in code.** Format changes use Alembic
migrations, so the table never holds mixed versions; widget props, which change
on each widget's own schedule, use SIP-231's in-memory migrators.
- **Declaring filter dependencies for cascading.** Scope already says which
filters reach a filter, so a second dependency graph could only disagree with
it.
- **Filter state in the URL itself, or in the document.** URLs outgrow
browser limits with a few multi-select filters, and state in the document would
change the canvas for every viewer; a permalink key keeps both small.
- **Persisted instances by default.** Every text block and container would
be a row with its own permissions, and a canvas couldn't travel as one
document; promote covers sharing.
- **Query results in snapshots.** Opening a snapshot would show data under
the exporter's access and row-level security, not the reader's.
- **Customizations as runtime filters.** A group-by changes what is grouped,
which an additive filter can't express; letting callers send arbitrary
group-bys would let a viewer reshape the query. `x-runtime` limits choices to
what the stored instance offers.
- **Validating the whole document on every write.** A tightened widget rule
would then block unrelated edits.
</details>
### Prototype
The
[`msyavuz/feat/canvas`](https://github.com/apache/superset/tree/msyavuz/feat/canvas)
branch implements this proposal on top of the SIP-231 prototype's widget
framework: the document, readable placement ids, inline placements and
`set_props`, widget behavior and UI declarations with the core containers,
scopes, concurrency and live updates, the REST API and MCP tools behind
`CANVAS`, and a read-only canvas page that renders the core containers and
markdown.
The prototype's widget framework is the SIP-231 prototype, so it differs
from SIP-231 in names: widgets register with `@widget(widget_type=…)` rather
than `id`, the API is `/api/v1/widgets` with `Chart` permissions rather than
`/api/v1/widget` with `Widget`, its schema routes are `/types` and
`/type/<id>/control-schema`, its discovery tool is `list_widget_types` rather
than `list_widgets`, and `CANVAS` gates its widget API and tools as well as the
canvas. Renderers register through a canvas-only
`canvas.registerWidgetRenderer` without schema versions, and receive props,
instance ids and filter values as props (`filters`, `setFilterValue` and so on)
instead of through hooks. `add` and `set_props` validate props against the
schema only.
Not yet built:
- **Promote, inline and snapshot export** need the `WidgetInstance` table.
- **Persisted placements** are modelled and validated through
`InstanceResolver`, but no resolver is registered until the `WidgetInstance`
table exists, so adding one is refused.
- **The placement data endpoint** needs `QueryMixin` and the server-side
query pipeline.
- **Chart and filter renderers** need `useWidgetData` and field options;
until then chart and `filter.select` placements render a placeholder, so
filters can't yet be driven from the page.
- **Permalinks**, SIP-231's hooks and renderer registry, per-placement
datasources for the bus, and `x-runtime` / `runtime_props` for customizations.
- **`patch_props`**, `get_widget_info`, `get_widget_field_options`, and
validation beyond the props schema (datasource access, columns and metrics,
queries).
### Open questions
1. **Changes needed from SIP-231.** Runtime choices for customizations
(`x-runtime`, `runtime_props`) and one shared flag; see "Changes this SIP asks
of SIP-231".
2. **Unfinished instances.** Every write is validated in full, so a chart
can't be built up field by field on a canvas. Either writes must carry complete
props (agents and the property panel send whole, valid props), or a placement
may hold an incomplete instance with a visible "not configured" state that
renders no data.
3. **Previewing before it's live.** Every accepted op is visible to all
viewers, so an agent or person can't look at a configuration's data before
others see it. Options: a preview data route that takes props without storing
them, or accepting that iteration happens live (see 4).
4. **Live layout edits.** Every accepted op is the stored canvas, so placing
a widget can push others down for every viewer at once. An edit mode would
batch a person's changes before others see them, at the cost of the "no Save"
model agents rely on.
5. **Op log visibility.** Viewers can read operations from before they had
access through `…/definition/changes`. Options: filter the log by the viewer's
access date, or return only revisions and have clients refetch.
6. **History and undo.** The op log records every change with its author,
but keeps only 500 revisions and canvases aren't versioned. Decide whether
history comes from the op log, from row versioning, or both, and whether undo
is an inverse op.
7. **Filters across datasources.** A filter reaches only placements on its
own datasource. A later, additive field on scope overrides could map across
datasources, either by column name or by explicit column mapping.
8. **Conversion from dashboards.** Which dashboard features that a canvas
doesn't model (legacy filter scopes, `expanded_slices`, `async_mode`) block a
dashboard from converting, and which are only listed in the conversion report.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
