hughhhh opened a new pull request, #43832: URL: https://github.com/apache/superset/pull/43832
### SUMMARY Fifth of the partition filter mapping stack, and the PR that makes the feature reachable. Until now a mapping could only be configured with a `PUT /api/v1/dataset/<pk>`, and the preview endpoint added in #pfm-3 had no caller — its own commit message says the visual controls "are not in this PR". This implements them: wireframes **1a, 1b, 1c, 1g, 1h**. Stacked on `hughhhh/pfm-4-docs`. Review the [compare against pfm-4](https://github.com/apache/superset/compare/hughhhh/pfm-4-docs...hughhhh/pfm-5-editor-ui) rather than the diff against master. **What's new in the editor** - **Partition column** select in the Columns tab's Default Column Settings, plus a read-only computed **Maps to partition**. It's deliberately not a peer dropdown — it reflects `partition_mapped_column ?? main_dttm_col`, and a second select would let it drift from the default datetime column silently. - The partition column's row is muted, carries a `PARTITION` tag, and defaults `Is filterable` / `Is dimension` off — still manually togglable. - A **Partition filter mapping** section in the row expand with three states: the mapped column holds the transform, every other column offers to take the mapping over, and the partition column itself shows nothing. - A live **Preview** panel showing sample input → emitted predicate, and a destructive **Remove mapping**. **Two places the spec and the merged backend disagreed** 1. The mockups have no monotonicity control, but the query path only mirrors ranges when the transform is declared order-preserving — so 1d/1e's own headline example (a time range producing two `dt_epoch` bounds) was unreachable through the UI. This adds the **Transform preserves ordering** checkbox that the docs in pfm-4 already describe. 2. The PRD asks for `unix_timestamp(:value)` as the temporal default, but that's Hive syntax and would not parse on Postgres, Trino or BigQuery. The default moves to the engine spec (`partition_value_transform_default`, set on Hive/Impala/Spark); engines without one offer no pre-fill rather than a wrong one. **Backend changes** - The preview endpoint takes `sample_values` + an `operator` and builds its predicate with `build_mirrored_predicates` — the same function the query path uses — so what the panel shows is what a chart emits, `IN` included. It also accepts a candidate `partition_column`, because the editor previews a mapping the owner hasn't saved yet and a preview that requires saving first isn't a preview. - Failed probes carry the engine's own message through an opt-in `errors` sink (the hot query path passes nothing and stays silent). sqlglot parses unknown functions happily, so a misspelled one is an *engine* error and was previously reported as an unexplained blank. - Parse failures now name a position, mapped back through the `SELECT` prefix and the `:value` → `NULL` substitution so it points at what the owner actually typed. **Two silent read-path bugs this uncovered** Neither was reachable from unit tests; both needed the running app. - `columns.partition_value_transform` and its monotonic flag were in the model, the export fields and the PUT schema but **not** in `show_columns` — so the editor reopened a saved mapping as if it had none, *and the next save wrote that emptiness back*. Related-model fields must be listed in `show_columns`, not only `show_select_columns` (`columns.advanced_data_type` is in both for the same reason). - `partition_value_transform_default` needed the same treatment to reach the pre-fill. Both are now pinned by tests in `partition_mapping_serialization_test.py`, whose whole premise is that a field missing from any one layer is dropped without a sound. **One small shared change:** `Field` gains an opt-in `passItemToControl`. The row-expand section keys off the whole column record, not just the transform it edits, and handing an unknown `item` prop to every `TextControl` and `Select` wasn't worth the convenience. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF _Screenshots captured during validation; attaching separately._ - `1a` Partition column + Maps to partition, and `1b` the muted `PARTITION` row - `1c` Row expand with transform, ordering checkbox and a valid preview - `1c` Parse error state — only one of preview/error is ever visible - `1g` Partition column set with nothing mapped - `1h` Non-temporal `country → region_key` ### TESTING INSTRUCTIONS Automated: `1800` backend unit tests and `225` frontend Datasource tests pass; `tsc`, `ruff`, `ruff format` and `oxlint` are clean (oxlint warnings in `src/components/Datasource` go 32 → 30, since the duplicated column-name renderer was folded into one helper). Manually, against a real engine: 1. Enable the flag: `FEATURE_FLAGS = {"PARTITION_FILTER_MAPPING": True}`. 2. Create a physical dataset on a table with both a business column and a partition column, where the partition column really is the transform of the other. On Postgres: ```sql CREATE TABLE partition_demo (event_time TIMESTAMP, dt_epoch BIGINT, country TEXT, region_key TEXT, revenue DOUBLE PRECISION); INSERT INTO partition_demo SELECT ts, EXTRACT(epoch FROM ts)::bigint, t.c, lower(t.c), (random()*100)::numeric(10,2) FROM generate_series(timestamp '2026-07-01', timestamp '2026-08-15', interval '6 hour') ts, (VALUES ('US'),('CA'),('MX')) AS t(c); ``` 3. Edit the dataset → **Columns** → set **Partition column** to `dt_epoch`. Confirm *Maps to partition* shows `event_time` tagged *Default datetime column*, and that `dt_epoch`'s row is muted with a `PARTITION` tag and its toggles off. 4. Follow **Map a different column instead →**. Set the value transform to `cast(extract(epoch from cast(:value as timestamp)) as bigint)` and check **Transform preserves ordering**. The preview should read `event_time >= '2026-01-15 00:00:00'` → `dt_epoch >= 1768435200`. Unchecking it drops the preview back to `=`. 5. Break the transform (`lower(:value))`) → *Can't parse transform* with a position, and the preview panel disappears. 6. Save, reopen — everything round-trips. 7. Build a chart with a time range and open **View query**: ```sql WHERE event_time >= TO_TIMESTAMP('2026-07-01 …') AND event_time < TO_TIMESTAMP('2026-08-01 …') AND dt_epoch >= 1782864000 AND dt_epoch < 1785542400 ``` 8. Non-temporal: point **Partition column** at `region_key`, move the mapping to `country`, set `lower(:value)`. Preview shows `country IN ('US', 'CA')` → `region_key IN ('us', 'ca')`, and a chart filtered on `country` emits `AND region_key IN ('us', 'ca')`. 9. Negative: turn the flag off. The controls disappear, the preview endpoint 404s, and no mirrored predicate is added. **Merge order.** pfm-4's docs describe this UI, so if pfm-4 merges first the published docs tell operators to click controls that don't exist yet. Merging this before or together with pfm-4 avoids that window; no docs edit is needed either way. **Known limitation, not introduced here:** SQLAlchemy's `text()` misparses Postgres `::` casts, so a transform written as `:value::timestamp` leaves the placeholder unbound. This comes from `build_probe_sql` in pfm-2 and is harmless on Hive/Impala, which have no `::` syntax — the ANSI `cast(:value as timestamp)` form works. Worth a follow-up for Postgres/Redshift users. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [x] Required feature flags: `PARTITION_FILTER_MAPPING` - [x] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [x] Introduces new feature or API - [ ] Removes existing feature or API 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
