msyavuz opened a new pull request, #43518:
URL: https://github.com/apache/superset/pull/43518
### SUMMARY
Explore's filter value picker fetched an unfiltered `SELECT DISTINCT col
FROM tbl LIMIT 10000` once and matched it in the browser, so a value past that
limit on a high-cardinality column simply could not be found. Dashboard native
filters already search server-side via `searchAllOptions`; Explore had no
equivalent.
`values_for_column` now takes a `search` term and applies a case-insensitive
containment predicate with LIKE wildcards escaped (a typed `%` matches a
literal percent instead of every row). The endpoint accepts `?q=`, includes it
in the cache key, caps the TTL for searched results, and returns the applied
`limit` so the client can tell a short list from a truncated one. The
comparator control moves to `AsyncSelect`, which debounces and queries the
server.
Three fixes to dashboard native filters in the same area:
- `allowNewOptions` was switched off whenever `searchAllOptions` was on,
leaving no way to apply a value the bounded search page did not return —
enabling search made things strictly worse.
- Searching sorted by the searched column, so the engine sorted every match
before applying the row limit. Dropped while a search is active; a sort metric
still selects which rows come back, so it stays.
- The term sent to the server is normalised the same way as the creatable
option shown in the dropdown.
Both dropdowns now say when a list is capped, rather than a partial list
reading as the whole set.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before: typing a value beyond the row limit yields no match and no
explanation.
After: the query reaches the database, and a capped list says so — "Only the
first 10000 values are listed. Type to search all of them, or enter a value
that is not listed."
### TESTING INSTRUCTIONS
Set `FILTER_SELECT_ROW_LIMIT = 5` and `NATIVE_FILTER_DEFAULT_ROW_LIMIT = 5`
so any column truncates against the examples DB.
1. Explore, chart on `birth_names`, filter on `name` with `IN`. Five values
plus the notice. Type `Michael` — a `/values/?q=Michael` request fires and the
value is selectable. Search `%`: matches nothing, not everything.
2. Pick a value, close and reopen the popover, remove it. Works.
3. Dashboard native select filter on `name`: notice appears. With
*Dynamically search all filter values* on, an unlisted value can still be
entered.
`pytest tests/unit_tests/models/helpers_test.py`, `npm run test --
AdhocFilterEditPopoverSimpleTabContent`, `npm run test --
src/filters/components/Select`.
Not verified: the performance gain from dropping `ORDER BY` needs a real
high-cardinality warehouse column; SQLite shows the behaviour but not the win.
Substring search remains unsargable by nature.
### ADDITIONAL INFORMATION
- [ ] Has associated issue: No
- [ ] Required feature flags: None
- [x] Changes UI — dropdown gains a header line when the list is capped;
Explore's comparator becomes async
- [ ] Includes DB Migration: No — no schema change
- [ ] Migration is atomic, supports rollback & is backwards-compatible: N/A
- [ ] Confirm DB migration upgrade and downgrade tested: N/A
- [ ] Runtime estimates and downtime expectations provided: N/A
- [x] Introduces new feature or API — optional `?q=` on the existing
column-values endpoint; omitting it preserves current behaviour
- [ ] Removes existing feature or API: No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
