sadpandajoe opened a new pull request, #42821:
URL: https://github.com/apache/superset/pull/42821
### SUMMARY
Rebased/conflict-resolved re-post of #42203 (by @massucattoj), which went
stale
against `master` and could not be pushed to the original fork branch due to
missing `workflow` scope on the pushing token. All commit content is
unchanged
from #42203; this PR only merges in current `master` and resolves the merge
conflict in `superset/db_engine_specs/mysql.py`, where `master` had
independently added a `var_string` → `VARCHAR` mapping. That entry is kept as
already merged, and #42203's remaining new mappings (`newdecimal`, `tiny`,
`short`, `blob`, `year`, `enum`, `set`) are appended after it; one test
assertion was updated to match (`VAR_STRING` now expects `types.VARCHAR`
instead of `types.String`, matching what `master` already established).
Original description from #42203:
Columns of MySQL virtual datasets (created from SQL Lab queries) showed a "?"
type icon in the chart panel instead of ABC/#, which also hid type-dependent
controls such as X-Axis Sort By.
Physical datasets get column types from the SQLAlchemy inspector as DDL names
(`VARCHAR(100)`, `DECIMAL(15,2)`), which match
`MySQLEngineSpec.column_type_mappings`.
Virtual dataset columns instead get their type from the driver's cursor
description via `get_datatype`, which returns the raw wire-protocol
`FIELD_TYPE` constant names (`VAR_STRING` for VARCHAR, `NEWDECIMAL` for
DECIMAL, `TINY` for TINYINT, `BLOB` for TEXT, and so on). None of those names
matched any mapping regex, so `type_generic` resolved to `None`.
This PR adds mappings for the wire-protocol names that had no match:
`NEWDECIMAL`, `TINY`, `SHORT`, `BLOB`, `YEAR`, `ENUM` and `SET` (`VAR_STRING`
is already covered on `master`). Regexes are anchored so they don't collide
with existing DDL patterns (`^tiny$` vs `^tinytext`), and `^enum\b`/`^set\b`
also cover the DDL forms (`ENUM('a','b')`), which were equally unresolved.
Fixing the type resolution exposed a second, latent bug: MySQL registers a
`column_type_mutators` entry that converts DECIMAL string values to Python
`Decimal`, keyed on the resolved column type. It had never fired because
`NEWDECIMAL` never resolved. Once it did, `BaseEngineSpec.fetch_data` crashed
with `TypeError: 'tuple' object does not support item assignment`, because
the
mutation loop assigns rows back into the `fetchall()` result and mysqlclient
returns a tuple of rows (psycopg2 returns a list, which is why the same code
works for Postgres). Fixed by copying the result into a list before mutating.
### TESTING INSTRUCTIONS
1. Start a MySQL instance and create a table exercising the affected types:
```sql
CREATE TABLE sample_data (
id INT AUTO_INCREMENT PRIMARY KEY,
label VARCHAR(100),
amount DECIMAL(15, 2)
);
INSERT INTO sample_data (label, amount) VALUES ('Ships', 10.50),
('Planes', 22.30);
```
2. Connect the MySQL database to Superset.
3. In SQL Lab, run `SELECT * FROM sample_data` and use "Save or Overwrite
Dataset" to save it as a new (virtual) dataset.
4. Create a chart from that dataset.
5. Verify `label` shows the string indicator, `amount` and `id` show the
numeric indicator, and the chart renders data instead of a type error.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
Supersedes #42203, which has the same commit content but is
stale/conflicting and can't be updated by maintainers due to token scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
