Herestothegamers opened a new pull request, #42603:
URL: https://github.com/apache/superset/pull/42603
### SUMMARY
Fixes #36305.
Uploading a CSV/Excel/Columnar file with the **Schema** field left empty
fails on PostgreSQL with `psycopg2.errors.InvalidSchemaName: schema "undefined"
does not exist`, and uploads that survive with no schema create a dataset with
`schema=NULL` that duplicates an existing dataset over the same physical table
("2 datasets for 1 table").
Root causes and fixes, one per layer:
1. **Frontend** (`UploadDataModel/index.tsx`): an unselected schema is JS
`undefined`, and `FormData.append('schema', undefined)` stringifies it to the
literal `"undefined"`. `appendFormData` now skips **all** `undefined`/`null`
values instead of a two-field allowlist. This also fixes `sheet_name` being
sent as `"undefined"` on the Excel path, and cleared numeric inputs being sent
as `"null"`.
2. **Backend contract** (`UploadPostSchema`): a `@post_load` hook normalizes
empty/whitespace/`"undefined"`/`"null"` schema values to unset — defense in
depth for cached pre-fix bundles and other broken clients.
3. **Backend command** (`UploadCommand.validate()`): an unset schema is
resolved to the database's actual default schema via the existing
`Database.get_default_schema()` machinery (e.g. `public` on PostgreSQL) before
the allow-list check, the dataset lookup, and `df_to_sql`, so uploaded datasets
always carry an explicit schema and can no longer create NULL-schema
duplicates. Resolution failures are logged and degrade to the previous
no-schema behavior, so no currently-working upload can start failing. This
restores the v4 UX where the default schema was applied.
**Intended behavior change:** with a non-empty
`schemas_allowed_for_file_upload` allow-list, an upload without a schema
previously always raised `DatabaseSchemaUploadNotAllowed` (`None not in
[...]`); it now succeeds when the database's default schema is allow-listed.
`test_csv_upload_schema_not_allowed` was updated accordingly.
**Known limitation (out of scope):** a pre-existing dataset with
`schema=NULL` over table X is not matched by the new explicit-schema lookup, so
one more dataset (with explicit schema) is created on the next upload, after
which uploads are stable. Making the lookup NULL-aware (mirroring
`_catalog_identity_filter`'s catalog normalization in `DatasetDAO`) is a
possible follow-up.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before: upload with blank schema fails with `schema "undefined" does not
exist` (see issue).
After: upload succeeds and the dataset is created under the database's
default schema (`public`).
### TESTING INSTRUCTIONS
1. Connect a PostgreSQL database and enable **Allow file uploads to
database**.
2. Upload a CSV via **+ → Data → Upload CSV to database**, leaving
**Schema** blank.
3. The upload succeeds; the created dataset shows schema `public`.
4. Re-upload with "already exists → replace": still exactly one dataset
(previously a NULL-schema duplicate could appear).
5. Optionally set `"schemas_allowed_for_file_upload": ["public"]` in the
database Extra: blank-schema upload still succeeds; a non-allow-listed schema
is still rejected.
New/updated tests: `UploadDataModal.test.tsx` (blank-schema submission omits
`schema` from FormData; Excel omits `sheet_name`), `upload_command_test.py`
(default-schema resolution, explicit schema kept, failure fallback, allow-list
sees resolved value), `api_test.py` (`""`/`"undefined"` payloads normalize to
`None`; `UploadPostSchema` normalization matrix), `upload_test.py`
(integration, runs on PostgreSQL).
### ADDITIONAL INFORMATION
- [x] Has associated issue: Fixes #36305
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01HhThrhzmtMYfdKaud7QZwk
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
