morningman opened a new pull request, #69351:
URL: https://github.com/apache/doris/pull/69351
### What problem does this PR solve?
Issue Number: None
Related PR: #67856, #68640
Problem Summary:
**In short.** Tests only. #68640 shows required Iceberg columns as NOT NULL
on master, as #67856 does on branch-4.1 and branch-4.2. The two implementations
behave the same (master also covers the HTTP schema APIs and the MySQL protocol
column metadata), but master's tests miss several cases that the 4.x tests
check. This PR adds them; it changes no production code.
Original author: @hubgeter (the 4.x tests these cases come from)
**Background.** An Iceberg column declared `NOT NULL` (required) is read by
Doris as nullable, so scans and expressions keep their nullable semantics,
while DESC, SHOW CREATE TABLE, SHOW [FULL] COLUMNS and information_schema show
the declared constraint. On master, `ConnectorColumn.sourceNullable` carries
the declared nullability, and `PluginDrivenExternalTable` projects a display
view next to the read schema in the same schema-cache entry.
**The problem, and what it cost.** These behaviors had no test on master, so
a regression in any of them would go unnoticed:
| Case | 4.x test | master before this PR |
|---|---|---|
| A table without any snapshot shows its required columns | yes | no (the
table was written before the first DESC) |
| Writing NULL into a required column is still rejected | yes | no |
| Relaxing a column creates no snapshot; information_schema, SHOW CREATE
TABLE and later writes follow it | yes | DESC only |
| SHOW FULL COLUMNS with the column comment, SHOW COLUMNS LIKE, `WHERE Null
= 'YES'`, information_schema filtered by IS_NULLABLE | yes | no |
| A hidden column keeps its declared nullability and is listed only in the
full view | yes | no (only the visible view) |
**How this PR fixes it.**
- `test_iceberg_show_nullable` (REST catalog): checks the empty table (0
snapshots) before the first write; adds SHOW FULL COLUMNS, SHOW COLUMNS LIKE,
`SHOW FULL COLUMNS ... WHERE Null = 'YES'` and information_schema filtered by
IS_NULLABLE; checks the column comment in SHOW CREATE TABLE; writes NULL into
`id` and `value` and expects "Column 'id' is declared non-nullable but contains
nulls" (the message the 4.x test expects too); relaxes `value`, checks that the
snapshot count stays 1 and that information_schema and SHOW CREATE TABLE
follow, then writes NULL into `value` and still expects the rejection for `id`.
- `PluginDrivenExternalTableTest`: a new test with a hidden required column
checks that `getBaseSchemaForDisplay(full)` lists exactly the columns of
`getBaseSchema(full)`, in the same order, for both values of `full`, and that
the hidden column is NOT NULL only in the display view. The setup of the
existing display test moves into a helper both tests use.
**Results.** Both pass locally. The blocks already in
`test_iceberg_show_nullable.out` are unchanged; 11 blocks are added. The new
unit test fails if the display view ignores `full`, or if a copied display
column loses its visibility.
### Release note
None
### Check List (For Author)
- Test
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
