Mryange opened a new pull request, #70562: URL: https://github.com/apache/doris/pull/70562
### Background Tablet write validation can truncate strings, round legacy decimals, and deduplicate map keys. Expression outputs can share column trees, so validation must preserve copy-on-write ownership before changing them. ### What problem does this PR solve? Issue Number: None Related PR: #70421 (superseded) PR Type: Refactor Problem Summary: Tablet validation is embedded in the block converter as a large recursive function that mixes type dispatch, nullable handling, row filtering, and mutation. Extract it into a standalone, testable column validator with per-type helpers and a shared validation context. The new entry consumes and returns a column, applying COW once before recursively borrowing writable child slots; this also fixes shared-column corruption during nested validation. ### Details - **Change map:** Roughly 500 lines of embedded validation are replaced by a 454-line implementation and 61-line interface; the converter keeps block orchestration. Add 479 lines of ownership/validation UTs and a 55-line regression with generated output; add one nullable child-slot accessor. - **Moves vs. logic changes:** Move scalar checks and decimal bound caches into per-type helpers/context. Centralize physical nullable handling and original-row mapping. Replace const-cast mutation with one COW boundary and mutable recursive traversal; visit each nested map once for deduplication. - **Behavior change:** Validation no longer modifies shared source columns or retains stale views after nested string replacement. Inherited NULLs skip nested payload checks and rejected elements filter their original outer row. - **Compatibility:** No wire or storage format changes. DecimalV2 keeps historical execution precision/scale (27, 9); enforcing declared schema parameters and resolving map-key collisions after string truncation remain outside this change. This refactoring targets master without release pick requests. - **Performance impact:** Unique columns retain object/data/offset/null-map addresses and reference counts, verified by UTs. Shared nodes detach at entry; recursive child-slot transfers do not repeat COW. No throughput benchmark was run. - **Rollback:** Revert this commit; no metadata or data migration is needed. ### Testing The following checks passed on the development machine before squashing. The squashed commit has an identical source tree; no tests were rerun for history cleanup: - `BUILD_TYPE=ASAN sh build.sh --be -j48`: passed for the final revision. - `BUILD_TYPE_UT=ASAN sh run-be-ut.sh --run --filter='TabletColumnValidatorTest.*:ColumnNullableTest.*:ColumnArrayTest.*:ColumnStructTest.*:ColumnMapTest.*' -j48`: 107 tests passed, including all 22 new validator tests. - Repository clang-format 16 scripts, build hygiene, `git diff --check`, and clang-tidy on both changed source TUs and the new UT: passed. - The existing shared-map regression and generated output are retained; that regression was not rerun after this refactoring. ### Release note Fix incorrect results or failures when inserting shared complex columns that require tablet validation. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Partially verified: ASAN build and focused/related UTs passed; end-to-end regression not rerun after refactoring. ### AI Disclosure - Tool: Codex (GPT-6). - Generated: Code, tests, and this description. - Human review: The author reviewed the design and an earlier source version; the final changes remain available for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
