suxiaogang223 opened a new pull request, #70438:
URL: https://github.com/apache/doris/pull/70438
### What problem does this PR solve?
Issue Number: None
Related PR: #69448 (backport to branch-4.2)
PR Type: Bugfix
Problem Summary:
Backport the merged Parquet conversion fix from #69448. When a mixed-schema
Parquet scan infers BIGINT and then reads an invalid string such as "xx", the
logical converter writes a failed-cast NULL before the destination null map has
allocated entries, which can crash the BE.
Copy the appended source null-map slice into the destination before logical
conversion. Preserve the existing destination prefix and source NULLs, allow
the converter to mark additional failed casts, and retain copy-on-write
isolation. Non-nullable conversion errors stay unchanged.
### Details
- **Frequency:** Triggered by nullable Parquet schema conversions that
produce NULL, including invalid strings and numeric overflow.
- **Root cause:** The destination null map was aligned after logical
conversion, while converters can write NULL flags during conversion.
- **Why existing tests missed it:** The conversion boundary lacked coverage
for failed nullable casts into empty destinations, append batches and shared
column aliases.
- **Fix:** Move the existing null-map preparation before the converter call;
no new production conditional, protocol or storage-format changes.
- **Other paths and branches:** Includes both BYTE_ARRAY and
FIXED_LEN_BYTE_ARRAY unit coverage. The existing S3 TVF suite compares complete
ordered File Scanner V1/V2 results over a BIGINT seed plus 4096 STRING rows,
using 1024-row batches and non-strict casts. Session settings are restored.
This PR targets branch-4.2.
The backport preserves branch-local tests and inserts only the three new
source unit tests at their available insertion point. The production and S3
regression changes match the merged source PR.
### Testing
Passed locally on branch-4.2: clang-format 16.0.6 checks for both changed
C++ files, `git diff --cached --check`, Groovy syntax/AST conversion, and
checks that the new unit-test block matches the merged source PR and all merge
conflicts are resolved.
Full Groovy semantic compilation was blocked by the host Java 25 / cached
Groovy 4.0.19 combination (unsupported class-file major version 69); the
syntax/AST conversion check passed. No Doris compilation, BE unit-test
execution or runtime regression was performed. CI test pipelines are deferred
at the author's request. No submodules were initialized for this work.
### Notes for Reviewers
This is a backport of master commit
`1bb35373ee076af701e9070d472f7ee47073b565` from #69448. Keep this PR in draft
while pipeline testing is deferred. The commit includes a CI skip directive,
and no test-triggering comments were posted.
### Release note
Fix a BE crash when Parquet schema conversion produces NULL for an invalid
value.
### Check List (For Author)
- Test:
- [ ] Regression test executed
- [ ] Unit Test executed
- [x] Manual/static checks: clang-format 16, whitespace, Groovy
syntax/AST conversion, source-patch and conflict checks.
- Behavior changed:
- [ ] No.
- [x] Yes. Failed nullable conversions produce NULL without accessing an
unallocated null map.
- Does this need documentation?
- [x] No.
- [ ] Yes.
- Pre-submission status: Partially verified: compilation, UT, sanitizer and
runtime regression validation deferred.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
### AI Disclosure
- Tool: Codex (GPT-6)
- Generated: Backport conflict resolution and this description.
- Human review: Pending.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
