suxiaogang223 opened a new pull request, #70439:
URL: https://github.com/apache/doris/pull/70439
### What problem does this PR solve?
Issue Number: None
Related PR: #69448 (backport to branch-4.1)
PR Type: Bugfix
Problem Summary:
Backport the merged Parquet conversion fix from #69448. When a mixed-schema
Parquet scan infers BIGINT and then reads an invalid string such as "xx", the
logical converter writes a failed-cast NULL before the destination null map has
allocated entries, which can crash the BE.
Copy the appended source null-map slice into the destination before logical
conversion. Preserve the existing destination prefix and source NULLs, allow
the converter to mark additional failed casts, and retain copy-on-write
isolation. Non-nullable conversion errors stay unchanged.
### Details
- **Frequency:** Triggered by nullable Parquet schema conversions that
produce NULL, including invalid strings and numeric overflow.
- **Root cause:** The destination null map was aligned after logical
conversion, while converters can write NULL flags during conversion.
- **Why existing tests missed it:** The conversion boundary lacked coverage
for failed nullable casts into empty destinations, append batches and shared
column aliases.
- **Fix:** Move the existing null-map preparation before the converter call;
no new production conditional, protocol or storage-format changes.
- **Other paths and branches:** Includes both BYTE_ARRAY and
FIXED_LEN_BYTE_ARRAY unit coverage. The existing S3 TVF suite compares complete
ordered File Scanner V1/V2 results over a BIGINT seed plus 4096 STRING rows,
using 1024-row batches and non-strict casts. Session settings are restored.
This PR targets branch-4.1.
Branch-4.1 has no existing parquet_column_convert_test.cpp. Add a focused
file containing only the three source unit tests with explicit includes; the
existing CMake format/*.cpp glob discovers it. The production and S3 regression
changes match the merged source PR.
### Testing
Passed locally on branch-4.1: clang-format 16.0.6 checks for both changed
C++ files, `git diff --cached --check`, Groovy syntax/AST conversion, and
checks that the new unit-test block matches the merged source PR and all merge
conflicts are resolved.
The shared full Groovy semantic compilation check was blocked by the host
Java 25 / cached Groovy 4.0.19 combination (unsupported class-file major
version 69); the syntax/AST conversion check passed. No Doris compilation, BE
unit-test execution or runtime regression was performed. CI test pipelines are
deferred at the author's request. No submodules were initialized for this work.
### Notes for Reviewers
This is a backport of master commit
`1bb35373ee076af701e9070d472f7ee47073b565` from #69448. Keep this PR in draft
while pipeline testing is deferred. The commit includes a CI skip directive,
and no test-triggering comments were posted.
### Release note
Fix a BE crash when Parquet schema conversion produces NULL for an invalid
value.
### Check List (For Author)
- Test:
- [ ] Regression test executed
- [ ] Unit Test executed
- [x] Manual/static checks: clang-format 16, whitespace, Groovy
syntax/AST conversion, source-patch and conflict checks.
- Behavior changed:
- [ ] No.
- [x] Yes. Failed nullable conversions produce NULL without accessing an
unallocated null map.
- Does this need documentation?
- [x] No.
- [ ] Yes.
- Pre-submission status: Partially verified: compilation, UT, sanitizer and
runtime regression validation deferred.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
### AI Disclosure
- Tool: Codex (GPT-6)
- Generated: Backport conflict resolution and this description.
- Human review: Pending.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
