Gabriel39 opened a new pull request, #69148:
URL: https://github.com/apache/doris/pull/69148
### What problem does this PR solve?
Related PR: #69127 (merged into branch-4.1)
Annotated Parquet Variant files may contain a required BYTE_ARRAY fallback
in an array element without a typed carrier. The same physical layout was
accepted through Paimon schema overrides but rejected for VARIANT(1)-annotated
files, including files read through Iceberg.
Port #69127: remove `allow_paimon_shredded_layout` and share the existing
compatibility for fallback-only arrays, optional shredded metadata, and
optional wrappers. Missing metadata and null array elements remain invalid at
materialization; object fields and arrays with typed carriers still require
optional fallbacks.
The schema acceptance tests call NativeFieldDescriptor directly because
diagnostic PR #69119 is still open. The file-reader regression still covers
actual Parquet footers. The diagnostic rejection fixture in #69119 is updated
separately to remain invalid after this compatibility fix, so the PRs can merge
in either order.
The port includes schema tests, a native file-reader regression, and two
entirely synthetic six-row Parquet fixtures with a reproducible generator. No
external data is included.
### Release note
Fix reading annotated Parquet Variant columns with compatible shredded
layouts previously accepted only through Paimon schema overrides.
### Check List (For Author)
- Test
- [x] Unit Test (ported coverage; target-branch execution pending CI)
- [x] Manual test: regenerated both fixtures byte-for-byte and verified
VARIANT(1), six rows and three row groups per file.
Repository clang-format 16 and `git diff --check` passed. BE unit tests and
full builds were not rerun locally for this port; `run buildall` is requested.
The original PR contains the prior focused ASAN and native reader validation
results.
- Behavior changed:
- [x] Yes. Annotated Variant columns share the existing shredded-layout
compatibility rules.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
