Gabriel39 opened a new pull request, #69147:
URL: https://github.com/apache/doris/pull/69147
### What problem does this PR solve?
Related PR: #69127 (merged into branch-4.1)
Annotated Parquet Variant files may contain a required BYTE_ARRAY fallback
in an array element without a typed carrier. The same physical layout was
accepted through Paimon schema overrides but rejected for VARIANT(1)-annotated
files, including files read through Iceberg.
Port #69127: remove `allow_paimon_shredded_layout` and share the existing
compatibility for fallback-only arrays, optional shredded metadata, and
optional wrappers. Missing metadata and null array elements remain invalid at
materialization; object fields and arrays with typed carriers still require
optional fallbacks.
The port includes schema tests, a native file-reader regression, and two
entirely synthetic six-row Parquet fixtures with a reproducible generator. No
external data is included.
### Release note
Fix reading annotated Parquet Variant columns with compatible shredded
layouts previously accepted only through Paimon schema overrides.
### Check List (For Author)
- Test
- [x] Unit Test (ported coverage; target-branch execution pending CI)
- [x] Manual test: regenerated both fixtures byte-for-byte and verified
VARIANT(1), six rows and three row groups per file.
Master build hygiene checks also passed. Repository clang-format 16 and `git
diff --check` passed. BE unit tests and full builds were not rerun locally for
this port; `run buildall` is requested. The original PR contains the prior
focused ASAN and native reader validation results.
- Behavior changed:
- [x] Yes. Annotated Variant columns share the existing shredded-layout
compatibility rules.
- Does this need documentation?
- [x] No.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
