csun5285 opened a new pull request, #68609:
URL: https://github.com/apache/doris/pull/68609
### What problem does this PR solve?
Issue Number: None
Related PR: #68608, #62854
Problem Summary:
The first piece split out of #68608, which writes segments straight from
IColumn and removes OlapBlockDataConvertor. It changes no behavior: it pins
the
page bytes the later pieces must keep, and removes code they would otherwise
carry along. Each commit message has the details.
1. Remove dead storage write-path code: `get_zone_map_value()` in
column_predicate.h, `WindowFunnelState::NativeType`, the never-used
`ColumnWriter::_null_bitmap`, the unreachable `append_nulls()` of the
struct
and map column writers (they now return NotSupported, as the variant
writers
already do), and an unused OlapBlockDataConvertor include in schema
change.
2. Drop the OFFSET_ONLY guard for CHAR columns in StringFileColumnIterator.
#62854 rejected an OFFSET_ONLY access path on a CHAR column because the
lengths read without the chars would be the zero-padded ones. A CHAR page
is
rewritten without its padding by its strip pre-decoder before the iterator
reads any offset, so the lengths are the logical ones, and the FE never
plans
this access path for CHAR anyway.
3. Pin the on-disk pages of every page encoding with golden files.
PageFormatTest appends deterministic rows block by block to a
ScalarColumnWriter, the way a segment writer does, with small page
budgets,
for every registered (storage type, page encoding) pair: 86 cases, DICT
once
per dictionary page format and AGG_STATE once per serialized shape. The
pages PageIO writes are kept as they are on disk, one golden file per
encoding plus layout.txt, which lists every page, so a failure names the
page
that changed. Each column is also read back through ColumnReader and must
hold the rows written. FOR is left out: nothing has written FOR pages
since
bitmap indexes were removed, FrameOfReferencePageDecoder cannot read the
values back, and a frame that keeps its original values carries bytes
ForEncoder never initializes.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- PageFormatTest (new) passes against its goldens, also when run after
BitmapValueTest, which leaves bitmap configs changed (46 tests).
- SegmentFlusherFormatTest and SegmentFlusherTransformFormatTest (strict
golden comparison), ColumnReaderTest and CharTypePaddingTest pass
(80 tests).
- Behavior changed: No
- Does this need documentation: No
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
