mrhhsg opened a new pull request, #70769:
URL: https://github.com/apache/doris/pull/70769
### Background
Backend literal expressions use constant columns with one stored physical
value and an independent logical row count. The generic data-type factories
create the physical column, insert a value or default, and then wrap it as a
constant column. Large literal lists can keep many such columns alive at once.
### What problem does this PR solve?
Issue Number: None
Related PR: None
PR Type: Performance
Problem Summary:
For POD-backed columns, the first insertion into an empty column can
allocate a 4096-byte buffer even though a constant column stores only one
physical value. A query with many independent literals can retain many of these
buffers. Both generic constant-column factories now reserve one physical value
before insertion; logical row counts and SQL results are unchanged. Most of
this diff adds unit and SQL regression coverage for the allocation and
semantics.
### Details
- **Why it was slow:** The first insertion into an empty POD-backed numeric
or offset column uses its default 4096-byte allocation. Each literal owns a
separate one-value physical column, so the allocation repeats across a large
list.
- **Optimization:** Reserve one physical value before `insert` or
`insert_default` in the two generic constant-column factories. Keep the
existing constant wrapper and logical row count.
- **Results:** On one local host with an ASAN build, one focused unit-test
run per version kept 4096 `Int64` literal columns alive and summed their
physical columns' `allocated_bytes()`:
| Measurement | Before | After |
|---|---:|---:|
| Allocated bytes for 4096 physical columns | 16,777,216 | 262,144 |
The baseline failed the new allocation assertions and the candidate
passed. This measures retained physical-column allocation, not query peak
memory, RSS, CPU, or latency. Host hardware was not recorded; no Release
benchmark was run.
- **Applicability and cost:** The saving applies when one-value constant
columns would otherwise take a larger initial POD allocation, including large
literal lists. Recursive reservation may allocate small buffers in nested
columns that do not need them for a particular value, while long strings may
incur an extra short-lived allocation. No configuration or SQL behavior changes.
### Testing
- Local ASAN BE unit tests: the four new focused tests failed on the
unmodified baseline and passed 4/4 after the change, checking physical
allocation, values, defaults, nullable and complex types, strings, and
logical-row expansion. A post-commit run on the reviewed head also passed 4/4.
- Local ASAN BE unit tests: 60/60 related literal and data-type tests passed.
- Local P0 SQL regression for small and large `IN`/`NOT IN` lists: one suite
passed during expected-output generation and passed again in an isolated-port
comparison; a final comparison after removing only a terminal blank line from
the generated output also passed. The suite checks ordered results, including
NULL behavior.
### Notes for Reviewers
Only three production lines change; the remaining 251 added lines are tests
and generated expected output. Reserve one row rather than the logical row
count because a constant column stores exactly one physical value. The
allocation result above does not establish query-level memory or latency gains.
### Release note
Reduce physical-column allocation when constructing many constant literals.
Query results are unchanged.
### Check List (For Author)
- Behavior changed: No
- Does this need documentation: No
- Pre-submission status: Partially verified: query-level peak memory,
Release timing, and CI have not been measured or run.
### AI Disclosure
- Tool: OpenAI Codex (model not stated)
- Generated: Code, tests, and this description.
- Human review: not stated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]