mrhhsg opened a new pull request, #70769:
URL: https://github.com/apache/doris/pull/70769

   ### Background
   
   Backend literal expressions use constant columns with one stored physical 
value and an independent logical row count. The generic data-type factories 
create the physical column, insert a value or default, and then wrap it as a 
constant column. Large literal lists can keep many such columns alive at once.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: None
   PR Type: Performance
   
   Problem Summary:
   
   For POD-backed columns, the first insertion into an empty column can 
allocate a 4096-byte buffer even though a constant column stores only one 
physical value. A query with many independent literals can retain many of these 
buffers. Both generic constant-column factories now reserve one physical value 
before insertion; logical row counts and SQL results are unchanged. Most of 
this diff adds unit and SQL regression coverage for the allocation and 
semantics.
   
   ### Details
   
   - **Why it was slow:** The first insertion into an empty POD-backed numeric 
or offset column uses its default 4096-byte allocation. Each literal owns a 
separate one-value physical column, so the allocation repeats across a large 
list.
   - **Optimization:** Reserve one physical value before `insert` or 
`insert_default` in the two generic constant-column factories. Keep the 
existing constant wrapper and logical row count.
   - **Results:** On one local host with an ASAN build, one focused unit-test 
run per version kept 4096 `Int64` literal columns alive and summed their 
physical columns' `allocated_bytes()`:
   
     | Measurement | Before | After |
     |---|---:|---:|
     | Allocated bytes for 4096 physical columns | 16,777,216 | 262,144 |
   
     The baseline failed the new allocation assertions and the candidate 
passed. This measures retained physical-column allocation, not query peak 
memory, RSS, CPU, or latency. Host hardware was not recorded; no Release 
benchmark was run.
   - **Applicability and cost:** The saving applies when one-value constant 
columns would otherwise take a larger initial POD allocation, including large 
literal lists. Recursive reservation may allocate small buffers in nested 
columns that do not need them for a particular value, while long strings may 
incur an extra short-lived allocation. No configuration or SQL behavior changes.
   
   ### Testing
   
   - Local ASAN BE unit tests: the four new focused tests failed on the 
unmodified baseline and passed 4/4 after the change, checking physical 
allocation, values, defaults, nullable and complex types, strings, and 
logical-row expansion. A post-commit run on the reviewed head also passed 4/4.
   - Local ASAN BE unit tests: 60/60 related literal and data-type tests passed.
   - Local P0 SQL regression for small and large `IN`/`NOT IN` lists: one suite 
passed during expected-output generation and passed again in an isolated-port 
comparison; a final comparison after removing only a terminal blank line from 
the generated output also passed. The suite checks ordered results, including 
NULL behavior.
   
   ### Notes for Reviewers
   
   Only three production lines change; the remaining 251 added lines are tests 
and generated expected output. Reserve one row rather than the logical row 
count because a constant column stores exactly one physical value. The 
allocation result above does not establish query-level memory or latency gains.
   
   ### Release note
   
   Reduce physical-column allocation when constructing many constant literals. 
Query results are unchanged.
   
   ### Check List (For Author)
   
   - Behavior changed: No
   - Does this need documentation: No
   - Pre-submission status: Partially verified: query-level peak memory, 
Release timing, and CI have not been measured or run.
   
   ### AI Disclosure
   
   - Tool: OpenAI Codex (model not stated)
   - Generated: Code, tests, and this description.
   - Human review: not stated


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to