morningman opened a new pull request, #435:
URL: https://github.com/apache/doris-thirdparty/pull/435

   ### Background
   
   Independent ORC readers use separate streams and predicate appliers. Stripe 
statistics live in a file-level Metadata protobuf, separate from the serialized 
file tail.
   
   ### What problem does this PR solve?
   
   Issue Number: None
   Related PR: apache/doris#70664 (physical stripe readers)
   PR Type: Bugfix
   
   Problem Summary:
   
   Physical stripe readers repeat complete Metadata parsing and all-file SARG 
evaluation. Eight one-stripe readers plus their planning reader can parse 
Metadata nine times and evaluate 72 stripe statistics. Share lazy immutable 
Metadata and add bounded stripe evaluation while preserving reader-local 
predicates and row-group filtering.
   
   ### Details
   
   - **Frequency:** Found by code review; triggered by independent readers with 
SARG over the same multi-stripe file.
   - **Root cause:** Serialized tails omit Metadata, and getNeedReadStripes 
evaluates the whole file regardless of the reader's range.
   - **Why existing tests missed it:** Predicate tests check pruning results 
without counting repeated setup across independent readers.
   - **Fix:** Add an opaque thread-safe lazy statistics handle, an interval 
overload and operation counters. A successful load publishes immutable 
Metadata; failed loads remain retryable. The handle retains no streams, 
allocators, metrics or SARG appliers.
   - **Other paths and branches:** The existing full-file API delegates to the 
interval implementation. No format or default changes; native consumers must 
rebuild for the extended interface.
   
   ### Testing
   
   Local Doris ASAN BE build passed with this SDK candidate. Standard BE unit 
tests, including the new concurrent cold/warm-handle test, are running; results 
will be updated before merge. The test checks one parse after parent 
destruction, bounded evaluation, independent predicates, row-group pruning and 
absolute row positions.
   
   ### Release note
   
   None
   
   ### Check List (For Author)
   
   - Behavior changed: No
   - Does this need documentation: No; SDK interface contracts are documented 
in the header.
   - Pre-submission status: Partially verified: unit tests are running.
   
   ### AI Disclosure
   
   - Tool: Codex
   - Generated: SDK implementation, tests and description.
   - Human review: not stated
   - Prompts: Fix F-01 from the review of apache/doris#70664 and update that PR.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to