airborne12 opened a new pull request, #69811:
URL: https://github.com/apache/doris/pull/69811
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Segment::_footer_pb is a std::weak_ptr to the footer owned
by the storage page
cache. _get_segment_footer() calls lock() on it and, once the cache entry
has been evicted,
parses the footer again and assigns the weak_ptr; get_metadata_size() calls
lock() as well. A
Segment is shared by every reader of its rowset (queries, compaction,
statistics), so lock() and
the assignment run on the same weak_ptr object concurrently. That is a data
race: the standard
only allows concurrent use of distinct weak_ptr objects, and the control
block counts can be
corrupted so that a footer still held through a shared_ptr gets destroyed.
Every access to _footer_pb now goes through two small accessors that take a
mutex.
The new test fetches the footer from 8 threads while half of them keep
evicting its cache page.
On master's segment.cpp, ASAN reported a heap-use-after-free within 20 runs:
a thread read the
footer it had just obtained (SegmentFooterPB::ByteSizeLong) after another
thread had freed it in
std::weak_ptr assignment (__weak_count::operator= -> _M_weak_release ->
_M_destroy). With the
change, 20 of 20 runs passed without an ASAN report.
The race was found while investigating BE crashes on large VARIANT tables
under heavy
compaction; those crashes had a different cause (the host kernel with
jemalloc MADV_FREE).
### Release note
Fix a data race on the cached segment footer that could free a footer still
in use.
### Check List (For Author)
- Test: Unit Test
(SegmentFooterCacheTest.ConcurrentGetSegmentFooterWithEviction; ASAN
heap-use-after-free without the change within 20 runs, 20/20 clean with
it; the other
SegmentFooterCacheTest cases pass)
- Behavior changed: No
- Does this need documentation: No
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
