Gabriel39 commented on PR #435: URL: https://github.com/apache/doris-thirdparty/pull/435#issuecomment-6967335487
Review of `b3ee845`: **[P1] The retryable Metadata loader can block permanently after an exception with pthread_once-backed runtime combinations** At [`ReaderImpl::readMetadata()`](https://github.com/apache/doris-thirdparty/blob/b3ee845417888bc60a009f1610d41219e4b5fb54/c%2B%2B/src/Reader.cc#L1000), the `std::call_once` callback performs I/O, decompression, allocation, and protobuf parsing, all of which can throw. The implementation relies on exceptional execution resetting the once flag so another reader can retry. I tested this through the SDK: export a cold statistics handle, destroy the parent, inject one Metadata read exception in the first child, catch that exception, and retry through a second child sharing the handle. The first exception was caught, but the second child blocked in `__pthread_once_slow`. A sequential diagnostic control that bypassed `call_once` completed the retry. This is **toolchain/runtime dependent**: a separate system compiler/runtime control succeeded, and I have not reproduced it in a running Doris BE. The underlying compatibility issue is documented in [GCC's explanation of the pthread_once-backed implementation](https://gcc.gnu.org/pipermail/libstdc%2B%2B-cvs/2021q1/035565.html). This minimal program also distinguishes affected runtime combinations: ```cpp #include <cstdio> #include <mutex> #include <stdexcept> int main() { std::once_flag flag; try { std::call_once(flag, [] { throw std::runtime_error("injected metadata read failure"); }); } catch (const std::runtime_error&) { std::puts("first failure caught"); std::fflush(stdout); } std::call_once(flag, [] { std::puts("retry succeeded"); }); } ``` On the affected combination, the second call never returns. Since siblings share the flag, a recoverable read or parse failure can leave subsequent readers waiting indefinitely. Please consider a mutex-protected loading state, publishing the immutable Metadata and marking it loaded only after successful parsing. Add a failure-then-retry test using two readers with the same handle; the new test currently exercises successful cold/warm loads only. Normal-path validation passed: 119 selected ASAN tests, 25 repetitions of the new concurrent test, and all 45 valid intervals of an eight-stripe fixture with mixed pruning decisions. One downstream integration note: the currently published `add70a4` head of apache/doris#70664 still passes only the serialized tail and calls the full-file `getNeedReadStripes(opts)`. It needs to propagate the shared statistics handle and use the interval overload, including its relative result indexing, before the intended Doris optimization takes effect. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
