airborne12 commented on code in PR #69882:
URL: https://github.com/apache/doris/pull/69882#discussion_r4422851046
##########
be/src/storage/index/inverted/snii/snii_doris_adapter.cpp:
##########
@@ -307,7 +312,7 @@ Status DorisSniiFileReader::read_batch(const
std::vector<::doris::snii::io::Rang
// Same shape as the hedged-read path in
cached_remote_file_reader.cpp:500-505.
const std::shared_ptr<ResourceContext> parent_resource_ctx =
thread_context()->resource_ctx();
- for (size_t base = 0; base < num_segs; base += kMaxConcurrentReads) {
+ for (size_t base = first; base < num_segs; base +=
kMaxConcurrentReads) {
const size_t wave_end = std::min(base + kMaxConcurrentReads,
num_segs);
::doris::CountDownLatch latch(cast_set<int>(wave_end - base));
for (size_t s = base; s < wave_end; ++s) {
Review Comment:
This is existing code and not reachable, so I am leaving it out of this PR.
The submit and the inline fallback in `fan_out` are unchanged: master runs
the same code for every batch that goes to the pool (`snii_doris_adapter.cpp`,
`submit_func` and the `!submit_st.ok()` branch). This PR only calls it for the
rest of a batch.
`ThreadPool::do_submit` returns an error with the task still queued only
when `create_thread()` fails and the pool has no thread at all.
`BufferedReaderPrefetchThreadPool` is built with
`num_buffered_reader_prefetch_thread_pool_min_thread` (16, and
`get_num_threads` turns 0 into the core count), a worker only exits while the
pool is above that minimum, and after shutdown `do_submit` returns
`_pool_status` before it queues anything. So the pool can only be empty if none
of its threads could start when the BE came up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
