leaves12138 commented on code in PR #62:
URL:
https://github.com/apache/paimon-vector-index/pull/62#discussion_r3651804509
##########
include/paimon_vindex.hpp:
##########
@@ -54,11 +56,59 @@ struct OutputFile {
using ReadRequest = PaimonVindexReadRequest;
struct InputFile {
+ // DiskANN batch search may invoke this callback from multiple worker
threads.
std::function<int(ReadRequest* requests, size_t request_count)>
read_ranges_fn;
+ // Optional positional-read capabilities. Zero leaves the policy
unspecified.
+ uint64_t estimated_random_read_latency_nanos = 0;
+ size_t preferred_window_bytes = 0;
+ size_t max_ranges_per_read = 0;
};
namespace detail {
+class NativeHandleMutex {
+public:
+ void lock() {
+ const auto current = std::this_thread::get_id();
+ {
+ std::lock_guard<std::mutex> state_lock(state_mutex_);
+ if (owner_ == current) {
Review Comment:
The owner-thread check still deadlocks when the storage callback is invoked
by a native worker thread rather than by the thread that entered the Reader. I
reproduced this with `RAYON_NUM_THREADS=4`, a DiskANN `search_batch`, and an 8
KiB Reader budget so adjacency reads remain cold. A worker-thread
`read_ranges_fn` callback called `reader.metadata()`; the outer search held
`operation_mutex_` on the caller thread and waited for the worker, while the
worker saw a different `owner_` and blocked on the same mutex. The process
timed out after five seconds. Since DiskANN explicitly invokes callbacks from
separate query workers, could reentry be tracked for the active native
operation/callback context rather than only by comparing thread IDs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
