Akash3121 commented on code in PR #11254:
URL: https://github.com/apache/paimon/pull/11254#discussion_r4618079170


##########
paimon-python/pypaimon/execution/python/read/search/vector_search_read.py:
##########
@@ -356,20 +369,27 @@ def _read_raw_search(self, raw_row_ranges, pre_filter, 
query_vector,
         if not raw_row_ranges:
             return DictBasedScoredIndexResult({})
 
-        table = self._read_raw_arrow(raw_row_ranges, include_filter, snapshot)
-        if table is None or table.num_rows == 0:
-            return DictBasedScoredIndexResult({})
+        table_read: "TableRead"
+        splits: "List[Split]"
+        table_read, splits = self._plan_raw_read(raw_row_ranges, 
include_filter, snapshot)
+        metric = self._search_metric(index_type)
+        workers = min(len(splits), table_read._resolve_parallelism(None, 
len(splits)))
+        if workers <= 1:
+            source = _open_raw_batch_reader(table_read, splits)
+        else:
+            source = closing(_parallel_raw_batches(table_read, splits, 
workers))
 
-        return self._score_raw_batches(
-            table.to_batches(max_chunksize=_score_block_size(query_vector)),
-            query_vector, self._search_metric(index_type), score_candidates)
+        with source as batches:
+            return self._score_raw_batches(batches, query_vector, metric, 
score_candidates)
 
     def _score_raw_batches(self, batches, query_vector, metric, 
score_candidates=None, reject_nan=False):
         """Score bounded Arrow batches, shared by local and distributed 
reads."""
         import math
 
         top_k_heap = []
-        for block in batches:
+        # Bound the Arrow take as well as the scoring matrix for wide vectors.
+        blocks = batches if score_candidates is None else 
_slice_batches(batches, _score_block_size(query_vector))
+        for block in blocks:

Review Comment:
   In `_parallel_raw_batches` (line 902), `del batch, future` is used after 
`yield batch` so the just-scored `RecordBatch` can be freed before waiting on 
the next read. However, in Python a `for block in blocks:` loop keeps `block`, 
`row_ids`, and `vectors` bound in `_score_raw_batches`'s frame while 
`blocks.__next__()` is executing to fetch the next batch (and similarly `batch` 
remains bound in `_slice_batches` on line 1137 while `batches.__next__()` runs).
    
    Because `block` / `vectors` (and `batch` in `_slice_batches`) share the 
underlying Arrow buffers, the previous `RecordBatch` stays pinned in memory 
throughout the next batch read until the loop variable is reassigned.
    
    Consider explicitly deleting the loop references at the end of each 
iteration (matching `del batch, row_ids, vectors` in `_score_raw_batch_queries` 
on line 868), and doing `del batch` at the end of the outer loop in 
`_slice_batches`:
    
    ```python
            for block in blocks:
                row_ids = block.column(SpecialFields.ROW_ID.name).to_pylist()
                if score_candidates is not None:
                    positions = [i for i, row_id in enumerate(row_ids)
                                 if row_id in score_candidates]
                    if not positions:
                        del block, row_ids
                        continue
                    vectors = 
block.column(self._vector_column.name).take(positions)
                    row_ids = [row_ids[i] for i in positions]
                else:
                    vectors = block.column(self._vector_column.name)
                for row_id, score in zip(
                    row_ids, _iter_arrow_scores(vectors, query_vector, metric)
                ):
                    if score is not None:
                        if reject_nan and math.isnan(score):
                            raise ValueError("Ray vector search cannot rank NaN 
scores.")
                        _offer_score(top_k_heap, self._limit, row_id, score)
                del block, row_ids, vectors
    ```



##########
paimon-python/pypaimon/execution/python/read/search/vector_search_read.py:
##########
@@ -848,6 +868,67 @@ def _score_raw_batch_queries(self, batches, metric):
         return [_scored_result(heap) for heap in heaps]
 
 
+def _open_raw_batch_reader(table_read: "TableRead", splits: "List[Split]") -> 
"_ClosableArrowBatchReader":
+    from pypaimon.read.read_support import _ClosableArrowBatchReader
+
+    reader, source = table_read._new_arrow_batch_reader(splits)
+    return _ClosableArrowBatchReader(reader, source)
+
+
+def _parallel_raw_batches(
+    table_read: "TableRead", splits: "List[Split]", workers: int,
+) -> "_BatchStream":
+    """Yield ready batches, with at most one outstanding read per split 
group."""
+    # Readers may be suspended after yielding. Join all in-flight reads
+    # before closing their iterators, including when scoring fails.
+    with ExitStack() as streams, ThreadPoolExecutor(max_workers=workers) as 
executor:
+        pending: "Dict[Future[Optional[pa.RecordBatch]], 
Iterator[pa.RecordBatch]]" = {}
+        try:
+            for i in range(workers):
+                group = splits[i::workers]
+                batches = _open_prefetched_batches(table_read, group, streams)
+                pending[executor.submit(next, batches, None)] = batches
+            while pending:
+                completed = wait(pending, return_when=FIRST_COMPLETED).done
+                while completed:
+                    future = completed.pop()
+                    batches = pending.pop(future)
+                    batch = future.result()
+                    if batch is None:
+                        continue
+                    yield batch
+                    # Keep completed batches bounded, and let ready
+                    # readers progress while another reader is slow.
+                    del batch, future
+                    pending[executor.submit(next, batches, None)] = batches
+        finally:
+            for future in pending:
+                future.cancel()
+
+
+def _open_prefetched_batches(
+    table_read: "TableRead", splits: "List[Split]", streams: ExitStack,
+) -> "_BatchStream":
+    """Open on the caller thread and register cleanup before submitting the 
first read."""
+    reader_scope = streams.enter_context(ExitStack())
+    reader = reader_scope.enter_context(_open_raw_batch_reader(table_read, 
splits))

Review Comment:
   When a split group yields multiple batches (`>= 2` batches across its 
splits/row groups), combining `_read_one_batch_ahead` with submitting 
`next(batches, None)` *after* `yield batch` has two side effects:
    
    1. **First-batch latency & eager second-batch read:** On the initial 
`executor.submit(next, batches, None)`, `_read_one_batch_ahead` reads `current 
= next(reader, None)` (batch 0) and then immediately enters `for following in 
reader:` to read batch 1 *before* yielding batch 0. The caller cannot start 
scoring batch 0 until two full batches have been read by that worker, and if 
scoring fails on the first batch, every multi-batch worker has already read a 
second batch.
    2. **No I/O–compute overlap within a split group:** Because 
`pending[executor.submit(next, batches, None)] = batches` happens *after* 
`yield batch`, the worker thread for that split group sits idle while the 
caller scores `batch` (holding `following` in its suspended generator frame). 
Then, once scoring finishes, the next `next(batches, None)` call cannot return 
the already-buffered `following` batch until it finishes reading the 
*subsequent* batch from storage.
    
    Would it be simpler to have `batches` yield one batch per `next()` call 
(closing `reader_scope` in `finally` on exhaustion) and submit 
`executor.submit(next, batches, None)` *before* `yield batch`? That way batch 0 
can be scored after a single batch read, reading batch `k + 1` (or hitting EOF 
and closing the reader after the last batch) overlaps on the worker thread 
while the caller scores batch `k`, and each split group still has at most 1 
batch being scored + 1 read in flight.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to