TheR1sing3un opened a new pull request, #10585: URL: https://github.com/apache/paimon/pull/10585
Ray hybrid search executes routes sequentially even when independent routes leave search-task slots idle. For example, a single vector split and a single full-text shard each use one slot despite concurrency=2. Run independent routes on driver threads after resolving their shared snapshot. Divide the existing concurrency budget between active routes, retain route order for ranking, and preserve the serial path for concurrency=1 or one route. A route failure signals sibling task iterators to cancel their outstanding Ray work; the cancellation event stays on the driver and is not serialized into worker context. The allocation is deliberately conservative: floor(concurrency / active_routes) slots per route. It never multiplies the search-task budget, though a remainder can remain unused. Scalar-filter dependency tasks retain their existing scheduling behavior. Validation: - 71 Ray hybrid/full-text/vector tests passed, covering overlap, concurrency bounds, ordering, all three rankers, retries, failures, and task cleanup. - Project flake8 and git diff --check passed. - Local experiment with 30,000 rows, two Ray CPUs, warm workers and seven alternating repetitions: vector raw scan + indexed text median 59.3 ms -> 42.9 ms; two indexed text routes 82.8 ms -> 50.0 ms. All returned row IDs and scores matched sequential execution. These are local query-execution timings, not a cluster scalability claim. No public API changes or benchmark code are included. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
