mayya-sharipova commented on code in PR #16794:
URL: https://github.com/apache/lucene/pull/16794#discussion_r4261471319
##########
lucene/core/src/java/org/apache/lucene/util/hnsw/HnswConcurrentMergeBuilder.java:
##########
@@ -114,23 +182,60 @@ public OnHeapHnswGraph build(int maxOrd) throws
IOException {
worker.setMergeStartTimeNs(mergeStartTimeNs);
worker.setCumulativeWorkTimeNs(cumulativeWorkTimeNs);
}
+ // The join plans only read the graphs being joined, so they are computed
while the repaired
+ // base graph is rebalanced, which runs on a single thread.
+ List<Callable<JoinPlan>> tasks = new ArrayList<>(graphs.size() + 1);
+ for (int g = 0; g < graphs.size(); g++) {
+ IOSupplier<HnswGraph> graph = graphs.get(g);
+ int[] ordMap = ordMaps[g];
+ tasks.add(() -> JoinPlan.create(graph.get(), ordMap));
+ }
+ int repairedNodes = 0;
+ long repairNs = 0;
+ AtomicLong rebalanceNs = new AtomicLong();
if (prunedGraph != null) {
long repairStartNs = System.nanoTime();
- int repairedNodes = repairDisconnectedNodes();
- long rebalanceStartNs = System.nanoTime();
- prunedGraph.builder().rebalanceGraph();
- long rebalanceEndNs = System.nanoTime();
- if (infoStream.isEnabled(HNSW_COMPONENT)) {
+ repairedNodes = repairDisconnectedNodes();
+ repairNs = System.nanoTime() - repairStartNs;
+ tasks.add(
Review Comment:
Since invokeAll starts tasks in list order, could the `rebalance` task go
first? It's the long single-threaded one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
