bitflicker64 commented on code in PR #430:
URL: https://github.com/apache/hugegraph-ai/pull/430#discussion_r4556782081
##########
hugegraph-llm/src/hugegraph_llm/nodes/hugegraph_node/graph_query_node.py:
##########
@@ -361,17 +366,24 @@ def _subgraph_query(self, context: Dict[str, Any]) ->
Dict[str, Any]:
vertex_knowledge =
self._format_graph_from_vertex(query_result=vertexes)
paths: List[Any] = []
- # TODO: use generator or asyncio to speed up the query logic
- for matched_vid in matched_vids:
+ # Batch the per-vid neighbour traversals: one gremlin exec per
+ # group instead of one per matched vid, so a large id set no longer
+ # triggers one server round-trip per id. Batch data is concatenated
+ # in matched_vids order, so the path list keeps the per-vid order.
+ batch_size = min(NEIGHBOR_QUERY_BATCH_SIZE, len(matched_vids))
+ for start in range(0, len(matched_vids), batch_size):
+ batch = matched_vids[start : start + batch_size]
gremlin_query = VID_QUERY_NEIGHBOR_TPL.format(
- keywords=f"'{matched_vid}'",
+ keywords=", ".join(f"'{vid}'" for vid in batch),
Review Comment:
Important: `bothE().limit({edge_limit})`, `dedup()` and
`.limit({max_items})` in `VID_QUERY_NEIGHBOR_TPL` are global steps, so the vids
in one batch share a single edge budget and dedup set; on TinkerGraph 3.5.1
with one edge label, 10 vids with 8 neighbours each return 8 paths, all from
the first vid, instead of 80. Wrap the traversal after `g.V(...)` in
`local(...)` so each vid keeps its own limits and dedup, and add a test that
checks the generated query keeps that per-vid scope.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]