imbajin commented on code in PR #424:
URL: https://github.com/apache/hugegraph-ai/pull/424#discussion_r4614366678


##########
hugegraph-llm/src/hugegraph_llm/utils/embedding_utils.py:
##########
@@ -45,20 +40,34 @@ async def get_embeddings_parallel(embedding: BaseEmbedding, 
vids: list[str]) ->
                   order as the input vids list.
 
     Note:
-        - Note: Uses a semaphore to limit maximum concurrency if we need
-        - Processes texts in batches of 500
+        - Processes texts in batches of 500 with at most 10 batches in flight
+        - Fetches through get_texts_embeddings on the loop's executor: callers 
run this
+          helper on short-lived loops (asyncio.run per index update), and an 
async
+          client's pooled connections outlive those loops
         - Displays progress using a progress bar that updates as each batch 
completes
         - Uses asyncio.gather() to preserve order correspondence between input 
and output
     """
     batch_size = 500
+    # Bound concurrent batches so a large input cannot fan out one request per 
batch.
+    sem = asyncio.Semaphore(10)
+
+    async def get_batch_with_progress(batch: list[str], pbar: tqdm) -> 
list[Any]:
+        async with sem:
+            loop = asyncio.get_running_loop()
+            # Callers run this helper on a fresh loop per update, while the 
embedding
+            # instance (and its clients) outlives those loops. The synchronous 
client
+            # holds no loop-bound state, so repeated runs on one instance stay 
safe.
+            result = await loop.run_in_executor(None, 
embedding.get_texts_embeddings, batch)

Review Comment:
   **⚠️ Stop remaining batch work after a failure**
   
   If one batch fails while other batches are running, `asyncio.run()` waits 
for its executor threads to finish; cancelling the futures cannot stop their 
synchronous work. For Gremlin and vector indexing this replaces cancellable 
async requests, and an OpenAI batch can continue issuing up to 50 sequential 
requests after the overall operation has failed, delaying the error and 
incurring unnecessary API usage. Please make remaining batches cooperatively 
stop on failure or use cancellable requests with a client lifecycle scoped to 
the event loop.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to