imbajin commented on code in PR #424:
URL: https://github.com/apache/hugegraph-ai/pull/424#discussion_r4614366678
##########
hugegraph-llm/src/hugegraph_llm/utils/embedding_utils.py:
##########
@@ -45,20 +40,34 @@ async def get_embeddings_parallel(embedding: BaseEmbedding,
vids: list[str]) ->
order as the input vids list.
Note:
- - Note: Uses a semaphore to limit maximum concurrency if we need
- - Processes texts in batches of 500
+ - Processes texts in batches of 500 with at most 10 batches in flight
+ - Fetches through get_texts_embeddings on the loop's executor: callers
run this
+ helper on short-lived loops (asyncio.run per index update), and an
async
+ client's pooled connections outlive those loops
- Displays progress using a progress bar that updates as each batch
completes
- Uses asyncio.gather() to preserve order correspondence between input
and output
"""
batch_size = 500
+ # Bound concurrent batches so a large input cannot fan out one request per
batch.
+ sem = asyncio.Semaphore(10)
+
+ async def get_batch_with_progress(batch: list[str], pbar: tqdm) ->
list[Any]:
+ async with sem:
+ loop = asyncio.get_running_loop()
+ # Callers run this helper on a fresh loop per update, while the
embedding
+ # instance (and its clients) outlives those loops. The synchronous
client
+ # holds no loop-bound state, so repeated runs on one instance stay
safe.
+ result = await loop.run_in_executor(None,
embedding.get_texts_embeddings, batch)
Review Comment:
**⚠️ Stop remaining batch work after a failure**
If one batch fails while other batches are running, `asyncio.run()` waits
for its executor threads to finish; cancelling the futures cannot stop their
synchronous work. For Gremlin and vector indexing this replaces cancellable
async requests, and an OpenAI batch can continue issuing up to 50 sequential
requests after the overall operation has failed, delaying the error and
incurring unnecessary API usage. Please make remaining batches cooperatively
stop on failure or use cancellable requests with a client lifecycle scoped to
the event loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]