jamesnetherton commented on issue #8864: URL: https://github.com/apache/camel-quarkus/issues/8864#issuecomment-5355467599
@jomin7 thanks for investigating, that's an interesting finding. Here's an AI agent summary about it: > This is a confirmed bug in Quarkus LangChain4j, not a design limitation. Here's the breakdown: > > How Camel gets token counts: LangChain4jEmbeddingsProducer calls model.embed(in), gets back a Response<Embedding>, and checks result.tokenUsage(). It's a straight passthrough — if the EmbeddingModel implementation provides TokenUsage, Camel exposes it as message headers. Nothing special on Camel's side. > > Where QL4J loses it: The issue is provider-specific in QL4J's embedding model implementations: > > - Ollama — OllamaEmbeddingModel calls Response.from(embeddings) (single-arg), which sets tokenUsage to null. The EmbeddingResponse DTO only deserializes the embeddings field — it drops prompt_eval_count from the Ollama /api/embed JSON response entirely. Compare with ChatResponse which properly deserializes promptEvalCount/evalCount and constructs TokenUsage from them. > - HuggingFace — Same pattern: Response.from(embeddings) with no token usage. > - Azure OpenAI — This one actually works correctly. It uses Response.from(embeddings, new TokenUsage(inputTokenCount)), extracting from response.usage().promptTokens(). > > Root cause: The Ollama /api/embed endpoint does return prompt_eval_count in its JSON — QL4J's DTO just doesn't deserialize it. The fix would be straightforward: add prompt_eval_count to the EmbeddingResponse DTO and pass it to Response.from(embeddings, tokenUsage) instead of the single-arg overload. > For now in the tests maybe we just have to expect missing token count info. We can open an issue with Quarkus LangChain4j to fix the issue, then update our tests later when it's available. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
