wenjin272 commented on code in PR #1047:
URL: https://github.com/apache/flink-agents/pull/1047#discussion_r3912075877
##########
api/src/main/java/org/apache/flink/agents/api/embedding/model/BaseEmbeddingModelSetup.java:
##########
@@ -93,9 +94,52 @@ public String getModel() {
return model;
}
+ /**
+ * Record embedding token usage metrics for the given model on this
setup's bound metric group.
+ *
+ * <p>Mirrors {@code BaseChatModelSetup#recordTokenMetrics} but records
input-side tokens only,
+ * since embeddings have no completion tokens. Counters are placed under
the same {@code model}
+ * key-value group used by chat metrics, so embedding and chat usage for a
model share one
+ * dimension.
+ *
+ * <p>Unlike the chat path, embedding calls do not run inside a plan
action that hands in a
+ * request-scoped metric group (vector-store, RAG, and direct calls reach
this setup directly),
+ * so the resource-bound metric group injected via {@link #setMetricGroup}
is used instead.
+ *
+ * @param modelName the name of the model used
+ * @param promptTokens the number of prompt tokens
+ * @param totalTokens the total number of tokens reported by the provider
+ */
+ public void recordTokenMetrics(String modelName, long promptTokens, long
totalTokens) {
+ Preconditions.checkArgument(
+ modelName != null && !modelName.isBlank(), "Model name must
not be null or blank.");
+ FlinkAgentsMetricGroup metricGroup = getMetricGroup();
Review Comment:
For context, this is the same recording-boundary issue that led #870 to
remove automatic metric recording and retain only usage transport. @joeyutong
is working on the framework-level fix, so this PR may need to wait for that
work and then adopt the resulting API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
