jlon commented on code in PR #861:
URL: https://github.com/apache/flink-agents/pull/861#discussion_r3541595356
##########
python/flink_agents/api/chat_models/chat_model.py:
##########
@@ -273,8 +278,12 @@ def _record_token_metrics(
The number of prompt tokens
completion_tokens : int
The number of completion tokens
+ metric_group : MetricGroup | None
+ The metric group captured when the request was initiated. If not
provided,
+ this resource's currently bound metric group is used.
"""
- metric_group = self.metric_group
+ if metric_group is None:
Review Comment:
Updated in ecd96c5. `_record_token_metrics` now requires the metric group
argument and no longer falls back to the resource-bound `self.metric_group`.
The action path also skips token metric recording when no request/action metric
group is available.
##########
api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelSetup.java:
##########
@@ -116,7 +116,22 @@ public void open() throws Exception {
* @param completionTokens the number of completion tokens
*/
public void recordTokenMetrics(String modelName, long promptTokens, long
completionTokens) {
Review Comment:
Updated in ecd96c5. Removed the 3-arg Java `recordTokenMetrics(...)`
overload and updated the tests to pass an explicit request/action metric group.
##########
api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelSetup.java:
##########
@@ -116,7 +116,22 @@ public void open() throws Exception {
* @param completionTokens the number of completion tokens
*/
public void recordTokenMetrics(String modelName, long promptTokens, long
completionTokens) {
- FlinkAgentsMetricGroup metricGroup = getMetricGroup();
+ recordTokenMetrics(getMetricGroup(), modelName, promptTokens,
completionTokens);
+ }
+
+ /**
+ * Record token usage metrics for the given model on the provided metric
group.
+ *
+ * @param metricGroup the metric group captured when the request was
initiated
+ * @param modelName the name of the model used
+ * @param promptTokens the number of prompt tokens
+ * @param completionTokens the number of completion tokens
+ */
+ public void recordTokenMetrics(
+ @Nullable FlinkAgentsMetricGroup metricGroup,
Review Comment:
Updated in ecd96c5. The Java `recordTokenMetrics(...)` metric group
parameter is no longer nullable, and `ChatModelAction` now skips token metric
recording before calling it when the action metric group is absent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
