richardchen874-sys commented on issue #858: URL: https://github.com/apache/flink-agents/issues/858#issuecomment-5018300297
I like the way this issue frames tracking token usage metrics for embedding models, not only chat models. In real workloads, the hard part is usually not sending one request; it is keeping cost, latency, and response shape predictable once users switch providers. I would record provider, model, dimensions, input batch size, token usage, and latency for embeddings just like chat responses. That makes embedding-heavy workflows visible before they become a hidden cost center. I am testing an OpenAI-compatible multi-model API layer around official Chinese models, so this kind of routing, usage, and compatibility boundary is very relevant. Who do you expect to consume these metrics first: developers debugging agents, operators managing spend, or end users? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
