BjornPrime commented on issue #21437:
URL: https://github.com/apache/beam/issues/21437#issuecomment-1270552212
I'm just imagining a situation where we want to update one metric and not
all of the others. For example, we could have a method
`update_metrics(metrics)`, where "metrics" is a dict with names of metrics as
keys and numerical values that are interpreted according to the metric they're
associated with. So if I run `update_metrics({"_num_failed_inferences": 2})`,
it would increment _num_failed_inferences twice, but if the key was
_inference_batch_latency_micro_secs, it would know to just set that equal to 2.
And you could run both together with one call if you include both key-value
pairs.
Maybe that's too generic for our purposes. It depends on how likely we are
to want to update specific metrics but not others. Valentyn has also pointed
out that none of this is directly exposed so if we need to change it later, we
shouldn't end up breaking anybody else.
I think we do need to consider how a failure affects the other metrics
though. For example, the inference counter basically just adds the number of
items in the batch after the batch finishes, but if the batch is interrupted by
failures, that number won't be accurate will it? Or maybe I'm misunderstanding
what's going on under the hood of run_inference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
