Can the system which controls the tasks collect the information when a task finishes and then add it to metrics it exposes? Total number of tasks completed, total time spent, etc.
On 27 February 2020 13:29:28 GMT, Paul van der Linden <[email protected]> wrote: >I'm trying to scrape metrics from tasks. The current task system spins >up a >process per task, which means I can't really get these scraped by >prometheus (most of these metrics are produced at the end of the tasks >(like running time of the task), so they would disappear immediately >after >they are available. Instead I have a sidecar(kubernetes) pushgateway >running. This solves the issue partially. But when I have an update, >the >pod will be replaced, and the pushgateway would also disappear. This >means >it's difficult to see differences in these metrics as they are spread >out >and disconnected (gaps in the line). If I run a single pushgateway I >would >have metrics sticking around forever, and would have to clean these up >if >something changes. What would be the best solution for recording >metrics on >jobs/tasks (processes/pods which disappear quickly after (some of) the >metrics became available), without running into these issues? > >-- >You received this message because you are subscribed to the Google >Groups "Prometheus Users" group. >To unsubscribe from this group and stop receiving emails from it, send >an email to [email protected]. >To view this discussion on the web visit >https://groups.google.com/d/msgid/prometheus-users/001e5d66-8ea5-4138-840b-b6cbce20ef7d%40googlegroups.com. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/85FE578E-9F7E-497A-B134-93206C15E0FB%40Jahingo.com.
