Hi, We are trying to understand metrics that are being populated by our samza job and are a little confused what each of these metrics mean especially since we’re running the job with a thread pool.
· We have 3 input streams · job.container.thread.pool.size=3 · 1 container per partition · Using a RocksDB backed store with changelogging · process-ns = 120,000 · get-ns ~ 30,000 · put-ns ~ 90,000 · block-ns ~ 300,000 · choose-ns ~ 500,000 Metrics are avg(metric) for all containers/partitions. Process-envelopes ~ 7000/sec. If I back the math out, this correlates quite closely to process-ns. (1/7000 ~ 0.15ms). What I don’t understand is that the event loop is single threaded. Even though, each process call is a separate thread, the main even loop is blocking (block-ns) and choosing (choose-ns) every time and these times are quite high. Given these metrics, how is it that we are consistently seeing the above metrics? Also, lag (messages behind high watermark) is ~ 0. Thanks Ankit
