arne-alex opened a new issue, #23262: URL: https://github.com/apache/beam/issues/23262
### What would you like to happen? We'll extend Google Cloud Dataflow runners with new latency metrics. The new metrics will track time spent in various processing states in the runner. In particular, they allow to distinguish between time spent running pipeline code and time spent waiting for Google Cloud Dataflow backends. Planned work: 1. extend the Windmill protocol with new fields in KeyedGetDataRequest, defined at https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/windmill/src/main/proto/windmill.proto 2. add another state that represents waiting for further data here: https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java#L1169 3. populate the new fields from google-cloud-dataflow-runner code here: https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java#L2378 ### Issue Priority Priority: 2 ### Issue Component Component: runner-dataflow -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
