zentol commented on pull request #18566: URL: https://github.com/apache/flink/pull/18566#issuecomment-1031229601
> Should the deploying time also include SCHEDULED time? I don't think so; I'd rather have separate metrics for the scheduling (and initialization for that matter). This gives us insights into where we spend the time (e.g., scheduling strategies being slow). In the end we can think about introducing a new upTime metric that is derived from the time the job is in a RUNNING state and these new metrics, or we just leave this up to users. > the 2 period together makes a period before the job starts working This isn't quite correct since you're also missing the task initialization; although opinions may differ as to whether that should be considered as the job running or not. (Hence why I want to break down the metrics as far as possible) > What do you think of use the same definition for batch jobs, and we simply say that the deploying time ticks whenever there is an ongoing deployment? Overall I'm very torn on this topic. In some ways every approach makes sense. Even for streaming jobs, if 1/10 tasks has been deployed, then that task is consuming/producing data, so in a way the job is running, similar to the proposed semantics for batch. On the other hand, maybe a job should only be considered as running if it runs with the desired parallelism, even for batch. I really don't know what the best option is; I also thought about making it configurable. I couldn't come to a final conclusion, so I went with the proposed semantics as they are consistent with how jobs make progress (batch = data is being processed; stream = checkpoints are happening), and we can see how users react to them. But these will be marked as experimental, so we can also change things later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
