potiuk commented on issue #18630: URL: https://github.com/apache/airflow/issues/18630#issuecomment-931163639
It's an interesting on. Statsd client uses UDP packet to send metrics and I think lack of those metrics (which are emited **JUST** before the task completes) that indicate that the UDP packet are either not sent at all by the process running the task or not received by statsd deamon. I think most likely is the first case. Are you using Kubernetes or Celery Executor? I'd say if that's the former, the the most likely case is that the process finishes more often than not before the UDP packet is actually sent (in Celery Executor the process that sends the metric are not finished after task end, they should be (I am not 100% sure of the forking strategy there) runing all the time, but in Kubernetes executor the process might exit faster than the UDP packet is actually sent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
