Hi, there
From the Spark UI, we can monitor the following two metrics:
• Processing Time - The time to process each batch of data.
• Scheduling Delay - the time a batch waits in a queue for the
processing of previous batches to finish.
However, what is the best way to monitor them proactively? For example, if
processing time/scheduling delay exceed certain threshold, send alert to the
admin/developer?
Lan
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
