Re: Effective ways monitor and identify that a Streaming job has been failing for the last 5 minutes

Any documentation/sample code on how to use Ganglia with Spark? On Sat, Dec 5, 2015 at 10:29 PM, manasdebashiskar wrote: > spark has capability to report to ganglia, graphite or jmx. > If none of that works for you you can register your own spark extra > listener > that

Re: Effective ways monitor and identify that a Streaming job has been failing for the last 5 minutes

spark has capability to report to ganglia, graphite or jmx. If none of that works for you you can register your own spark extra listener that does your bidding. ..Manas -- View this message in context:

Effective ways monitor and identify that a Streaming job has been failing for the last 5 minutes

Hi, We need to monitor and identify if the Streaming job has been failing for the last 5 minutes and restart the job accordingly. In most cases our Spark Streaming with Kafka direct fails with leader lost errors. Or offsets not found errors for that partition. What is the most effective way to