Perhaps receiver needs to be failure-resistant itself. Spark does not know what your application requirements are and what should be done in case of network stream error. I would implement akka actor which switches into connection retry mode upon network error and switches back into receiving mode when connection restored.
Vadim. On Fri, Sep 20, 2013 at 9:07 AM, Russell Cardullo <[email protected] > wrote: > Hello All, > > When running Spark streaming 24/7, we will occasionally get errors in our > receivers which causes the Spark driver program to de-register that > receiver, yet continue running. > > For instance, in the example 'spark.streaming.examples.NetworkWordCount' > program, if the source for the network receiver is temporarily unavailable > the driver will de-register the receiver but the driver will continue to > run indefinitely. If the source later becomes available again the receiver > will not re-register. > > Is there some easy means to catch this and re-register the receiver if it > becomes available later? > > Alternatively is there some way to catch the case where the receiver is > lost and call stop() on the streaming context? > > Cheers, > --Russell -- >From RFC 2631: In ASN.1, EXPLICIT tagging is implicit unless IMPLICIT is explicitly specified
