Hi TD, Thanks for your reply.
Attachment in previous email was from Master. Below is the log message from one of the worker. ----------------------------------------------------------------------------------------------- 2014-10-01 01:49:22,348 ERROR akka.remote.EndpointWriter: AssociationError [akka.tcp://sparkWorker@*<worker4>*:7078] -> [akka.tcp://sparkExecutor@ *<worker4>*:34010]: Error [Association failed with [akka.tcp://sparkExecutor@*<worker4>*:34010]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkExecutor@*<worker4>*:34010] Caused by: akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused: *<worker4>*:34010 ] 2014-10-01 02:14:54,868 ERROR akka.remote.EndpointWriter: AssociationError [akka.tcp://sparkWorker@*<worker4>*:7078] -> [akka.tcp://sparkExecutor@ *<worker4>*:33184]: Error [Association failed with [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkExecutor@*<worker4>*:33184] Caused by: akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused: *<worker4>*:33184 ] 2014-10-01 02:14:54,878 ERROR akka.remote.EndpointWriter: AssociationError [akka.tcp://sparkWorker@*<worker4>*:7078] -> [akka.tcp://sparkExecutor@ *<worker4>*:33184]: Error [Association failed with [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkExecutor@*<worker4>*:33184] Caused by: akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused: *<worker4>*:33184 ] 2014-10-01 02:14:54,887 ERROR akka.remote.EndpointWriter: AssociationError [akka.tcp://sparkWorker@*<worker4>*:7078] -> [akka.tcp://sparkExecutor@ *<worker4>*:33184]: Error [Association failed with [akka.tcp://sparkExecutor@*<worker4>*:33184]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkExecutor@*<worker4>*:33184] Caused by: akka.remote.transport.netty.NettyTransport$$anonfun$associate$1$$anon$2: Connection refused: *<worker4>*:33184 ] ----------------------------------------------------------------------------- Your support will be highly appreciated. Regards, Riyaz On Wed, Oct 1, 2014 at 1:16 AM, Tathagata Das <[email protected]> wrote: > Is this the logs of the worker where the failure occurs? I think issues > similar to these have since been solved in later versions of Spark. > > TD > > On Tue, Sep 30, 2014 at 11:33 AM, Shaikh Riyaz <[email protected]> > wrote: > >> Dear All, >> >> We are using Spark streaming version 1.0.0 in our Cloudea Hadoop cluster >> CDH 5.1.3. >> >> Spark streaming is reading messages from Kafka using >> https://github.com/dibbhatt/kafka-spark-consumer. >> >> We have allocated 4gb memory to executor and 5gb to each workers. We have >> total 6 workers spread across 6 machines. >> >> Please find the attach log file for detailed error messages. >> >> >> Thanks in advance. >> >> -- >> Regards, >> >> Riyaz >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > -- Regards, Riyaz
