Hi Team, We had created a simulator (akka based) that is sending 400-500 msg/sec. Same is consumed by subscriber using Spark Streaming and write to HDFS
*Environment details:* 1) Akka: 2.2.3 2) Spark: 1.0.0-cdh5.1.3 3) Scala: 2.10.4 *Cloudera Details 5.1.3:* 1) Master Node: INHYDBILXHDP01 2) Worker node: INHYDBILXHDP02 All nodes are free from firewall and have access to each-other *Prog runs well when we run it with below command :* spark-submit --class com.deere.springxd.domaineventstream.source.StartApp --master *local[2] *domain-event-stream-source-cdh5.1.3-Ankur.jar *Same Prog fails when we run it with below command* spark-submit --class com.deere.springxd.domaineventstream.source.StartApp --master *spark://INHYDBILXHDP01:7077* domain-event-stream-source-cdh5.1.3-Ankur.jar *Errors * 1. *Cloudera worker log file* AssociationError [akka.tcp://sparkWorker@INHYDBILXHDP02:7078] -> [akka.tcp://sparkExecutor@INHYDBILXHDP02:55625]: Error [Association failed with [akka.tcp://sparkExecutor@INHYDBILXHDP02:55625]] [ akka.remote.EndpointAssociationException: Association failed with [akka.tcp://sparkExecutor@INHYDBILXHDP02:55625] 2. *Spark Worker log *java.lang.ClassNotFoundException: com.deere.fieldstreams.commons.serviceapi.EquipmentEvent at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 3. *Simulator:* [WARN] [10/01/2014 13:54:21.243] [EventDispatcherSimulator-akka.remote.default-remote-dispatcher-7] [akka.tcp://[email protected] ash.com:2552/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2Fspark%40INHYDBILXHDP02%3A46504-0] Association with remote system [akka.tcp ://spark@INHYDBILXHDP02:46504] has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. *Error Logs: (Attached)* Please guide us as to what needs to be done to get this fixed. Also if anything else is required than plz do let me know. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
spark-worker.log
Description: Binary data
cloudera worker.log
Description: Binary data
simulator.log
Description: Binary data
