Hi Patrik, I did the upgrade to the version 2.3.0 and the joinSeedNodeProcess issue is fixed. For the current setup, 2 singletons and 1 router clusters are deployed inside the same Akka system. I presume this is causing the repeated calls the joinSeedNodeProcess actor.
What do recommend for this setup? Should it be a unique cluster for every Akka system? We faced a new issue after the upgrade. The thread count on Tomcat increase due to 500 New I/O worker threads. Are you aware of any issue with the 2.3.0 that could cause this? Many thanks! Ryadh The thread dump of I/O threads: Thread 10466: (state = IN_NATIVE) - sun.nio.ch.EPollArrayWrapper.epollWait(long, int, long, int) @bci=0 (Compiled frame; information may be imprecise) - sun.nio.ch.EPollArrayWrapper.poll(long) @bci=18, line=228 (Compiled frame) - sun.nio.ch.EPollSelectorImpl.doSelect(long) @bci=28, line=81 (Compiled frame) - sun.nio.ch.SelectorImpl.lockAndDoSelect(long) @bci=37, line=87 (Compiled frame) - sun.nio.ch.SelectorImpl.select(long) @bci=30, line=98 (Compiled frame) - org.jboss.netty.channel.socket.nio.SelectorUtil.select(java.nio.channels.Selector) @bci=4, line=68 (Compiled frame) - org.jboss.netty.channel.socket.nio.AbstractNioSelector.select(java.nio.channels.Selector) @bci=1, line=409 (Compiled frame) - org.jboss.netty.channel.socket.nio.AbstractNioSelector.run() @bci=49, line=206 (Compiled frame) - org.jboss.netty.channel.socket.nio.AbstractNioWorker.run() @bci=1, line=90 (Interpreted frame) - org.jboss.netty.channel.socket.nio.NioWorker.run() @bci=1, line=178 (Interpreted frame) - org.jboss.netty.util.ThreadRenamingRunnable.run() @bci=55, line=108 (Interpreted frame) - org.jboss.netty.util.internal.DeadLockProofWorker$1.run() @bci=14, line=42 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) @bci=46, line=1110 (Interpreted frame) - java.util.concurrent.ThreadPoolExecutor$Worker.run() @bci=5, line=603 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=722 (Interpreted frame) On Wednesday, April 2, 2014 11:23:40 AM UTC+1, Patrik Nordwall wrote: > > Hi Ryadh, > > I looked at the implementation and this is bug that can happen if you call > joinSeedNodes repeatedly. I don't see a reason why you would need to do > that, but it is supported usage and should be fixed. Ticket: > https://www.assembla.com/spaces/akka/tickets/3973 > > Regards, > Patrik > > > On Sat, Mar 29, 2014 at 6:28 PM, Ryadh khsib <[email protected]<javascript:> > > wrote: > >> Hi Patrik, >> I have upgraded to the version 2.3.0 (the current version is the 2.2.0). >> The fix is going to production next week. Hopefully this will fix it and >> will stabilize the cluster setup in general. >> >> I will keep you informed ;) >> >> Thanks, >> Ryadh >> >> >> On Friday, March 28, 2014 7:55:43 PM UTC, Patrik Nordwall wrote: >> >>> Ryadh, did that solve the problem? Otherwise I would like to see a >>> ticket so that we can fix it. >>> http://doc.akka.io/docs/akka/current/project/issue-tracking.html >>> >>> /Patrik >>> >>> On Mar 21, 2014, at 18:52, √iktor Ҡlang <[email protected]> wrote: >>> >>> Always upgrade to at least the latest bugfix release for the version of >>> Akka you are using and see if that resolves your problem. >>> Did that solve it? >>> >>> >>> On Fri, Mar 21, 2014 at 6:37 PM, Ryadh khsib <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> >>>> We are getting intermittently a "[actor name [joinSeedNodeProcess] is not >>>> unique!]" exception at the akka Cluster system start up. >>>> >>>> The production setup is a akka Singleton cluster deployed on 12 tomcats >>>> across 3 physical machines (4 nodes on every machine). >>>> >>>> >>>> Do you have any idea how we could fix this? >>>> >>>> >>>> >>>> Many thanks, >>>> >>>> Ryadh >>>> >>>> >>>> >>>> Exception trace: >>>> >>>> >>>> ERROR akka.cluster.ClusterCoreSupervisor 2014-03-21 08:59:28,092 >>>> [build=null] [System-akka.actor.default-dispatcher-8] - Cluster node >>>> [akka.tcp://System@host084:2551] crashed, [actor name >>>> [joinSeedNodeProcess] is not unique!] - shutting down... >>>> >>>> [exceptionClass=akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer, >>>> exceptionMethod=reserve, exceptionID=74a5d13f, ] >>>> akka.actor.InvalidActorNameException: actor name [joinSeedNodeProcess] is >>>> not unique! >>>> at >>>> akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.reserve(ChildrenContainer.scala:130) >>>> ~[akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.dungeon.Children$class.reserveChild(Children.scala:77) >>>> ~[akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.ActorCell.reserveChild(ActorCell.scala:338) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.dungeon.Children$class.makeChild(Children.scala:186) >>>> ~[akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.dungeon.Children$class.actorOf(Children.scala:38) >>>> ~[akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.ActorCell.actorOf(ActorCell.scala:338) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at >>>> akka.cluster.ClusterCoreDaemon.joinSeedNodes(ClusterDaemon.scala:361) >>>> ~[akka-cluster_2.10-2.2.0.jar:na] >>>> at >>>> akka.cluster.ClusterCoreDaemon$$anonfun$tryingToJoin$1.applyOrElse(ClusterDaemon.scala:302) >>>> ~[akka-cluster_2.10-2.2.0.jar:na] >>>> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.ActorCell.invoke(ActorCell.scala:456) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.dispatch.Mailbox.run(Mailbox.scala:219) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at >>>> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) >>>> [scala-library-2.10.2.jar:na] >>>> at >>>> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) >>>> [scala-library-2.10.2.jar:na] >>>> at >>>> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) >>>> [scala-library-2.10.2.jar:na] >>>> at >>>> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) >>>> [scala-library-2.10.2.jar:na] >>>> >>>> ERROR akka.actor.OneForOneStrategy 2014-03-21 08:59:28,093 [build=null] >>>> [System-akka.actor.default-dispatcher-8] - actor name >>>> [joinSeedNodeProcess] is not unique! >>>> >>>> [exceptionClass=akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer, >>>> exceptionMethod=reserve, exceptionID=74a5d13f, ] >>>> akka.actor.InvalidActorNameException: actor name [joinSeedNodeProcess] is >>>> not unique! >>>> at >>>> akka.actor.dungeon.ChildrenContainer$NormalChildrenContainer.reserve(ChildrenContainer.scala:130) >>>> ~[akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.dungeon.Children$class.reserveChild(Children.scala:77) >>>> ~[akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.ActorCell.reserveChild(ActorCell.scala:338) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.dungeon.Children$class.makeChild(Children.scala:186) >>>> ~[akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.dungeon.Children$class.actorOf(Children.scala:38) >>>> ~[akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.ActorCell.actorOf(ActorCell.scala:338) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at >>>> akka.cluster.ClusterCoreDaemon.joinSeedNodes(ClusterDaemon.scala:361) >>>> ~[akka-cluster_2.10-2.2.0.jar:na] >>>> at >>>> akka.cluster.ClusterCoreDaemon$$anonfun$tryingToJoin$1.applyOrElse(ClusterDaemon.scala:302) >>>> ~[akka-cluster_2.10-2.2.0.jar:na] >>>> at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.actor.ActorCell.invoke(ActorCell.scala:456) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at akka.dispatch.Mailbox.run(Mailbox.scala:219) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at >>>> akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) >>>> [akka-actor_2.10-2.2.0.jar:2.2.0] >>>> at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) >>>> [scala-library-2.10.2.jar:na] >>>> at >>>> scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) >>>> [scala-library-2.10.2.jar:na] >>>> at >>>> scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) >>>> [scala-library-2.10.2.jar:na] >>>> at >>>> scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) >>>> [scala-library-2.10.2.jar:na] >>>> >>>> -- >>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>>> current/additional/faq.html >>>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>>> group/akka-user >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Akka User List" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> >>>> Visit this group at http://groups.google.com/group/akka-user. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Cheers, >>> √ >>> >>> * ——————— **Viktor Klang* >>> *Chief Architect - **Typesafe <http://www.typesafe.com/>* >>> >>> Twitter: @viktorklang >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>> current/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>> group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected]<javascript:> >> . >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > Patrik Nordwall > Typesafe <http://typesafe.com/> - Reactive apps on the JVM > Twitter: @patriknw > JOIN US. REGISTER TODAY! <http://www.scaladays.org/> > Scala <http://www.scaladays.org/> > Days <http://www.scaladays.org/> > June 16th-18th, <http://www.scaladays.org/> > Berlin <http://www.scaladays.org/> > > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
