It seems that we found a way around the problem. It is a normal behaviour to serialize LocalActorRef without address, however to Cluster Sharding to work, we do need to persist messages with RemoteActorRef. If you serialize PersistentRepr, MessageSerializer will serialize with RemoteActorRef. However we were serializing payload separately and reconstructing PersistentRepr on recovery in our custom journal. Changing the journal to serialize whole PersistentRepr instead solved the issue. I hope someone will find this useful.
On Monday, 26 January 2015 09:58:19 UTC, Patrik Nordwall wrote: > > > > On Mon, Jan 26, 2015 at 10:57 AM, Saparbek Smailov <[email protected] > <javascript:>> wrote: > >> Hello Patrik, >> >> Just for your information, one of my friends asked the same question: >> https://github.com/akka/akka/pull/2117# (Akrome) >> >> Thank you for your time and consideration! >> > > Thanks. I will take a look at the debug log. > /Patrik > > >> >> >> >> >> On Monday, January 26, 2015 at 9:54:33 AM UTC, Saparbek Smailov wrote: >>> >>> 2.3.4 >>> >>> On Monday, January 26, 2015 at 9:36:17 AM UTC, Patrik Nordwall wrote: >>>> >>>> What version of Akka do you use? >>>> /Patrik >>>> >>>> On Mon, Jan 26, 2015 at 10:23 AM, Saparbek Smailov < >>>> [email protected]> wrote: >>>> >>>>> Hello everyone, >>>>> >>>>> In our project we use Cluster Sharding with Cluster Singletons. Our >>>>> Cluster is set up on AWS Elastic Beanstalk. >>>>> The set up works perfectly until you start removing instances from the >>>>> cluster. The issue arises when a shard is recreated using Akka >>>>> persistence: during the recovery we get an error saying "Shard already >>>>> allocated". >>>>> >>>>> I went through the logs and realised that some of >>>>> ShardRegionTerminated events are persisted using LocalRef. >>>>> Let me elaborate on this a little bit: >>>>> - During the recovery I get ShardRegionRegistered event with >>>>> RemoteActorRef. >>>>> - Then I get ShardHomeAllocated event which allocates shard on the >>>>> region >>>>> - I get ShardRegionTerminated with LocalActorRef, which does not >>>>> remove the Shard and Region from Maps, as RemoteActorRef is not equal to >>>>> LocalActorRef >>>>> - Then I get ShardRegionRegistered with Ref haveing different IP >>>>> address >>>>> - Finally when next ShardHomeAllocated is replayed I get "Shard >>>>> already allocated" error. >>>>> >>>>> How can I avoid the problem? Is there any way to force Akka use only >>>>> RemoteActorRef or at least serialize using RemoteActorRef? Am I not using >>>>> the library properly? >>>>> I would appreciate any help/hint on this issue. >>>>> >>>>> Thank you very much! >>>>> >>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ >>>>> current/additional/faq.html >>>>> >>>>>>>>>> Search the archives: https://groups.google.com/ >>>>> group/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at http://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Patrik Nordwall >>>> Typesafe <http://typesafe.com/> - Reactive apps on the JVM >>>> Twitter: @patriknw >>>> >>>> -- >> >>>>>>>>>> Read the docs: http://akka.io/docs/ >> >>>>>>>>>> Check the FAQ: >> http://doc.akka.io/docs/akka/current/additional/faq.html >> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user >> --- >> You received this message because you are subscribed to the Google Groups >> "Akka User List" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at http://groups.google.com/group/akka-user. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > Patrik Nordwall > Typesafe <http://typesafe.com/> - Reactive apps on the JVM > Twitter: @patriknw > > -- Notice: This email is confidential and may contain copyright material of members of the Ocado Group. Opinions and views expressed in this message may not necessarily reflect the opinions and views of the members of the Ocado Group. If you are not the intended recipient, please notify us immediately and delete all copies of this message. Please note that it is your responsibility to scan this message for viruses. Fetch and Sizzle are trading names of Speciality Stores Limited, a member of the Ocado Group. References to the “Ocado Group” are to Ocado Group plc (registered in England and Wales with number 7098618) and its subsidiary undertakings (as that expression is defined in the Companies Act 2006) from time to time. The registered office of Ocado Group plc is Titan Court, 3 Bishops Square, Hatfield Business Park, Hatfield, Herts. AL10 9NE. -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
