No change -- for the time being, I'm still coping with a few seconds of downtime per deployment. (On the grounds that that is better than risking shard duplication between old and new clusters) That's not a major pain point while the company is small, but will eventually prove to be more of an issue.
I've been thinking a bit about what might be required for a plausible zero-downtime deployment, but still don't have a design that I'm fully happy with. (Which is part of why I haven't pursued it yet.) It really is a pretty tricky problem to address in a theoretically-correct way; I suspect the application and the deployment system (and maybe the Cluster Sharding mechanism itself) have to work together to make it at all plausible. On Sun, Dec 3, 2017 at 1:59 PM, Gytis G <[email protected]> wrote: > Bumping an old thread, Justin du coeur, can you perhaps share what > approach did you take and how did you succeed ? > > On Friday, June 10, 2016 at 9:30:27 PM UTC+3, Justin du coeur wrote: >> >> Denis' definition is about right, although I think of it slightly >> differently: so that an end user doesn't see any hiccup beyond somewhat >> longer latency than normal. I can think of various ways to achieve this, >> but all of them look like a fairly major pain in the tuchus in one way or >> another, so I'm curious about whether you have recommendations. >> >> We're using ConductR, but I'm curious about approaches both with and >> without it. Architecturally, you can think of Querki as a Play >> application, where 95% of the serious code is under sharded Actors in an >> Akka cluster, which is started under but separate from Play's built-in >> ActorSystem. (That's wildly oversimplified, but a decent thousand-foot >> block diagram description.) >> >> On Thu, Jun 9, 2016 at 4:31 PM, Viktor Klang <[email protected]> wrote: >> >>> Can we please define "zero" in this context? >>> >>> On Thu, Jun 9, 2016 at 9:05 PM, Justin du coeur <[email protected]> >>> wrote: >>> >>>> +1 to this question. For the moment I'm coping with a few seconds of >>>> downtime for releases, but we're going to have to become >>>> downtime-intolerant before long. And zero downtime does look challenging >>>> in a heavily-sharded application. >>>> >>>> Personally, I've been wondering if I should be trying to deal with >>>> intermixed releases -- bringing the new release up on one node at a time, >>>> in the *same* cluster as the old one, and gradually shutting the old ones >>>> down. That seems to make sense in theory, but also seems bloody dangerous >>>> -- it requires that the releases be 100% wire-compatible, which is hard to >>>> test and presents evolutionary challenges -- and I'm not sure if there are >>>> gotchas to be aware of... >>>> >>>> On Thu, Jun 9, 2016 at 3:44 AM, Denis Mikhaylov <[email protected]> >>>> wrote: >>>> >>>>> Hi, hakkers! >>>>> >>>>> I have a distributed app that uses `akka-persistence` and >>>>> `akka-sharding`. >>>>> For some of my shard coordinators I use `remember entities` feature. >>>>> I heard (from Bonér or Kuhn, can't say for sure) that in production >>>>> it's better to use blue/green deployments. >>>>> As I see, the steps are: >>>>> 1. Deploy and start fresh application, let it form cluster (don't >>>>> start any ShardRegions yet) >>>>> 2. Stop ShardRegions that do not use `remember entities` features >>>>> (e.g. Aggregate Roots). >>>>> 3. Start these ShardRegions on freshly deployed system >>>>> 4. Switch external traffic to new system (so that we already can >>>>> accept external commands). >>>>> 5. Stop ShardRegions that use `remember entities` features (e.g. Long >>>>> Running Processes, that react to events from Aggregate Roots). >>>>> 6. Start these ShardRegions on freshly deployed system (here I need >>>>> akka sharding to restart entities that were alive on previous system) >>>>> 7. Shut down old system >>>>> >>>>> So the questions are: >>>>> 1. Is there any improvements to the deployment process? >>>>> 2. Wouldn't this scenario corrupt Sharding related or any Akka >>>>> internal data in journal? >>>>> 3. How do you handle deployments in production? >>>>> >>>>> Thanks a lot, >>>>> Denis. >>>>> >>>>> -- >>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >>>>> urrent/additional/faq.html >>>>> >>>>>>>>>> Search the archives: https://groups.google.com/grou >>>>> p/akka-user >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Akka User List" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/akka-user. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >>>> urrent/additional/faq.html >>>> >>>>>>>>>> Search the archives: https://groups.google.com/grou >>>> p/akka-user >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Akka User List" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/akka-user. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Cheers, >>> √ >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/c >>> urrent/additional/faq.html >>> >>>>>>>>>> Search the archives: https://groups.google.com/grou >>> p/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/ > current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
