On Mon, Feb 1, 2016 at 10:10 AM, 'Martin Krasser' via Akka User List < [email protected]> wrote:
> > > On 01.02.16 08:44, Roland Kuhn wrote: > > > That's definitely not the case. For tracking potential causality O(n) data > size is required where each participant has an entry in a vector clock > <http://rbmhtechnology.github.io/eventuate/architecture.html#vector-clocks>. > This can be further reduced by using plausible clocks > <https://github.com/RBMHTechnology/eventuate/issues/68> where > participants that share an event log at a given location may also share a > clock entry. See #68 > <https://github.com/RBMHTechnology/eventuate/issues/68> and #103 > <https://github.com/RBMHTechnology/eventuate/issues/103> for details. > > > Hmm, I’ll need to swap all those details back in, but I’m pretty sure that > when discussing the difference between full causality and E-order Mark > Miller came to the conclusion that there are cases that require a vector > matrix to be tracked correctly (reading #68 I think we discussed that > approach and concluded that it collapses too much of the history, i.e. it > leads to pessimistically assuming a lot more causality than there is; plus > having a central counter at each location limits scalability of what a > location is). > > > Saying plausible clocks "collapse too much of history" is somewhat > meaningless when not putting it into context of requirements. For example, > in 90% of our requirements, they give us exactly what we need, for the > remaining ones we let persistent actors opt-in to their own clock entry (as > explained in #103) to obtain what we need. So in this context, "collapse > too much of history" is not applicable. Let's rather first identify *what* > should be supported and question if exotic use cases should be supported at > all. > > Intuitively that also makes sense: the common meaning of the word implies > that we have to keep track of who knows what when a message is sent, and > “what” is a version vector while “who” takes that one dimension up. And > that information needs to be sent with every message. > > > The "what" is just local time at each unit of concurrency, and the "who" > is the identifier of a concurrency unit, so you have all you need in a > single vector timestamp. > > > There may be ways to restrict the notion of causality itself or the kind > of actions that can be performed in order to reduce this overhead to a > single version vector, but even that would need to be sent with every > message and therefore over the course of the conversation it would consume > quadratic network bandwidth. > > > This is rather a problem of all-to-all replication, not one of a causality > tracking mechanism. > > Besides that, I'm also thinking about supporting explicit causality > tracking in Eventuate i.e. let the application define the causal > relationships rather than tracking them implicitly. Both approaches could > even co-exist, if needed. > > > E-order on the other hand has a constant overhead per message send but > restricts the notion of causality to only apply to what is effected by a > single source. > > > > - Having causality alone does not allow your agents to be programmed > such that they can keep even simple invariants like “do not allow the > creation of more than 500 blog entries”—if two concurrent histories both > create the 500th post then conflict resolution becomes a hassle. > > > Right, global invariants require coordination which limits availability. > However, this is not in conflict with an AP-by-default approach as > coordination can be added on top where needed. > > > This is the question that needs answering—or that will spur further > development: are consistency and availability really that opposed in an > absolute sense? > > > You just have to make a decision under failure conditions. I'm not saying > that AP is the better default choice, only what Eventuate is doing. > > And if yes, does it make sense to pick a default, or are these two > completely segregated problem spaces? > > > > The latter point is what users of this abstraction need to consider when > coming from a strongly consistent RDBMS background, it fundamentally > changes the world view. The apologies cannot in all cases be contained > within the system, business processes involving human personnel may need to > be created to deal with the fallout—this is of course very desirable > because otherwise these processes would need to be improvised when the hair > is on fire (a.k.a. the network is split) but it should be mentioned both as > a cost and a benefit when introducing such a system. > > > When it comes to interactions spanning multiple microservices I tend to > favor the Saga approach: creating an external entity that manages the > causal relationship between the changes effected at different services and > the failure handling (i.e. apologies) allows the compression of the > required data for causality tracking—because it is no longer generic but > tailored to the use-case—as well as the colocation of action and > compensating action in a single unit. As a bonus these units are highly > approachable for non-programmers as well because they directly correspond > to business processes. > > > I don't want to enumerate over all the advantages and disadvantages of > central coordination vs. federated collaboration here but central > coordination may be in conflict with availability requirements. Also, > central coordination is often the entry point to monolithic solutions :-) > > > I don’t consider the use of sagas as being related to central > coordination, a saga is nothing but an agent just like everything else in > the system, it does not possess special powers or represent a central > authority—all data are owned by some component and that ownership must > always be local where consistency is demanded. This is the point of «Life > Beyond Distributed Transactions», and it is the same point Garcia-Molina & > Salem make: split up the responsibility so that you can distribute. > Foregoing local consistency may be necessary in specific cases, but it > makes for a so much harder programming model that I remain unconvinced that > it should be recommended often, let alone be the default. > > One thing that strikes me just now is that the user of Eventuate currently > has the responsibility of constructing a correct semi-lattice for their > data type—why not use CRDTs to make this safer? > > > For that purpose, Eventuate also provides CRDTs. But there are situations > where custom domain logic makes it easier to construct a correct > semi-lattice compared to implementing that functionality on top of generic > CRDTs. We are practicing this with a globally distributed workflow > management system where business logic is needed to decide how to handle > concurrent (and potentially conflicting) job lifecycle events, to mention > only one example. > Is the causality tracking (the vector clocks) used for anything else than supporting "interactive conflict resolution"? Are they needed when using CRDTs? Based on my (incomplete) understanding of Riak's "failed experiments" of exposing vector clocks and letting application/user decide what version to continue with I'm skeptical to that approach. I think the conflict resolution must be automatic, i.e. CRDTs, and then I wonder what the vector clocks adds? /Patrik > > > > > > Compare this to plain at-least-once based messaging between persistent > actors or services (in different datacenters, for example) where you cannot > make assumptions on message ordering and duplicates. It usually makes > writing correct business logic much harder. Furthermore, if you want to > decouple a service from the availability of others you are again faced with > the problem of detecting and resolving conflicts. I should mention here > that rejecting commands != being available :-). As soon as distributed > services/applications shall become more resilient to network partitions, > you'll anyway have to deal with many of the issues that Eventuate is > already adressing. > > Eventuate meanwhile emerged from a proof-of-concept in early 2015 to a > production ready toolkit for building (globally) distributed, > service-oriented CQRS/ES applications. As Eventuate is also almost a > functional superset of akka-persistence, I wonder if combining efforts > would make sense. Please let me know if this sounds interesting to you. > > > I am very much interested in attacking this problem, we need to give users > the tools that reduce their responsibility to the essence of their business > problem. I must confess that I have not yet studied the Eventuate source > code, all statements above are based on my incomplete understanding of the > problem domain and some very interesting conversations at ECOOP last year. > On this basis my current impression is that while we hold some promising > pieces in our hands we have not yet found the golden hammer—if it exists at > all. Perhaps it is time to plan a get-together to assemble the pieces we > know now and explore the landscape that emerges? > > > Good idea. Let's start that initiative. > > > Okay, so how do we go about this? > > > I'll also try to contact you this week regarding that ... > > > I’m in central Europe (as I believe you are) and would propose a small > meeting of people who have an interest in implementing such a system. Who > else should participate? Who on this list would want to come? > > Regards, > > Roland > > > > Regards, > > Roland > > Regards, > Martin > > > So, you can use Akka Persistence with the same store in different > locations, but you’ll have to make sure that you don’t emit events to the > same log from different places—there can only be one running source of > truth for each persistenceId at any given time. > > > Regards, > > Roland > > 22 jan 2016 kl. 14:52 skrev Paul Cleary < <[email protected]> > [email protected]>: > > Will Akka Persistence work if you have two different clusters pointing to > the same data store? > > Imagine I have 2 data centers that point at the same database. > > If I have updates happening in both data centers at the same time, will > akka persistence stomp all over the journal / snapshots? > > I know that akka persistence has sequence numbers, and I am not sure how > those are managed. > > It would be great if this just worked, but I am thinking I need to > implement my own persistence plugin and / or persistence layer in my app to > make sure that there are no collisions. > > -- > >>>>>>>>>> Read the docs: <http://akka.io/docs/>http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > <http://doc.akka.io/docs/akka/current/additional/faq.html> > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: > <https://groups.google.com/group/akka-user> > https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to <[email protected]> > [email protected]. > To post to this group, send email to <[email protected]> > [email protected]. > Visit this group at <https://groups.google.com/group/akka-user> > https://groups.google.com/group/akka-user. > For more options, visit <https://groups.google.com/d/optout> > https://groups.google.com/d/optout. > > > > > *Dr. Roland Kuhn* > *Akka Tech Lead* > Typesafe <http://typesafe.com/> – Reactive apps on the JVM. > twitter: @rolandkuhn > <http://twitter.com/#%21/rolandkuhn> > > -- > >>>>>>>>>> Read the docs: <http://akka.io/docs/>http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > <http://doc.akka.io/docs/akka/current/additional/faq.html> > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: > <https://groups.google.com/group/akka-user> > https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to <[email protected]> > [email protected]. > To post to this group, send email to <[email protected]> > [email protected]. > Visit this group at <https://groups.google.com/group/akka-user> > https://groups.google.com/group/akka-user. > For more options, visit <https://groups.google.com/d/optout> > https://groups.google.com/d/optout. > > > -- > Martin Krasser > > blog: http://krasserm.github.io > code: http://github.com/krasserm > twitter: http://twitter.com/mrt1nz > > > -- > >>>>>>>>>> Read the docs: <http://akka.io/docs/>http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > <http://doc.akka.io/docs/akka/current/additional/faq.html> > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: > <https://groups.google.com/group/akka-user> > https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to <[email protected]> > [email protected]. > To post to this group, send email to <[email protected]> > [email protected]. > Visit this group at <https://groups.google.com/group/akka-user> > https://groups.google.com/group/akka-user. > For more options, visit <https://groups.google.com/d/optout> > https://groups.google.com/d/optout. > > > > > *Dr. Roland Kuhn* > *Akka Tech Lead* > Typesafe <http://typesafe.com/> – Reactive apps on the JVM. > twitter: @rolandkuhn > <http://twitter.com/#%21/rolandkuhn> > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to <[email protected]> > [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > > -- > Martin Krasser > > blog: http://krasserm.github.io > code: http://github.com/krasserm > twitter: http://twitter.com/mrt1nz > > > -- > >>>>>>>>>> Read the docs: <http://akka.io/docs/>http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > <http://doc.akka.io/docs/akka/current/additional/faq.html> > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to <[email protected]> > [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > > > > > *Dr. Roland Kuhn* > *Akka Tech Lead* > Typesafe <http://typesafe.com/> – Reactive apps on the JVM. > twitter: @rolandkuhn > <http://twitter.com/#%21/rolandkuhn> > > -- > >>>>>>>>>> Read the docs: <http://akka.io/docs/>http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > <http://doc.akka.io/docs/akka/current/additional/faq.html> > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: > <https://groups.google.com/group/akka-user> > https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > > > -- > Martin Krasser > > blog: http://krasserm.github.io > code: http://github.com/krasserm > twitter: http://twitter.com/mrt1nz > > -- > >>>>>>>>>> Read the docs: http://akka.io/docs/ > >>>>>>>>>> Check the FAQ: > http://doc.akka.io/docs/akka/current/additional/faq.html > >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user > --- > You received this message because you are subscribed to the Google Groups > "Akka User List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/akka-user. > For more options, visit https://groups.google.com/d/optout. > -- Patrik Nordwall Typesafe <http://typesafe.com/> - Reactive apps on the JVM Twitter: @patriknw -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
