Hi Roland,
a few clarifications/additions inline ...
On 31.01.16 10:17, Roland Kuhn wrote:
Hi Paul,
unfortunately it is impossible to make this “just work”—not least
because you would first have to define what that means. Volker
mentioned Eventuate as a possible solution, but this also is not
something that “just works”, it requires your events to be structured
such that your defined state update functions have all the right
properties to make it work.
The features and the guarantees provided by Eventuate actually do *not*
depend on the structure of events. Eventuate provides means to
distinguish causally related from concurrent events and it is just the
responsibility of the application to ensure that derived state does not
depend on the order of concurrent events. Causally related events are
always delivered in the same causal order at all locations (datacenters,
for example). Relaxing strict order to causal order is what gives you
availability and partition-tolerance in multi-datacenter setups.
Imagine there being two copies of yourself running around and doing
things: it would not be enough for one to tell the other what it has
done, there can be real conflicts that arise from these independent
actions (like one of your selves telling your wife that you love her
and shortly thereafter—without having caught up to that point yet—the
other files for divorce). The key here is coordination, without that
certain actions cannot be taken. And coordination can be impossible,
e.g. due to network partitions, which means that you’ll have to decide
whether to be cautious or reckless.
Coordination is just an option. If you want to *prevent* concflicts (you
called it being "cautious"), only then you need coordination. Here you
give up availability in favor of consistency. The other option is to
*allow* conflicts, and resolve them later. This does not require
coordination but rather means to track, detect and resolve conflicts.
With this option, replicas at different datacenters remain writeable,
even if they are partitioned from others (you called that being
"reckless"). However, you "apologize" for being "reckless" with conflict
resolution :-) A great introductory read on this is Pat Helland's paper
Building on Quicksand
<http://db.cs.berkeley.edu/cs286/papers/quicksand-cidr2009.pdf>. With
that second option, you choose availability over (strong) consistency.
The second option is not only relevant for multi-datacenter replication
but also more generally for collaboration between (micro)services where
you usually don't want to couple the availability of one service to that
of others. The price for that is that applications must be able to deal
with conflicts, rather than trying to prevent them. A consistent
approach for that is often missing in distributed applications and often
solved with error-prone ad-hoc solutions. Eventuate tries to change that
by not only
providing APIs to track, detect, and resolve conflicts in an automated
or interactive way but also by providing an infrastructure where
distributed services can communicate via events in a causally consistent
and reliable way. The underlying event logs give you the following
guarantees:
- the order of events in a local event log is consistent with causal
order i.e. consumers will never see an effect before its cause.
- event replication across different locations is reliable and
idempotent i.e. consumers will never see duplicates when reading from a
local log.
Compare this to plain at-least-once based messaging between persistent
actors or services (in different datacenters, for example) where you
cannot make assumptions on message ordering and duplicates. It usually
makes writing correct business logic much harder. Furthermore, if you
want to decouple a service from the availability of others you are again
faced with the problem of detecting and resolving conflicts. I should
mention here that rejecting commands != being available :-). As soon as
distributed services/applications shall become more resilient to network
partitions, you'll anyway have to deal with many of the issues that
Eventuate is already adressing.
Eventuate meanwhile emerged from a proof-of-concept in early 2015 to a
production ready toolkit for building (globally) distributed,
service-oriented CQRS/ES applications. As Eventuate is also almost a
functional superset of akka-persistence, I wonder if combining efforts
would make sense. Please let me know if this sounds interesting to you.
Regards,
Martin
So, you can use Akka Persistence with the same store in different
locations, but you’ll have to make sure that you don’t emit events to
the same log from different places—there can only be one running
source of truth for each persistenceId at any given time.
Regards,
Roland
22 jan 2016 kl. 14:52 skrev Paul Cleary <[email protected]
<mailto:[email protected]>>:
Will Akka Persistence work if you have two different clusters
pointing to the same data store?
Imagine I have 2 data centers that point at the same database.
If I have updates happening in both data centers at the same time,
will akka persistence stomp all over the journal / snapshots?
I know that akka persistence has sequence numbers, and I am not sure
how those are managed.
It would be great if this just worked, but I am thinking I need to
implement my own persistence plugin and / or persistence layer in my
app to make sure that there are no collisions.
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google
Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
*Dr. Roland Kuhn*
/Akka Tech Lead/
Typesafe <http://typesafe.com/> – Reactive apps on the JVM.
twitter: @rolandkuhn
<http://twitter.com/#%21/rolandkuhn>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google
Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
--
Martin Krasser
blog: http://krasserm.github.io
code: http://github.com/krasserm
twitter: http://twitter.com/mrt1nz
--
Read the docs: http://akka.io/docs/
Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
