On 05.08.14 21:25, ahjohannessen wrote:
"How do you define the order? Is it based on time stamps in the persistent actors?
Is it based on some feature in the backend store?"
I do not think time stamp precision is important for this, I would imagine a
logical position / offset as EventStore/Kafka do.
Kafka maintains an offset for each partition separately and a partition
is bound to a single node (disregarding replication). For example, if a
Kafka topic is configured to have 2 partitions, each partition starts
with offset=0, and, if you consume from that topic you only obtain a
partially ordered stream because Kafka doesn't define any ordering
across partitions (see Kafka docs for details). This situation is
comparable to other distributed datastores. For example, Cassandra only
maintains an ordering for entries with the same partition key (i.e. for
entries that reside on the same node).
In general, if you want to maintain an ordering of entries, you either
have to use
- a single writer in the whole cluster (which is the case for persistent
actors) or
- keep entries (that are generated by multiple producers) on a single
node so that the server is able to maintain a local counter (which is
what Kafka does with offsets for each partition separately)
Both limits scalability (as already mentioned by Patrik) for both write
throughput and data volume. It may well be that some applications are
fine with these limitations and benefit from a total ordering of entries
per "tag" but this should not be the default in akka-persistence. IMO,
it would make sense if akka-persistence allows applications to configure
an optional ordering per "tag" so that users can decide to sacrifice
scalability if total ordering is needed for a given tag (and it is up to
journal implementations how to implement that ordering).
As already mentioned in a previous post, causal ordering could be a
later extension to akka-persistence that goes beyond the limits of a
single writer or co-located storage *and* allows for better scalability.
I wish I had more time for hacking on a prototype that tracks causalities :)
I imagine those are based on integers / longs.
I think it depends on the journal, something simple like leveldb would need
help from journal, whereas something like kafka / eventstore would probably
have something that one could adapt and get easier implemented.
--
Martin Krasser
blog: http://krasserm.blogspot.com
code: http://github.com/krasserm
twitter: http://twitter.com/mrt1nz
--
Read the docs: http://akka.io/docs/
Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to akka-user+unsubscr...@googlegroups.com.
To post to this group, send email to akka-user@googlegroups.com.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
