On 25.07.14 18:19, Konrad 'ktoso' Malawski wrote:
Hello there,
Ah, much better when able to communicate in full sentences without 140
chars limit! ;-)
( https://twitter.com/mrt1nz/status/492676432410447872 )
So, now that it’s spelled out as full sentences, I’ll gladly dig into
your points:
1)
Has been already/proposed and accepted/ in
https://github.com/akka/akka/issues/15004,
including your +1, so I guess you’re aware that it’s in our backlog.
The module is experimental and published “early” exactly in order to
gather,
and implement these features before stabilising the APIs.
So it’s coming, we simply have not yet gotten to implementing it -
it’s holiday season, which isn’t helping development speed :-)
2)
For the benefit of the discussion, example in EventStore:
http://geteventstore.com/blog/20130707/catch-up-subscriptions-with-the-event-store/
One thing to keep in mind here is that some Journals would have no
problem implementing this, such as Kafka or EventStore - because it’s
a built in mechanism to “subscribe to something” in them… See Martin’s
Kafka journal and how one can subscribe to a event stream there:
https://github.com/krasserm/akka-persistence-kafka#journal-topics On
the other hand implementing this in other Journals would be pretty
painful / inefficient (cassandra, hbase, …).
A general approach for supporting this in other journals would require:
- a materialized view (on journal entries) in the backend store (compare
to user-defined topics
<https://github.com/krasserm/akka-persistence-kafka#user-defined-topics>
in the Kafka journal). Updating this view (or several) can either be
done by the plugin itself or by backend store triggers, for example. In
the case of the Kafka journal, the plugin updates the materialized view
(= user-defined topic). Alternatively, user-defined views don't
necessarily need to be materialized, if a backend store can generate
them on-the-fly in a very efficient way.
- a consumer that keeps track of what has already been consumed from the
materialized view. Kafka consumers, for example, do this automatically
(the consumer position can optionally be written to Zookeeper for
fault-tolerance). PersistentViews (in Akka Persistence) also track the
consumer position on the client side. Storing that position is only
possible via snapshotting at the moment. Extending PersistentViews to
track the position in a more efficient way (instead of snapshotting)
could make sense.
IMO, materialized views together with pull-based consumers that can
write checkpoints shouldn't be a big deal for any backend store to
support. What Akka Persistence should offer in the future is an
abstraction over how custom (materialized) views can be defined and over
pull-based consumers. Furthermore, an a.p.PersistentView could then
abstract over processor journals and custom views (which would finally
cover issue #15004 <https://github.com/akka/akka/issues/15004>).
Thoughts?
We were playing around with some ideas to expose optional db specific
journal functionalities. This would be a good candidate for this.
This request seems to depend on these things by the way:
* changes in the journal plugins (we some changes there anyway
https://github.com/krasserm/akka-persistence-kafka#journal-topics ),
* views over “tags" (as this would essentially be a view over “all”),
* and lastly reactive-streams (views exposed as streams of events).
Thanks for your feedback and keep in mind that no-one said that this
module is “done”.
It’s still experimental and this kind of feature requests are exacly
what we need and will have to provide to make it stable and usable in
all kinds of apps.
Lastly, would you mind creating a ticket for the 2) feature?
Thanks in advance, have a nice weekend :-)
<https://github.com/akka/akka/issues/15004>
--
Konrad 'ktoso' Malawski
hAkker @ typesafe
http://akka.io
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google
Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
--
Martin Krasser
blog: http://krasserm.blogspot.com
code: http://github.com/krasserm
twitter: http://twitter.com/mrt1nz
--
Read the docs: http://akka.io/docs/
Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
