Re: [akka-user] master/master or disconnected mode example ?

Martin Krasser Tue, 11 Feb 2014 22:52:07 -0800

Hi Jospeh,

On 11.02.14 23:48, Joseph Pachod wrote:

Hi Oleg
Thanks for your answer :)
Well, I worked on some homegrown EDA framework which had support formaster/master and events replaying, so I guess I was starting with toomuch expectation.
Anyway, regarding akka-persistence, there's no support out of the boxfor master/master.

I'm still missing an answer to Rolands question: what *exactly* do youwant to achieve?. Anyway, here are a few thoughts.


1.) master/master

Using a replicated journal<https://gist.github.com/krasserm/8612920#file-akka-persistence-plugins-md>,processors with different processor ids can of course write to differentnodes. Not all writes have to go through the same node, this restrictiononly applies to processors with the same processor id. So it reallydepends on what you mean by master/master support.

When updating processors with the same processor id on different nodes,you have to answer at least these questions:

- does it matter if processors with the same processor id on differentnodes see updates in different order. If yes, you'll need coordinationamong these processors (= expensive)- what data structures for processor state shall I use if my processorsmay receive updates in any order? You may want to look at CRDTs andrelated work <https://github.com/jboner/akka-crdt> done by Jonas.

Addressing these questions goes beyond the intended scope ofakka-persistence. It was intended for fast and scalable single actorstate persistence (of course, with migration/fail-over support in acluster). If you want to maintain read-only replicas of a processor youshould use views<http://doc.akka.io/docs/akka/snapshot/scala/persistence.html#views>.

Keeping state consistent across (writable) processors is anapplication-level concern. You can implement that yourself but Irecommend that you take a look at the numerous replicated NoSQL and SQLdatabases out there as an alternative.


2.) Offline features

You probably want to use a replicated journal that is able to have someof its nodes temporarily separated from others and still support writes(e.g. look at CouchDB's offline features, to mention only one example).If those journal nodes are re-connected they would synchronize loggedmessages (captured offline) with the other journal nodes. Afterre-connecting, remote views would be automatically updated then.


Hope that helps,

Cheers,
Martin

If a local instance was to create new events and then, way later,pushed them to some other instance, I would have to handle all thepersistence of this already existing event myself, and it's a hardtopic I feel. Shall I keep track of the original instance in the eventstored ? Is it actually the same event or something else... ? Plentyof questions : to have them already tackled, if only partly, would beof a great help :$ It's almost to the point where having to do it allfrom scratch make me wonder whether akka would be right the right toolfor the job. There may be easier ways...


Thanks again !
joseph

On Tue, Feb 11, 2014 at 12:27 AM, Oleg Zhurakousky<[email protected] <mailto:[email protected]>> wrote:


    I don't think there is anything to support. May be Akka-eng will
    chime in, but IMHO Akka is not a product but a library and a
    framework which allows you to build products. Git is a product and
    in theory you can easily emulate its architecture using Akka.
    Also, what Akka does provide is akka-persistence module which
    essentially allows you to preserve/persist the state of the
    mailbox, so it could be replayed (restored) for messages that
    didn't get a chance to process before system went down. So you may
    want to look at it and see if this is something that fits your use
    case.

    Cheers
    Oleg


    On Mon, Feb 10, 2014 at 6:00 PM, Joseph Pachod
    <[email protected] <mailto:[email protected]>> wrote:

        Hi Oleg

        That's part of it indeed. However, I haven't seen anything
        related to replaying history with the new events in akka so
        far, so I was planning to only push events which don't need
        replaying.  Actually I haven't seen something similar in Akka
        at all to be honest. This kind of "disconnected/extra master"
        doesn't seem to be supported by akka AFAIK. Am I right ?

        Best,


        On Mon, Feb 10, 2014 at 2:11 PM, Oleg Zhurakousky
        <[email protected]
        <mailto:[email protected]>> wrote:

            I think what you are talking about is classical event
            sourcing or a variation of it. Each new piece of data is a
            new event persisted somewhere (e.g., file) on the local
            system. Then when remote system comes online the event
            source stored up until this point would have to be
            replayed for the remote system, thus preserving the
            over-time state of the event stream allowing access to
            both current state as well as point-in-time.
            Is that what you trying to accomplish or your pieces of
            data are completely independent from one another (no
            historically significant timeline)?

            Cheers
            Oleg


            On Mon, Feb 10, 2014 at 4:54 AM, joseph
            <[email protected] <mailto:[email protected]>>
            wrote:

                Hi Roland

                Thanks a lot for your answer.

                The end target would be to have the data on the local
                system and allowing creation of new data while
                offline. Then, on being online again, the diff would
                be pushed and resolved. I had played with
                akka-persistence regarding this application and I was
                wondering how I would implement such an use case with it.

                Thanks again

                best




                On Monday, February 10, 2014 8:03:32 AM UTC+1, rkuhn
                wrote:

                    Hi Joseph,

                    what exactly do you want to achieve? I’m asking
                    because Akka itself does not really have any
                    end-user features, it is only a library for
                    building your own functionality. The precise
                    meaning of master/master or “disconnected mode”
                    depends very much on what you are building with
                    Akka, not on Akka itself.

                    Regards,

                    Roland

                    9 feb 2014 kl. 21:57 skrev joseph
                    <[email protected]>:

                    Hi

                    While working on the train the other day I
                    realized how much I missed git disconnected
                    capacities.

                    Ever since, this has stuck into my mind and I'm
                    thinking about how to implement such mechanism.

                    I looked around regarding akka but apparently
                    master/master and disconnected stuff aren't too
                    much spoken of. Did I miss something obvious ?
                    Are they some best practices around or, even
                    better, examples ?

                    Thanks in advance
                    joseph