Thanks John. It sounds like you're basically describing akka-persistence,
with Kafka taking the place of the journal and the "queue" drain similar to
an akka-persistence "view". In fact, Kafka sounds like it could be plugged
into akka-persistence as an alternate journal backend ("event sourcing" is
listed as one of the use cases on their site). I wonder if anyone has
looked into this?
I'll need to determine if persisting the updates to disk, even with a fast
append-only mechanism such as LevelDB/Kafka, has a low enough latency for
my use case.
Regards,
Raman
On Wednesday, March 19, 2014 4:44:42 PM UTC-4, John McNulty wrote:
>
> You might consider persisting to a local Kafka cluster so you can ACK
> quickly and with confidence and then have a separate process draining the
> Kafka "queue" and writing to Mongo at its leisure if that works for your
> use case. Kafka is persistent and replicated so you can have high
> confidence that your ACK is good and it takes advantage of the fact that
> write-only/append-only disk access is several orders of magnitude faster
> than a disk seek to get staggeringly high throughput
>
> The one gotcha is the same one Patrick mentioned -- what if your entire
> Kafka cluster is unavailable -- then all of "receiver" nodes (taking
> requests from browsers) will be useless and all useless at the same time.
> In that case to have both the availability and the reliability you'd need
> to spool to local disk at each node during a Kafka unavailability event and
> then play back from local disk once Kafka became available again
>
> One could argue that's a lot of backups on top of backups but if you
> really truly need your ACK to mean it, that's getting closer
>
> On Tuesday, March 18, 2014 11:11:50 AM UTC-5, Raman Gupta wrote:
>>
>> Patrik -- great suggestions, thank you.
>>
>> On Tuesday, March 18, 2014 4:18:31 AM UTC-4, Patrik Nordwall wrote:
>>>
>>> Looks good. If Play 2 is unreachable you might introduce unwanted
>>> latency. You can consider replicating to more than one, or start the insert
>>> immediately and if that is quicker than then the ack from Play 2 you can
>>> respond to client immediately.
>>>
>>> Regards,
>>> Patrik
>>>
>>>
>>> On Mon, Mar 17, 2014 at 6:32 PM, Raman Gupta <[email protected]> wrote:
>>>
>>>> I am building a transactional system requiring low latency, but is
>>>> highly available and fault tolerant. I can ack the transaction
>>>> immediately,
>>>> as soon as I can guarantee that it won't be lost due to a single-node
>>>> failure. Initial thoughts around the main components are Play Framework to
>>>> receive requests, Akka (of course), and MongoDB as a data store.
>>>>
>>>> To avoid the latency cost of writing to disk, I was considering sending
>>>> the received message to a second server for temporary storage in an
>>>> actor's
>>>> memory state:
>>>>
>>>> Play 1 -> receive message from browser
>>>> Play 1 -> forward message to "journal actor" on Play 2
>>>> Play 2 -> ack receipt to Play 1
>>>> Play 1 -> ack receipt to browser (now that the message is "safe" in
>>>> memory on two servers) <-- user-perceived latency up to here
>>>> Play 1 -> insert into MongoDB
>>>> Play 1 -> upon confirmation of MongoDB insert, send "delete" to Play 2
>>>> Play 2 -> deletes message in memory
>>>>
>>>> If Play 1 fails to insert into MongoDB within the time that Play 2
>>>> expects the delete, Play 2 attempts an "upsert" into MongoDB (upsert
>>>> handles the situation in which the initial insert did work, but no delete
>>>> was received).
>>>>
>>>> Akka-persistence could even be used on Play 2 to write the data to disk
>>>> asynchronously for greater consistency -- though not perfect -- in case of
>>>> multi-node failure.
>>>>
>>>> Thoughts? Is there a better / simpler way to accomplish this?
>>>>
>>>> Regards,
>>>> Raman
>>>>
>>>> --
>>>> >>>>>>>>>> Read the docs: http://akka.io/docs/
>>>> >>>>>>>>>> Check the FAQ:
>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>> >>>>>>>>>> Search the archives:
>>>> https://groups.google.com/group/akka-user
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Akka User List" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to [email protected].
>>>> To post to this group, send email to [email protected].
>>>> Visit this group at http://groups.google.com/group/akka-user.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Patrik Nordwall
>>> Typesafe <http://typesafe.com/> - Reactive apps on the JVM
>>> Twitter: @patriknw
>>>
>>>
--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ:
>>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka
User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.
