You might consider persisting to a local Kafka cluster so you can ACK quickly and with confidence and then have a separate process draining the Kafka "queue" and writing to Mongo at its leisure if that works for your use case. Kafka is persistent and replicated so you can have high confidence that your ACK is good and it takes advantage of the fact that write-only/append-only disk access is several orders of magnitude faster than a disk seek to get staggeringly high throughput
The one gotcha is the same one Patrick mentioned -- what if your entire Kafka cluster is unavailable -- then all of "receiver" nodes (taking requests from browsers) will be useless and all useless at the same time. In that case to have both the availability and the reliability you'd need to spool to local disk at each node during a Kafka unavailability event and then play back from local disk once Kafka became available again One could argue that's a lot of backups on top of backups but if you really truly need your ACK to mean it, that's getting closer On Tuesday, March 18, 2014 11:11:50 AM UTC-5, Raman Gupta wrote: > > Patrik -- great suggestions, thank you. > > On Tuesday, March 18, 2014 4:18:31 AM UTC-4, Patrik Nordwall wrote: >> >> Looks good. If Play 2 is unreachable you might introduce unwanted >> latency. You can consider replicating to more than one, or start the insert >> immediately and if that is quicker than then the ack from Play 2 you can >> respond to client immediately. >> >> Regards, >> Patrik >> >> >> On Mon, Mar 17, 2014 at 6:32 PM, Raman Gupta <[email protected]> wrote: >> >>> I am building a transactional system requiring low latency, but is >>> highly available and fault tolerant. I can ack the transaction immediately, >>> as soon as I can guarantee that it won't be lost due to a single-node >>> failure. Initial thoughts around the main components are Play Framework to >>> receive requests, Akka (of course), and MongoDB as a data store. >>> >>> To avoid the latency cost of writing to disk, I was considering sending >>> the received message to a second server for temporary storage in an actor's >>> memory state: >>> >>> Play 1 -> receive message from browser >>> Play 1 -> forward message to "journal actor" on Play 2 >>> Play 2 -> ack receipt to Play 1 >>> Play 1 -> ack receipt to browser (now that the message is "safe" in >>> memory on two servers) <-- user-perceived latency up to here >>> Play 1 -> insert into MongoDB >>> Play 1 -> upon confirmation of MongoDB insert, send "delete" to Play 2 >>> Play 2 -> deletes message in memory >>> >>> If Play 1 fails to insert into MongoDB within the time that Play 2 >>> expects the delete, Play 2 attempts an "upsert" into MongoDB (upsert >>> handles the situation in which the initial insert did work, but no delete >>> was received). >>> >>> Akka-persistence could even be used on Play 2 to write the data to disk >>> asynchronously for greater consistency -- though not perfect -- in case of >>> multi-node failure. >>> >>> Thoughts? Is there a better / simpler way to accomplish this? >>> >>> Regards, >>> Raman >>> >>> -- >>> >>>>>>>>>> Read the docs: http://akka.io/docs/ >>> >>>>>>>>>> Check the FAQ: >>> http://doc.akka.io/docs/akka/current/additional/faq.html >>> >>>>>>>>>> Search the archives: >>> https://groups.google.com/group/akka-user >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Akka User List" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/akka-user. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> >> Patrik Nordwall >> Typesafe <http://typesafe.com/> - Reactive apps on the JVM >> Twitter: @patriknw >> >> -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
