I read through the file store example. If I understand correctly, in the write method we should take the request and apply it right there in the method into the state machine? So it only gets persisted into the state machine and not the raft log?
How will syncing up and backups work then? If there is no 'raft log' anymore since we are overriding the write method to only apply writes to the state machine. How will the other servers be able to sync and catch up? ________________________________ From: Tsz Wo Sze <[email protected]> Sent: Tuesday, December 14, 2021 8:16:23 PM To: [email protected] <[email protected]> Subject: Re: Raft log persistance Hi Asad, By default, all the write requests (including the data inside) are recorded in the raft log. This is specified in the standard RAFT algorithm so that any server can replay the log in order to sync up to the current state. If the state machine keeps another persistent copy of the data, you are right that there will be two copies. In this sense, the standard RAFT algorithm is not suitable for data intensive applications. In Ratis, we want to support data intensive applications such as Apache Ozone. State machine implementations may choose to manage the data itself. In that case, the data won't be recorded in the raft log. In order to use this feature, state machine implementations must override the read and write methods defined in StateMachine.DataApi. The FileStore example indeed has overridden these two methods. Hope it helps. Tsz-Wo On Wed, Dec 15, 2021 at 3:40 AM Asad Awadia <[email protected]<mailto:[email protected]>> wrote: Hello, Is ratis keeping a copy of all the writes made? On top of what is persisted in my own DB? So ~2x storage used I read that the file system example avoids that but I don't see how that is being done in the code? Regards, Asad
