> ... what is the difference between StateMachineLogEntryProto#logData and StateMachineLogEntryProto#stateMachineEntry#stateMachineData ?
StateMachineLogEntryProto#logData is the data that will be written to RaftLog. In contrast, StateMachineLogEntryProto#stateMachineEntry#stateMachineData won't be written to the RaftLog. The state machine is going to store it. > In the counter example the log data is the 'increment request' and in filestore the stateMachineData is the stateMachineLogEntry is being used? In the counter example, the log data contains all the data in the request and the request is just the "increment request". In filestore, a client may send a request to write a file containing data D to a particular path P. The stateMachineData is the file data D and the log data is the meta data such as file path P. > What should be used where/when? StateMachineLogEntryProto#logData is the logData specified in the RAFT algorithm. StateMachineLogEntryProto#stateMachineEntry#stateMachineData is the data that the state machine wants to store the data itself only but not in the RaftLog in order to avoid storing two copies of data. Please feel free to let me know if you have more questions. Tsz-Wo On Sat, Dec 18, 2021 at 11:42 PM Asad Awadia <[email protected]> wrote: > I think I am starting to understand. > > > Reading the raft.proto file - what is the difference between > StateMachineLogEntryProto#logData and > StateMachineLogEntryProto#stateMachineEntry#stateMachineData ? > > In the counter example the log data is the 'increment request' and in > filestore the stateMachineData is the stateMachineLogEntry is being used? > > What should be used where/when? > ------------------------------ > *From:* Tsz Wo Sze <[email protected]> > *Sent:* Saturday, December 18, 2021 1:36:18 AM > *To:* [email protected] <[email protected]> > *Subject:* Re: Raft log persistance > > The file store example is just a simple example. It does not intend to > support all the features. It indeed does not work for the case that the > data is deleted -- it will fail to read the file. > > You are right that, for real applications supporting multiple versions of > data, it should retrieve the data from the version corresponding to the > given log index. > > Tsz-Wo > > > On Sat, Dec 18, 2021 at 1:05 PM Asad Awadia <[email protected]> wrote: > > In file store example read method - it looks like it is returning the file > at the path without taking the raft index in consideration i.e it would > keep returning the same file for many entry indices provided the file > hadn't changed? > > What is going on there? Shouldn't it return the data snapshot at instant > entry.getIndex? > > ------------------------------ > *From:* Tsz Wo Sze <[email protected]> > *Sent:* Wednesday, December 15, 2021 7:37:20 PM > *To:* [email protected] <[email protected]> > *Subject:* Re: Raft log persistance > > > > If I understand correctly, in the write method we should take the > request and apply it right there in the method into the state machine? So > it only gets persisted into the state machine and not the raft log? > > The write method should write the stateMachineData > (i.e. > LogEntryProto.getStateMachineLogEntry().getStateMachineEntry().getStateMachineData()) > to its storage. The stateMachineData will be removed from the > logEntryProto. The logEntryProto with removed stateMachineData will be > written to the raft log. Yes, stateMachineData won't be written to the > raft log. > > > How will syncing up and backups work then? If there is no 'raft log' > anymore since we are overriding the write method to only apply writes to > the state machine. How will the other servers be able to sync and catch up? > > When reading from the raft log, the server in addition calls the > DataApi.read(LogEntryProto) method. The state machine should return back > the stateMachineData corresponding to the given logEntryProto as a > CompletableFuture<ByteString>. Then, the server will add > the stateMachineData back to the logEntryProto and then send > the logEntryProto to the other servers. > > Tsz-Wo > > > On Thu, Dec 16, 2021 at 3:13 AM Asad Awadia <[email protected]> wrote: > > I read through the file store example. > > If I understand correctly, in the write method we should take the request > and apply it right there in the method into the state machine? So it only > gets persisted into the state machine and not the raft log? > > > How will syncing up and backups work then? If there is no 'raft log' > anymore since we are overriding the write method to only apply writes to > the state machine. How will the other servers be able to sync and catch up? > > ------------------------------ > *From:* Tsz Wo Sze <[email protected]> > *Sent:* Tuesday, December 14, 2021 8:16:23 PM > *To:* [email protected] <[email protected]> > *Subject:* Re: Raft log persistance > > Hi Asad, > > By default, all the write requests (including the data inside) are > recorded in the raft log. This is specified in the standard RAFT algorithm > so that any server can replay the log in order to sync up to the current > state. If the state machine keeps another persistent copy of the data, you > are right that there will be two copies. In this sense, the standard RAFT > algorithm is not suitable for data intensive applications. > > In Ratis, we want to support data intensive applications such as Apache > Ozone. State machine implementations may choose to manage the data > itself. In that case, the data won't be recorded in the raft log. In > order to use this feature, state machine implementations must override the > read and write methods defined in StateMachine.DataApi. The FileStore > example indeed has overridden these two methods. > > Hope it helps. > Tsz-Wo > > > On Wed, Dec 15, 2021 at 3:40 AM Asad Awadia <[email protected]> wrote: > > Hello, > > Is ratis keeping a copy of all the writes made? On top of what is > persisted in my own DB? So ~2x storage used > > I read that the file system example avoids that but I don't see how that > is being done in the code? > > Regards, > Asad > >
