> How would error handling work then? ... What kinds of errors?
Not sure if you are talking about this kind of error: In the FileStore example, suppose the user does not have the permission (for discussion only, the permission feature is not implemented.) to create the file. The StateMachine should just complete the future exceptionally with permission-denied. Ratis will wrap the exception and then return it back to the client. It won't affect the other transactions. > Do you know of a way to get a distributed datastore but keep it multi threaded all the way down to disk writes? What would I have to trade off to get that requirement? It depends on the consistency model and other requirements such as the fault tolerance model. For eventual consistency, it can achieve such parallelism. This is a hard question in general and probably is a Ph.D. thesis. Tsz-Wo On Sat, Feb 26, 2022 at 3:04 PM Asad Awadia <[email protected]> wrote: > How would error handling work then? Is the processing of a Queue > completely halted when an error occurs? > > > Do you know of a way to get a distributed datastore but keep it multi > threaded all the way down to disk writes? What would I have to trade off to > get that requirement? > ------------------------------ > *From:* Tsz Wo Sze <[email protected]> > *Sent:* Saturday, February 26, 2022 12:19:00 AM > *To:* [email protected] <[email protected]> > *Subject:* Re: Log applying threading > > Hi Asad, > > These are good questions. > > This depends on the StateMachine implementations. Like most of the Ratis > APIs, the StateMachine interface is asynchronous. In particular, the > applyTransaction(..) method returns a future as shown below. Ratis > *submits* the async calls to StateMachine sequentially. The StateMachine > may choose to *process* those calls in parallel. > > //StateMachine.java > CompletableFuture<Message> applyTransaction(TransactionContext trx); > > For example, the FileStoreStateMachine processes the write requests in > parallel when writing to different files. Of course, the writes within the > same file are sequential. In more detail, each file has its own > TaskQueue. In applyTransaction, FileStoreStateMachine only puts on the > write requests to the corresponding TaskQueue. There is a separated thread > pool to process the tasks from the queues. Therefore, the tasks in > different queues are processed in parallel. > > Thanks. > Tsz-Wo > > On Sat, Feb 26, 2022 at 7:06 AM Asad Awadia <[email protected]> wrote: > > Hello, > > Since raft is dependent on applying the raft log in order does that mean > that only a single thread is looping over them and applying it? > > If yes, does that not cause performance bottle necks? > > Regards, > Asad > >
