How would error handling work then? Is the processing of a Queue completely halted when an error occurs?
Do you know of a way to get a distributed datastore but keep it multi threaded all the way down to disk writes? What would I have to trade off to get that requirement? ________________________________ From: Tsz Wo Sze <[email protected]> Sent: Saturday, February 26, 2022 12:19:00 AM To: [email protected] <[email protected]> Subject: Re: Log applying threading Hi Asad, These are good questions. This depends on the StateMachine implementations. Like most of the Ratis APIs, the StateMachine interface is asynchronous. In particular, the applyTransaction(..) method returns a future as shown below. Ratis *submits* the async calls to StateMachine sequentially. The StateMachine may choose to *process* those calls in parallel. //StateMachine.java CompletableFuture<Message> applyTransaction(TransactionContext trx); For example, the FileStoreStateMachine processes the write requests in parallel when writing to different files. Of course, the writes within the same file are sequential. In more detail, each file has its own TaskQueue. In applyTransaction, FileStoreStateMachine only puts on the write requests to the corresponding TaskQueue. There is a separated thread pool to process the tasks from the queues. Therefore, the tasks in different queues are processed in parallel. Thanks. Tsz-Wo On Sat, Feb 26, 2022 at 7:06 AM Asad Awadia <[email protected]<mailto:[email protected]>> wrote: Hello, Since raft is dependent on applying the raft log in order does that mean that only a single thread is looping over them and applying it? If yes, does that not cause performance bottle necks? Regards, Asad
