Hi Asad, > what kind of serialization (json/protobuf) and transport (http/tcp/websocket) is used to send messages between nodes.
We mainly use protobuf for serialization and grpc/netty for transport, although the answer is RPC dependent since the RPC implementation is pluggable. > Any optimizations being done in those areas to improve perf and stability/reliability? Ratis supports asynchronous RPC which greatly improves the performance. It also supports streaming large data ( https://issues.apache.org/jira/browse/RATIS-979 ). > Should app devs be concerned about the chattiness among the nodes in terms of perf, network congestion/bandwidth, and/or disk usage? By design, Raft writes all the data and metadata to the log. If the state machine also keeps the data in the storage, then the data is written twice to the disk. It is inefficient when the data size is large. In Ratis, the state machine can choose to manage the data itself so that the data won't be written to the log. In addition, we can use Ratis streaming to write large data sets. Other than that, the network communication in Raft/Ratis is quite simple -- mainly the leader forwards log entries to the followers. Hope it helps! Tsz-Wo As mentioned above, On Tue, May 31, 2022 at 2:12 PM Asad Awadia <[email protected]> wrote: > Hello, > > Since raft relies on being able to communicate across the network between > nodes, I wanted to know what kind of serialization (json/protobuf) and > transport (http/tcp/websocket) is used to send messages between nodes. Any > optimizations being done in those areas to improve perf and > stability/reliability? > > > Should app devs be concerned about the chattiness among the nodes in terms > of perf, network congestion/bandwidth, and/or disk usage? > > Regards, > Asad >
