GitHub user numinnex added a comment to the discussion: Metadata layer in cluster.
Thanks Chiradip. Yeah after thinking about it more, I agree that a single log, with one atomic snapshot is the easiest path to achieve strict serializability, considering that metadata defines all the necessary elements required for the clients to operate the server, SS is a must. > logical domain separation inside the SM > modular snapshot format Yeah those can be achieved by having MuxStateMachine, while still having one atomic snapshot, seems that redpanda does exactly that. > A circular buffer with “upper-bounded ops” will break in production when > metadata spikes. What do you mean by that ? What I've thought is, since our metadata ops have an "upperbound" size (which isn't large it's few thousands bytes maximum), we can easily estimate the size of the log and limit it's size by the snapshot interval -- log_size = snapshot_interval * max_op_size, and utilize the snapshot sequence number to distinguish between the "junk" log entries and "new" log entries, without paying the cost of physically truncating the log, the log entries become sort of "copy-on-write". Tigerbeetle uses that approach for their log and it seems to work really great (they even add an extra "lag" when snapshotting the log, to allow for extra pipelinening), I don't think so we need to go this far, as metadata isn't the "hot" path of our system. GitHub link: https://github.com/apache/iggy/discussions/2346#discussioncomment-14978311 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
