GitHub user chiradip added a comment to the discussion: Metadata layer in cluster.
Thanks Greg — great write-up. A couple of focused points from building clustered metadata systems with SMR (Raft/VSR): 1. Since Iggy is using VSR, start with one VSR group + one ordered log A single VSR log keeps all metadata ops (streams, topics, consumer groups, users) linearizable and naturally atomic. It avoids every cross-SM coordination problem that appears the moment you introduce multiple VSR groups. You can still keep logical namespaces inside the state machine — just not separate consensus groups yet. 2. Make snapshots modular, not multi-group The “MuxStateMachine” idea is useful inside one VSR log, but running multiple SMRs now creates 2PC-style coordination for deletes/renames/permissions. Instead: one snapshot manifest with per-domain sub-snapshots. Easy to install, easy to stream, and future-proof if you ever need to split domains. 3. The journal needs real segmented logs A circular buffer with “upper-bounded ops” will break in production when metadata spikes. Use append-only segmented logs + snapshot-triggered truncation + streaming snapshot install for follower catch-up. 4. Define semantics first Before choosing architecture, clarify: - Is metadata strictly linearizable? - Should cross-domain metadata changes be atomic? - What’s the recovery path for a cold follower? These decisions determine whether multiple SMs are even viable long-term. TL;DR Greg — the cleanest and safest path is: - one VSR group - one log - logical domain separation inside the SM - modular snapshot format It gives you correctness, simple recovery, atomic metadata ops, and a clean way to scale later without committing to multi-group coordination now. Happy to dig deeper if helpful. GitHub link: https://github.com/apache/iggy/discussions/2346#discussioncomment-14977379 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
