spetz opened a new pull request, #4092: URL: https://github.com/apache/iggy/pull/4092
Producer acknowledgments could precede recoverable disk storage. enforce_fsync synchronized segment flushes but did not make PrepareOk durable on replicas. Consumer-offset synchronization was also global, preventing independent policies for mixed workloads. Add create-only durability and consumer_offset_durability options with replicated and persisted values. Both independently default to replicated, without inheritance. Replicate the options with topic metadata so every replica applies the same policy. Keep VSR quorum commit as the acknowledgment rule. Persisted additionally requires recoverable stable-storage copies on the required quorum. Both policies retain ordinary disk persistence. Name the policies by their guarantees so a future Direct I/O backend can preserve the same API contract. Add a bounded partition prepare WAL when either policy is persisted. Use 4 KiB-padded records, checksums, parent-chain validation and a durable frontier. Retain mixed-operation predecessors so durable offsets cannot leave gaps in the recoverable message history. Forward prepares before waiting for local persistence and deliver durable completion through shard lifecycle messages. Fence stale completions by instance and generation. Replicated/replicated topics stay off the disk WAL path and avoid extra prepare cloning. Single-node persisted operations synchronize committed local state before replying, without a prepare WAL. Reclaim WAL history only after materialized state is durable. Cover truncation, purge, repair and delete/recreate, and retain rollback state across destructive transfer installation. Restore checksum anchors after checkpoint-only restart and carry them in ICO3 state-transfer artifacts. Fail closed on missing history or storage errors. Flatten system.* into root runtime, logging, encryption, stream, topic, partition, memory_pool and sharding tables. Move the data root to path and IGGY_PATH, merge partition storage settings with partition limits, and remove unsupported archive_expired and recreate_missing_state settings. Update Rust plumbing, environment mappings, CI, Helm, scripts and fixtures. Expose idiomatic enums in Rust, Java, C#, Go, Node, Python, PHP and C++. Update CLI and benchmark flags, option catalogs, generated declarations, documentation and cross-SDK golden vectors. Preserve HTTP option provenance and report replicated or persisted on awaited produce, with none on early dispatch acceptance. Keep poll auto-commit and existing no-ack dispatch asynchronous. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
