This is an automated email from the ASF dual-hosted git repository. hubcio pushed a commit to branch docs/align-with-topic-durability in repository https://gitbox.apache.org/repos/asf/iggy-website.git
commit e355b6628ac3171939327b146116865d374014a0 Author: hubcio <[email protected]> AuthorDate: Sat Sep 12 02:37:17 2026 +0200 fix(docs): qualify FAQ behavior and guarantees Correct transport choices, resource requirements, rebalancing and processing guarantees. Scope historical benchmarks and distinguish connector telemetry from the broker export limitations. --- content/docs/faq/faq.mdx | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/content/docs/faq/faq.mdx b/content/docs/faq/faq.mdx index 72f6b2bb..17caaf64 100644 --- a/content/docs/faq/faq.mdx +++ b/content/docs/faq/faq.mdx @@ -5,7 +5,7 @@ description: "Common questions about Apache Iggy, including how it compares to K ## Q: What is the difference between Iggy and traditional message brokers like Kafka? -Iggy is a persistent message streaming platform that stores messages in an append-only log format, similar to Kafka. However, Iggy is designed for high performance and low latency using `io_uring` and a thread-per-core architecture. Iggy doesn't use kafka protocol, instead it has its own binary protocol optimized for speed and efficiency, which means that clients need to use our native client libraries. +Iggy is a persistent message streaming platform that stores messages in an append-only log format, similar to Kafka. However, Iggy is designed for high performance and low latency using `io_uring` and a thread-per-core architecture. Iggy uses its own binary protocol rather than the Kafka protocol. Clients can use an Iggy SDK, implement the documented binary protocol, or use the HTTP API. ## Q: Are there plans to support Kafka protocol in Iggy? @@ -13,31 +13,33 @@ Currently, Iggy does not support the Kafka protocol. Our focus is on providing a ## Q: What transport protocol should I use? -For maximum throughput and lowest latency, use **TCP**. If you need built-in encryption without configuring TLS separately, **QUIC** is a good choice. **WebSocket** works well for browser-based clients. **HTTP** is the most accessible but has the highest overhead due to JSON serialization and stateless connections (consumer groups can be created, inspected, and deleted, but there is no join/leave membership over HTTP). +For maximum throughput and lowest latency, use **TCP**. If you need built-in encryption without configuring TLS separately, **QUIC** is a good choice. **WebSocket** works well for browser-based clients. **HTTP** is the most accessible but has the highest overhead due to JSON serialization. HTTP connections can be reused. Consumer groups can be created, inspected, and deleted, but there is no join/leave membership over HTTP. ## Q: What are the system requirements? -Iggy is a single binary with no external dependencies. On Linux, it uses `io_uring` for maximum performance, which requires kernel 5.19+. The server starts with around 20 MB of RAM (plus the configured memory pool, default 4 GiB). For Docker, you need to set `SYS_NICE` capability, disable seccomp, and set unlimited memlock. +Iggy runs as a single server process without an external coordination service; native library requirements depend on the build. The provided Linux images include hwloc and udev libraries. The Linux shard runtime requires `io_uring` flags introduced in kernel 5.19. The server starts with around 20 MB of RAM; the default 4 GiB memory-pool setting is a budget, with buffers allocated on demand and fallback allocations outside the pool. + +Docker must allow the required `io_uring` calls and provide enough locked-memory allowance. The supplied Compose configuration uses `SYS_NICE`, `seccomp:unconfined` and unlimited memlock; those settings are not universal requirements for every host. See [Docker](/docs/server/docker) and [configuration](/docs/server/configuration). ## Q: How does consumer group rebalancing work? -When consumers join or leave a consumer group, the server triggers cooperative partition rebalancing. Partitions are redistributed among active members. During rebalancing, there is a pending revocation phase (configurable timeout, default 30s) to ensure in-progress message processing can complete before partitions move to new owners. +When consumers join or leave a consumer group, the server triggers cooperative partition rebalancing. Partitions are redistributed among active members. During rebalancing, a pending revocation phase fences new polls from the previous owner. Its configurable timeout defaults to 30s, after which the server can force a transfer. This coordinates polling ownership; it does not prove that application processing has finished. ## Q: Does Iggy support exactly-once delivery? -Iggy supports **at-most-once** (with auto-commit) and **at-least-once** (without auto-commit, manual offset management). **Exactly-once** semantics can be achieved at the application level, e.g. by attaching unique IDs to messages and deduplicating on the consumer side. +Committing an offset before application processing, including poll-time auto-commit, can provide **at-most-once** processing: a crash can lose unprocessed messages. Committing only after successful processing supports **at-least-once** processing, with duplicates possible after a crash. Recovery also depends on retention and the configured message and consumer-offset durability. Iggy does not provide an atomic transaction between a consumer offset and an external side effect. **Exactly-o [...] ## Q: How do I secure my Iggy deployment? -Iggy supports TLS on all transport protocols, Argon2id password hashing, granular per-stream/per-topic permissions, Personal Access Tokens for programmatic access, and optional AES-256-GCM encryption at rest. For the HTTP API, JWT tokens are used for session management. See the [Security](/docs/server/security) documentation for details. +Iggy supports TLS on all transport protocols, Argon2id password hashing, granular per-stream/per-topic permissions, Personal Access Tokens for programmatic access, and optional AES-256-GCM message-payload encryption. For the HTTP API, JWT tokens are used for session management. See the [Security](/docs/server/security) documentation for details. ## Q: Can I use Iggy with my existing tooling? -Iggy provides a [Model Context Protocol (MCP)](/docs/ai/mcp) server with 40+ tools for LLM integration, [connectors](/docs/connectors/introduction) for piping data to/from external systems (e.g. PostgreSQL, MongoDB, Elasticsearch, ClickHouse, InfluxDB, S3, Delta Lake, Apache Iceberg, Quickwit), Prometheus metrics, and OpenTelemetry traces/logs. The HTTP API works with any REST client. +Iggy provides a [Model Context Protocol (MCP)](/docs/ai/mcp) server with 40+ tools for LLM integration, [connectors](/docs/connectors/introduction) for piping data to/from external systems (e.g. PostgreSQL, MongoDB, Elasticsearch, ClickHouse, InfluxDB, S3, Delta Lake, Apache Iceberg, Quickwit), Prometheus metrics, and OpenTelemetry traces/logs in the connectors runtime. Broker OpenTelemetry export has runtime limitations described in [configuration](/docs/server/configuration#telemetry). [...] ## Q: What happened to the Tokio-based runtime? -Iggy migrated from Tokio to a thread-per-core architecture with `compio` (which uses `io_uring` on Linux) starting with version 0.6.0. The migration delivered major performance improvements, including up to 92% better P9999 tail latency, and an 18% throughput improvement when fsync is enabled. You can read the full story in the [thread-per-core io_uring blog post](https://iggy.apache.org/blogs/2026/02/27/thread-per-core-io_uring/). +Iggy migrated from Tokio to a thread-per-core architecture with `compio` (which uses `io_uring` on Linux) starting with version 0.6.0. The published v0.5.0 versus v0.7.0 comparison reports 92% lower P9999 latency for its 16-producer, 16-stream, 40-million-message workload, and 18% higher throughput for the corresponding fsync workload. These are historical workload results, not a guarantee for 0.9.0. You can read the full story in the [thread-per-core io_uring blog post](https://iggy.apa [...] ## Q: Is clustering/replication available?
