This is an automated email from the ASF dual-hosted git repository.
gkoszyk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/iggy-website.git
The following commit(s) were added to refs/heads/main by this push:
new 5c990bb1 update docs
5c990bb1 is described below
commit 5c990bb1344efb9f871d424237709299b49fb786
Author: numminex <[email protected]>
AuthorDate: Thu Dec 11 18:52:23 2025 +0100
update docs
---
docs/faq/_category_.json | 4 ++++
docs/faq/faq.md | 14 ++++++++++++++
docs/introduction/architecture.md | 18 ++++++++++++++++++
docs/server/configuration.md | 24 +++++++++++++++++++++++-
4 files changed, 59 insertions(+), 1 deletion(-)
diff --git a/docs/faq/_category_.json b/docs/faq/_category_.json
new file mode 100644
index 00000000..1b185d7c
--- /dev/null
+++ b/docs/faq/_category_.json
@@ -0,0 +1,4 @@
+{
+ "label": "FAQ",
+ "position": 9
+}
diff --git a/docs/faq/faq.md b/docs/faq/faq.md
new file mode 100644
index 00000000..ba2946b0
--- /dev/null
+++ b/docs/faq/faq.md
@@ -0,0 +1,14 @@
+---
+id: faq
+slug: /faq/faq
+title: FAQ
+sidebar_position: 1
+---
+
+## Q: What is the difference between Iggy and traditional message brokers like
Kafka?
+
+Iggy is a persistent message streaming platform that stores messages in an
append-only log format, similar to Kafka. However, Iggy is designed for high
performance and low latency using `io_uring` and a thread-per-core
architecture. Iggy doesn't use kafka protocol, instead it has its own binary
protocol optimized for speed and efficiency, which means that clients need to
use our native client libraries.
+
+## Q: Are there plans to support Kafka protocol in Iggy?
+
+Currently, Iggy does not support the Kafka protocol. Our focus is on providing
a high-performance native protocol that leverages the strengths of Iggy’s
architecture. However, we are open to community contributions (in fact there is
an [discussion](https://github.com/apache/iggy/discussions/6) about it).
diff --git a/docs/introduction/architecture.md
b/docs/introduction/architecture.md
index c5a708a1..46f2e1de 100644
--- a/docs/introduction/architecture.md
+++ b/docs/introduction/architecture.md
@@ -23,6 +23,24 @@ There are advantages and disadvantages of both approaches,
but the main differen
As you might've guessed by now, Iggy is the latter - the message streaming
platform.
+## Thread per core (shared nothing) + io_uring
+
+Iggy utilizies `io_uring` as the main I/O interface, which allows it to
achieve high performance and low latency. Each core (shard), runs it's own
instance of the server, which means that there is (almost) no shared state
between the cores. This allows Iggy to scale linearly with the number of cores,
as there is no contention between the cores. This architecture been proven by
systems such as `redpanda` or `scylladb`, which are known for their high
performance and low latency. Each shard [...]
+
+```rs
+pub fn calculate_shard_assignment(ns: &IggyNamespace, upperbound: u32) -> u16 {
+ let mut hasher = Murmur3Hasher::default();
+ hasher.write_u64(ns.inner());
+ let hash = hasher.finish32();
+ // Murmur3 has problems with weak lower bits for small integer inputs, so
we use bits from the middle.
+ return ((hash >> 16) % upperbound) as u16;
+}
+```
+
+`io_uring` is an ever-growing beast, to learn more about it please check the
[documentation](https://kernel.dk/io_uring.pdf) or the excellent
[presentation](https://www.youtube.com/watch?v=ToSRCSijRuE) by Jens Axboe, the
original author of `io_uring`.
+
+Check [benchmarking](../server/benchmarking.md) section to see some
performance numbers.
+
## Append-only log
The append-only log is the core concept of Iggy. It's a simple data structure,
which is optimized for the append-only operations. It's a sequence of records,
that are being appended to the end of the log. The records are immutable, so
that they can't be changed once they are written to the log. The records are
being written in the order they are received, which results in the log being is
ordered.
diff --git a/docs/server/configuration.md b/docs/server/configuration.md
index e3d5bcab..41bcbf32 100644
--- a/docs/server/configuration.md
+++ b/docs/server/configuration.md
@@ -452,7 +452,23 @@ validate_checksum = false
# The threshold of buffered messages before triggering a save to disk
(integer).
# Specifies how many messages accumulate before persisting to storage.
# Adjusting this can balance between write performance and data durability.
-messages_required_to_save = 5000
+# Together with `size_of_messages_required_to_save` it defines the threshold
of buffered messages.
+# Minimum value is 32. Value has to be a multiple of 32 due to minimum
+# direct I/O block size (512 bytes) and message index size (16 bytes per
message).
+# With direct I/O, writes must occur in blocks of at least 512 bytes, which
equals 32 message indices.
+# The direct I/O requirement was added as forward compatibility, keep in mind
that in case of
+# discovering better solution in the future, this requirement might be removed.
+messages_required_to_save = 1024
+
+# The size threshold of buffered messages before triggering a save to disk
(string).
+# Specifies how much size of messages accumulate before persisting to storage.
+# Adjusting this can balance between write performance and data durability.
+# This is soft limit, actual number of messages may be higher, depending on
last batch size.
+# Together with `messages_required_to_save` it defines the threshold of
buffered messages.
+# Minimum value is 512 B. Value has to be a multiple of 512 B due to direct
I/O requirements.
+# Direct I/O operations must align with the underlying storage block size
(typically 512 B or 4 KiB).
+# 512 B is also minimum size the block device can store atomically, so writing
less than that may lead to partial writes in case of a crash/power loss.
+size_of_messages_required_to_save = "1 MiB"
# Segment configuration
[system.segment]
@@ -491,6 +507,12 @@ expiry = "1 m"
recreate_missing_state = true
```
+### Durability
+
+The single node `Iggy` server by default provides low durability guarantees,
meaning that in case of a crash or power loss, some of messages might be lost.
+To fix that you can enable `enforce_fsync` option in `system.partition`
section together with setting `messages_required_to_save` and
`size_of_messages_required_to_save` to lowest allowed values.
+Since `messages_required_to_save` has to be at least 32 and
`size_of_messages_required_to_save` at least 512 B, in case if your message
size is not big enough, you might want to pad your message to 512 B boundary,
or alternatively send messages in batches.
+
### HTTP
An optional transport layer, which can be used to connect to the server with
any tool or application that supports HTTP. It is enabled by default, but you
can disable it by setting `enabled` to `false`. The `address` option specifies
the address and port to listen on. The `cors` section allows you to configure
the CORS policy, which internally uses [Tower
middleware](https://docs.rs/tower-http/latest/tower_http/cors/index.html).
