slbotbm commented on code in PR #3809: URL: https://github.com/apache/iggy/pull/3809#discussion_r3839676353
########## core/sdk/src/lib.rs: ########## @@ -15,6 +15,237 @@ // specific language governing permissions and limitations // under the License. +//! Apache Iggy is a high-performance, persistent message streaming platform written in Rust, +//! capable of processing millions of messages per second with ultra-low latency. +//! It is part of the [`Apache Incubating Program`] of the [`Apache Software Foundation`] (ASF). +//! +//! **This library is the Apache Iggy SDK.** +//! It exposes a low-level and a high-level API for the Apache Iggy message streaming infrastructure for the Rust programming language. +//! SDKs for other programming languages can be found in [`core/foreign`] of the root repository on GitHub. +//! +//! The core of the Iggy server is a persisted append-only log data structure. +//! It is concerned with allowing read and writes in the most efficient way. +//! Reading and writing to the server is the domain of this SDK. +//! The server exposes *commands* that can be triggered to change its state. +//! These commands allow administrative tasks, such as handling users, permissions and setting up streams and topics +//! or writing and reading messages from the log. +//! A comprehensive overview of commands can be found in the [`schema spec`] on the website or checking the [`server command enum`] within the source code. +//! +//! The SDK provides tools to build production ready message-streaming applications. +//! It exposes its functionality at two levels. The [high-level API](#high-level-api) +//! is transport-agnostic and ships with the batching, retry, offset-tracking, and +//! connection-management machinery a production application needs. The +//! [low-level API](#low-level-api) is the set of concrete transport clients that +//! speak the wire protocol directly and that the high-level API is built on top of. +//! It is recommended to start with the high-level API, and utilize the low level API +//! in case the high-level API cannot satisfy your requirements. +//! +//! # High-level API +//! +//! The high-level API is most likely what you are looking for, especially if you are new to building +//! message-streaming applications with Iggy. +//! Clients provided by the high-level API already provide common message-streaming features that +//! you would otherwise need to build yourself. +//! +//! There are three client types: +//! - [`IggyClient`] is the entry point and the full API surface. It owns the +//! connection and implements every domain trait, including [`MessageClient`] +//! with the raw [`send_messages`] and [`poll_messages`] primitives. Each call +//! is a single, stateless request: no batching, retries, offset tracking, or +//! polling loop. +//! - [`IggyProducer`] is a stateful helper for high-throughput sending, built on +//! [`send_messages`]. +//! - [`IggyConsumer`] is a stateful helper for continuous consumption, built on +//! [`poll_messages`]. +//! +//! You do not construct the producer and consumer independently. Spawn them +//! from an [`IggyClient`] with [`IggyClient::producer`] and +//! [`IggyClient::consumer`] so they share its connection. +//! +//! ## When to use each +//! +//! Reach for [`IggyClient`] directly for administrative tasks such as +//! creating streams, topics, users, and consumer groups, reading or storing +//! offsets, or sending and polling a handful of messages in a script. +//! Reach for [`IggyProducer`] and [`IggyConsumer`] when producing and consuming messages. +//! +//! The [`IggyProducer`] adds, on top of [`send_messages`]: +//! - **Background batching** that flushes by size, message count, or a linger +//! interval, instead of one network round-trip per send. +//! - **Retries** with a configurable count and interval (three attempts one +//! second apart by default). +//! - A pluggable **partitioning strategy**, so the target partition is not +//! passed on every call. +//! - **In-flight and ordering control**, optional payload **encryption**, and +//! `create_stream_if_not_exists` / `create_topic_if_not_exists` convenience. +//! +//! The [`IggyConsumer`] adds, on top of [`poll_messages`]: +//! - A [`futures::Stream`] implementation, so a `while let Some(message) = +//! consumer.next().await` loop drives polling, paging, and the poll interval +//! for you. +//! - A **polling strategy** (`next`, `offset`, or `timestamp`) that tracks +//! position instead of taking an offset on every call. +//! - **Auto-commit** and offset storage on an interval or after a number of +//! messages, so a restart resumes where it left off. +//! - **Auto-join** of consumer groupsa and assignment refresh should the server have +//! assigned the consumer another partition +//! - Reconnection handling should the client disconnect Review Comment: groups -> groups ########## core/sdk/src/lib.rs: ########## @@ -15,6 +15,237 @@ // specific language governing permissions and limitations // under the License. +//! Apache Iggy is a high-performance, persistent message streaming platform written in Rust, +//! capable of processing millions of messages per second with ultra-low latency. +//! It is part of the [`Apache Incubating Program`] of the [`Apache Software Foundation`] (ASF). +//! +//! **This library is the Apache Iggy SDK.** +//! It exposes a low-level and a high-level API for the Apache Iggy message streaming infrastructure for the Rust programming language. +//! SDKs for other programming languages can be found in [`core/foreign`] of the root repository on GitHub. +//! +//! The core of the Iggy server is a persisted append-only log data structure. +//! It is concerned with allowing read and writes in the most efficient way. +//! Reading and writing to the server is the domain of this SDK. +//! The server exposes *commands* that can be triggered to change its state. +//! These commands allow administrative tasks, such as handling users, permissions and setting up streams and topics +//! or writing and reading messages from the log. +//! A comprehensive overview of commands can be found in the [`schema spec`] on the website or checking the [`server command enum`] within the source code. +//! +//! The SDK provides tools to build production ready message-streaming applications. +//! It exposes its functionality at two levels. The [high-level API](#high-level-api) +//! is transport-agnostic and ships with the batching, retry, offset-tracking, and +//! connection-management machinery a production application needs. The +//! [low-level API](#low-level-api) is the set of concrete transport clients that +//! speak the wire protocol directly and that the high-level API is built on top of. +//! It is recommended to start with the high-level API, and utilize the low level API +//! in case the high-level API cannot satisfy your requirements. +//! +//! # High-level API +//! +//! The high-level API is most likely what you are looking for, especially if you are new to building +//! message-streaming applications with Iggy. +//! Clients provided by the high-level API already provide common message-streaming features that +//! you would otherwise need to build yourself. +//! +//! There are three client types: +//! - [`IggyClient`] is the entry point and the full API surface. It owns the +//! connection and implements every domain trait, including [`MessageClient`] +//! with the raw [`send_messages`] and [`poll_messages`] primitives. Each call +//! is a single, stateless request: no batching, retries, offset tracking, or +//! polling loop. +//! - [`IggyProducer`] is a stateful helper for high-throughput sending, built on +//! [`send_messages`]. +//! - [`IggyConsumer`] is a stateful helper for continuous consumption, built on +//! [`poll_messages`]. +//! +//! You do not construct the producer and consumer independently. Spawn them +//! from an [`IggyClient`] with [`IggyClient::producer`] and +//! [`IggyClient::consumer`] so they share its connection. +//! +//! ## When to use each +//! +//! Reach for [`IggyClient`] directly for administrative tasks such as +//! creating streams, topics, users, and consumer groups, reading or storing +//! offsets, or sending and polling a handful of messages in a script. +//! Reach for [`IggyProducer`] and [`IggyConsumer`] when producing and consuming messages. +//! +//! The [`IggyProducer`] adds, on top of [`send_messages`]: +//! - **Background batching** that flushes by size, message count, or a linger +//! interval, instead of one network round-trip per send. +//! - **Retries** with a configurable count and interval (three attempts one +//! second apart by default). +//! - A pluggable **partitioning strategy**, so the target partition is not +//! passed on every call. +//! - **In-flight and ordering control**, optional payload **encryption**, and +//! `create_stream_if_not_exists` / `create_topic_if_not_exists` convenience. +//! +//! The [`IggyConsumer`] adds, on top of [`poll_messages`]: +//! - A [`futures::Stream`] implementation, so a `while let Some(message) = +//! consumer.next().await` loop drives polling, paging, and the poll interval +//! for you. +//! - A **polling strategy** (`next`, `offset`, or `timestamp`) that tracks +//! position instead of taking an offset on every call. +//! - **Auto-commit** and offset storage on an interval or after a number of +//! messages, so a restart resumes where it left off. +//! - **Auto-join** of consumer groupsa and assignment refresh should the server have +//! assigned the consumer another partition +//! - Reconnection handling should the client disconnect +//! - Payload **decryption** +//! +//! # Stream builder API +//! +//! The stream builder API is a convenient way to use the high-level API. +//! [`IggyStream`], [`IggyStreamProducer`], and +//! [`IggyStreamConsumer`] construct everything at once. +//! You can pass an [`IggyClient`] (or just a connection string) together with a config, +//! and they hand back a ready, connected [`IggyProducer`] / [`IggyConsumer`]. +//! Compared to the **high-level API**, it changes how you construct +//! producers and consumers, not what they can do. Instead of chaining an +//! [`IggyProducerBuilder`] / [`IggyConsumerBuilder`] and setting each option +//! with a method call, you describe the whole setup once in an +//! [`IggyStreamConfig`] and build from it. The result is +//! the same [`IggyProducer`] and [`IggyConsumer`] the builders produce, backed +//! by the same [`IggyClient`]. Review Comment: change to ```rust //! Compared to the **high-level API**, the stream builder simplifies construction //! but exposes a smaller configuration surface. It returns the same //! [`IggyProducer`] and [`IggyConsumer`] types, backed by the same [`IggyClient`], //! but configures the producer in direct mode. Use [`IggyProducerBuilder`] when //! you need background batching, sharding, backpressure, or in-flight controls. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
