GitHub user slbotbm created a discussion: C++ Client Design and Implementation Details
# Overview The C++ client uses [CXX]([cxx.rs](http://cxx.rs/)) to build a low level API from the files in `src/*.rs`, which are then used to create a high-level client with the header file being `include/iggy.hpp` and implementations living in `src/*.cpp`. Bazel is used as the build-system. ## Concepts Shared vs opaque structs: In CXX, “shared” structs are structs whose internals are transparent to both the C++ as well as Rust side (the `Message` struct in `src/lib.rs` is an example of this). “Opaque” types are types that are only visible to one side, and we need to define functions so that the other side can fetch data from these (the `Client` or `Identifier` in `src/lib.rs` are examples of this). ## Design I iterated on the C++ client a few times and finally settled on the following ideas: - Having as few shared structs as possible. This is because I would like to entrust the validation of input to the rust side. Further, shared structs introduce latency and memory overhead since they need to be converted between rust and cpp, with data being copied multiple times. - Expose a low-level as well as a high-level client. The high level client being built from `src/*.cpp` aims to present a C++-native interface to the user, but does this by introducing latency by converting data between Rust and C++. If the user is alright with working with non-native C++ code, they can use the low-level client. - For the opaque types, instead of Box’ing them, I prefer to handle raw pointers. This is because I would like to give users using the low-level client explicit control of these types’ lifetimes. For the high-level client, I wrap these pointers in RAII classes that safely handle the destruction when the type goes out of scope. - The current client is a blocking one. This is because CXX requires C++20 and up for asynchronicity. Looking at the [data](https://lp.jetbrains.com/the-state-of-cpp-2025/), C++17 still has 43% of the market share. If we utilize C++20, there may be compatibility issues and could end up alienating some users. Hubert has suggested some libraries in the PR’s comments, and I’ll take a look at them as well. - We use Bazel 9 as the build system for generating the final binaries, but for generating the CXX-related header files, we do not use `rules_rust` as the primary build path; instead `BUILD.bazel` invokes `cargo build` with a Bazel-provisioned Rust toolchain. Since Bazel requires an isolated environment, this approach helps us preserve the local path dependency `iggy = { path = "../../core/sdk" }` in Cargo.toml without having to convert the whole project into one managed using Bazel instead of cargo. ## Implementation - `include/iggy.hpp` is the main header file defining all of the structs and methods available to the user. The implementations live in `src/*.cpp` for the high-level client, and `src/*.rs` for the rust bindings. - On the C++ side, `CompressionAlgorithm`, `Expiry`, `MaxTopicSize` and `PollingStrategy` provide higher-level structures that enable the user to specify their preferred configurations. On the Rust side, some of these structs accept inputs for certain options like `Expiry::duration(value)`, and do not for some like `Expiry::server_default()`. To avoid defining a shared struct for each of them, the Rust side accepts two values (_kind and _value) wherever such a pattern is observed and the code builds the required type internally. For options that do not accept a value like `Expiry::server_default`, the C++-side client sends 0 as the default value, and it is discarded when building the required type on the Rust side. - The `IggyMessage` class on the C++ side utilizes the shared struct `Message` and presents a unified class for receiving and sending messages. The `send_message_` flag controls what the user can access when sending or receiving messages. Further, since message ids are 128-bit numbers, the client uses abseil to work with them. The `PolledMessages` class is similarly defined and utilized the `PolledMessages` shared struct to handle messages. - The four opaque types defined on the Rust side (`Client`, `Identifier`, `StreamDetails`, and `TopicDetails`) are handled on the C++-side using RAII classes to safely deal with raw pointers. These also expose various methods for the user to interact with the Rust side, which are similar to the Python ffi. - `IggyException` is the unified error class that our library presents to the user. All `rust:Error` and `std::exception` are cast to `IggyException` before being presented to the user. ## Limitations - The client is currently blocking, preventing users from taking full advantage of Iggy. - The generated binaries are very large (iggy-cpp is ~500 MB, the example in `examples/example.cpp` is ~200 MB) ## Future Development / Ideas - Make the client asynchronous - Bring the client up to parity with the canonical Rust version - Currently, the client assumes a lot for the user. For example, it uses tcp by default, and the user cannot use a different method. To increase its flexibility, I was thinking of implementing a shared struct that carries commands from the C++ side, is parsed on the Rust side, and the required configurations are created accordingly. ## Credits The initial iteration of the client and the build system settings were largely inspired by [fluss-rust’s C++ bindings](https://github.com/apache/fluss-rust/tree/main/bindings/cpp) GitHub link: https://github.com/apache/iggy/discussions/2691 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
