Hi xuanwo On Tue, Mar 25, 2025 at 2:36 PM Xuanwo <xua...@apache.org> wrote:
> After a deeper discussion with friends in the Slack #cpp channel, we > reached the following consensus: > > - iceberg-rust won't maintain any c/cpp related code. > - iceberg-cpp will have an internal crate based on iceberg-rust and > invokes cxx/cbingen to build bindings > - And finally, we will have a libiceberg-api and libiceberg-cpp , > libicberg-rust in the iceberg-cpp repo. > > In which, iceberg-cpp will provide: > > - libiceberg-api : An headers that defines APIs available to users. > - libiceberg-cpp : An impl of libiceberg-api based on pure cpp (based on > nanoarrow, json, libz and so on) > - libicberg-rust : An impl of libiceberg-api based on iceberg-rust. > > What do you think? > Thank you for the summary! I think this approach will work well. Iceberg-cpp might also consider providing a C API wrapper to support pure C users. Since we are still at the early stage, we can experiment with this approach and revisit it for further discussion if any issues arise. > > On Tue, Mar 18, 2025, at 16:56, Fokko Driesprong wrote: > > Hi Xuanwo, > > Thanks for bringing this up. I'm not a cpp expert, but following this with > great interest. I think this is a very promising approach since the memory > model is already the same across projects (Arrow). > > Kind regards, > Fokko > > Op vr 14 mrt 2025 om 08:53 schreef Xuanwo <xua...@apache.org>: > > > Hi, everyone > > Today, Renjie Liu and I had a discussion with some of the iceberg-cpp > contributors—Gang Wu, Junwang Zhao, and David Li—about ways to advance the > project. We came up with some great ideas, so I'm sharing them here to > gather more feedback. > > ## Background > > The community continues to show interest in a C++ implementation of > Iceberg. Some users are interested in C++ bindings with a well-designed API > for data types and manifests, but they have their own runtime and I/O, so > they prefer a binding that exclude these components. However, there are > also users who require an all-in-one solution. > > The iceberg-cpp project aims to provide a well-designed interface for > users to plug in file I/O, readers, writers, and other components while > offering a default implementation in other modules. > > Luckily, we now have iceberg-rust, which is not fully feature-complete but > undergoing rapid development. We can develop a C++ binding based on > iceberg-rust and establish a robust C++ API within the iceberg-cpp project. > The C++ project will define public interfaces and offer various build > targets for users to choose from. The overall process will be similar to > how pyiceberg interacts with iceberg-rust. > > ## Plan > > > 1. Use cxx[^1] to build cpp bindings for iceberg-rust which exposes core > APIs. > > > The binding will support different build profiles that users can choose to > use fileio from iceberg-rust or not. > > 2. iceberg-cpp will define public interfaces and types for cpp users. > > After this separation, iceberg-rust can focus on implementing features and > exposing them through simpler, more direct bindings. Meanwhile, iceberg-cpp > will be responsible for designing, testing, and maintaining the user-level > APIs, allowing users to select the parts they need or plug in their own > file I/O components. > > ## Conclusion > > We believe this design combines the best parts of two projects and helps > us iterate iceberg-cpp more quickly. > > Welcome to leave your comments here! > > [^1]: https://cxx.rs/ > > Xuanwo > > https://xuanwo.io/ > > Xuanwo > > https://xuanwo.io/ > > -- Regards Junwang Zhao
