Thanks Xuanwo for the summary. I think it's a good idea to move c/cpp related code to iceberg-cpp repo, and have a try on the cxx binding approach.
On Wed, Mar 26, 2025 at 6:54 PM Junwang Zhao <zhjw...@gmail.com> wrote: > Hi xuanwo > > On Tue, Mar 25, 2025 at 2:36 PM Xuanwo <xua...@apache.org> wrote: > >> After a deeper discussion with friends in the Slack #cpp channel, we >> reached the following consensus: >> >> - iceberg-rust won't maintain any c/cpp related code. >> - iceberg-cpp will have an internal crate based on iceberg-rust and >> invokes cxx/cbingen to build bindings >> - And finally, we will have a libiceberg-api and libiceberg-cpp , >> libicberg-rust in the iceberg-cpp repo. >> >> In which, iceberg-cpp will provide: >> >> - libiceberg-api : An headers that defines APIs available to users. >> - libiceberg-cpp : An impl of libiceberg-api based on pure cpp (based on >> nanoarrow, json, libz and so on) >> - libicberg-rust : An impl of libiceberg-api based on iceberg-rust. >> >> What do you think? >> > > Thank you for the summary! > > I think this approach will work well. Iceberg-cpp might also consider > providing a > C API wrapper to support pure C users. > > Since we are still at the early stage, we can experiment with this approach > and revisit it for further discussion if any issues arise. > > >> >> On Tue, Mar 18, 2025, at 16:56, Fokko Driesprong wrote: >> >> Hi Xuanwo, >> >> Thanks for bringing this up. I'm not a cpp expert, but following this >> with great interest. I think this is a very promising approach since the >> memory model is already the same across projects (Arrow). >> >> Kind regards, >> Fokko >> >> Op vr 14 mrt 2025 om 08:53 schreef Xuanwo <xua...@apache.org>: >> >> >> Hi, everyone >> >> Today, Renjie Liu and I had a discussion with some of the iceberg-cpp >> contributors—Gang Wu, Junwang Zhao, and David Li—about ways to advance the >> project. We came up with some great ideas, so I'm sharing them here to >> gather more feedback. >> >> ## Background >> >> The community continues to show interest in a C++ implementation of >> Iceberg. Some users are interested in C++ bindings with a well-designed API >> for data types and manifests, but they have their own runtime and I/O, so >> they prefer a binding that exclude these components. However, there are >> also users who require an all-in-one solution. >> >> The iceberg-cpp project aims to provide a well-designed interface for >> users to plug in file I/O, readers, writers, and other components while >> offering a default implementation in other modules. >> >> Luckily, we now have iceberg-rust, which is not fully feature-complete >> but undergoing rapid development. We can develop a C++ binding based on >> iceberg-rust and establish a robust C++ API within the iceberg-cpp project. >> The C++ project will define public interfaces and offer various build >> targets for users to choose from. The overall process will be similar to >> how pyiceberg interacts with iceberg-rust. >> >> ## Plan >> >> >> 1. Use cxx[^1] to build cpp bindings for iceberg-rust which exposes core >> APIs. >> >> >> The binding will support different build profiles that users can choose >> to use fileio from iceberg-rust or not. >> >> 2. iceberg-cpp will define public interfaces and types for cpp users. >> >> After this separation, iceberg-rust can focus on implementing features >> and exposing them through simpler, more direct bindings. Meanwhile, >> iceberg-cpp will be responsible for designing, testing, and maintaining the >> user-level APIs, allowing users to select the parts they need or plug in >> their own file I/O components. >> >> ## Conclusion >> >> We believe this design combines the best parts of two projects and helps >> us iterate iceberg-cpp more quickly. >> >> Welcome to leave your comments here! >> >> [^1]: https://cxx.rs/ >> >> Xuanwo >> >> https://xuanwo.io/ >> >> Xuanwo >> >> https://xuanwo.io/ >> >> > > -- > Regards > Junwang Zhao >
