Hi Team, The File Format API is merged, so that should not block the progress on the Comet integration progress anymore. If you need help, let me know. Thanks, Peter
Andy Grove <[email protected]> ezt írta (időpont: 2026. jan. 22., Cs, 21:51): > I added another comment [1] on the issue [2], but will share here as well > for maximum visibility. > > I have two PRs in Comet that I am looking for reviews on. > > The first PR adds an @IcebergApi annotation to all Java > classes/methods/fields that are currently referenced from the Iceberg main > branch, and also adds documentation to the contributors guide. > #3237 <https://github.com/apache/datafusion-comet/pull/3237> > > The second PR builds on this and adds new dedicated unit tests for API > stability. This PR is more consequential because it discusses > deprecating/removing Comet's native_comet scan and how that relates to > the Iceberg API. > #3240 <https://github.com/apache/datafusion-comet/pull/3240> > > I am not an expert on the current integration, so it is possible that I > may have misunderstood things. I would appreciate reviews from both the > Iceberg and Comet communities! > Thanks, > > Andy. > > [1] > https://github.com/apache/datafusion-comet/issues/2921#issuecomment-3785878915 > [2] https://github.com/apache/datafusion-comet/issues/2921 > > On Tue, Dec 23, 2025 at 1:20 PM Kevin Liu <[email protected]> wrote: > >> Thanks for starting the thread! I've added a comment on the Github issue. >> Super exciting to see the iceberg-rust integration with Comet. >> As mentioned, I'll help take a look at >> https://github.com/apache/iceberg/pull/13786 and see if we can bring it >> to the finish line. >> >> Best, >> Kevin Liu >> >> On Thu, Dec 18, 2025 at 6:30 PM Renjie Liu <[email protected]> >> wrote: >> >>> Thanks Andy for raising this. >>> >>> +1 for the iceberg-rust solution. This could benefit the whole oss >>> community in the long term and avoid duplicated efforts. As with the >>> technique challenges you mentioned, this seems more like a missing feature >>> rather than limitation. >>> >>> On Thu, Dec 18, 2025 at 5:58 AM huaxin gao <[email protected]> >>> wrote: >>> >>>> Thanks Andy for starting this discussion! >>>> >>>> I agree we should prioritize the iceberg-rust path as the long term >>>> default for Comet/Iceberg scans, while it's still valuable to keep the >>>> Iceberg Java integration as an option, especially for users who need >>>> JVM-specific features. I'm hoping we can get consensus on a path to resolve >>>> the shading issues (e.g. the approach proposed in >>>> github.com/apache/iceberg/pull/13786) so the java option can move >>>> forward. >>>> >>>> Thanks, >>>> Huaxin >>>> >>>> On Wed, Dec 17, 2025 at 12:23 PM Shawn Chang <[email protected]> >>>> wrote: >>>> >>>>> Thanks Andy for starting this thread! >>>>> >>>>> I've replied to the issue above but am posting here as well for better >>>>> visibility: >>>>> >>>>> I’m leaning toward prioritizing the iceberg-rust path to avoid the >>>>> shading and circular dependency headaches of the Java approach. Easier >>>>> maintenance typically lowers the entry barrier and encourages more >>>>> community involvement, which is important for the project’s long-term >>>>> health. >>>>> >>>>> It also makes sense to keep the Iceberg Java integration available as >>>>> an experimental option for users who need its richer JVM-specific features >>>>> while Rust becomes the long-term default. We don’t want to block >>>>> contributions from anyone who truly needs the Java path. >>>>> >>>>> Best, >>>>> Shawn >>>>> >>>>> On Wed, Dec 17, 2025 at 11:00 AM Andy Grove <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> There is a discussion in the DataFusion Comet community about the >>>>>> future direction of our Iceberg support [1]. For example, should we >>>>>> continue the efforts to integrate with the Iceberg Java project or focus >>>>>> more on the iceberg-rust project. >>>>>> >>>>>> It would be great to also get input from the Iceberg community. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Andy. >>>>>> >>>>>> [1] https://github.com/apache/datafusion-comet/issues/2921 >>>>>> >>>>>
