Hi Peter It makes sense to me. I'm happy to help on Flink and Kafka Connect connector to use FileWriterFactory. We can already flag with deprecated and change in Flink and Kafka Connect in the meantime (I don't think we have a strong dependency between the two actions).
Thanks! Regards JB On Fri, Oct 10, 2025 at 11:18 AM Péter Váry <peter.vary.apa...@gmail.com> wrote: > > Hi everyone, > > Long time ago, Anton introduced FileWriterFactory [1] as a replacement for > FileAppenderFactory for creating Iceberg data and delete files. While Spark > has been migrated to use the new factory, both Flink and the Kafka connector > still rely on FileAppenderFactory. > > As part of the ongoing discussions [2] around the File Format API proposal, > the topic of these factories came up [3]. The current plan is to migrate only > FileWriterFactory to the new API. To support this, the suggestion is to > deprecate FileAppenderFactory and migrate Flink and Kafka to use > FileWriterFactory. I’ve opened a PR to migrate Flink to use > FlinkFileWriterFactory [4] > The change is relatively straightforward, and we can create a similar PR for > Kafka as well. > > Questions for the community: > > Are you aware of any users currently relying on FileAppenderFactory? > Are there any blockers preventing them from switching to FileWriterFactory? > > > Deprecation timeline: > If we decide to deprecate FileAppenderFactory, we have a couple of options: > > Deprecate in 1.11.0 and remove in 1.12.0 > Or keep it until 2.0 > > > What are your thoughts on the timeline? > > We’ll also discuss this topic during the next File Format API sync on Monday, > October 13 at 9:00 AM PST. Join the meeting here: > https://meet.google.com/fow-bauj-mmg > > Looking forward to your feedback! > > Thanks, > Peter > > [1] - PR: Core: Add WriterFactory - > https://github.com/apache/iceberg/pull/2873 > [2] - Dev list discussion - > https://lists.apache.org/thread/ovyh52m2b6c1hrg4fhw3rx92bzr793n2 > [3] - Iceberg DataFile reader and writer API proposal document / Open > questions - > https://docs.google.com/document/d/1sF_d4tFxJsZWsZFCyCL9ZE7YuI7-P3VrzMLIrrTIxds/edit?pli=1&tab=t.0#heading=h.fntuqbcl74w1 > [4] - PR: Flink: Move write from AppenderFactory to FileWriterFactory - > https://github.com/apache/iceberg/pull/14271 >
