Hi Bryan, Any update on this thread ? Can I help somehow ?
Thanks, Regards JB On Mon, Oct 2, 2023 at 7:39 PM Bryan Keller <brya...@gmail.com> wrote: > > Hi all, > > We at Tabular would like to contribute our Kafka Connect Iceberg sink to the > Iceberg project. It would be great to give Iceberg users another option for > landing data from Kafka into Iceberg tables that is supported by the Iceberg > community. Kafka Connect is a part of systems from AWS, Confluent, Redpanda, > and so on, so it can make landing data from Kafka into Iceberg much easier > for those without a Flink or Spark infrastructure. > > There are a few Iceberg sink implementations out there for Kafka Connect, but > we feel this one covers most of the features users have requested, such as > exactly-once processing, schema evolution, and multi-table fanout. And having > the sink backed by the Iceberg community will help it to evolve and improve > over time. > > If this sounds like something everyone would like to see added to Iceberg, > I've opened a PR that includes some initial pieces of the sink. The thought > was to break up the submission into parts so each could be reviewed more > easily. Some design docs and notes can be found in the original repo here: > https://github.com/tabular-io/iceberg-kafka-connect/tree/main/docs > > We'd like to get feedback if others approve of moving forward with this or > not. > > Thanks, > Bryan >
