I hope we include Spark structured streaming read as well in the next release; that was proposed in Feb this year and still around. Quoting my comment on benefit of the streaming read on Spark;
This would be the major feature to cover the gap on use case for structured > streaming between Delta Lake and Iceberg. There's a technical limitation on > Spark structured streaming itself (global watermark), which requires > workaround via splitting query into multiple queries & intermediate storage > supporting end-to-end exactly once. Delta Lake covers the case, and I > really would like to see the case also covered by Iceberg. > I see there're lots of works in progress on the milestone (and these are > great features which should be done), but after this we cover both batch > and streaming workloads being done with Spark, which is a huge step forward > on Spark users. Thanks, Jungtaek Lim (HeartSaVioR) On Thu, Aug 27, 2020 at 1:13 AM Ryan Blue <rb...@netflix.com.invalid> wrote: > Hi Marton, > > 0.9.0 was released about 6 weeks ago, so I don't think we've planned when > the next release will be yet. I think it's a good idea to release soon, > though. The Flink sink is close to being ready as well and I'd like to get > both of those released so that the contributors can start using them. > > Seems like a good question for the broader community: how about a release > in the next month or so for Hive reads and the Flink sink? > > rb > > On Wed, Aug 26, 2020 at 8:58 AM Marton Bod <marton....@gmail.com> wrote: > >> Hi Team, >> >> I was wondering whether there is a release cadence already in place for >> Iceberg, e.g. how often releases will take place approximately? Which >> commits/features as release candidates in the near term? >> >> We're looking to integrate Iceberg into Hive, however, the current 0.9.1 >> release does not yet contain the StorageHandler code in iceberg-mr. Knowing >> the approximate release timelines would help greatly with our integration >> planning. >> >> Of course, happy to get involved with ongoing dev/stability efforts to >> help achieve a new release of this module. >> >> Thanks a lot, >> Marton >> > > > -- > Ryan Blue > Software Engineer > Netflix >
