Re: [PROPOSAL] Release process to improve communication of the Iceberg project

I'm +1 to the proposal, I think it would greatly help the community in seeing what will be shipped with the next and future release(s). On Wed, Nov 1, 2023 at 4:47 PM Brian Olsen wrote: > Hey Iceberg Nation, > > Last proposal from me today I promise! Another issue I've seen as I've > looked over

Re: [PROPOSAL] Use Microsoft Style Guide for documentation

+1 to the proposal On Wed, Nov 1, 2023 at 11:14 PM Yufei Gu wrote: > +1 Love the following example. Not sure if Vale can catch this and provide > suggestions. It may be only possible with LLM. > >> Replace this: If you're ready to purchase Office 365 for your >> organization, contact your Micros

Re: [Discussion] Move `iceberg-parquet` and `iceberg-orc` modules into `iceberg-core`

Hey Ajantha, Wouldn't this require a major version bump considering this is a breaking change for users depending on iceberg-parquet or iceberg-orc now? Gabor On Thu, Nov 2, 2023 at 3:01 AM Ajantha Bhat wrote: > Hi Everyone, > > At present, Iceberg exclusively utilizes Avro, JSON, and Puffin f

Re: [Discussion] Move `iceberg-parquet` and `iceberg-orc` modules into `iceberg-core`

Hey Gabor, Given that we are not deprecating or altering any classes, and there are no changes to package names, I believed it would be acceptable to include these changes in minor versions (with appropriate updates in the release notes). Moreover, everyone is already reliant on the `iceberg-core`

Re: [Discussion] Move `iceberg-parquet` and `iceberg-orc` modules into `iceberg-core`

Hi: Could you provide concrete cases to elaborate this change? On Thu, Nov 2, 2023 at 4:22 PM Gabor Kaszab wrote: > Hey Ajantha, > > Wouldn't this require a major version bump considering this is a breaking > change for users depending on iceberg-parquet or iceberg-orc now? > > Gabor > > On Thu

Re: [Discussion] Move `iceberg-parquet` and `iceberg-orc` modules into `iceberg-core`

Hi Renjie, I have highlighted the use case from the above mail, > > *However, with the addition of partition statistics > , > Iceberg's metadata (stats file) will berepresented in Parquet or ORC > formats.* > T

Re: [PROPOSAL] Use Microsoft Style Guide for documentation

+1 > On Nov 1, 2023, at 6:13 PM, Yufei Gu wrote: > > +1 Love the following example. Not sure if Vale can catch this and provide > suggestions. It may be only possible with LLM. >> Replace this: If you're ready to purchase Office 365 for your organization, >> contact your Microsoft account repr

Re: [Discussion] Move `iceberg-parquet` and `iceberg-orc` modules into `iceberg-core`

Is there an alternative where we do an implementation similar to how Position Deletes and Data Files are currently written? Like we have the more generic "writers" in core but the actual implementations still live in iceberg-parquet or iceberg-orc? > On Nov 2, 2023, at 9:38 AM, Ajantha Bhat wr

Re: [VOTE] Release Apache Iceberg 1.4.2 RC0

Thanks Ajantha, I've reached out to a few more PMCs to see about getting some more votes casted. A few folks are voting on it, really appreciate everyone's help on this release. Thanks, Amogh Jahagirdar On Wed, Nov 1, 2023 at 9:47 PM Ajantha Bhat wrote: > Friendly reminder: We are still in nee

Re: [VOTE] Release Apache Iceberg 1.4.2 RC0

+1 Thanks for getting this fix out, Amogh! On Thu, Nov 2, 2023 at 9:19 AM Amogh Jahagirdar wrote: > Thanks Ajantha, I've reached out to a few more PMCs to see about getting > some more votes casted. A few folks are voting on it, really appreciate > everyone's help on this release. > > Thanks, >

Updating the Iceberg table architecture diagram

Hey all, The current architecture diagram for an iceberg table hasn't been updated in over 3 years, and there's are some aspects to the architecture of an iceberg table that have changed, most notably delete files and puffin files. since this d

Re: [VOTE] Release Apache Iceberg 1.4.2 RC0

+1 - Checked all the normal things (Checksum, Tests, Rat) > On Nov 2, 2023, at 12:38 PM, Ryan Blue wrote: > > +1 > > Thanks for getting this fix out, Amogh! > > On Thu, Nov 2, 2023 at 9:19 AM Amogh Jahagirdar > wrote: >> Thanks Ajantha, I've reached out to a few more

Re: [PROPOSAL] Use Microsoft Style Guide for documentation

@Yufei, Regarding: > Love the following example. Not sure if Vale can catch this and provide > suggestions. It may be only possible with LLM. > >> Replace this: If you're ready to purchase Office 365 for your >> organization, contact your Microsoft account representative. >> With this: Ready to b

[ANNOUNCE] Apache Iceberg release 1.4.2

I'm pleased to announce the release of Apache Iceberg 1.4.2! Apache Iceberg is an open table format for huge analytic datasets. Iceberg delivers high query performance for tables with tens of petabytes of data, along with atomic commits, concurrent writes, and SQL-compatible table evolution. This

Re: [VOTE] Release Apache Iceberg 1.4.2 RC0

Thanks everyone who participated in the vote for Release Apache Iceberg 1.4.2 RC0. The vote result is: +1: 4 (binding), 5 (non-binding) +0: 0 (binding), 0 (non-binding) -1: 0 (binding), 0 (non-binding) Therefore, the release candidate is passed. On Thu, Nov 2, 2023 at 11:19 AM Russell Spitzer

Re: [Discussion] Move `iceberg-parquet` and `iceberg-orc` modules into `iceberg-core`

> > Is there an alternative where we do an implementation similar to how > Position Deletes and Data Files are currently written? Like we have the > more generic "writers" in core but the actual implementations still live in > iceberg-parquet or iceberg-orc? Hi Russell, Let me explore this path a

Re: [Discussion] Move `iceberg-parquet` and `iceberg-orc` modules into `iceberg-core`

> > Is there an alternative where we do an implementation similar to how > Position Deletes and Data Files are currently written? Like we have the > more generic "writers" in core but the actual implementations still live in > iceberg-parquet or iceberg-orc? +1. What I'm thinking is also extracti

MOR CDC view support

Hi community, I wonder if anyone is interested in having a MOR CDC view feature? My organization is interested in using Flink upsert (MOR) into the Iceberg table, but currently the MOR CDC view is not supported. If we were to support it, do you know how much work it will be? How difficult will th

Re: MOR CDC view support

Feature request ticket: https://github.com/apache/iceberg/issues/8975 On Thu, Nov 2, 2023 at 9:16 PM Pucheng Yang wrote: > Hi community, > > I wonder if anyone is interested in having a MOR CDC view feature? My > organization is interested in using Flink upsert (MOR) into the Iceberg > table, bu