Thanks Russell for following up on this. On Wed, Jan 29, 2025 at 2:50 PM Russell Spitzer <russell.spit...@gmail.com> wrote:
> Given that I think we are all in agreement, and Micah has signed off on > the PR I think we can go ahead and merge this one. > > On Tue, Dec 10, 2024 at 12:16 AM Aihua Xu <aihu...@gmail.com> wrote: > >> Thank you all for the discussion. Micah raised a valid concern about >> including a specification that has not yet been finalized in Parquet. As we >> discussed earlier, the community has shown interest in introducing the >> basic variant type and shredding in V3. >> >> From my perspective, while the change is relatively simple and largely >> references the Parquet variant spec, it will still take time to get it >> thoroughly reviewed. My approach is to update the Iceberg spec to align >> with the current version, iteratively refine it as we make changes to the >> Parquet variant spec, and proceed with implementation in parallel. Hope >> that makes sense. >> >> Thanks, >> >> Aihua >> >> On Tue, Nov 26, 2024 at 11:10 AM Jack Ye <yezhao...@gmail.com> wrote: >> >>> I am +1 on adding it to the spec and not waiting for Parquet. It feels >>> like a better 2-way door decision compared to being blocked by Parquet >>> ratification timeline. >>> >>> -Jack >>> >>> On Tue, Nov 26, 2024 at 10:05 AM Micah Kornfield <emkornfi...@gmail.com> >>> wrote: >>> >>>> 2. We aren't going to formally close V3 Spec yet, so if we do end up in >>>>> a situation where we want to close the spec and Parquet has not removed >>>>> the >>>>> tag, we can remove the variant from the spec then. (I think that scenario >>>>> is unlikely) >>>> >>>> >>>> This seems like it puts the effort at the wrong side of things. While >>>> I agree it is probably low probability reviewing the whole V3 spec for >>>> completeness and making sure there are no loose ends makes it more likely >>>> to miss things like this. And if Variant ends up being the long poll of >>>> the release, it seems like we are just adding effort to shipping V3 (which >>>> already has a lot of other valuable additions). >>>> >>>> >>>>> 3. There is very little in our change set here that specifically >>>>> references the Parquet spec except for our reference link to it. >>>> >>>> >>>> This cuts both ways? What is the rush to get this into V3 if it can >>>> easily be merged once the Parquet side is official? >>>> >>>> Cheers, >>>> Micah >>>> >>>> On Mon, Nov 25, 2024 at 9:21 AM Russell Spitzer < >>>> russell.spit...@gmail.com> wrote: >>>> >>>>> I'm +1, >>>>> >>>>> 1. I don't think we are going to change our decision on whether to >>>>> include variants based on the timing of Parquet ratification >>>>> 2. We aren't going to formally close V3 Spec yet, so if we do end up >>>>> in a situation where we want to close the spec and Parquet has not removed >>>>> the tag, we can remove the variant from the spec then. (I think that >>>>> scenario is unlikely) >>>>> 3. There is very little in our change set here that specifically >>>>> references the Parquet spec except for our reference link to it. >>>>> >>>>> I don't think there is anything that will happen in the spec that will >>>>> change what we would include in the Iceberg Spec (especially in this PR) >>>>> >>>>> On Fri, Nov 22, 2024 at 5:10 PM Micah Kornfield <emkornfi...@gmail.com> >>>>> wrote: >>>>> >>>>>> My (non-binding) vote is -1 until the variant spec is formally >>>>>> adopted in Parquet. >>>>>> >>>>>> On Fri, Nov 22, 2024 at 2:51 PM Aihua Xu <aihu...@gmail.com> wrote: >>>>>> >>>>>>> Hi everyone, >>>>>>> >>>>>>> I've updated the Iceberg spec to include the new Variant type as >>>>>>> part of #10831 <https://github.com/apache/iceberg/pull/10831>. The >>>>>>> changes are basically complete. This is a heads-up about the upcoming >>>>>>> change. Please review and +1 to acknowledge, so we will merge. >>>>>>> >>>>>>> Thanks, >>>>>>> Aihua >>>>>>> >>>>>>
