So this vote is actually a vote for: "accept donation of variant code from spark project"
That's very different (to me) from "adopt variant from spark". I'm +1 (non-binding) for accepting donation of code from Spark. I'm -1 (non-binding) for adopting variant because I don't really understand what that means. On Wed, Sep 11, 2024 at 10:34 PM Antoine Pitrou <anto...@python.org> wrote: > > +0 on accepting Variant into the Parquet *project*, but that's not an > approval for sharing repos with the current Parquet format and > implementations. > > Also, I have the same impression of this vote being a bit prematurate. > Is the Variant type as proposed performant enough? Is it flexible > enough to enable interoperability with other systems than Spark? > > Regards > > Antoine. > > > On Thu, 12 Sep 2024 12:15:13 +0800 > Gang Wu <ust...@gmail.com> wrote: > > Sorry for the confusion. The intention of this vote is to formally accept > > the adoption from Spark and is a formal answer to the corresponding vote > > on the Spark side: > > https://lists.apache.org/thread/gqy02x1r5dj73woj4l8r0xxkrztd5qos > > Both parties should officially agree on the move before discussing the > > details. > > > > On Thu, Sep 12, 2024 at 7:21 AM Jacques Nadeau < > jacques-1odqgaof3lkdnm+yrof...@public.gmane.org> wrote: > > > > > > My personal opinion is that we've been jumping the gun on voting a > bit. I > > > do appreciate the enthusiasm though. :) > > > > > > Yeah, felt the same to me (non-binding). It's hard to fully understand > the > > > specific actions that are outcomes of this vote. > > > > > > On Wed, Sep 11, 2024 at 1:11 PM Julien Le Dem < > julien-1odqgaof3lkdnm+yrof...@public.gmane.org> wrote: > > > > > > > +1 (binding) on agreeing on principle to add Variant to Parquet. > > > > Now on the specific plan, > > > > > > > > > For repositories to host the Variant specification and library: > > > > > - apache/parquet-format will add documentation for the > specification > > > > > - apache/parquet-java will add a new module for the Java > implementation > > > > > > > > Gene has posted his doc with a plan "[DISCUSS] Moving Variant to > Parquet > > > > Details" to collect feedback. > > > > Once he's done integrating the feedback and it's finalized, that > will be > > > > the plan on how to do it. > > > > The doc itself is a better reference on how it's going to happen. > Maybe > > > > that's a better artifact to vote on. > > > > > > > > My personal opinion is that we've been jumping the gun on voting a > bit. I > > > > do appreciate the enthusiasm though. :) > > > > Voting is more of a procedural mechanism to formally record that > we've > > > > achieved consensus. > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 11:45 AM Micah Kornfield < > emkornfi...@gmail.com> > > > > wrote: > > > > > > > > > +1 (binding) in principle on adding it. I think there are still > a > > > number > > > > > of issues to be worked out and we should try to come to a > consensus in > > > > > Gene's doc [1] + discussion thread on the nitty gritty of what this > > > > > proposal actually means. > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1guEzBQjzOEEZvvibeZjNraKmZHWtxQR95O_DvtZU0xw/edit#heading=h.5ad5xy8ox6bp > > > > > > > > > > > On Wed, Sep 11, 2024 at 9:21 AM Nong Li < > nongli-re5jqeeqqe8avxtiumw...@public.gmane.org> wrote: > > > > > > > > > > > +1 > > > > > > > > > > > > On Wed, Sep 11, 2024 at 8:53 AM Gang Wu < > ustcwg-re5jqeeqqe8avxtiumw...@public.gmane.org> wrote: > > > > > > > > > > > > > Let's just vote for the adoption in this thread and discuss > the > > > > > location > > > > > > in > > > > > > > > https://lists.apache.org/thread/xwd3mqjr9bdpg3jcnlprbyb4x09c9ymj > > > > > > > > > > > > > > Cast my own vote: +1 for adding the variant spec to parquet > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > Gang > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 5:27 PM Eduard Tudenhöfner < > > > > > > > etudenhoef...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > > > +1 (non-binding) for adding the variant spec to parquet > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 11:08 PM Daniel Weeks < > dwe...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > +1 on adding the variant spec to Parquet > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 12:23 PM Russell Spitzer < > > > > > > > > > russell.spit...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > +1 (Non-binding) This will be great for universal > adoption of > > > > the > > > > > > > > variant > > > > > > > > > > type > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 2:14 PM > rdblue-re5jqeeqqe8avxtiumw...@public.gmane.org < > > > > > rdb...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > +1 for adding the variant spec to Parquet. I'm > looking > > > > forward > > > > > to > > > > > > > > > working > > > > > > > > > > > on the addition of shredding. > > > > > > > > > > > > > > > > > > > > > > As for the details, I think I also prefer a separate > > > > > repository, > > > > > > > > > > > `parquet-variant`, but I don't think we necessarily > need to > > > > > > > determine > > > > > > > > > > that > > > > > > > > > > > question up front. > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 9:05 AM Gang Wu < > ust...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Antoine, > > > > > > > > > > > > > > > > > > > > > > > > A separate project was my 1st proposal in the > original > > > > > > discussion > > > > > > > > > > > > on the dev@iceberg ML :). > > > > > > > > > > > > > > > > > > > > > > > > TBH, I'm open to putting them either in existing > repos > > > or a > > > > > > > > dedicated > > > > > > > > > > > > parquet-variant repo. The intention of this thread > is to > > > > try > > > > > to > > > > > > > > push > > > > > > > > > > the > > > > > > > > > > > > discussion to reach a consensus. > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 11:54 PM Antoine Pitrou < > > > > > > > > anto...@python.org> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Gang, > > > > > > > > > > > > > > > > > > > > > > > > > > Assuming we do want to adopt this in Parquet, I > would > > > > very > > > > > > much > > > > > > > > > > > > > recommend separate repositories for this. Putting > the > > > > spec > > > > > > > inside > > > > > > > > > > > > > `parquet-format` breeds confusion, IMHO, and may > > > > discourage > > > > > > > third > > > > > > > > > > > > > parties from considering this standalone, > non-Parquet, > > > > data > > > > > > > > format. > > > > > > > > > > > > > > > > > > > > > > > > > > (but for the same reason, I would recommand a > separate > > > > > > project > > > > > > > as > > > > > > > > > > well > > > > > > > > > > > > > :-)) > > > > > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 10 Sep 2024 09:48:03 +0800 > > > > > > > > > > > > > Gang Wu < > ustcwg-re5jqeeqqe8avxtiumw...@public.gmane.org> wrote: > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > I’d like to start a vote for adopting the > Variant > > > > > > > specification > > > > > > > > > and > > > > > > > > > > > > > library > > > > > > > > > > > > > > from > > > > > > > > > > > > > > the Spark project. This allows the Variant > binary > > > > format > > > > > > and > > > > > > > > > > > shredding > > > > > > > > > > > > > > format > > > > > > > > > > > > > > to be more broadly used by other interested > projects > > > > and > > > > > > > > systems. > > > > > > > > > > > > > > > > > > > > > > > > > > > > For repositories to host the Variant > specification > > > and > > > > > > > library: > > > > > > > > > > > > > > - apache/parquet-format will add documentation > for > > > the > > > > > > > > > > specification > > > > > > > > > > > > > > - apache/parquet-java will add a new module for > the > > > > Java > > > > > > > > > > > implementation > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please refer to the discussion thread: > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread/6h58hj39lhqtcyd2hlsyvqm4lzdh4b9z > > > > > > > > > > > > > > > > > > > > > > > > > > > > This vote will be open for at least 72 hours. > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ ] +1: Accept the proposal > > > > > > > > > > > > > > [ ] +0 > > > > > > > > > > > > > > [ ] -1: I don’t think this is a good idea > because … > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
