To eliminate any confusion, let's CANCEL this vote. I will not send a separate thread with [VOTE][RESULT] title because we have actually received enough binding votes to accept the donation of variant spec and code from Spark. The Parquet and Spark communities will work together to sort things out. A new vote email will be sent once we have reached consensus. Thanks everyone for voting and feedback!
Best, Gang On Mon, Sep 16, 2024 at 5:42 PM Antoine Pitrou <anto...@python.org> wrote: > > So, I agree with Jacques that: > > 1) there should be two separate votes *at least*: one for adopting > the spec itself, one for each donated implementation > > 2) there should be more explanation of what this implies for people not > really familiar with Spark > > Regards > > Antoine. > > > On Fri, 13 Sep 2024 15:22:43 -0700 > "rdb...@gmail.com" > <rdb...@gmail.com> wrote: > > Yes, I think this is a vote for maintaining the variant spec here in the > > Parquet community and accepting the code that goes along with it. > > > > On Fri, Sep 13, 2024 at 12:18 PM Jacques Nadeau < > jacques-1odqgaof3lkdnm+yrof...@public.gmane.org> wrote: > > > > > So this vote is actually a vote for: > > > "accept donation of variant code from spark project" > > > > > > That's very different (to me) from "adopt variant from spark". > > > > > > I'm +1 (non-binding) for accepting donation of code from Spark. I'm -1 > > > (non-binding) for adopting variant because I don't really understand > what > > > that means. > > > > > > > > > > > > On Wed, Sep 11, 2024 at 10:34 PM Antoine Pitrou <anto...@python.org> > > > wrote: > > > > > > > > > > > +0 on accepting Variant into the Parquet *project*, but that's not an > > > > approval for sharing repos with the current Parquet format and > > > > implementations. > > > > > > > > Also, I have the same impression of this vote being a bit > prematurate. > > > > Is the Variant type as proposed performant enough? Is it flexible > > > > enough to enable interoperability with other systems than Spark? > > > > > > > > Regards > > > > > > > > Antoine. > > > > > > > > > > > > On Thu, 12 Sep 2024 12:15:13 +0800 > > > > Gang Wu <ust...@gmail.com> wrote: > > > > > Sorry for the confusion. The intention of this vote is to > formally > > > accept > > > > > the adoption from Spark and is a formal answer to the > corresponding > > > vote > > > > > on the Spark side: > > > > > https://lists.apache.org/thread/gqy02x1r5dj73woj4l8r0xxkrztd5qos > > > > > Both parties should officially agree on the move before discussing > the > > > > > details. > > > > > > > > > > On Thu, Sep 12, 2024 at 7:21 AM Jacques Nadeau < > > > > > jacques-1odqgaof3lkdnm+yrofe0a-xmd5yjdbdmrexy1tmh2...@public.gmane.org> > wrote: > > > > > > > > > > > > My personal opinion is that we've been jumping the gun on > voting a > > > > bit. I > > > > > > do appreciate the enthusiasm though. :) > > > > > > > > > > > > Yeah, felt the same to me (non-binding). It's hard to fully > > > understand > > > > the > > > > > > specific actions that are outcomes of this vote. > > > > > > > > > > > > On Wed, Sep 11, 2024 at 1:11 PM Julien Le Dem < > > > > julien-1odqgaof3lkdnm+yrof...@public.gmane.org> wrote: > > > > > > > > > > > > > +1 (binding) on agreeing on principle to add Variant to > Parquet. > > > > > > > Now on the specific plan, > > > > > > > > > > > > > > > For repositories to host the Variant specification and > library: > > > > > > > > - apache/parquet-format will add documentation for the > > > > specification > > > > > > > > - apache/parquet-java will add a new module for the Java > > > > implementation > > > > > > > > > > > > > > Gene has posted his doc with a plan "[DISCUSS] Moving Variant > to > > > > Parquet > > > > > > > Details" to collect feedback. > > > > > > > Once he's done integrating the feedback and it's finalized, > that > > > > will be > > > > > > > the plan on how to do it. > > > > > > > The doc itself is a better reference on how it's going to > happen. > > > > Maybe > > > > > > > that's a better artifact to vote on. > > > > > > > > > > > > > > My personal opinion is that we've been jumping the gun on > voting a > > > > bit. I > > > > > > > do appreciate the enthusiasm though. :) > > > > > > > Voting is more of a procedural mechanism to formally record > that > > > > we've > > > > > > > achieved consensus. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 11:45 AM Micah Kornfield < > > > > emkornfi...@gmail.com> > > > > > > > wrote: > > > > > > > > > > > > > > > +1 (binding) in principle on adding it. I think there are > still > > > > a > > > > > > number > > > > > > > > of issues to be worked out and we should try to come to a > > > > consensus in > > > > > > > > Gene's doc [1] + discussion thread on the nitty gritty of > what > > > this > > > > > > > > proposal actually means. > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://docs.google.com/document/d/1guEzBQjzOEEZvvibeZjNraKmZHWtxQR95O_DvtZU0xw/edit#heading=h.5ad5xy8ox6bp > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 9:21 AM Nong Li < > > > > nongli-re5jqeeqqe8avxtiumw...@public.gmane.org> wrote: > > > > > > > > > > > > > > > > > +1 > > > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 8:53 AM Gang Wu < > > > > ustcwg-re5jqeeqqe8avxtiumw...@public.gmane.org> wrote: > > > > > > > > > > > > > > > > > > > Let's just vote for the adoption in this thread and > discuss > > > > the > > > > > > > > location > > > > > > > > > in > > > > > > > > > > > > > > https://lists.apache.org/thread/xwd3mqjr9bdpg3jcnlprbyb4x09c9ymj > > > > > > > > > > > > > > > > > > > > Cast my own vote: +1 for adding the variant spec to > parquet > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > On Wed, Sep 11, 2024 at 5:27 PM Eduard Tudenhöfner < > > > > > > > > > > etudenhoef...@apache.org> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > +1 (non-binding) for adding the variant spec to parquet > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 11:08 PM Daniel Weeks < > > > > dwe...@apache.org > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > +1 on adding the variant spec to Parquet > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 12:23 PM Russell Spitzer < > > > > > > > > > > > > > russell.spitzer-re5jqeeqqe8avxtiumw...@public.gmane.org> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > +1 (Non-binding) This will be great for universal > > > > adoption of > > > > > > > the > > > > > > > > > > > variant > > > > > > > > > > > > > type > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 2:14 PM > > > > rdblue-re5jqeeqqe8avxtiumw...@public.gmane.org < > > > > > > > > rdb...@gmail.com > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > +1 for adding the variant spec to Parquet. I'm > > > > looking > > > > > > > forward > > > > > > > > to > > > > > > > > > > > > working > > > > > > > > > > > > > > on the addition of shredding. > > > > > > > > > > > > > > > > > > > > > > > > > > > > As for the details, I think I also prefer a > separate > > > > > > > > repository, > > > > > > > > > > > > > > `parquet-variant`, but I don't think we > necessarily > > > > need to > > > > > > > > > > determine > > > > > > > > > > > > > that > > > > > > > > > > > > > > question up front. > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 9:05 AM Gang Wu < > > > > ust...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Antoine, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > A separate project was my 1st proposal in the > > > > original > > > > > > > > > discussion > > > > > > > > > > > > > > > on the dev@iceberg ML :). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > TBH, I'm open to putting them either in > existing > > > > repos > > > > > > or a > > > > > > > > > > > dedicated > > > > > > > > > > > > > > > parquet-variant repo. The intention of this > thread > > > > is to > > > > > > > try > > > > > > > > to > > > > > > > > > > > push > > > > > > > > > > > > > the > > > > > > > > > > > > > > > discussion to reach a consensus. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 10, 2024 at 11:54 PM Antoine > Pitrou < > > > > > > > > > > > anto...@python.org> > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Gang, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Assuming we do want to adopt this in > Parquet, I > > > > would > > > > > > > very > > > > > > > > > much > > > > > > > > > > > > > > > > recommend separate repositories for this. > Putting > > > > the > > > > > > > spec > > > > > > > > > > inside > > > > > > > > > > > > > > > > `parquet-format` breeds confusion, IMHO, and > may > > > > > > > discourage > > > > > > > > > > third > > > > > > > > > > > > > > > > parties from considering this standalone, > > > > non-Parquet, > > > > > > > data > > > > > > > > > > > format. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > (but for the same reason, I would recommand > a > > > > separate > > > > > > > > > project > > > > > > > > > > as > > > > > > > > > > > > > well > > > > > > > > > > > > > > > > :-)) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 10 Sep 2024 09:48:03 +0800 > > > > > > > > > > > > > > > > Gang Wu < > > > > ustcwg-re5jqeeqqe8avxtiumw...@public.gmane.org> wrote: > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I’d like to start a vote for adopting the > > > > Variant > > > > > > > > > > specification > > > > > > > > > > > > and > > > > > > > > > > > > > > > > library > > > > > > > > > > > > > > > > > from > > > > > > > > > > > > > > > > > the Spark project. This allows the > Variant > > > > binary > > > > > > > format > > > > > > > > > and > > > > > > > > > > > > > > shredding > > > > > > > > > > > > > > > > > format > > > > > > > > > > > > > > > > > to be more broadly used by other > interested > > > > projects > > > > > > > and > > > > > > > > > > > systems. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > For repositories to host the Variant > > > > specification > > > > > > and > > > > > > > > > > library: > > > > > > > > > > > > > > > > > - apache/parquet-format will add > documentation > > > > for > > > > > > the > > > > > > > > > > > > > specification > > > > > > > > > > > > > > > > > - apache/parquet-java will add a new > module for > > > > the > > > > > > > Java > > > > > > > > > > > > > > implementation > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Please refer to the discussion thread: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread/6h58hj39lhqtcyd2hlsyvqm4lzdh4b9z > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This vote will be open for at least 72 > hours. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [ ] +1: Accept the proposal > > > > > > > > > > > > > > > > > [ ] +0 > > > > > > > > > > > > > > > > > [ ] -1: I don’t think this is a good idea > > > > because … > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > Gang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
