The steps from here are for you all to get the codebase into a state that is ready for donation, including ASF license headers, etc. A pull request into apache/arrow would be the best thing
Then we have to do the following * Vote on the Arrow mailing list * Receive ICLAs from contributors * Complete IP clearance * Merge codebase Let me know when you are ready to move forward. From start to finish that can get done in approximately 6 days if the code is ready Thanks On Wed, Nov 21, 2018 at 9:45 AM Andy Grove <andygrov...@gmail.com> wrote: > > Renjie, > > Can you remind us all why nightly is required? > > My personal feeling is that stable is a nice-to-have, but Rust is still > moving fast and we are on the bleeding edge here so I'm OK with Arrow relying > on nightly for now. Maybe we can have a plan to transition back to stable for > a future release if we go with nightly now. > > Thanks, > > Andy. > > > On Wed, Nov 21, 2018 at 7:37 AM Renjie Liu <liurenjie2...@gmail.com> wrote: >> >> That sounds great. But parquet-rs currently relies on nightly rust, that >> would be the first problem to resolve. >> >> On Wed, Nov 21, 2018 at 4:49 AM Andy Grove <andygrov...@gmail.com> wrote: >> >> > This sounds like a great idea. >> > >> > With support for both CSV and Parquet in the Arrow crate, it would be nice >> > to design a standard interface for Arrow data sources. Maybe this is as >> > simple as implementing `Iterator<RecordBatch>`. >> > >> > Andy. >> > >> > On Tue, Nov 20, 2018 at 11:46 AM Chao Sun <sunc...@apache.org> wrote: >> > >> > > Yes, we'd be interested to move forward. I'm inclined to merge this into >> > > Arrow because of the issues that you pointed out with parquet c++ merge, >> > > and I do see a tight relationship between the two projects, and potential >> > > sharing of common libraries. @Ivan Sadikov <ivan.sadi...@gmail.com> what >> > > do you think? >> > > >> > > Chao >> > > >> > > On Tue, Nov 20, 2018 at 10:23 AM Wes McKinney <wesmck...@gmail.com> >> > wrote: >> > > >> > >> hi folks, >> > >> >> > >> Would you all be interested in moving forward the parquet-rs project? >> > >> I have a little more bandwidth to help with the code donation in the >> > >> next month or two. >> > >> >> > >> I know we voted on the Parquet mailing list about the donation >> > >> already. One big question is whether you want to create an >> > >> apache/parquet-rs repository or whether you want to co-develop >> > >> parquet-rs together with Arrow in Rust, similar to what we are doing >> > >> with C++. It's possible you might run into the same kinds of issues >> > >> that led us to consider the monorepo arrangement. >> > >> >> > >> Thanks >> > >> Wes >> > >> On Sun, Aug 19, 2018 at 11:11 PM Renjie Liu <liurenjie2...@gmail.com> >> > >> wrote: >> > >> > >> > >> > Hi, Chao: >> > >> > I've opened an jira issue for that and planning to work on that. >> > >> > >> > >> > On Mon, Aug 20, 2018 at 11:03 AM Renjie Liu <liurenjie2...@gmail.com> >> > >> wrote: >> > >> > >> > >> > > Yes, it's a mistake, sorry for that >> > >> > > >> > >> > > >> > >> > > On Mon, Aug 20, 2018 at 10:57 AM Chao Sun <sunc...@apache.org> >> > wrote: >> > >> > > >> > >> > >> (s/flink/arrow - it is a mistake?) >> > >> > >> >> > >> > >> Thanks Renjie for your interest. Yes, one of the next step in >> > >> parquet-rs >> > >> > >> is to integrate with Apache Arrow. Actually we just had a >> > discussion >> > >> > >> <https://github.com/sunchao/parquet-rs/issues/140> about this >> > >> recently. >> > >> > >> Feel free to share your comments on the github. >> > >> > >> >> > >> > >> Best, >> > >> > >> Chao >> > >> > >> >> > >> > >> On Sun, Aug 19, 2018 at 7:39 PM, Renjie Liu < >> > liurenjie2...@gmail.com >> > >> > >> > >> > >> wrote: >> > >> > >> >> > >> > >>> cc:Sunchao and Any >> > >> > >>> >> > >> > >>> >> > >> > >>> ---------- Forwarded message --------- >> > >> > >>> From: Uwe L. Korn <uw...@xhochy.com> >> > >> > >>> Date: Sun, Aug 19, 2018 at 5:08 PM >> > >> > >>> Subject: Re: [DISCUSS] Rust add adapter for parquet >> > >> > >>> To: <dev@arrow.apache.org> >> > >> > >>> >> > >> > >>> >> > >> > >>> Hello, >> > >> > >>> >> > >> > >>> you might also want to raise this with the >> > >> > >>> https://github.com/sunchao/parquet-rs project. The overlap >> > between >> > >> the >> > >> > >>> developers of this project and the Arrow Rust implementation is >> > >> quite large >> > >> > >>> but still it may make sense to also start a discussion there. >> > >> > >>> >> > >> > >>> Uwe >> > >> > >>> >> > >> > >>> On Thu, Aug 16, 2018, at 9:14 AM, Renjie Liu wrote: >> > >> > >>> > Hi, all: >> > >> > >>> > >> > >> > >>> > Now the rust component is approaching a stable state and rust >> > >> reader >> > >> > >>> for >> > >> > >>> > parquet is ready. I think it maybe a good time to start an >> > >> adapter for >> > >> > >>> > parquet, just like adapter for orc in cpp. How you guys think >> > >> about it? >> > >> > >>> > -- >> > >> > >>> > Liu, Renjie >> > >> > >>> > Software Engineer, MVAD >> > >> > >>> -- >> > >> > >>> Liu, Renjie >> > >> > >>> Software Engineer, MVAD >> > >> > >>> >> > >> > >> >> > >> > >> -- >> > >> > > Liu, Renjie >> > >> > > Software Engineer, MVAD >> > >> > > >> > >> > -- >> > >> > Liu, Renjie >> > >> > Software Engineer, MVAD >> > >> >> > > >> > >> -- >> Renjie Liu >> Software Engineer, MVAD
