> Can you remind us all why nightly is required? Here's a tracking issue <https://github.com/sunchao/parquet-rs/issues/119> for all the unstable features parquet-rs uses. I'm personally inclined to use nightly since some new features such as specialization makes development much easier. Like Andy mentioned, as we are still developing arrow + parquet I think we can stay with nightly and transition back to stable once major features are implemented and more and more people start to use it (hopefully by that time the unstable features are stabilized). Moreover, I've seen quite a few popular projects rely on nightly such as rocket, tikv, etc., so seems it is not uncommon in the Rust world.
> The steps from here are for you all to get the codebase into a state that is ready for donation, including ASF license headers, etc. A pull request into apache/arrow would be the best thing Sure. I'll prepare a pull request in the next few days, and then we can proceed to the voting, ICLA, etc. Thanks. Chao On Wed, Nov 21, 2018 at 7:55 AM Wes McKinney <wesmck...@gmail.com> wrote: > The steps from here are for you all to get the codebase into a state > that is ready for donation, including ASF license headers, etc. A pull > request into apache/arrow would be the best thing > > Then we have to do the following > > * Vote on the Arrow mailing list > * Receive ICLAs from contributors > * Complete IP clearance > * Merge codebase > > Let me know when you are ready to move forward. From start to finish > that can get done in approximately 6 days if the code is ready > > Thanks > On Wed, Nov 21, 2018 at 9:45 AM Andy Grove <andygrov...@gmail.com> wrote: > > > > Renjie, > > > > Can you remind us all why nightly is required? > > > > My personal feeling is that stable is a nice-to-have, but Rust is still > moving fast and we are on the bleeding edge here so I'm OK with Arrow > relying on nightly for now. Maybe we can have a plan to transition back to > stable for a future release if we go with nightly now. > > > > Thanks, > > > > Andy. > > > > > > On Wed, Nov 21, 2018 at 7:37 AM Renjie Liu <liurenjie2...@gmail.com> > wrote: > >> > >> That sounds great. But parquet-rs currently relies on nightly rust, that > >> would be the first problem to resolve. > >> > >> On Wed, Nov 21, 2018 at 4:49 AM Andy Grove <andygrov...@gmail.com> > wrote: > >> > >> > This sounds like a great idea. > >> > > >> > With support for both CSV and Parquet in the Arrow crate, it would be > nice > >> > to design a standard interface for Arrow data sources. Maybe this is > as > >> > simple as implementing `Iterator<RecordBatch>`. > >> > > >> > Andy. > >> > > >> > On Tue, Nov 20, 2018 at 11:46 AM Chao Sun <sunc...@apache.org> wrote: > >> > > >> > > Yes, we'd be interested to move forward. I'm inclined to merge this > into > >> > > Arrow because of the issues that you pointed out with parquet c++ > merge, > >> > > and I do see a tight relationship between the two projects, and > potential > >> > > sharing of common libraries. @Ivan Sadikov <ivan.sadi...@gmail.com> > what > >> > > do you think? > >> > > > >> > > Chao > >> > > > >> > > On Tue, Nov 20, 2018 at 10:23 AM Wes McKinney <wesmck...@gmail.com> > >> > wrote: > >> > > > >> > >> hi folks, > >> > >> > >> > >> Would you all be interested in moving forward the parquet-rs > project? > >> > >> I have a little more bandwidth to help with the code donation in > the > >> > >> next month or two. > >> > >> > >> > >> I know we voted on the Parquet mailing list about the donation > >> > >> already. One big question is whether you want to create an > >> > >> apache/parquet-rs repository or whether you want to co-develop > >> > >> parquet-rs together with Arrow in Rust, similar to what we are > doing > >> > >> with C++. It's possible you might run into the same kinds of issues > >> > >> that led us to consider the monorepo arrangement. > >> > >> > >> > >> Thanks > >> > >> Wes > >> > >> On Sun, Aug 19, 2018 at 11:11 PM Renjie Liu < > liurenjie2...@gmail.com> > >> > >> wrote: > >> > >> > > >> > >> > Hi, Chao: > >> > >> > I've opened an jira issue for that and planning to work on that. > >> > >> > > >> > >> > On Mon, Aug 20, 2018 at 11:03 AM Renjie Liu < > liurenjie2...@gmail.com> > >> > >> wrote: > >> > >> > > >> > >> > > Yes, it's a mistake, sorry for that > >> > >> > > > >> > >> > > > >> > >> > > On Mon, Aug 20, 2018 at 10:57 AM Chao Sun <sunc...@apache.org> > >> > wrote: > >> > >> > > > >> > >> > >> (s/flink/arrow - it is a mistake?) > >> > >> > >> > >> > >> > >> Thanks Renjie for your interest. Yes, one of the next step in > >> > >> parquet-rs > >> > >> > >> is to integrate with Apache Arrow. Actually we just had a > >> > discussion > >> > >> > >> <https://github.com/sunchao/parquet-rs/issues/140> about this > >> > >> recently. > >> > >> > >> Feel free to share your comments on the github. > >> > >> > >> > >> > >> > >> Best, > >> > >> > >> Chao > >> > >> > >> > >> > >> > >> On Sun, Aug 19, 2018 at 7:39 PM, Renjie Liu < > >> > liurenjie2...@gmail.com > >> > >> > > >> > >> > >> wrote: > >> > >> > >> > >> > >> > >>> cc:Sunchao and Any > >> > >> > >>> > >> > >> > >>> > >> > >> > >>> ---------- Forwarded message --------- > >> > >> > >>> From: Uwe L. Korn <uw...@xhochy.com> > >> > >> > >>> Date: Sun, Aug 19, 2018 at 5:08 PM > >> > >> > >>> Subject: Re: [DISCUSS] Rust add adapter for parquet > >> > >> > >>> To: <dev@arrow.apache.org> > >> > >> > >>> > >> > >> > >>> > >> > >> > >>> Hello, > >> > >> > >>> > >> > >> > >>> you might also want to raise this with the > >> > >> > >>> https://github.com/sunchao/parquet-rs project. The overlap > >> > between > >> > >> the > >> > >> > >>> developers of this project and the Arrow Rust implementation > is > >> > >> quite large > >> > >> > >>> but still it may make sense to also start a discussion there. > >> > >> > >>> > >> > >> > >>> Uwe > >> > >> > >>> > >> > >> > >>> On Thu, Aug 16, 2018, at 9:14 AM, Renjie Liu wrote: > >> > >> > >>> > Hi, all: > >> > >> > >>> > > >> > >> > >>> > Now the rust component is approaching a stable state and > rust > >> > >> reader > >> > >> > >>> for > >> > >> > >>> > parquet is ready. I think it maybe a good time to start an > >> > >> adapter for > >> > >> > >>> > parquet, just like adapter for orc in cpp. How you guys > think > >> > >> about it? > >> > >> > >>> > -- > >> > >> > >>> > Liu, Renjie > >> > >> > >>> > Software Engineer, MVAD > >> > >> > >>> -- > >> > >> > >>> Liu, Renjie > >> > >> > >>> Software Engineer, MVAD > >> > >> > >>> > >> > >> > >> > >> > >> > >> -- > >> > >> > > Liu, Renjie > >> > >> > > Software Engineer, MVAD > >> > >> > > > >> > >> > -- > >> > >> > Liu, Renjie > >> > >> > Software Engineer, MVAD > >> > >> > >> > > > >> > > >> -- > >> Renjie Liu > >> Software Engineer, MVAD >
