Re: Rust Arrow status and plans for this week

2018-04-10 Thread Renjie Liu
Hello Uwe:
My JIRA id is liurenjie1024 and it seems that I have been given contibutor
permission.

On Tue, Apr 10, 2018 at 3:00 PM Uwe L. Korn  wrote:

> Hello Andy,
>
> this is very exciting. Once we have basic documentation, we should have a
> look at streamlining the release process in the ASF infrastructure so
> making releases is straight-forward. We have a small collection of scripts
> to do this for the main release and the JS release that we should be able
> to adapt to the Rust part of the project. I could simply make the
> respective JIRAs for that or we have a small chat first about the ASF
> release process.
>
> > My next area of interest personally is the IPC mechanism and interop
> > testing with other languages, especially Java.
>
> This is a very important step for all our implementations. We have an
> integration test setup in
> https://github.com/apache/arrow/tree/master/integration where we test the
> compatibility of all Arrow implementations to each other to verify that
> they all have the same understanding of the data structures.
>
> Uwe
>
> On Mon, Apr 9, 2018, at 3:26 PM, Andy Grove wrote:
> > Over the weekend I added preliminary Parquet support to DataFusion (it
> only
> > supports int/float primitives and UTF8 so far). This was possible due to
> > the great work happening with the parquet-rs crate.
> >
> > Integrating this with the current Rust version of Arrow was simple and I
> > have now started running benchmarks (and we now have some benchmark code
> > checked into the Arrow project).
> >
> > Now that the basic functionality is stable enough to support this use
> case
> > I am going to focus on quality this week and start improving unit tests
> and
> > adding documentation.
> >
> > I think we might be at the point where it makes sense to start
> discussing a
> > first official release and maybe a roadmap for the Rust library?
> >
> > My next area of interest personally is the IPC mechanism and interop
> > testing with other languages, especially Java.
> >
> > Thanks,
> >
> > Andy.
>
-- 
Liu, Renjie
Software Engineer, MVAD


Re: Rust Arrow status and plans for this week

2018-04-10 Thread Uwe L. Korn
Hello Andy,

this is very exciting. Once we have basic documentation, we should have a look 
at streamlining the release process in the ASF infrastructure so making 
releases is straight-forward. We have a small collection of scripts to do this 
for the main release and the JS release that we should be able to adapt to the 
Rust part of the project. I could simply make the respective JIRAs for that or 
we have a small chat first about the ASF release process.

> My next area of interest personally is the IPC mechanism and interop
> testing with other languages, especially Java.

This is a very important step for all our implementations. We have an 
integration test setup in 
https://github.com/apache/arrow/tree/master/integration where we test the 
compatibility of all Arrow implementations to each other to verify that they 
all have the same understanding of the data structures.

Uwe

On Mon, Apr 9, 2018, at 3:26 PM, Andy Grove wrote:
> Over the weekend I added preliminary Parquet support to DataFusion (it only
> supports int/float primitives and UTF8 so far). This was possible due to
> the great work happening with the parquet-rs crate.
> 
> Integrating this with the current Rust version of Arrow was simple and I
> have now started running benchmarks (and we now have some benchmark code
> checked into the Arrow project).
> 
> Now that the basic functionality is stable enough to support this use case
> I am going to focus on quality this week and start improving unit tests and
> adding documentation.
> 
> I think we might be at the point where it makes sense to start discussing a
> first official release and maybe a roadmap for the Rust library?
> 
> My next area of interest personally is the IPC mechanism and interop
> testing with other languages, especially Java.
> 
> Thanks,
> 
> Andy.


Re: Rust Arrow status and plans for this week

2018-04-10 Thread Uwe L. Korn
Hello Renjie,

I can give you contributor permissions on JIRA so you can assign issues to 
yourself. I would need to know your JIRA id for that.

Code contributions happen per pull request on github. Just fork the project, 
open a new branch and once it's ready: make a pull request to the main arrow 
repository.

Cheers
Uwe

On Tue, Apr 10, 2018, at 4:38 AM, Renjie Liu wrote:
> Cool!
> I'm also trying to use arrow-rs in my project and would like to contribute
> to arrow-rs, can anybody give me contributor permission?
> 
> On Tue, Apr 10, 2018 at 10:31 AM Jacques Nadeau  wrote:
> 
> > Super cool, congrats on the progress!
> >
> > The IPC/interop is top priority for me as well.
> >
> > On Mon, Apr 9, 2018 at 6:26 AM, Andy Grove  wrote:
> >
> > > Over the weekend I added preliminary Parquet support to DataFusion (it
> > only
> > > supports int/float primitives and UTF8 so far). This was possible due to
> > > the great work happening with the parquet-rs crate.
> > >
> > > Integrating this with the current Rust version of Arrow was simple and I
> > > have now started running benchmarks (and we now have some benchmark code
> > > checked into the Arrow project).
> > >
> > > Now that the basic functionality is stable enough to support this use
> > case
> > > I am going to focus on quality this week and start improving unit tests
> > and
> > > adding documentation.
> > >
> > > I think we might be at the point where it makes sense to start
> > discussing a
> > > first official release and maybe a roadmap for the Rust library?
> > >
> > > My next area of interest personally is the IPC mechanism and interop
> > > testing with other languages, especially Java.
> > >
> > > Thanks,
> > >
> > > Andy.
> > >
> >
> -- 
> Liu, Renjie
> Software Engineer, MVAD


Re: Rust Arrow status and plans for this week

2018-04-09 Thread Renjie Liu
Cool!
I'm also trying to use arrow-rs in my project and would like to contribute
to arrow-rs, can anybody give me contributor permission?

On Tue, Apr 10, 2018 at 10:31 AM Jacques Nadeau  wrote:

> Super cool, congrats on the progress!
>
> The IPC/interop is top priority for me as well.
>
> On Mon, Apr 9, 2018 at 6:26 AM, Andy Grove  wrote:
>
> > Over the weekend I added preliminary Parquet support to DataFusion (it
> only
> > supports int/float primitives and UTF8 so far). This was possible due to
> > the great work happening with the parquet-rs crate.
> >
> > Integrating this with the current Rust version of Arrow was simple and I
> > have now started running benchmarks (and we now have some benchmark code
> > checked into the Arrow project).
> >
> > Now that the basic functionality is stable enough to support this use
> case
> > I am going to focus on quality this week and start improving unit tests
> and
> > adding documentation.
> >
> > I think we might be at the point where it makes sense to start
> discussing a
> > first official release and maybe a roadmap for the Rust library?
> >
> > My next area of interest personally is the IPC mechanism and interop
> > testing with other languages, especially Java.
> >
> > Thanks,
> >
> > Andy.
> >
>
-- 
Liu, Renjie
Software Engineer, MVAD


Re: Rust Arrow status and plans for this week

2018-04-09 Thread Jacques Nadeau
Super cool, congrats on the progress!

The IPC/interop is top priority for me as well.

On Mon, Apr 9, 2018 at 6:26 AM, Andy Grove  wrote:

> Over the weekend I added preliminary Parquet support to DataFusion (it only
> supports int/float primitives and UTF8 so far). This was possible due to
> the great work happening with the parquet-rs crate.
>
> Integrating this with the current Rust version of Arrow was simple and I
> have now started running benchmarks (and we now have some benchmark code
> checked into the Arrow project).
>
> Now that the basic functionality is stable enough to support this use case
> I am going to focus on quality this week and start improving unit tests and
> adding documentation.
>
> I think we might be at the point where it makes sense to start discussing a
> first official release and maybe a roadmap for the Rust library?
>
> My next area of interest personally is the IPC mechanism and interop
> testing with other languages, especially Java.
>
> Thanks,
>
> Andy.
>