Re: Rust Arrow status and plans for this week
Hello Uwe: My JIRA id is liurenjie1024 and it seems that I have been given contibutor permission. On Tue, Apr 10, 2018 at 3:00 PM Uwe L. Kornwrote: > Hello Andy, > > this is very exciting. Once we have basic documentation, we should have a > look at streamlining the release process in the ASF infrastructure so > making releases is straight-forward. We have a small collection of scripts > to do this for the main release and the JS release that we should be able > to adapt to the Rust part of the project. I could simply make the > respective JIRAs for that or we have a small chat first about the ASF > release process. > > > My next area of interest personally is the IPC mechanism and interop > > testing with other languages, especially Java. > > This is a very important step for all our implementations. We have an > integration test setup in > https://github.com/apache/arrow/tree/master/integration where we test the > compatibility of all Arrow implementations to each other to verify that > they all have the same understanding of the data structures. > > Uwe > > On Mon, Apr 9, 2018, at 3:26 PM, Andy Grove wrote: > > Over the weekend I added preliminary Parquet support to DataFusion (it > only > > supports int/float primitives and UTF8 so far). This was possible due to > > the great work happening with the parquet-rs crate. > > > > Integrating this with the current Rust version of Arrow was simple and I > > have now started running benchmarks (and we now have some benchmark code > > checked into the Arrow project). > > > > Now that the basic functionality is stable enough to support this use > case > > I am going to focus on quality this week and start improving unit tests > and > > adding documentation. > > > > I think we might be at the point where it makes sense to start > discussing a > > first official release and maybe a roadmap for the Rust library? > > > > My next area of interest personally is the IPC mechanism and interop > > testing with other languages, especially Java. > > > > Thanks, > > > > Andy. > -- Liu, Renjie Software Engineer, MVAD
Re: Rust Arrow status and plans for this week
Hello Andy, this is very exciting. Once we have basic documentation, we should have a look at streamlining the release process in the ASF infrastructure so making releases is straight-forward. We have a small collection of scripts to do this for the main release and the JS release that we should be able to adapt to the Rust part of the project. I could simply make the respective JIRAs for that or we have a small chat first about the ASF release process. > My next area of interest personally is the IPC mechanism and interop > testing with other languages, especially Java. This is a very important step for all our implementations. We have an integration test setup in https://github.com/apache/arrow/tree/master/integration where we test the compatibility of all Arrow implementations to each other to verify that they all have the same understanding of the data structures. Uwe On Mon, Apr 9, 2018, at 3:26 PM, Andy Grove wrote: > Over the weekend I added preliminary Parquet support to DataFusion (it only > supports int/float primitives and UTF8 so far). This was possible due to > the great work happening with the parquet-rs crate. > > Integrating this with the current Rust version of Arrow was simple and I > have now started running benchmarks (and we now have some benchmark code > checked into the Arrow project). > > Now that the basic functionality is stable enough to support this use case > I am going to focus on quality this week and start improving unit tests and > adding documentation. > > I think we might be at the point where it makes sense to start discussing a > first official release and maybe a roadmap for the Rust library? > > My next area of interest personally is the IPC mechanism and interop > testing with other languages, especially Java. > > Thanks, > > Andy.
Re: Rust Arrow status and plans for this week
Hello Renjie, I can give you contributor permissions on JIRA so you can assign issues to yourself. I would need to know your JIRA id for that. Code contributions happen per pull request on github. Just fork the project, open a new branch and once it's ready: make a pull request to the main arrow repository. Cheers Uwe On Tue, Apr 10, 2018, at 4:38 AM, Renjie Liu wrote: > Cool! > I'm also trying to use arrow-rs in my project and would like to contribute > to arrow-rs, can anybody give me contributor permission? > > On Tue, Apr 10, 2018 at 10:31 AM Jacques Nadeauwrote: > > > Super cool, congrats on the progress! > > > > The IPC/interop is top priority for me as well. > > > > On Mon, Apr 9, 2018 at 6:26 AM, Andy Grove wrote: > > > > > Over the weekend I added preliminary Parquet support to DataFusion (it > > only > > > supports int/float primitives and UTF8 so far). This was possible due to > > > the great work happening with the parquet-rs crate. > > > > > > Integrating this with the current Rust version of Arrow was simple and I > > > have now started running benchmarks (and we now have some benchmark code > > > checked into the Arrow project). > > > > > > Now that the basic functionality is stable enough to support this use > > case > > > I am going to focus on quality this week and start improving unit tests > > and > > > adding documentation. > > > > > > I think we might be at the point where it makes sense to start > > discussing a > > > first official release and maybe a roadmap for the Rust library? > > > > > > My next area of interest personally is the IPC mechanism and interop > > > testing with other languages, especially Java. > > > > > > Thanks, > > > > > > Andy. > > > > > > -- > Liu, Renjie > Software Engineer, MVAD
Re: Rust Arrow status and plans for this week
Cool! I'm also trying to use arrow-rs in my project and would like to contribute to arrow-rs, can anybody give me contributor permission? On Tue, Apr 10, 2018 at 10:31 AM Jacques Nadeauwrote: > Super cool, congrats on the progress! > > The IPC/interop is top priority for me as well. > > On Mon, Apr 9, 2018 at 6:26 AM, Andy Grove wrote: > > > Over the weekend I added preliminary Parquet support to DataFusion (it > only > > supports int/float primitives and UTF8 so far). This was possible due to > > the great work happening with the parquet-rs crate. > > > > Integrating this with the current Rust version of Arrow was simple and I > > have now started running benchmarks (and we now have some benchmark code > > checked into the Arrow project). > > > > Now that the basic functionality is stable enough to support this use > case > > I am going to focus on quality this week and start improving unit tests > and > > adding documentation. > > > > I think we might be at the point where it makes sense to start > discussing a > > first official release and maybe a roadmap for the Rust library? > > > > My next area of interest personally is the IPC mechanism and interop > > testing with other languages, especially Java. > > > > Thanks, > > > > Andy. > > > -- Liu, Renjie Software Engineer, MVAD
Re: Rust Arrow status and plans for this week
Super cool, congrats on the progress! The IPC/interop is top priority for me as well. On Mon, Apr 9, 2018 at 6:26 AM, Andy Grovewrote: > Over the weekend I added preliminary Parquet support to DataFusion (it only > supports int/float primitives and UTF8 so far). This was possible due to > the great work happening with the parquet-rs crate. > > Integrating this with the current Rust version of Arrow was simple and I > have now started running benchmarks (and we now have some benchmark code > checked into the Arrow project). > > Now that the basic functionality is stable enough to support this use case > I am going to focus on quality this week and start improving unit tests and > adding documentation. > > I think we might be at the point where it makes sense to start discussing a > first official release and maybe a roadmap for the Rust library? > > My next area of interest personally is the IPC mechanism and interop > testing with other languages, especially Java. > > Thanks, > > Andy. >