[jira] [Created] (ARROW-3960) [Rust] remove extern crate for Rust 2018

2018-12-07 Thread Chao Sun (JIRA)
Chao Sun created ARROW-3960: --- Summary: [Rust] remove extern crate for Rust 2018 Key: ARROW-3960 URL: https://issues.apache.org/jira/browse/ARROW-3960 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-3959) [Rust] Time and Timestamp Support

2018-12-07 Thread nevi_me (JIRA)
nevi_me created ARROW-3959: -- Summary: [Rust] Time and Timestamp Support Key: ARROW-3959 URL: https://issues.apache.org/jira/browse/ARROW-3959 Project: Apache Arrow Issue Type: New Feature

[jira] [Created] (ARROW-3958) [Plasma] Reduce number of IPCs

2018-12-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3958: - Summary: [Plasma] Reduce number of IPCs Key: ARROW-3958 URL: https://issues.apache.org/jira/browse/ARROW-3958 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-3957) pyarrow.hdfs.connect fails silently

2018-12-07 Thread Jim Fulton (JIRA)
Jim Fulton created ARROW-3957: - Summary: pyarrow.hdfs.connect fails silently Key: ARROW-3957 URL: https://issues.apache.org/jira/browse/ARROW-3957 Project: Apache Arrow Issue Type: Bug

[jira] [Created] (ARROW-3956) [Python] ParquetWriter.write_table isn't working

2018-12-07 Thread David Lee (JIRA)
David Lee created ARROW-3956: Summary: [Python] ParquetWriter.write_table isn't working Key: ARROW-3956 URL: https://issues.apache.org/jira/browse/ARROW-3956 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-3955) [GLib] Add (transfer full) to free when no longer needed

2018-12-07 Thread Yosuke Shiro (JIRA)
Yosuke Shiro created ARROW-3955: --- Summary: [GLib] Add (transfer full) to free when no longer needed Key: ARROW-3955 URL: https://issues.apache.org/jira/browse/ARROW-3955 Project: Apache Arrow

[jira] [Created] (ARROW-3954) [Rust] Add Slice to Array and ArrayData

2018-12-07 Thread Chao Sun (JIRA)
Chao Sun created ARROW-3954: --- Summary: [Rust] Add Slice to Array and ArrayData Key: ARROW-3954 URL: https://issues.apache.org/jira/browse/ARROW-3954 Project: Apache Arrow Issue Type: Improvement

Re: [RUST] [DISCUSS] Changing type of array lengths

2018-12-07 Thread Marco Neumann
On windows it depends if it's a 32 or 64 bit binary, like on every other system as well. usize is usually used by Rust containers for indexing (see for example Vec in the standard library) and I found it personally very annoying if libraries break that rule, because in Rust you have to be

Re: [RUST] [DISCUSS] Changing type of array lengths

2018-12-07 Thread Wes McKinney
What would be the argument for using usize over i64/u64? Is usize 64 bits in Rust when compiling on Windows? On Fri, Dec 7, 2018 at 9:48 AM Andy Grove wrote: > > I am in favor of using usize. > > Thanks. > > On Thu, Dec 6, 2018 at 7:20 PM paddy horan wrote: > > > All, > > > > As part of the PR

Re: [RUST] [DISCUSS] Changing type of array lengths

2018-12-07 Thread Andy Grove
I am in favor of using usize. Thanks. On Thu, Dec 6, 2018 at 7:20 PM paddy horan wrote: > All, > > As part of the PR for ARROW-3347 there was a discussion regarding the type > that should be used for anything that measures the length of an array, > i.e. len and capacity. > > The result of

Re: [RUST] [DISCUSS] Changing type of array lengths

2018-12-07 Thread Marco Neumann
One question here is: do we want to support datasets with more than 4G entries on 32bit systems? If so, how would this even be possible (since you cannot just fit that much data in any addressable memory chunk in Rust)? So I would say: usize is idiomic and supports large enough datasets on the

Re: [RUST] [DISCUSS] Changing type of array lengths

2018-12-07 Thread Wes McKinney
Thanks for raising the issue, Paddy. In C++/Python/R we often work with vary large contiguous datasets, so having support for 64-bit lengths is important. If supporting this in Rust is not a hardship, I think it's a good idea. For IPC (shared memory) or RPC (Flight / gRPC), in many cases it would

Re: Thoughts about 2019 Arrow development focus areas

2018-12-07 Thread Andy Grove
Wes, This is very exciting. Thanks for writing up the detailed document. I think it is time for me to start brushing up on modern C++. Andy. On Thu, Dec 6, 2018 at 8:37 PM Wes McKinney wrote: > hi folks, > > I jotted down some high level ideas about directions I'd like to push > the various

Re: Timeline for Arrow 0.12.0 release

2018-12-07 Thread Andy Grove
I've added one PR to the list (https://github.com/apache/arrow/pull/3119) to update the project to use Rust 2018 Edition. I'm also considering removing one PR from the list and would like to get opinions here. I have a PR (https://github.com/apache/arrow/pull/3033) to add some basic math and

[jira] [Created] (ARROW-3953) Pandas MultiIndex renamed labels to codes (pd 0.24)

2018-12-07 Thread Joris Van den Bossche (JIRA)
Joris Van den Bossche created ARROW-3953: Summary: Pandas MultiIndex renamed labels to codes (pd 0.24) Key: ARROW-3953 URL: https://issues.apache.org/jira/browse/ARROW-3953 Project: Apache

[jira] [Created] (ARROW-3952) [Rust] Specify edition="2018" in Cargo.toml

2018-12-07 Thread Andy Grove (JIRA)
Andy Grove created ARROW-3952: - Summary: [Rust] Specify edition="2018" in Cargo.toml Key: ARROW-3952 URL: https://issues.apache.org/jira/browse/ARROW-3952 Project: Apache Arrow Issue Type:

[Go] Npyio / Arrow support

2018-12-07 Thread Sebastien Binet
hi there, Back in the days, for my particle physics work, I had to be able to read and write numpy data files[1] (for interop with existing analysis pipelines.) I was wondering whether I couldn't integrate this Go package with Apache Arrow: - https://github.com/sbinet/npyio and put it under

[jira] [Created] (ARROW-3951) [Go] implement a CSV writer

2018-12-07 Thread Sebastien Binet (JIRA)
Sebastien Binet created ARROW-3951: -- Summary: [Go] implement a CSV writer Key: ARROW-3951 URL: https://issues.apache.org/jira/browse/ARROW-3951 Project: Apache Arrow Issue Type: Improvement