Re: [VOTE] Release Apache Arrow 10.0.0 - RC0

2022-10-24 Thread Neville Dipale
;Re: [VOTE] Release Apache Arrow 10.0.0 - RC0" on Sun, 23 Oct 2022 > 10:31:27 +0300, > Benson Muite wrote: > > > WIP but source verification fails for me on CentOS 7 due to unsigned > > key from Neville Dipale: > > > > TEST_DEFAULT=0 TEST_SOURCE=1 dev/re

Re: [VOTE][RUST] Release Apache Arrow Rust 25.0.0 RC1

2022-10-16 Thread Neville Dipale
+1 (binding) Verified on Ubuntu (WSL2) On Sun, 16 Oct 2022 at 13:50, Remzi Yang <1371656737...@gmail.com> wrote: > +1 (non-binding) Verified on M1 Mac. > > Thank you Andrew. > > On Sun, 16 Oct 2022 at 11:18, L. C. Hsieh wrote: > > > +1 (binding) > > > > Verified on Intel Mac. > > > > Thanks

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 10.0.0 RC1

2022-07-15 Thread Neville Dipale
I've logged https://github.com/apache/arrow-datafusion/issues/2916 On Fri, 15 Jul 2022 at 08:34, Neville Dipale wrote: > +1 (binding) > > There are 3 test failures because the hardcoded strings expect > "ARROW_TEST_DATA" but we get "privateARROW_TEST_DA

Re: [VOTE][RUST][DataFusion] Release Apache Arrow DataFusion 10.0.0 RC1

2022-07-15 Thread Neville Dipale
+1 (binding) There are 3 test failures because the hardcoded strings expect "ARROW_TEST_DATA" but we get "privateARROW_TEST_DATA" when verifying the RC. An example can be seen on this diff https://www.diffchecker.com/B8i8w4GC. My conclusion from looking at it is that it's likely a quirk that we

Re: [VOTE][RUST] Release Apache Arrow Rust 17.0.0 RC1

2022-06-27 Thread Neville Dipale
+1 (binding) Verified on aarch64 macos On Sat, 25 Jun 2022 at 07:00, QP Hou wrote: > +1 (binding) > > On Fri, Jun 24, 2022 at 7:36 PM Remzi Yang <1371656737...@gmail.com> > wrote: > > > > +1 (non-binding). Verified on Mac M1. > > Thanks Andrew. > > > > Remzi > > > > On Sat, 25 Jun 2022 at

Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1

2022-02-22 Thread Neville Dipale
ers. > > On Tue, Feb 22, 2022 at 11:13 AM Neville Dipale > wrote: > > > Thanks Andy, I didn't get invites for parquet, parquet-derive and > > arrow-flight > > > > On Tue, 22 Feb 2022 at 19:58, Andy Grove wrote: > > > > > Hi Neville, > > > >

Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1

2022-02-22 Thread Neville Dipale
Thanks Andy, I didn't get invites for parquet, parquet-derive and arrow-flight On Tue, 22 Feb 2022 at 19:58, Andy Grove wrote: > Hi Neville, > > I have invited you to become an owner on the Arrow/DataFusion crates. > > Thanks, > > Andy. > > > > On Tue, Feb 22

Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1

2022-02-22 Thread Neville Dipale
org/repos/dist/release/arrow/KEYS > > > > ? > > > > > > > > The release/arrow/KEYS content is distributed as > > > > https://downloads.apache.org/arrow/KEYS . > > > > > > > > > > > > Thanks, > > > > -- > > >

Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1

2022-02-20 Thread Neville Dipale
+1 binding Verified on macos aarch64 + TEST_SUCCESS=yes + echo 'Release candidate looks good!' Release candidate looks good! On Sun, 20 Feb 2022 at 10:43, Neville Dipale wrote: > Hi Kou, > > Thanks, I've just managed to add my key through some fiddling before I saw > your

Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1

2022-02-20 Thread Neville Dipale
t 19 Feb 2022 10:01:41 AM MST > > gpg:using EDDSA key > 3905F254F9E504B40FFF6CF6000488D7717D3FB2 > > gpg: Can't check signature: No public key > > + cleanup > > + '[' no = yes ']' > > + echo 'Failed to verify release candidate. See /tmp/arrow-

Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1

2022-02-19 Thread Neville Dipale
echo 'Failed to verify release candidate. See /tmp/arrow-9.1.0.62KW4 for > details.' > > > On Sat, Feb 19, 2022 at 10:04 AM Neville Dipale > wrote: > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Implementation, &g

[VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1

2022-02-19 Thread Neville Dipale
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 9.1.0. This release candidate is based on commit: ecba7dc0830dbde6aa6dd9432519b776e40c1e85 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download,

Re: [ANNOUNCE] New Arrow PMC member: QP Hou

2022-02-19 Thread Neville Dipale
Congratulations QP! 拾 On Fri, 18 Feb 2022 at 09:42, Daniël Heres wrote: > Congratulations QP! > > On Fri, Feb 18, 2022, 06:55 Benson Muite > wrote: > > > Congratulations QP! > > On 2/18/22 8:35 AM, Jiayu Liu wrote: > > > Congratulations QP! > > > > > > On Fri, Feb 18, 2022 at 1:32 PM Micah

Re: [VOTE][RUST][Datafusion] Release Apache Arrow Datafusion 7.0.0 RC2

2022-02-15 Thread Neville Dipale
+1 binding, verified the RC on macos aarch64 On Tue, 15 Feb 2022 at 09:18, Yijie Shen wrote: > Thanks, Andrew! > > +1 non-binding > > > On Tue, Feb 15, 2022 at 11:51 AM QP Hou wrote: > > > +1 non-binding, went release script on Linux arm64. > > > > It failed at the end when executing the

Re: [VOTE][RUST] Release Apache Arrow Rust 9.0.3 RC3

2022-02-11 Thread Neville Dipale
> More details can be found here [1] > >> > > >> > Andrew > >> > > >> > [1] > >> https://github.com/apache/arrow-rs/pull/1297#issuecomment-1034888108 > >> > > >> > On Thu, Feb 10, 2022 at 2:43 AM Neville Dipale >

Re: [VOTE][RUST] Release Apache Arrow Rust 9.0.3 RC3

2022-02-10 Thread Neville Dipale
tails can be found here [1] > > Andrew > > [1] https://github.com/apache/arrow-rs/pull/1297#issuecomment-1034888108 > > On Thu, Feb 10, 2022 at 2:43 AM Neville Dipale > wrote: > > > Sorry, that should be a -1 because of the bug > > > > On Thu, 10 Feb 2022 at

Re: [VOTE][RUST] Release Apache Arrow Rust 9.0.3 RC3

2022-02-09 Thread Neville Dipale
Sorry, that should be a -1 because of the bug On Thu, 10 Feb 2022 at 09:42, Neville Dipale wrote: > [X] +0 binding > > There's a test failure on aarch64, I've opened > https://github.com/apache/arrow-rs/issues/1294 > > thread > 'util::bit_chunk_iterator::tests::test_unalign

Re: [VOTE][RUST] Release Apache Arrow Rust 9.0.3 RC3

2022-02-09 Thread Neville Dipale
[X] +0 binding There's a test failure on aarch64, I've opened https://github.com/apache/arrow-rs/issues/1294 thread 'util::bit_chunk_iterator::tests::test_unaligned_bit_chunk_iterator' panicked at 'assertion failed: ALIGNMENT > 64', arrow/src/util/bit_chunk_iterator.rs:470:9 On Wed, 9 Feb 2022

Re: VOTE][RUST] Release Apache Arrow Rust 8.0.0 RC1

2022-01-25 Thread Neville Dipale
Good day +1 (binding), verified on macos-aarch64 On Tue, 25 Jan 2022 at 05:08, Wang Xudong wrote: > +1(non-binding) > Verified on macos > "Release candidate looks good!" > > -- > xudong963 > > Andy Grove 于2022年1月25日周二 09:07写道: > > > +1 (binding). Verified on Ubuntu 20.04.3 LTS. > > > > On

Re: [VOTE][RUST][Datafusion] Release Apache Arrow Datafusion 5.0.0 RC3

2021-08-13 Thread Neville Dipale
+1 (bniding) i verified the RC on aarch64-macos On Fri, 13 Aug 2021 at 23:29, Jorge Cardoso Leitão wrote: > +1 > > Great work everyone! > > > On Fri, Aug 13, 2021, 22:19 Daniël Heres wrote: > > > +1 (non binding). Looking good. > > > > > > On Fri, Aug 13, 2021, 07:49 QP Hou wrote: > > > > >

Re: [VOTE][RUST] Release Apache Arrow Rust 5.2.0 RC1

2021-08-13 Thread Neville Dipale
+1 (binding) I ran the verification script on aarch64 macos On Fri, 13 Aug 2021 at 07:45, QP Hou wrote: > +1 (non-binding) > > ran the verification script on Linux 5.4.0 x86_64 > > On Thu, Aug 12, 2021 at 12:44 PM Andrew Lamb wrote: > > > > Hi, > > > > I would like to propose a release of

Re: [VOTE][RUST] Release Apache Arrow Rust 5.1.0 RC1

2021-08-02 Thread Neville Dipale
+1 Ran ./dev/release/verify-release-candidate.sh 5.1.0 1 on aarch64-apple-darwin Got + TEST_SUCCESS=yes + echo 'Release candidate looks good!' Release candidate looks good! + exit 0 + cleanup On Fri, 30 Jul 2021 at 15:53, Wayne Xia wrote: > +1 > > I ran this on Intel macOS Catalina: >

Re: [ANNOUNCE] New Arrow PMC member: Neville Dipale

2021-07-30 Thread Neville Dipale
at 3:20 PM Wes McKinney > > > wrote: > > > > > > > > > > The Project Management Committee (PMC) for Apache Arrow has invited > > > > > Neville Dipale to become a PMC member and we are pleased to > announce > > > > > that Neville has accepted. > > > > > > > > > > Congratulations and welcome! > > > > > > > > >

Re: Delta Lake support for DataFusion

2021-06-09 Thread Neville Dipale
The correct approach might be to improve DataFusion support in delta-rs. TableProvider is already implemented here: https://github.com/delta-io/delta-rs/blob/main/rust/src/delta_datafusion.rs I've pinged QP to ask for their advice. Neville On Wed, 9 Jun 2021 at 19:58, Andrew Lamb wrote: > I

Re: [DISCUSS] Moving the format directory to arrow-format repository

2021-05-02 Thread Neville Dipale
gt; > in > > > apache/arrow, even if there were a separate apache/arrow-format > > repository. > > > The format files are slow-moving enough that I don't think it's > > burdensome > > > to mirror these into satellite repositories like arrow/arrow-rs. > > > > >

[DISCUSS] Moving the format directory to arrow-format repository

2021-04-27 Thread Neville Dipale
Hi Arrow devs, Andy noticed that we carry a copy of the format directory in arrow-rs, which is bound to get outdated in the future. We would like to propose creating an arrow-format repository, similar to parquet-format, so that arrow-rs and other future separate repositories could add this as a

Re: [VOTE] Move Rust components to new repos and process

2021-04-14 Thread Neville Dipale
Hi Arrow devs, +1 On Thu, 15 Apr 2021 at 05:28, Jorge Cardoso Leitão wrote: > +1 binding. > > On Thu, Apr 15, 2021 at 4:10 AM Micah Kornfield > wrote: > > > +1 binding. Does this also cover other changes (issue management) +1 to > > those as well. > > > > On Wednesday, April 14, 2021, Andy

Re: [VOTE] Accept donation of Rust Ballista project

2021-03-21 Thread Neville Dipale
+1 (binding) On Sun, 21 Mar 2021 at 19:48, Neal Richardson wrote: > +1 (binding) > > Neal > > On Sun, Mar 21, 2021 at 10:38 AM Francis Du wrote: > > > I have been following Ballista for a long time. This is an exciting > thing. > > I am also very happy to contribute to this. > > > > So +1 > >

Re: [RUST] Fields and schema metadata

2021-02-06 Thread Neville Dipale
We had to use the BTreeMap because a HashMap doesn't implement Hash, so can't be used in the Field. The easiest way to see this, is to replace it with a HashMap, and try compile the arrow crate. Neville On Sat, 06 Feb 2021, 13:50 Fernando Herrera, wrote: > Hi all, Is there a reason why the

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Neville Dipale
B 1 On Fri, 22 Jan 2021 at 15:14, Krisztián Szűcs wrote: > Thanks Neville for testing it! > > There should be more context about the failures above the summary. > Could you please post the errors? > > > On Fri, Jan 22, 2021 at 2:05 PM Neville Dipale > wrote: > > > &

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Neville Dipale
. Neville On Fri, 22 Jan 2021 at 14:22, Neville Dipale wrote: > This is my first time verifying, do I also need to set the env vars below? > > ARROW_GANDIVA=0 ARROW_PLASMA=0 TEST_DEFAULT=0 > TEST_SOURCE=1 TEST_CPP=1 TEST_PYTHON=1 TEST_JAVA=1 > TEST_INTEGRATION_CPP=1 TEST_INTEGRATION_JAVA=

Re: [VOTE] Release Apache Arrow 3.0.0 - RC2

2021-01-22 Thread Neville Dipale
This is my first time verifying, do I also need to set the env vars below? ARROW_GANDIVA=0 ARROW_PLASMA=0 TEST_DEFAULT=0 TEST_SOURCE=1 TEST_CPP=1 TEST_PYTHON=1 TEST_JAVA=1 TEST_INTEGRATION_CPP=1 TEST_INTEGRATION_JAVA=1 Otherwise, I'm currently running: ./dev/release/verify-release-candidate.bat

Re: Release 3.0 timeline?

2021-01-16 Thread Neville Dipale
Hi Arrow devs, There's some bugs in the Parquet implementation which affect reading of data: - https://issues.apache.org/jira/browse/ARROW-11269, which was opened today, and I just saw now. - an issue with list schema nulls from the parquet-format's logical types. In this case, we misinterpret

Re: Arrow 3.0 release

2021-01-13 Thread Neville Dipale
Good day, I was hoping to complete the parquet list writer PR in time, but even though I've been burning the midnight oil addressing the remaining issues in the PR, I won't make it in time :( I've removed the Rust blocker from the milestone, so there should be nothing on my side blocking us.

Re: [ANNOUNCE] New Arrow committer: Andrew Lamb

2020-11-10 Thread Neville Dipale
Congrats Andrew! Look forward to working more with you On Tue, 10 Nov 2020, 18:07 Jorge Cardoso Leitão, wrote: > Congrats, Andrew! > > Andrew has been doing an amazing job, both on the implementation but also > at reviewing and helping others. He taught me a lot, I am having a great > time

[Rust] Merging the Parquet Arrow Branch

2020-10-26 Thread Neville Dipale
Hi Arrow devs, We've been working on a Parquet writer on a separate branch, mainly to expedite merging PRs in case there weren't enough reviewers.I wasn't comfortable merging the work by 2.0, so I opted not to get it merged for the release. There are people who have expressed an interest in using

Re: [VOTE] Accept donation of Julia implementation for Apache Arrow

2020-10-12 Thread Neville Dipale
Good day  +1 On Mon, 12 Oct 2020, 22:35 Neal Richardson, wrote: > Hi all, > Last month [1] Jacob Quinn proposed donating Arrow.jl, a Julia > implementation of Arrow, to the Apache Arrow project. The community has had > an opportunity to discuss this and there do not seem to be objections. >

Re: [VOTE][Format] Allow for 256-bit Decimal's in the Arrow specification

2020-09-29 Thread Neville Dipale
+1 (non-binding) Rust support is behind, but we'll catch up at some point. On Wed, 30 Sep 2020 at 03:10, Holden Karau wrote: > +1 (non-binding) > > On Tue, Sep 29, 2020 at 6:08 PM Sutou Kouhei wrote: > > > +1 > > > > In > > "Re: [VOTE][Format] Allow for 256-bit Decimal's in the Arrow > >

Re: 2.0.0 release timeline: October 9

2020-09-29 Thread Neville Dipale
Hi Neal, I've pruned the Rust backlog a bit, but only changed PRs that I've either opened, or those that I presume nobody is currently working on. There's 2 major features I've been working on: 1. Writing Arrow data to Parquet (separate branch) 2. Integration testing I'll prioritise 2 as

Re: [Rust] Arrow SQL Adapters/Connectors

2020-09-27 Thread Neville Dipale
tation of this trait by different providers > > would give support to be used in Arrow and DataFusion. > > > > Besides postgres, one idea is to pick the top from this list > > <https://db-engines.com/en/ranking>: > > > > * Oracle > > * MySQL &g

[Rust] Arrow SQL Adapters/Connectors

2020-09-26 Thread Neville Dipale
Hi Arrow developers I would like to gauge the appetite for an Arrow SQL connector that: * Reads and writes Arrow data to and from SQL databases * Reads tables and queries into record batches, and writes batches to tables (either append or overwrite) * Leverages binary SQL formats where available

Re: Rust conversion

2020-09-16 Thread Neville Dipale
Hi Roland, For primitive types, there are value_slice methods which would allow you to get the array's contents as a vector, but you have to handle the null values as the vectors wouldn't return an Option. We haven't seen demand yet in converting arrays to JSON, but the integration crate (

[Rust] Creating a separate branch for Arrow Partuet writer until next release

2020-08-08 Thread Neville Dipale
Good day, This relates to https://github.com/apache/arrow/pull/7319 (ARROW-8289) In the past few months we haven't had enough review bandwidth on Rust's Parquet implementation (mostly relying on Chao for non-trivial reviews), and given the amount of work needed for an Arrow writer + the interest

Re: Status of Rust Integration Testing

2020-07-11 Thread Neville Dipale
. One question, we have "gold" files for 0.14.0 > checked into the test-data repo and run integration tests on those to > ensure we can read them in a few implementations. Does Rust at least read > those correctly? > > Thanks, > Micah > > On Fri, Jul 10, 2020 at 1:03 PM

Status of Rust Integration Testing

2020-07-10 Thread Neville Dipale
Good day Arrow devs, I've spent a few evenings looking into the issues that we're experiencing with Rust integration testing. In summary, none of our tests pass (zero batch doesn't count :) ). This is mainly because of changes from the legacy padding in the 0.15.0 release, which we never made in

Re: [Integration] Errors running archery integration on Windows

2020-07-06 Thread Neville Dipale
> > Numpy on windows has different default bitwidth than on linux. Perhaps > this > > is causing the issue? (see: > > > https://stackoverflow.com/questions/36278590/numpy-array-dtype-is-coming-as-int32-by-default-in-a-windows-10-64-bit-machine > > ) > > > >

[Integration] Errors running archery integration on Windows

2020-07-06 Thread Neville Dipale
Hi Arrow devs, I'm trying to run archery integration tests in Windows 10 (Python 3.7.7; conda 4.8.3), but I'm getting an error *ValueError: low is out of bounds for int32* (https://gist.github.com/nevi-me/4946eabb2dc111e10b98c074b45b73b1 ). Has someone else encountered this problem before?

Re: Bot to set "In Progress" status in JIRA

2020-06-30 Thread Neville Dipale
Thanks Wes, I noticed this today, but was a bit confused as to why you reassigned a JIRA to yourself, then back to me. This clarifies what happened :) Neville On Tue, 30 Jun 2020 at 15:39, Wes McKinney wrote: > hi, > > Yesterday I set up a bot to set issues to In Progress if they have an >

Re: [VOTE] Increment MetadataVersion in Schema.fbs from V4 to V5 for 1.0.0 release

2020-06-30 Thread Neville Dipale
+1 (non-binding) On Tue, 30 Jun 2020 at 06:29, Ben Kietzman wrote: > +1 (non binding) > > On Tue, Jun 30, 2020, 00:25 Wes McKinney wrote: > > > +1 (binding) > > > > On Mon, Jun 29, 2020 at 10:49 PM Micah Kornfield > > wrote: > > > > > > +1 (binding) > > > > > > On Mon, Jun 29, 2020 at 2:43 PM

[jira] [Created] (ARROW-9095) [Rust] Fix NullArray to comply with spec

2020-06-10 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-9095: - Summary: [Rust] Fix NullArray to comply with spec Key: ARROW-9095 URL: https://issues.apache.org/jira/browse/ARROW-9095 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-9053) [Rust] Add sort for lists and structs

2020-06-06 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-9053: - Summary: [Rust] Add sort for lists and structs Key: ARROW-9053 URL: https://issues.apache.org/jira/browse/ARROW-9053 Project: Apache Arrow Issue Type: Sub

[jira] [Created] (ARROW-9007) [Rust] Support appending arrays by merging array data

2020-06-02 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-9007: - Summary: [Rust] Support appending arrays by merging array data Key: ARROW-9007 URL: https://issues.apache.org/jira/browse/ARROW-9007 Project: Apache Arrow

[jira] [Created] (ARROW-8883) [Rust] [Integration Testing] Disable unsupported tests

2020-05-21 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-8883: - Summary: [Rust] [Integration Testing] Disable unsupported tests Key: ARROW-8883 URL: https://issues.apache.org/jira/browse/ARROW-8883 Project: Apache Arrow

[jira] [Created] (ARROW-8881) [Rust] Add large list and binary support

2020-05-21 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-8881: - Summary: [Rust] Add large list and binary support Key: ARROW-8881 URL: https://issues.apache.org/jira/browse/ARROW-8881 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-8308) [Rust] [Flight] Implement DoExchange on examples

2020-04-01 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-8308: - Summary: [Rust] [Flight] Implement DoExchange on examples Key: ARROW-8308 URL: https://issues.apache.org/jira/browse/ARROW-8308 Project: Apache Arrow

Re: [DISCUSS] Adding "trivial" buffer compression option to IPC protocol (ARROW-300)

2020-03-01 Thread Neville Dipale
I also support compression at the buffer level, and making it an extra message. Talking about compression and flight, has anyone tested using grpc's compression to compress at the transport level (if that's a correct way to describe it)? I believe only gzip and brotli are currently supported, so

[jira] [Created] (ARROW-7924) [Rust] Add sort for float types

2020-02-23 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7924: - Summary: [Rust] Add sort for float types Key: ARROW-7924 URL: https://issues.apache.org/jira/browse/ARROW-7924 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-7705) [Rust] Initial sort implementation

2020-01-28 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7705: - Summary: [Rust] Initial sort implementation Key: ARROW-7705 URL: https://issues.apache.org/jira/browse/ARROW-7705 Project: Apache Arrow Issue Type: Sub

[jira] [Created] (ARROW-7704) [Rust] Support sort

2020-01-28 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7704: - Summary: [Rust] Support sort Key: ARROW-7704 URL: https://issues.apache.org/jira/browse/ARROW-7704 Project: Apache Arrow Issue Type: New Feature

Re: [Rust] Possible blocking issue for 0.16 release

2020-01-26 Thread Neville Dipale
Hi Andy, I think `cargo update` is the correct approach to resolve this issue. Yes, we've got quite a bit of backlog in Rust, but we should initiate the process to adopt parquet-format with Chao and Ivan's approval post 0.16. In the interim we could fix the thrift dependency issue upstream (as

[jira] [Created] (ARROW-7620) [Rust] Windows builds failing due to flatbuffer compile error

2020-01-20 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7620: - Summary: [Rust] Windows builds failing due to flatbuffer compile error Key: ARROW-7620 URL: https://issues.apache.org/jira/browse/ARROW-7620 Project: Apache Arrow

Re: [Discuss][Rust] Policy regarding "unsafe"

2020-01-17 Thread Neville Dipale
Hi Paddy, Arrow Developers, I've given this some thought, and I preliminarily think that perhaps we can audit our use of unsafe and evaluate where we can remove it, propagate it upwards (and provide safe alternatives) or provide some safety to callers. Looking at the 3 options that Paul Kernfeld

Re: [Discuss][Rust] Policy regarding "unsafe"

2020-01-17 Thread Neville Dipale
Hi Paddy, On Fri, 10 Jan 2020 at 04:13, paddy horan wrote: > Hi All, > > This time last year there was a brief discussion on the usage of unsafe in > Rust (a user on github raised the issue and I created the JIRA). [1] > > So far we mostly avoid unsafe in the public API's. The thinking here is

[jira] [Created] (ARROW-7521) [Rust] Remove tuple on FixedSizeList datatype

2020-01-08 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7521: - Summary: [Rust] Remove tuple on FixedSizeList datatype Key: ARROW-7521 URL: https://issues.apache.org/jira/browse/ARROW-7521 Project: Apache Arrow Issue

[jira] [Created] (ARROW-7475) [Rust] Create Arrow Stream writer

2019-12-29 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7475: - Summary: [Rust] Create Arrow Stream writer Key: ARROW-7475 URL: https://issues.apache.org/jira/browse/ARROW-7475 Project: Apache Arrow Issue Type: Sub

[jira] [Created] (ARROW-7460) [Rust] Improve arithmetic kernels with autovec

2019-12-22 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7460: - Summary: [Rust] Improve arithmetic kernels with autovec Key: ARROW-7460 URL: https://issues.apache.org/jira/browse/ARROW-7460 Project: Apache Arrow Issue

[jira] [Created] (ARROW-7364) [Rust] Add cast options to cast kernel

2019-12-10 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7364: - Summary: [Rust] Add cast options to cast kernel Key: ARROW-7364 URL: https://issues.apache.org/jira/browse/ARROW-7364 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-7324) [Rust] Add Timezone to Timestamp

2019-12-04 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7324: - Summary: [Rust] Add Timezone to Timestamp Key: ARROW-7324 URL: https://issues.apache.org/jira/browse/ARROW-7324 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-7207) [Rust] Update Generated Flatbuffer Files

2019-11-19 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7207: - Summary: [Rust] Update Generated Flatbuffer Files Key: ARROW-7207 URL: https://issues.apache.org/jira/browse/ARROW-7207 Project: Apache Arrow Issue Type

Re: [Help Needed] Arrow IPC Reader in Rust

2019-11-18 Thread Neville Dipale
___ > From: paddy horan > Sent: Saturday, November 16, 2019 1:03 PM > To: dev@arrow.apache.org > Subject: Re: [Help Needed] Arrow IPC Reader in Rust > > Hey Neville, > > I'll take a look if no-one beats me to it (I might not have time today or > tomorrow). > >

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-11-17 Thread Neville Dipale
ess most of my experience up to now is with Tonic > itself, I > > wanted to make myself available to you all for help or special > > implementations coming from the Tonic side. I am also willing to help > with > > the flight rust implementation. So please let me know a

[jira] [Created] (ARROW-7194) [Rust] CSV Writer causing recursion errors

2019-11-16 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-7194: - Summary: [Rust] CSV Writer causing recursion errors Key: ARROW-7194 URL: https://issues.apache.org/jira/browse/ARROW-7194 Project: Apache Arrow Issue Type

[Help Needed] Arrow IPC Reader in Rust

2019-11-15 Thread Neville Dipale
Hi Arrow developers, I'm "done" with the Arrow IPC Reader in Rust (for supported data types), but am having issues with reading some of the test data. Specifically, I've noticed that when reading the integration test data (primitve_generated), where I expect an array with 17 values, the arrow

[jira] [Created] (ARROW-6944) [Rust] Add StringType

2019-10-19 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-6944: - Summary: [Rust] Add StringType Key: ARROW-6944 URL: https://issues.apache.org/jira/browse/ARROW-6944 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-6928) [Rust] Add FixedSizeList type

2019-10-17 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-6928: - Summary: [Rust] Add FixedSizeList type Key: ARROW-6928 URL: https://issues.apache.org/jira/browse/ARROW-6928 Project: Apache Arrow Issue Type: Sub-task

Re: [DISCUSS] [Rust] Adding support for Flight protocol

2019-10-17 Thread Neville Dipale
> they might be interested in helping here too. > > Let me know how else I can help out with this effort. > > Andy. > > > > On Thu, Oct 17, 2019 at 2:42 PM Neville Dipale > wrote: > > > Good evening > > > > With support for testing against i

[jira] [Created] (ARROW-6650) [Rust] [Integration] Add method to generate JSON from RecordBatch

2019-09-20 Thread Neville Dipale (Jira)
Neville Dipale created ARROW-6650: - Summary: [Rust] [Integration] Add method to generate JSON from RecordBatch Key: ARROW-6650 URL: https://issues.apache.org/jira/browse/ARROW-6650 Project: Apache

[jira] [Created] (ARROW-5408) [Rust] Create struct array builder that creates null buffers

2019-05-23 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5408: - Summary: [Rust] Create struct array builder that creates null buffers Key: ARROW-5408 URL: https://issues.apache.org/jira/browse/ARROW-5408 Project: Apache Arrow

[jira] [Created] (ARROW-5400) [Rust] Test/ensure that reader and writer support zero-length record batches

2019-05-23 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5400: - Summary: [Rust] Test/ensure that reader and writer support zero-length record batches Key: ARROW-5400 URL: https://issues.apache.org/jira/browse/ARROW-5400 Project

[jira] [Created] (ARROW-5399) [Rust] [Testing] Add IPC test files to arrow-testing

2019-05-23 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5399: - Summary: [Rust] [Testing] Add IPC test files to arrow-testing Key: ARROW-5399 URL: https://issues.apache.org/jira/browse/ARROW-5399 Project: Apache Arrow

[jira] [Created] (ARROW-5367) [Rust] Add temporal kernels

2019-05-18 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5367: - Summary: [Rust] Add temporal kernels Key: ARROW-5367 URL: https://issues.apache.org/jira/browse/ARROW-5367 Project: Apache Arrow Issue Type: New Feature

[jira] [Created] (ARROW-5366) [Rust] Implement Duration and Interval Types

2019-05-18 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5366: - Summary: [Rust] Implement Duration and Interval Types Key: ARROW-5366 URL: https://issues.apache.org/jira/browse/ARROW-5366 Project: Apache Arrow Issue

[jira] [Created] (ARROW-5360) [Rust] Builds are broken by rustyline on nightly 2019-05-16+

2019-05-17 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5360: - Summary: [Rust] Builds are broken by rustyline on nightly 2019-05-16+ Key: ARROW-5360 URL: https://issues.apache.org/jira/browse/ARROW-5360 Project: Apache Arrow

[jira] [Created] (ARROW-5352) [Rust] BinaryArray filter loses replaces nulls with empty strings

2019-05-16 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5352: - Summary: [Rust] BinaryArray filter loses replaces nulls with empty strings Key: ARROW-5352 URL: https://issues.apache.org/jira/browse/ARROW-5352 Project: Apache

[jira] [Created] (ARROW-5351) [Rust] Add support for take kernel functions

2019-05-16 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5351: - Summary: [Rust] Add support for take kernel functions Key: ARROW-5351 URL: https://issues.apache.org/jira/browse/ARROW-5351 Project: Apache Arrow Issue

[jira] [Created] (ARROW-5350) [Rust] Support filtering on nested array types

2019-05-16 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5350: - Summary: [Rust] Support filtering on nested array types Key: ARROW-5350 URL: https://issues.apache.org/jira/browse/ARROW-5350 Project: Apache Arrow Issue

[Rust] [Discuss] Generalising Eq, Neq Kernel Functions Beyond Numeric Types

2019-05-13 Thread Neville Dipale
Hi Arrow[Rust] developers, I came across an instance where I wanted to compare 2 arrays that aren't numeric (bool, string, list?), and couldn't conveniently leverage the comparison array_ops for this. This is due to the trait bounds that require that PrimitiveArray satisfy T: ArrowNumericType.

Re: [ANNOUNCE] New Arrow committer: Neville Dipale

2019-05-13 Thread Neville Dipale
turday, May 11, 2019, paddy horan > > > > > wrote: > > > > > > > > > > > > > > > Congrats Neville! Thank you for your contributions! > > > > > > > > > > > > > > > > Get Outlook for iOS<https://aka.ms/

[jira] [Created] (ARROW-5303) [Rust] Add SIMD vectorization of numeric casts

2019-05-12 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5303: - Summary: [Rust] Add SIMD vectorization of numeric casts Key: ARROW-5303 URL: https://issues.apache.org/jira/browse/ARROW-5303 Project: Apache Arrow Issue

[Rust] [Format] Should Null Bitmaps be Padded to 8 or 64 Bits?

2019-04-26 Thread Neville Dipale
Hi Arrow developers, I'm currently working on IPC in Rust, specifically reading Arrow files. I've noticed that null buffers/bitmaps are always padded to 64 bits (from pyarrow, not sure about others), while in Rust we pad to 8 bits. 1. Is this fine re. Rust per the spec? I'm having issues with

Re: Proper way to retrigger Travis CI builds

2019-04-25 Thread Neville Dipale
To add here, sometimes builds for unrelated changes are caused by your branch being behind master. I've noticed that whenever I rebase my changes to latest master, I reliably only trigger the Rust jobs to run. Maybe that could also help non-Arrow commiters :) On Thu, 25 Apr 2019 at 14:11, Wes

[jira] [Created] (ARROW-5191) [Rust] Expose schema in readers (CSV, JSON) without reading batches

2019-04-21 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5191: - Summary: [Rust] Expose schema in readers (CSV, JSON) without reading batches Key: ARROW-5191 URL: https://issues.apache.org/jira/browse/ARROW-5191 Project: Apache

[jira] [Created] (ARROW-5188) [Rust] Add temporal builders for StructArray

2019-04-19 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5188: - Summary: [Rust] Add temporal builders for StructArray Key: ARROW-5188 URL: https://issues.apache.org/jira/browse/ARROW-5188 Project: Apache Arrow Issue

[jira] [Created] (ARROW-5187) [Rust] Ability to flatten StructArray into a RecordBatch

2019-04-19 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5187: - Summary: [Rust] Ability to flatten StructArray into a RecordBatch Key: ARROW-5187 URL: https://issues.apache.org/jira/browse/ARROW-5187 Project: Apache Arrow

[jira] [Created] (ARROW-5182) [Rust] Create Arrow File writer

2019-04-17 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5182: - Summary: [Rust] Create Arrow File writer Key: ARROW-5182 URL: https://issues.apache.org/jira/browse/ARROW-5182 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-5181) [Rust] Create Arrow File reader

2019-04-17 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5181: - Summary: [Rust] Create Arrow File reader Key: ARROW-5181 URL: https://issues.apache.org/jira/browse/ARROW-5181 Project: Apache Arrow Issue Type: Sub-task

[jira] [Created] (ARROW-5180) [Rust] IPC Support

2019-04-17 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-5180: - Summary: [Rust] IPC Support Key: ARROW-5180 URL: https://issues.apache.org/jira/browse/ARROW-5180 Project: Apache Arrow Issue Type: New Feature

[jira] [Created] (ARROW-4968) [Rust] StructArray builder and From<> methods should check that field types match schema

2019-03-19 Thread Neville Dipale (JIRA)
Neville Dipale created ARROW-4968: - Summary: [Rust] StructArray builder and From<> methods should check that field types match schema Key: ARROW-4968 URL: https://issues.apache.org/jira/browse/ARRO

Re: Timeline for 0.13 Arrow release

2019-03-19 Thread Neville Dipale
When is the cut-off for PRs? We have a public holiday on Thursday, and I want to use that to finish off my work on array casting. If that'll be too late I can defer to the next release. On Tue, 19 Mar 2019, 12:34 Antoine Pitrou, wrote: > > The only potential blocker from my POV is >

Re: Timeline for 0.13 Arrow release

2019-03-19 Thread Neville Dipale
the JIRAs (e.g., test case to reproduce). Array currently > doesn't expose a bitmask API, and I don't think we need specialized > implementations for struct & list. > > Chao > > On Mon, Mar 18, 2019 at 8:24 PM Neville Dipale > wrote: > > > Hi Wes, > > > >

  1   2   >