Re: Rust/Datafusion sort kernel issues

2020-08-28 Thread Andy Grove
Hi Jörn, I agree with your concerns about NaN. There was a discussion about this in https://github.com/apache/arrow/pull/7193 I will try and make time this weekend to look at the current implementation and your suggestions around DictionaryArray. Hopefully, other contributors that are more

Re: Arrow Dataset API on Ceph

2020-08-28 Thread Ivo Jimenez
Hi Antoine > Yes, that is our plan. Since this is going to be done on the storage-, > > server-side, this would be transparent to the client. So our main concern > > is whether this be OK from the design perspective, and could this > > eventually be merged upstream? > > Arrow datasets have no

Re: Subject: [VOTE] Release Apache Arrow 1.0.1 - RC0

2020-08-28 Thread Neal Richardson
CRAN has finally accepted the R package release, so I think that closes the post-release tasks for 1.0.1 Neal On Mon, Aug 24, 2020 at 1:58 PM Krisztián Szűcs wrote: > Thank you Neal! > > The conda packages have been updated as well. > > 2. [done] upload source > 3. [done] upload binaries >

Rust/Datafusion sort kernel issues

2020-08-28 Thread Jörn Horstmann
I ran into a few issues with the rust sort kernels and would like to discuss possible solutions. 1. When sorting by multiple columns (lexsort_to_indices) the Float32 and Float64 data types are not supported because the implementation relies on the OrdArray trait. This trait is not implemented

[NIGHTLY] Arrow Build Report for Job nightly-2020-08-28-0

2020-08-28 Thread Crossbow
Arrow Build Report for Job nightly-2020-08-28-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-08-28-0 Failed Tasks: - conda-osx-clang-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-08-28-0-azure-conda-osx-clang-py36 -

Re: Arrow Dataset API on Ceph

2020-08-28 Thread Antoine Pitrou
Le 27/08/2020 à 21:55, Ivo Jimenez a écrit : > Hi Antoine, > >> Our main concern is that this new arrow::dataset::RadosFormat class will >> be >>> deriving from the arrow::dataset::FileFormat class, which seems to raise >> a >>> conceptual mismatch as there isn’t really a RADOS format but

Re: Ursabot Benchmark framework for other languages

2020-08-28 Thread Kazuaki Ishizaki
Hi François, Thank you very much for your response with very helpful information. I realized that this command is launched as follows at https://ci.ursalabs.org/#/builders/73/builds/100/steps/3/logs/stdio $ archery benchmark diff --output=diff.json --suite-filter=parquet-encoding-benchmark