Re: Thank you

2020-08-27 Thread Lucas Pickup
This is an awesome sentimate, thank you Release orchestrstors and contributors! Cheers, Lucas On Thu, Aug 27, 2020 at 1:26 PM Jorge Cardoso Leitão < jorgecarlei...@gmail.com> wrote: > Hi, > > > > I am writing to just thank all those involved in the release process. > > Sometimes the work of

Thank you

2020-08-27 Thread Jorge Cardoso Leitão
Hi, I am writing to just thank all those involved in the release process. Sometimes the work of releases is not fully appreciated within development (where are the PRs ^_^?), but I find it impressive that the release is so smooth for such a complex project, and IMO that is to a large extent due

Re: Arrow Dataset API on Ceph

2020-08-27 Thread Ivo Jimenez
Hi Antoine, > Our main concern is that this new arrow::dataset::RadosFormat class will > be > > deriving from the arrow::dataset::FileFormat class, which seems to raise > a > > conceptual mismatch as there isn’t really a RADOS format but rather a > > formatting/serialization deferral that will be

Re: Ursabot Benchmark framework for other languages

2020-08-27 Thread Francois Saint-Jacques
Hello Kazuaki! I recommend you read and take a look at the benchmark sub-library [1] of archery and how it's glued [2]. You will need to implement: - A runner for the framework you intend to use [3] and [4], it also implies capturing the output into a class that implements the "Benchmark"

Ursabot Benchmark framework for other languages

2020-08-27 Thread Kazuaki Ishizaki
I am new to the Ursabot benchmark framework like https://github.com/apache/arrow/pull/7940#issuecomment-673183390. How can we run benchmark programs written in other languages (e.g. Java) beyond C++? If it is not supported yet, what do we need to support to run benchmark programs in other

[NIGHTLY] Arrow Build Report for Job nightly-2020-08-27-0

2020-08-27 Thread Crossbow
Arrow Build Report for Job nightly-2020-08-27-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-08-27-0 Failed Tasks: - conda-osx-clang-py36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-08-27-0-azure-conda-osx-clang-py36 -

Re: Writing parquet to new filesystem API

2020-08-27 Thread Joris Van den Bossche
Small correction, I of course meant to use "write_table" and not "write_to_dataset" in the code snippet (as the latter won't work that way). Corrected example below: But, if you already want to use the new filesystems for writing as well, > there is one workaround to create an output stream

Re: Writing parquet to new filesystem API

2020-08-27 Thread Joris Van den Bossche
Hi Weston, You are not missing something obvious, but this is a bit an unfortunate "transitional phase" where we have new filesystems, but they are not yet fully supported (on the reading side they are supported in pyarrow 1.0, but for the writing side we are actively working on that, which will

Re: Arrow Dataset API on Ceph

2020-08-27 Thread Antoine Pitrou
Hello Ivo, Le 27/08/2020 à 07:02, Ivo Jimenez a écrit : > > Our main concern is that this new arrow::dataset::RadosFormat class will be > deriving from the arrow::dataset::FileFormat class, which seems to raise a > conceptual mismatch as there isn’t really a RADOS format but rather a >