Re: Missing hypothesis module in PyArrow

2019-01-08 Thread Wes McKinney
hi Andrew, On Python 2.7 you need to run both pip install -r requirements.txt pip install -r requirements-test.txt It looks like your CMake version is old so ZSTD was disabled. zstd cannot be built automatically from source for CMake versions less than 3.7 You will have a better time if you use

[DRAFT] Apache Arrow board report January 2019

2019-01-08 Thread Wes McKinney
## Description: - Apache Arrow is a cross-language development platform for in-memory data. It specifies a standardized language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware. It also provides computational l

Compiling Arrow for RaspberryPi

2019-01-08 Thread Suvayu Ali
Hi everyone, I wanted run a long running data collection process on an RPi. But it has been proven difficult to install pyarrow with pip as it still needs to compile. Is it possible to I cross-compile it on my desktop? If so, could someone point me in the right direction? Cheers, -- Suvayu

Missing hypothesis module in PyArrow

2019-01-08 Thread Andrew Palumbo
Hello, I'm just building arrow from source from a fresh checkout; commit: 326015cfc66e1f657cdd6811620137e9e277b43d Everything seems to build against python 2.7: $python setup.py build_ext --build-type=$ARROW_BUILD_TYPE--with-parquet --with-plasma --inplace {...} Bundling includes: release

Re: Timeline for Arrow 0.12.0 release

2019-01-08 Thread Wes McKinney
Well, a mid-week release candidate isn't looking too likely. I've spent most of today working on the Gandiva Windows build and haven't been able to do much 0.12 backlog items yesterday or today so far. Krisztian -- is your GPG key in KEYS yet? If not, you cannot cut a release. Out of the remainin

Re: RecordBatchFile with no batches, Error: Pyarrow.lib.ArrowInvalid: File is smaller than indicated metadata size.

2019-01-08 Thread Wes McKinney
I think I fixed this in master. Are you able to build from source to try it out? I am hopeful that sometime this year my team and I can provide a conda channel with nightly Arrow builds to help with testing and development On Tue, Jan 8, 2019 at 1:49 PM White4, Ryan (STATCAN) wrote: > > Hi, > >

Re: Open Zulip server for the Ursa Labs organization

2019-01-08 Thread Wes McKinney
These are not Arrow community discussions from a governance / project decision-making perspective (those need to happen here or JIRA or GitHub -- "mailing list or it didn't happen"), but I don't have any problem with the logs being published publicly. I don't think Zulip offers a live public view l

RecordBatchFile with no batches, Error: Pyarrow.lib.ArrowInvalid: File is smaller than indicated metadata size.

2019-01-08 Thread White4, Ryan (STATCAN)
Hi, I get an error when writing a file with no record batches. I came across this when implementing a simple way to spill the buffer to disk automatically (this is potentially coming in release 0.12???). I'm using pyarrow 0.11. Is there a JIRA related to this, or is there a problem in this simp

Re: Open Zulip server for the Ursa Labs organization

2019-01-08 Thread Jason Grout
I'm curious: are the discussions (either live or as logs) going to be public? Jason On Tue, Jan 8, 2019 at 10:08 AM Wes McKinney wrote: > On Tue, Jan 8, 2019 at 11:48 AM Enrico Olivelli > wrote: > > > > Il mar 8 gen 2019, 18:23 Wes McKinney ha scritto: > > > > > hi folks, > > > > > > The kin

[jira] [Created] (ARROW-4206) [Gandiva] Implement decimal divide

2019-01-08 Thread Pindikura Ravindra (JIRA)
Pindikura Ravindra created ARROW-4206: - Summary: [Gandiva] Implement decimal divide Key: ARROW-4206 URL: https://issues.apache.org/jira/browse/ARROW-4206 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-4204) [Gandiva] implement decimal subtract

2019-01-08 Thread Pindikura Ravindra (JIRA)
Pindikura Ravindra created ARROW-4204: - Summary: [Gandiva] implement decimal subtract Key: ARROW-4204 URL: https://issues.apache.org/jira/browse/ARROW-4204 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-4205) [Gandiva] Implement decimal multiply

2019-01-08 Thread Pindikura Ravindra (JIRA)
Pindikura Ravindra created ARROW-4205: - Summary: [Gandiva] Implement decimal multiply Key: ARROW-4205 URL: https://issues.apache.org/jira/browse/ARROW-4205 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-4203) [Gandiva] use aliases when building expressions to simplify tests

2019-01-08 Thread Pindikura Ravindra (JIRA)
Pindikura Ravindra created ARROW-4203: - Summary: [Gandiva] use aliases when building expressions to simplify tests Key: ARROW-4203 URL: https://issues.apache.org/jira/browse/ARROW-4203 Project: Ap

[jira] [Created] (ARROW-4202) [Gandiva] use ArrayFromJson in tests

2019-01-08 Thread Pindikura Ravindra (JIRA)
Pindikura Ravindra created ARROW-4202: - Summary: [Gandiva] use ArrayFromJson in tests Key: ARROW-4202 URL: https://issues.apache.org/jira/browse/ARROW-4202 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-4201) [Gandiva] integrate test utils with arrow

2019-01-08 Thread Pindikura Ravindra (JIRA)
Pindikura Ravindra created ARROW-4201: - Summary: [Gandiva] integrate test utils with arrow Key: ARROW-4201 URL: https://issues.apache.org/jira/browse/ARROW-4201 Project: Apache Arrow Issu

Re: Open Zulip server for the Ursa Labs organization

2019-01-08 Thread Wes McKinney
On Tue, Jan 8, 2019 at 11:48 AM Enrico Olivelli wrote: > > Il mar 8 gen 2019, 18:23 Wes McKinney ha scritto: > > > hi folks, > > > > The kind folks at https://zulipchat.com/ have offered free hosting to > > our not-for-profit organization Ursa Labs > > > > https://ursalabs.zulipchat.com > > > > T

Re: Open Zulip server for the Ursa Labs organization

2019-01-08 Thread Enrico Olivelli
Il mar 8 gen 2019, 18:23 Wes McKinney ha scritto: > hi folks, > > The kind folks at https://zulipchat.com/ have offered free hosting to > our not-for-profit organization Ursa Labs > > https://ursalabs.zulipchat.com > > The Arrow developer mailing list is the primary mode of communication > for de

Open Zulip server for the Ursa Labs organization

2019-01-08 Thread Wes McKinney
hi folks, The kind folks at https://zulipchat.com/ have offered free hosting to our not-for-profit organization Ursa Labs https://ursalabs.zulipchat.com The Arrow developer mailing list is the primary mode of communication for development work and project governance, but we may set up various au

[jira] [Created] (ARROW-4200) [C++] conda_env_* files cannot be used to create a fresh conda environment on Windows

2019-01-08 Thread Wes McKinney (JIRA)
Wes McKinney created ARROW-4200: --- Summary: [C++] conda_env_* files cannot be used to create a fresh conda environment on Windows Key: ARROW-4200 URL: https://issues.apache.org/jira/browse/ARROW-4200 Pro

[jira] [Created] (ARROW-4199) [GLib] Add garrow_seekable_input_stream_peek()

2019-01-08 Thread Yosuke Shiro (JIRA)
Yosuke Shiro created ARROW-4199: --- Summary: [GLib] Add garrow_seekable_input_stream_peek() Key: ARROW-4199 URL: https://issues.apache.org/jira/browse/ARROW-4199 Project: Apache Arrow Issue Type:

[jira] [Created] (ARROW-4198) [Gandiva] Add support to cast timestamp

2019-01-08 Thread shyam narayan singh (JIRA)
shyam narayan singh created ARROW-4198: -- Summary: [Gandiva] Add support to cast timestamp Key: ARROW-4198 URL: https://issues.apache.org/jira/browse/ARROW-4198 Project: Apache Arrow Issu

[jira] [Created] (ARROW-4197) Emscripten compiler fails building Arrow

2019-01-08 Thread Timothy Paine (JIRA)
Timothy Paine created ARROW-4197: Summary: Emscripten compiler fails building Arrow Key: ARROW-4197 URL: https://issues.apache.org/jira/browse/ARROW-4197 Project: Apache Arrow Issue Type: Bug

Re: Compiling C++ Arrow Flight

2019-01-08 Thread Wes McKinney
That's great. Any feedback (whether feature needs for your use cases / security / configurability) would be useful as more development time in spent on Flight this year On Mon, Jan 7, 2019 at 6:24 PM Tim Bisson wrote: > > I was finally able to build and run the flight-test-server/flight-test > bi

[jira] [Created] (ARROW-4196) [Rust] Add explicit SIMD vectorization for ops in "array_ops"

2019-01-08 Thread Paddy Horan (JIRA)
Paddy Horan created ARROW-4196: -- Summary: [Rust] Add explicit SIMD vectorization for ops in "array_ops" Key: ARROW-4196 URL: https://issues.apache.org/jira/browse/ARROW-4196 Project: Apache Arrow

Re: [Rust] [DISCUSS] Donate DataFusion to Arrow project

2019-01-08 Thread Wes McKinney
hi Andy -- I'm supportive of the code donation. I see building in-memory, embeddable analytics and query processing as the natural next stage of this project. As I have described on this mailing list, I intend to work on this with my colleagues in C++ with the goal of making such functionality avai

[jira] [Created] (ARROW-4195) [Rust] Ability to create Arrow native types from Rust primitives/types

2019-01-08 Thread nevi_me (JIRA)
nevi_me created ARROW-4195: -- Summary: [Rust] Ability to create Arrow native types from Rust primitives/types Key: ARROW-4195 URL: https://issues.apache.org/jira/browse/ARROW-4195 Project: Apache Arrow

Re: [Rust] [DISCUSS] Donate DataFusion to Arrow project

2019-01-08 Thread Neville Dipale
Hi Andy, I can't comment on the voting process, but regarding the addition of DataFusion: I support the idea to donate the code, mainly as I think that will help us accelerate some work on Rust. Out of curiousity, I've been prototying a 'Rust dataframe' abstraction which (can/will) have various s

Re: [Rust] [DISCUSS] Donate DataFusion to Arrow project

2019-01-08 Thread Andy Grove
Bumping this thread ... I know everyone is busy with getting the 0.12 release out, but would be good to know the process for raising this for a vote. However, given the lack of comments on this thread I'm starting to suspect that maybe there isn't much of an appetite for this, which is fine, but wo

[jira] [Created] (ARROW-4194) [Format] Metadata.rst does not specify timezone for Timestamp type

2019-01-08 Thread Andy Grove (JIRA)
Andy Grove created ARROW-4194: - Summary: [Format] Metadata.rst does not specify timezone for Timestamp type Key: ARROW-4194 URL: https://issues.apache.org/jira/browse/ARROW-4194 Project: Apache Arrow

[jira] [Created] (ARROW-4192) "./dev/run_docker_compose.sh" is out of date

2019-01-08 Thread Antoine Pitrou (JIRA)
Antoine Pitrou created ARROW-4192: - Summary: "./dev/run_docker_compose.sh" is out of date Key: ARROW-4192 URL: https://issues.apache.org/jira/browse/ARROW-4192 Project: Apache Arrow Issue Typ

[jira] [Created] (ARROW-4193) [Rust] Add Decimal to DataType enum

2019-01-08 Thread Andy Grove (JIRA)
Andy Grove created ARROW-4193: - Summary: [Rust] Add Decimal to DataType enum Key: ARROW-4193 URL: https://issues.apache.org/jira/browse/ARROW-4193 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-4191) [C++] Use same CC and AR for jemalloc as for the main sources

2019-01-08 Thread Uwe L. Korn (JIRA)
Uwe L. Korn created ARROW-4191: -- Summary: [C++] Use same CC and AR for jemalloc as for the main sources Key: ARROW-4191 URL: https://issues.apache.org/jira/browse/ARROW-4191 Project: Apache Arrow

[jira] [Created] (ARROW-4190) [C++] memcpy_ssse3_back Segmentation fault

2019-01-08 Thread ShuMing Li (JIRA)
ShuMing Li created ARROW-4190: - Summary: [C++] memcpy_ssse3_back Segmentation fault Key: ARROW-4190 URL: https://issues.apache.org/jira/browse/ARROW-4190 Project: Apache Arrow Issue Type: Bug