Re: [DISCUSS] Big Endian support in Arrow (was: Re: [Java] Supporting Big Endian)

2020-09-13 Thread Micah Kornfield
If I read the responses so far it seems like the following might be a good compromise/summary: 1. It does not seem too invasive to support native endianness in implementation libraries. As long as there is appropriate performance testing and CI infrastructure to demonstrate the changes work. 2.

[PSA} PR Queue is at 93

2020-09-13 Thread Micah Kornfield
Could respective language maintainers try to look at PRs and triage/close them out? Thanks, Micah

Re: Julia implementation and integration with main apache arrow repository

2020-09-13 Thread Wes McKinney
Would be great to see Julia working together with the broader Apache Arrow community. I attempted unsuccessfully in the past several times to engage with other developers. Since this code was developed outside of the community, we need to decide whether the full IP clearance process is needed

Re: Julia implementation and integration with main apache arrow repository

2020-09-13 Thread Neal Richardson
Hi Jacob, Yes, this is exciting! My recommendation would be to fork apache/arrow, add a `julia` directory, copy the contents of https://github.com/JuliaData/Arrow.jl in there, and put up a pull request for review. Then we can discuss specifics there. There will be a few other steps to do that I

Re: MacOS CI issue

2020-09-13 Thread Neal Richardson
Yes, there are broader grpc packaging issues involved. See https://github.com/grpc/grpc/issues/24077. I worked around the re2 one in the PR that Kou cited, but that made it fail later on c-ares for a similar reason. Kou may further ideas for working around, but for now don't be concerned if these

Julia implementation and integration with main apache arrow repository

2020-09-13 Thread Jacob Quinn
Hello all, Hopefully this email works (I'm not super familiar with using mailing lists like this). Over the past few weeks, I've been working on a pure Julia implementation to support serializing/deserializing the arrow format for Julia. The code in its current state can be found here:

Re: Implementation independent __arrow_array__

2020-09-13 Thread Joris Van den Bossche
In addition to Wes' reference to the Arrow C data interface, I think it is also important to clarify some aspects. In numpy, you have the "array interface" (`__array_interface__` property) and the "array dunder method" (`__array__` method). When speaking about the array protocol typically the

[NIGHTLY] Arrow Build Report for Job nightly-2020-09-13-0

2020-09-13 Thread Crossbow
Arrow Build Report for Job nightly-2020-09-13-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-09-13-0 Failed Tasks: - homebrew-cpp: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-09-13-0-travis-homebrew-cpp -