Re: [DISCUSS] Big Endian support in Arrow (was: Re: [Java] Supporting Big Endian)

In case any one wants to comment further, I've opened https://github.com/apache/arrow/pull/8374 to canonicalize the details. On Mon, Sep 28, 2020 at 9:08 PM Micah Kornfield wrote: > OK, I will try to update documentation ref

Re: Dictionary key access in python/generally

I can't speak to whether Pandas conversion will ever change. Some one else can potentially chime in I don't recall any JIRAs recently changing this type of conversion, however currently for library functionality there aren't any hard guarantees for backwards compatibility (generally we try to do o

Re: 2.0.0 release timeline: October 9

There is still some work left to make the packaging builds pass on the PR. Considering how close we are to the release I find it risky to include that change to 2.0. So I'm in favor of postponing it to 3.0. On Wed, Oct 7, 2020 at 11:10 PM Micah Kornfield wrote: > > I agree with Antoine that we s

Re: 2.0.0 release timeline: October 9

I agree with Antoine that we shouldn't be making changes to dependency versions so close to a release. This is consistent with other types of changes that could have a potentially large blast radius I don't have a strong opinion on what version we end up with though (would need to do more resear

Re: 2.0.0 release timeline: October 9

On Wed, Oct 7, 2020 at 1:11 PM Antoine Pitrou wrote: > > Le 07/10/2020 à 21:55, Neal Richardson a écrit : > > * The only version that is a requirement is > > > https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516 > , > > and so that's the one we're concerned

Re: 2.0.0 release timeline: October 9

Le 07/10/2020 à 21:55, Neal Richardson a écrit : > * The only version that is a requirement is > https://github.com/apache/arrow/pull/8325/files#diff-2420b0c5b6bdad921f1d538f92d64b59R2516, > and so that's the one we're concerned about increasing. If we can keep it > low with an #ifdef, great. Tha

Re: 2.0.0 release timeline: October 9

On Wed, 7 Oct 2020 12:46:22 -0700 James Duong wrote: > I could add a #ifdef around this, however gRPC itself doesn't appear to > provide a version macro. Hmm, can you please report an issue upstream? This could be more generally useful. > We also need a macro to define what namespace we get the

Re: 2.0.0 release timeline: October 9

I am in no way in charge to make a decision here--I'm just the one nagging everyone to get their patches merged by Friday ;)--but my personal thoughts when looking at https://github.com/apache/arrow/pull/8325: * The code and CI config have gRPC versions in various places. Before a release is a gre

Re: 2.0.0 release timeline: October 9

I could add a #ifdef around this, however gRPC itself doesn't appear to provide a version macro. So the person building flight would have to define this option themselves. We also need a macro to define what namespace we get the new API from -- it changes in gRPC 1.32 and above. How about I go ah

Re: 2.0.0 release timeline: October 9

Or perhaps we (meaning James :-)) can add an #ifdef-based switch. There's no need to penalize users of older gRPCs for a rather optional feature (if you can call disabling a security verification a feature, of course ;-)). Regards Antoine. Le 07/10/2020 à 21:33, Wes McKinney a écrit : > Give

Re: 2.0.0 release timeline: October 9

Given Google's "live at head" mantra, in principle I don't have a problem with requiring a < 1 year old version of gRPC On Wed, Oct 7, 2020 at 2:23 PM Antoine Pitrou wrote: > > > Le 07/10/2020 à 21:19, James Duong a écrit : > > Hi Neal, > > > > Are you the release manager for 2.0? > > I've been w

Re: 2.0.0 release timeline: October 9

Le 07/10/2020 à 21:19, James Duong a écrit : > Hi Neal, > > Are you the release manager for 2.0? > I've been working on the task to disable server verification in Flight > clients, and it appears we'll need > to update the minimum gRPC version to at least 1.27 to support this. > > Would it be O

Re: 2.0.0 release timeline: October 9

Hi Neal, Are you the release manager for 2.0? I've been working on the task to disable server verification in Flight clients, and it appears we'll need to update the minimum gRPC version to at least 1.27 to support this. Would it be OK to do this for the 2.0. release? It looks like we also need t

Re: Dictionary key access in python/generally

> > But I'm also attaching table > metadata to each feather, which I'd hate to lose. > Note the arrow format allows attaching custom metadata at the column (field), schema, and message level, so it should be possible to retain any metadata this way. -Jacob On Wed, Oct 7, 2020 at 11:38 AM Benjami

Dictionary key access in python/generally

Hello, Exciting project, thanks for all your work. I gather it's appropriate to ask a use question here? Assuming so: I have a web application that serves portions of a dataset I've broken into a few thousand featherV2 files structured as a quadtree. The structure makes heavy use of text dictiona

[NIGHTLY] Arrow Build Report for Job nightly-2020-10-07-0

Arrow Build Report for Job nightly-2020-10-07-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-10-07-0 Failed Tasks: - conda-linux-gcc-py37-aarch64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-10-07-0-drone-conda-linux-gcc-py3