[jira] [Created] (ARROW-7224) [C++][Datasets] Partition level filters should be able to provide filtering to file systems

2019-11-20 Thread Micah Kornfield (Jira)
Micah Kornfield created ARROW-7224: -- Summary: [C++][Datasets] Partition level filters should be able to provide filtering to file systems Key: ARROW-7224 URL: https://issues.apache.org/jira/browse/ARROW-7224

Re: [Discuss][Java] Provide default for io.netty.tryReflectionSetAccessible to prevent errors

2019-11-20 Thread Bryan Cutler
I'm not sure what the best way to handle this is. Ideally we would use an alternative that doesn't require setting a property, but I don't know Netty well enough to make a recommendation. I also want to be careful not to introduce anything that would hurt performance or cause any other side

[jira] [Created] (ARROW-7223) [Java] Provide default setting of io.netty.tryReflectionSetAccessible=true

2019-11-20 Thread Bryan Cutler (Jira)
Bryan Cutler created ARROW-7223: --- Summary: [Java] Provide default setting of io.netty.tryReflectionSetAccessible=true Key: ARROW-7223 URL: https://issues.apache.org/jira/browse/ARROW-7223 Project:

[DISCUSS][C++] Pointer name aliasing

2019-11-20 Thread Micah Kornfield
A recent PR for datasets [1] seems to have introduced the convention of aliasing "std::shared_ptr" with "TypePtr" for some type. I think in the past we had decided not to use a convention like this but I can't find the thread. IMO, I think this generally makes the code less understandable but

[VOTE] Clarifications and forward compatibility changes for Dictionary Encoding (second iteration)

2019-11-20 Thread Micah Kornfield
Hello, As discussed on [1], I've proposed clarifications in a PR [2] that clarifies: 1. It is not required that all dictionary batches occur at the beginning of the IPC stream format (if a the first record batch has an all null dictionary encoded column, the null column's dictionary might not be

[jira] [Created] (ARROW-7222) [Python] Wipe any existing generated Python API documentation when updating website

2019-11-20 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7222: --- Summary: [Python] Wipe any existing generated Python API documentation when updating website Key: ARROW-7222 URL: https://issues.apache.org/jira/browse/ARROW-7222

Re: ConcatenateTables APIs

2019-11-20 Thread Wes McKinney
I agree with introducing ConcatenateTables with an options argument (which can have parameters added to it without disrupting public APIs too much). It would be good to do this sooner rather than later On Fri, Nov 15, 2019 at 12:22 AM Micah Kornfield wrote: > > This sounds like a reasonable

Re: Building Arrow 0.15.1 using dependencies in local source folder

2019-11-20 Thread Wes McKinney
I agree that the *_ROOT variables should be the way. If you find one that does not work, please open a JIRA issue. I don't think this is documented well enough in http://arrow.apache.org/docs/developers/cpp.html#build-dependency-management so I'm opening an issue

[jira] [Created] (ARROW-7221) [C++][Documentation] Document how to set installed location for individual toolchain components

2019-11-20 Thread Wes McKinney (Jira)
Wes McKinney created ARROW-7221: --- Summary: [C++][Documentation] Document how to set installed location for individual toolchain components Key: ARROW-7221 URL: https://issues.apache.org/jira/browse/ARROW-7221

Re: [NIGHTLY] Arrow Build Report for Job nightly-2019-11-20-0

2019-11-20 Thread Krisztián Szűcs
Currently I'm busy with migrating our development docker images to the apache organization [1], but afterwards I'll go through the failures. Any help is appreciated! [1]: https://issues.apache.org/jira/browse/ARROW-7116 On Wed, Nov 20, 2019 at 2:01 PM Crossbow wrote: > > > Arrow Build Report

[jira] [Created] (ARROW-7220) [CI] Docker compose (github actions) Mac Python 3 build is using Python 2

2019-11-20 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7220: Summary: [CI] Docker compose (github actions) Mac Python 3 build is using Python 2 Key: ARROW-7220 URL: https://issues.apache.org/jira/browse/ARROW-7220

[jira] [Created] (ARROW-7219) [CI][Python] Install pickle5 in the conda-python docker image for python version 3.6

2019-11-20 Thread Krisztian Szucs (Jira)
Krisztian Szucs created ARROW-7219: -- Summary: [CI][Python] Install pickle5 in the conda-python docker image for python version 3.6 Key: ARROW-7219 URL: https://issues.apache.org/jira/browse/ARROW-7219

[jira] [Created] (ARROW-7218) [Python] Conversion from boolean numpy scalars not working

2019-11-20 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7218: Summary: [Python] Conversion from boolean numpy scalars not working Key: ARROW-7218 URL: https://issues.apache.org/jira/browse/ARROW-7218 Project:

[NIGHTLY] Arrow Build Report for Job nightly-2019-11-20-0

2019-11-20 Thread Crossbow
Arrow Build Report for Job nightly-2019-11-20-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-11-20-0 Failed Tasks: - conda-osx-clang-py27: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2019-11-20-0-azure-conda-osx-clang-py27 -

Re: MIME type

2019-11-20 Thread Antoine Pitrou
If it's not standardized, shouldn't it be prefixed with x-? e.g. application/x-apache-arrow-stream Le 20/11/2019 à 08:29, Micah Kornfield a écrit : > I would propose: > application/apache-arrow-stream > application/apache-arrow-file > > I'm not attached to those names but I think there

Re: [Discuss][Java] Provide default for io.netty.tryReflectionSetAccessible to prevent errors

2019-11-20 Thread Fan Liya
Hi Bryan, Thanks for bringing this up. +1 for the change. I am not clear what is the right place to override the jvm property. It is possible that when we try to override it (possibly in a static block), the old property value has already been read by netty library. To avoid this problem, do we

[jira] [Created] (ARROW-7217) Docker compose / github actions ignores PYTHON env

2019-11-20 Thread Joris Van den Bossche (Jira)
Joris Van den Bossche created ARROW-7217: Summary: Docker compose / github actions ignores PYTHON env Key: ARROW-7217 URL: https://issues.apache.org/jira/browse/ARROW-7217 Project: Apache

[jira] [Created] (ARROW-7216) [Java] Improve the performance of setting/clearing individual bits

2019-11-20 Thread Liya Fan (Jira)
Liya Fan created ARROW-7216: --- Summary: [Java] Improve the performance of setting/clearing individual bits Key: ARROW-7216 URL: https://issues.apache.org/jira/browse/ARROW-7216 Project: Apache Arrow

[jira] [Created] (ARROW-7215) [C++][Gandiva] Implement castVARCHAR(integer_type) functions in Gandiva

2019-11-20 Thread Projjal Chanda (Jira)
Projjal Chanda created ARROW-7215: - Summary: [C++][Gandiva] Implement castVARCHAR(integer_type) functions in Gandiva Key: ARROW-7215 URL: https://issues.apache.org/jira/browse/ARROW-7215 Project: