Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
the discussion there. On Fri, Sep 10, 2021 at 1:45 AM Antoine Pitrou wrote: > > Le 10/09/2021 à 10:05, Philipp Moritz a écrit : > > Thanks for your answer Antoine! > > > > Considering your first comment, there is a section in > > https://www.python.org/dev/peps/pep-0571

Re: [Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
will need a bit more coordination across the Python community. I'm happy to get a discussion started over there. On Fri, Sep 10, 2021 at 12:40 AM Antoine Pitrou wrote: > > Le 10/09/2021 à 09:12, Philipp Moritz a écrit : > > Dear all, > > > > how do you think _GLIBCXX_U

[Python] manylinux2014 and _GLIBCXX_USE_CXX11_ABI setting

2021-09-10 Thread Philipp Moritz
Dear all, how do you think _GLIBCXX_USE_CXX11_ABI should be set for manylinux2014 (and manylinux2010) wheels? Should it be 0 or 1? Unfortunately I don't see https://www.python.org/dev/peps/pep-0599/ or https://github.com/pypa/manylinux specifying it. I think for manylinux1 the common wisdom was

[jira] [Created] (ARROW-7004) [Plasma] Make it possible to bump up object in LRU cache

2019-10-28 Thread Philipp Moritz (Jira)
Philipp Moritz created ARROW-7004: - Summary: [Plasma] Make it possible to bump up object in LRU cache Key: ARROW-7004 URL: https://issues.apache.org/jira/browse/ARROW-7004 Project: Apache Arrow

Re: [Discuss] C++ filenames: hyphens or underscores?

2019-08-06 Thread Philipp Moritz
I also have a small preference for underscores but would also be fine with dashes. It seems to be more common (therefore blends better with vendored code) and agrees with the styleguide and is closest to the exiting code. Also as an aside, having file_names names like variable_names is nice.

[jira] [Created] (ARROW-5904) [Java] [Plasma] Fix compilation of Plasma Java client

2019-07-10 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5904: - Summary: [Java] [Plasma] Fix compilation of Plasma Java client Key: ARROW-5904 URL: https://issues.apache.org/jira/browse/ARROW-5904 Project: Apache Arrow

[jira] [Created] (ARROW-5751) [Packaging][Python] Python 2.7 wheels broken on macOS: libcares.2.dylib not found

2019-06-26 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5751: - Summary: [Packaging][Python] Python 2.7 wheels broken on macOS: libcares.2.dylib not found Key: ARROW-5751 URL: https://issues.apache.org/jira/browse/ARROW-5751

[jira] [Created] (ARROW-5690) [Packaging] macOS wheels broken: libprotobuf.18.dylib missing

2019-06-22 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5690: - Summary: [Packaging] macOS wheels broken: libprotobuf.18.dylib missing Key: ARROW-5690 URL: https://issues.apache.org/jira/browse/ARROW-5690 Project: Apache Arrow

[jira] [Created] (ARROW-5671) [crossbow] mac os python wheels failing

2019-06-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5671: - Summary: [crossbow] mac os python wheels failing Key: ARROW-5671 URL: https://issues.apache.org/jira/browse/ARROW-5671 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-5670) [crossbow] mac os python 3.5 wheel failing

2019-06-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5670: - Summary: [crossbow] mac os python 3.5 wheel failing Key: ARROW-5670 URL: https://issues.apache.org/jira/browse/ARROW-5670 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-5669) [crossbow] manylinux1 wheel building failing

2019-06-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5669: - Summary: [crossbow] manylinux1 wheel building failing Key: ARROW-5669 URL: https://issues.apache.org/jira/browse/ARROW-5669 Project: Apache Arrow Issue

Re: [ANNOUNCE] New Arrow committer: Francois Saint-Jacques

2019-06-12 Thread Philipp Moritz
Congrats François :) On Wed, Jun 12, 2019 at 3:37 PM Antoine Pitrou wrote: > > Welcome on the team François :-) > > > Le 12/06/2019 à 17:45, Wes McKinney a écrit : > > On behalf of the Arrow PMC I'm happy to announce that Francois has > > accepted an invitation to become an Arrow committer! > >

Re: [ANNOUNCE] New Arrow committer: Neville Dipale

2019-05-11 Thread Philipp Moritz
Congrats Neville! On Sat, May 11, 2019 at 6:09 PM Renjie Liu wrote: > Congrats! > > Chao Sun 于 2019年5月12日周日 上午12:38写道: > > > Congrats Neville! > > > > On Sat, May 11, 2019 at 9:36 AM Micah Kornfield > > wrote: > > > > > Congrats!! > > > > > > On Saturday, May 11, 2019, paddy horan wrote: > >

Re: Use arrow as a general data serialization framework in distributed stream data processing

2019-04-25 Thread Philipp Moritz
Hey Shawn, Thanks for these benchmarks! This is indeed a workload we would like to support well in Arrow/Plasma/Ray (if you are using Ray, using Plasma as a shared memory transport but some of the issues this raises will apply more widely to Arrow and other possible IPC/RPC transports like Flight

Re: [VOTE] Add 64-bit offset list, binary, string (utf8) types to the Arrow columnar format

2019-04-25 Thread Philipp Moritz
+1 (binding) On Thu, Apr 25, 2019 at 1:34 PM Wes McKinney wrote: > +1 (binding) > > On Thu, Apr 25, 2019 at 3:33 PM Wes McKinney wrote: > > > > In a recent mailing list discussion [1] Micah Kornfield has proposed > > to add new list and variable-size binary and unicode types to the > > Arrow

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

2019-04-25 Thread Philipp Moritz
y PRs would need both C++ and Java implementation > before > > > being merged. > > > > > > Could a PMC member facilitate this? > > > > > > Philipp if approved, do you have bandwidth to finish up the PR for > > > LargeList? > > > > &

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

2019-04-16 Thread Philipp Moritz
@Micah: I wanted to make it possible to support serializing large objects (existing large pandas dataframes with an "object" column and also large python types with the pyarrow serialization). On Mon, Apr 15, 2019 at 8:22 PM Micah Kornfield wrote: > To summarize my understanding of the thread

Re: [DISCUSS] 64-bit offset variable width types (i.e.Large List, Last String, Large bytes)

2019-04-11 Thread Philipp Moritz
Thanks for getting the discussion started, Micah! I'm +1 on this change and also slightly prefer 1. As Antoine mentions, there doesn't seem to be a clear benefit from 2, unless we want to also support 8 or 16 bit indices in the future, which seems unlikely. So going with 1 is ok I think. Best,

Re: Support for numpy matrix

2019-03-30 Thread Philipp Moritz
Hey Mitar, It might be as simple as adding a handler here: https://github.com/apache/arrow/blob/master/python/pyarrow/serialization.py#L300 Do you want to try that? -- Philipp. On Sat, Mar 30, 2019 at 3:22 PM Mitar wrote: > Hi! > > I do not know where to start looking into this? Not sure if

[jira] [Created] (ARROW-5027) [Python] Add JSON Reader

2019-03-27 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5027: - Summary: [Python] Add JSON Reader Key: ARROW-5027 URL: https://issues.apache.org/jira/browse/ARROW-5027 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-5022) [C++] Implement more "Datum" types for AggregateKernel

2019-03-26 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5022: - Summary: [C++] Implement more "Datum" types for AggregateKernel Key: ARROW-5022 URL: https://issues.apache.org/jira/browse/ARROW-5022 Project: Ap

[jira] [Created] (ARROW-5002) [C++] Implement GroupBy

2019-03-24 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-5002: - Summary: [C++] Implement GroupBy Key: ARROW-5002 URL: https://issues.apache.org/jira/browse/ARROW-5002 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-4983) [Plasma] Unmap memory when the client is destroyed

2019-03-21 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4983: - Summary: [Plasma] Unmap memory when the client is destroyed Key: ARROW-4983 URL: https://issues.apache.org/jira/browse/ARROW-4983 Project: Apache Arrow

[jira] [Created] (ARROW-4958) [C++] Purely static linking broken

2019-03-18 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4958: - Summary: [C++] Purely static linking broken Key: ARROW-4958 URL: https://issues.apache.org/jira/browse/ARROW-4958 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-4912) [C++, Python] Allow specifying column names to CSV reader

2019-03-15 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4912: - Summary: [C++, Python] Allow specifying column names to CSV reader Key: ARROW-4912 URL: https://issues.apache.org/jira/browse/ARROW-4912 Project: Apache Arrow

[jira] [Created] (ARROW-4905) [C++][Plasma] Remove dlmalloc from client library

2019-03-15 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4905: - Summary: [C++][Plasma] Remove dlmalloc from client library Key: ARROW-4905 URL: https://issues.apache.org/jira/browse/ARROW-4905 Project: Apache Arrow

Re: CMake refactor Heads-up

2019-03-15 Thread Philipp Moritz
Thanks Uwe for doing this! It's amazing how well the arrow build system is working given it supports so many different configurations (which makes Uwe's refactor all the more impressive). Big thanks for everybody helping to maintain it. On Fri, Mar 15, 2019 at 8:08 AM Wes McKinney wrote: >

Re: [ANNOUNCE] New Arrow committer: Micah Kornfield

2019-03-08 Thread Philipp Moritz
Congrats Micah! On Fri, Mar 8, 2019 at 11:28 AM Wes McKinney wrote: > On behalf of the Arrow PMC, I'm happy to announce that Micah has > accepted an invitation to become a committer on Apache Arrow. > > Welcome, and thank you for your contributions! >

[jira] [Created] (ARROW-4797) [Plasma] Avoid store crash if not enough memory is available

2019-03-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4797: - Summary: [Plasma] Avoid store crash if not enough memory is available Key: ARROW-4797 URL: https://issues.apache.org/jira/browse/ARROW-4797 Project: Apache Arrow

[jira] [Created] (ARROW-4757) Nested chunked array support

2019-03-04 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4757: - Summary: Nested chunked array support Key: ARROW-4757 URL: https://issues.apache.org/jira/browse/ARROW-4757 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-4690) Building TensorFlow compatible wheels for Arrow

2019-02-26 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4690: - Summary: Building TensorFlow compatible wheels for Arrow Key: ARROW-4690 URL: https://issues.apache.org/jira/browse/ARROW-4690 Project: Apache Arrow Issue

Re: TensorFlow, PyTorch, and manylinux1

2019-02-06 Thread Philipp Moritz
manylinux2014 or whatever will support c++14. On Wed, Feb 6, 2019 at 9:14 AM Philipp Moritz wrote: > The problems arose if some functionality of C++11 were used. It > led to certain symbols being statically linked into the shared library > which clashed with other shared libraries

Re: TensorFlow, PyTorch, and manylinux1

2019-02-06 Thread Philipp Moritz
019 at 12:38 PM Antoine Pitrou > <mailto:anto...@python.org>> wrote: > > > > > > Le 06/02/2019 à 01:06, Philipp Moritz a écrit : > > > Thanks for the meeting! One question concerning a point that is > still > > > not super clear to me

[jira] [Created] (ARROW-4491) [Python] Remove usage of std::to_string and std::stoi

2019-02-05 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4491: - Summary: [Python] Remove usage of std::to_string and std::stoi Key: ARROW-4491 URL: https://issues.apache.org/jira/browse/ARROW-4491 Project: Apache Arrow

Re: TensorFlow, PyTorch, and manylinux1

2019-02-05 Thread Philipp Moritz
Thanks for the meeting! One question concerning a point that is still not super clear to me: Say we define a new manylinux standard based on gcc >=5 (with stable c++11 support). There will still be a lot of wheels form the manylinux1 days that are built against gcc 4.8 that might use the c++11

Re: [ANNOUNCE] New Arrow committer: Ravindra Pindikura

2019-02-04 Thread Philipp Moritz
Congratulations! On Mon, Feb 4, 2019 at 9:44 PM Ravindra Pindikura wrote: > Thanks everyone. It’s been a pleasure to work with the arrow community, > and I feel honored. > > > On Feb 5, 2019, at 9:28 AM, Bryan Cutler wrote: > > > > Congratulations Ravindra! > > > > On Mon, Feb 4, 2019, 7:40 PM

Re: [ANNOUNCE] New Arrow PMC member: Andy Grove

2019-02-04 Thread Philipp Moritz
Congratulations! On Mon, Feb 4, 2019 at 3:16 PM Krisztián Szűcs wrote: > Congrats Andy! :) > > On Mon, Feb 4, 2019 at 4:39 PM Wes McKinney wrote: > > > The Project Management Committee (PMC) for Apache Arrow has invited > > Andy Grove to become a PMC member and we are pleased to announce that

[jira] [Created] (ARROW-4453) [Python] Create Cython wrappers for sparse array

2019-02-01 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4453: - Summary: [Python] Create Cython wrappers for sparse array Key: ARROW-4453 URL: https://issues.apache.org/jira/browse/ARROW-4453 Project: Apache Arrow

[jira] [Created] (ARROW-4452) [Python] Serializing sparse torch tensors

2019-02-01 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4452: - Summary: [Python] Serializing sparse torch tensors Key: ARROW-4452 URL: https://issues.apache.org/jira/browse/ARROW-4452 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-4378) [Plasma] Release objects upon Create

2019-01-25 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4378: - Summary: [Plasma] Release objects upon Create Key: ARROW-4378 URL: https://issues.apache.org/jira/browse/ARROW-4378 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-4285) [Python] Use proper builder interface for serialization

2019-01-17 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4285: - Summary: [Python] Use proper builder interface for serialization Key: ARROW-4285 URL: https://issues.apache.org/jira/browse/ARROW-4285 Project: Apache Arrow

[jira] [Created] (ARROW-4269) [Python] AttributeError: module 'pandas.core' has no attribute 'arrays'

2019-01-15 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4269: - Summary: [Python] AttributeError: module 'pandas.core' has no attribute 'arrays' Key: ARROW-4269 URL: https://issues.apache.org/jira/browse/ARROW-4269 Project

Re: [VOTE] Release Apache Arrow 0.12.0 (RC2) ARROW

2019-01-13 Thread Philipp Moritz
+1 (non-binding) I didn't do very exhaustive tests, but the full Ray test suite passes based on that commit. On Sun, Jan 13, 2019 at 8:59 PM Wes McKinney wrote: > +1 (binding) > > Ran release verification scripts (for source release) on Ubuntu 14.04 > (with ARROW_HAVE_CUDA=yes) and Windows /

[jira] [Created] (ARROW-4249) [Plasma] Remove reference to logging.h from plasma/common.h

2019-01-13 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4249: - Summary: [Plasma] Remove reference to logging.h from plasma/common.h Key: ARROW-4249 URL: https://issues.apache.org/jira/browse/ARROW-4249 Project: Apache Arrow

[jira] [Created] (ARROW-4217) [Plasma] Remove custom object metadata

2019-01-09 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4217: - Summary: [Plasma] Remove custom object metadata Key: ARROW-4217 URL: https://issues.apache.org/jira/browse/ARROW-4217 Project: Apache Arrow Issue Type

Re: plan for Go implementation of Plasma

2018-12-19 Thread Philipp Moritz
Hey Dustin, Thanks for getting in touch! Here are two additional ways to do it: 5. Native go client library: If Go has support to ship file descriptors over unix domain sockets (which I think it has, see https://github.com/opencontainers/runc/blob/master/libcontainer/utils/cmsg.go) and interact

TensorFlow, PyTorch, and manylinux1

2018-12-15 Thread Philipp Moritz
Dear all, As some of you know, there is a standard in Python called manylinux ( https://www.python.org/dev/peps/pep-0513/) to package binary executables and libraries into a “wheel” in a way that allows the code to be run on a wide variety of Linux distributions. This is very convenient for

[jira] [Created] (ARROW-4025) [Python] TensorFlow/PyTorch arrow ThreadPool workarounds not working in some settings

2018-12-13 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4025: - Summary: [Python] TensorFlow/PyTorch arrow ThreadPool workarounds not working in some settings Key: ARROW-4025 URL: https://issues.apache.org/jira/browse/ARROW-4025

[jira] [Created] (ARROW-4024) [Python] Cython compilation error on cython==0.27.3

2018-12-13 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-4024: - Summary: [Python] Cython compilation error on cython==0.27.3 Key: ARROW-4024 URL: https://issues.apache.org/jira/browse/ARROW-4024 Project: Apache Arrow

[jira] [Created] (ARROW-3958) [Plasma] Reduce number of IPCs

2018-12-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3958: - Summary: [Plasma] Reduce number of IPCs Key: ARROW-3958 URL: https://issues.apache.org/jira/browse/ARROW-3958 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-3950) [Plasma] Don't force loading the TensorFlow op on import

2018-12-06 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3950: - Summary: [Plasma] Don't force loading the TensorFlow op on import Key: ARROW-3950 URL: https://issues.apache.org/jira/browse/ARROW-3950 Project: Apache Arrow

[jira] [Created] (ARROW-3934) [Gandiva] Don't compile precompiled tests if ARROW_GANDIVA_BUILD_TESTS=off

2018-12-03 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3934: - Summary: [Gandiva] Don't compile precompiled tests if ARROW_GANDIVA_BUILD_TESTS=off Key: ARROW-3934 URL: https://issues.apache.org/jira/browse/ARROW-3934 Project

[jira] [Created] (ARROW-3919) [Python] Support 64 bit indices for pyarrow.serialize and pyarrow.deserialize

2018-11-30 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3919: - Summary: [Python] Support 64 bit indices for pyarrow.serialize and pyarrow.deserialize Key: ARROW-3919 URL: https://issues.apache.org/jira/browse/ARROW-3919

[jira] [Created] (ARROW-3746) [Gandiva] [Python] Make it possible to list all functions registered with Gandiva

2018-11-09 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3746: - Summary: [Gandiva] [Python] Make it possible to list all functions registered with Gandiva Key: ARROW-3746 URL: https://issues.apache.org/jira/browse/ARROW-3746

Re: [ANNOUNCE] New Arrow PMC member: Krisztián Szűcs

2018-11-08 Thread Philipp Moritz
Congrats and welcome Krisztián! On Thu, Nov 8, 2018 at 11:48 AM Wes McKinney wrote: > The Project Management Committee (PMC) for Apache Arrow has invited > Krisztián Szűcs to become a PMC member and we are pleased to announce > that he has accepted. > > Congratulations and welcome, Krisztián! >

[jira] [Created] (ARROW-3718) [Gandiva] Remove spurious gtest include

2018-11-08 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3718: - Summary: [Gandiva] Remove spurious gtest include Key: ARROW-3718 URL: https://issues.apache.org/jira/browse/ARROW-3718 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-3721) [Gandiva] [Python] Support all Gandiva literals

2018-11-08 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3721: - Summary: [Gandiva] [Python] Support all Gandiva literals Key: ARROW-3721 URL: https://issues.apache.org/jira/browse/ARROW-3721 Project: Apache Arrow Issue

[jira] [Created] (ARROW-3659) Clang Travis build (matrix entry 2) might not actually be using clang

2018-10-30 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3659: - Summary: Clang Travis build (matrix entry 2) might not actually be using clang Key: ARROW-3659 URL: https://issues.apache.org/jira/browse/ARROW-3659 Project

[jira] [Created] (ARROW-3602) [Gandiva] [Python] Add preliminary Cython bindings for Gandiva

2018-10-23 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3602: - Summary: [Gandiva] [Python] Add preliminary Cython bindings for Gandiva Key: ARROW-3602 URL: https://issues.apache.org/jira/browse/ARROW-3602 Project: Apache Arrow

[jira] [Created] (ARROW-3589) [Gandiva] Make it possible to compile gandiva without JNI

2018-10-22 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3589: - Summary: [Gandiva] Make it possible to compile gandiva without JNI Key: ARROW-3589 URL: https://issues.apache.org/jira/browse/ARROW-3589 Project: Apache Arrow

[jira] [Created] (ARROW-3243) [C++] Upgrade jemalloc to version 5

2018-09-16 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3243: - Summary: [C++] Upgrade jemalloc to version 5 Key: ARROW-3243 URL: https://issues.apache.org/jira/browse/ARROW-3243 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-3199) [Plasma] Check for EAGAIN in recvmsg and sendmsg

2018-09-08 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3199: - Summary: [Plasma] Check for EAGAIN in recvmsg and sendmsg Key: ARROW-3199 URL: https://issues.apache.org/jira/browse/ARROW-3199 Project: Apache Arrow

[jira] [Created] (ARROW-3159) [Plasma] Plasma C++ and Python integration test for tensors

2018-09-01 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3159: - Summary: [Plasma] Plasma C++ and Python integration test for tensors Key: ARROW-3159 URL: https://issues.apache.org/jira/browse/ARROW-3159 Project: Apache Arrow

[jira] [Created] (ARROW-3157) [C++] Improve buffer creation for typed data

2018-09-01 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3157: - Summary: [C++] Improve buffer creation for typed data Key: ARROW-3157 URL: https://issues.apache.org/jira/browse/ARROW-3157 Project: Apache Arrow Issue

[jira] [Created] (ARROW-3116) [Plasma] Add "ls" to object store

2018-08-24 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3116: - Summary: [Plasma] Add "ls" to object store Key: ARROW-3116 URL: https://issues.apache.org/jira/browse/ARROW-3116 Project: Apache Arrow

[jira] [Created] (ARROW-3105) [Plasma] Improve flushing error message

2018-08-21 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3105: - Summary: [Plasma] Improve flushing error message Key: ARROW-3105 URL: https://issues.apache.org/jira/browse/ARROW-3105 Project: Apache Arrow Issue Type

Re: [VOTE] Combining Arrow C++ development process with Apache Parquet C++

2018-08-21 Thread Philipp Moritz
+1 for the monorepo plan and push access to Parquet C++ committers -- Philipp. On Tue, Aug 21, 2018 at 3:12 PM, Antoine Pitrou wrote: > > +1 from me. > > Regards > > Antoine. > > > Le 21/08/2018 à 18:49, Wes McKinney a écrit : > > hi folks, > > > > As discussed on the Arrow and Parquet mailing

Re: [ANNOUNCE] New Arrow PMC member: Antoine Pitrou

2018-08-21 Thread Philipp Moritz
Awesome, congratulations and welcome! On Tue, Aug 21, 2018 at 1:28 PM, Phillip Cloud wrote: > Welcome! Great to have you on the team! > > On Tue, Aug 21, 2018 at 4:27 PM Wes McKinney wrote: > > > The Project Management Committee (PMC) for Apache Arrow has invited > > Antoine Pitrou to become a

Re: [VOTE] Accept donation of Gandiva to Apache Arrow

2018-08-16 Thread Philipp Moritz
+1 On Thu, Aug 16, 2018, 10:02 AM Siddharth Teotia wrote: > +1 > > On Thu, Aug 16, 2018 at 9:57 AM, Julian Hyde wrote: > > > +1 > > On Thu, Aug 16, 2018 at 8:56 AM Wes McKinney > wrote: > > > > > > Dear all, > > > > > > The developers of Gandiva, an LLVM-based vectorized expression > > >

[jira] [Created] (ARROW-3062) [Python] Extend fast libtensorflow_framework.so compatibility workaround to Python 2.7

2018-08-15 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3062: - Summary: [Python] Extend fast libtensorflow_framework.so compatibility workaround to Python 2.7 Key: ARROW-3062 URL: https://issues.apache.org/jira/browse/ARROW-3062

[jira] [Created] (ARROW-3018) [Plasma] Improve random ObjectID generation

2018-08-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-3018: - Summary: [Plasma] Improve random ObjectID generation Key: ARROW-3018 URL: https://issues.apache.org/jira/browse/ARROW-3018 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2976) [Python] Directory in pyarrow.get_library_dirs() on Travis doesn't contain libarrow.so

2018-08-03 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2976: - Summary: [Python] Directory in pyarrow.get_library_dirs() on Travis doesn't contain libarrow.so Key: ARROW-2976 URL: https://issues.apache.org/jira/browse/ARROW-2976

[jira] [Created] (ARROW-2975) [Plasma] TensorFlow op: Compilation only working if arrow found by pkg-config

2018-08-03 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2975: - Summary: [Plasma] TensorFlow op: Compilation only working if arrow found by pkg-config Key: ARROW-2975 URL: https://issues.apache.org/jira/browse/ARROW-2975

[jira] [Created] (ARROW-2954) [Plasma] Store object_id only once in object table

2018-07-31 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2954: - Summary: [Plasma] Store object_id only once in object table Key: ARROW-2954 URL: https://issues.apache.org/jira/browse/ARROW-2954 Project: Apache Arrow

[jira] [Created] (ARROW-2953) [Plasma] Store memory usage

2018-07-31 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2953: - Summary: [Plasma] Store memory usage Key: ARROW-2953 URL: https://issues.apache.org/jira/browse/ARROW-2953 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-2940) [Python] Import error with pytorch 0.3

2018-07-30 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2940: - Summary: [Python] Import error with pytorch 0.3 Key: ARROW-2940 URL: https://issues.apache.org/jira/browse/ARROW-2940 Project: Apache Arrow Issue Type

Re: [DISCUSS] Solutions for improving the Arrow-Parquet C++ development morass

2018-07-29 Thread Philipp Moritz
I do not claim to have insight into parquet-cpp development. However, from our experience developing Ray, I can say that the monorepo approach (for Ray) has improved things a lot. Before we tried various schemes to split the project into multiple repos, but the build system and test infrastructure

[jira] [Created] (ARROW-2920) [Python] Segfault with pytorch 0.4

2018-07-26 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2920: - Summary: [Python] Segfault with pytorch 0.4 Key: ARROW-2920 URL: https://issues.apache.org/jira/browse/ARROW-2920 Project: Apache Arrow Issue Type: Bug

Re: [DISCUSS] Contribution of Gandiva to Apache Arrow

2018-07-25 Thread Philipp Moritz
+1 on merging it and also agreed with Uwe that we will need to deal with LLVM version conflicts. In addition it would be good to come up with a plan on how it can be useful for other DataFrame open source projects. Having end-to-end applications that let people profit from this code will help

Re: Plasma Java API to read RecordBatch from Python process

2018-07-20 Thread Philipp Moritz
Hey Jieun, Currently we can only transfer RecordBatch objects using Plasma between C++ and Python unfortunately. I just opened a JIRA for doing it with Java too in https://issues.apache.org/jira/browse/ARROW-2892. The necessary pieces are there (in particular there is a low level API to access

[jira] [Created] (ARROW-2892) [Plasma] Implement interface to get Java arrow objects from Plasma

2018-07-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2892: - Summary: [Plasma] Implement interface to get Java arrow objects from Plasma Key: ARROW-2892 URL: https://issues.apache.org/jira/browse/ARROW-2892 Project: Apache

[jira] [Created] (ARROW-2890) [Plasma] Make Python PlasmaClient.release private

2018-07-20 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2890: - Summary: [Plasma] Make Python PlasmaClient.release private Key: ARROW-2890 URL: https://issues.apache.org/jira/browse/ARROW-2890 Project: Apache Arrow

Re: Pyarrow Plasma client.release() fault

2018-07-20 Thread Philipp Moritz
Also you should avoid calling release directly, because it will also be called automatically here: https://github.com/apache/arrow/blob/master/python/pyarrow/_plasma.pyx#L222 Instead, you should call "del buffer" on the PlasmaBuffer. I'll submit a PR to make the release method private. The only

[jira] [Created] (ARROW-2866) [Plasma] TensorFlow op: Investiate outputting multiple output Tensors for the reading op

2018-07-17 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2866: - Summary: [Plasma] TensorFlow op: Investiate outputting multiple output Tensors for the reading op Key: ARROW-2866 URL: https://issues.apache.org/jira/browse/ARROW-2866

Re: Passing Arrow object across language

2018-07-16 Thread Philipp Moritz
Hey Yuri, you can use the Arrow IPC mechanism to do this: - https://github.com/apache/arrow/blob/master/format/IPC.md - Python: https://arrow.apache.org/docs/python/ipc.html - C++: https://arrow.apache.org/docs/cpp/namespacearrow_1_1ipc.html - For Java, see the org.apache.arrow.vector.ipc

[jira] [Created] (ARROW-2811) [Python] Test serialization for determinism

2018-07-07 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2811: - Summary: [Python] Test serialization for determinism Key: ARROW-2811 URL: https://issues.apache.org/jira/browse/ARROW-2811 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2805) [Python] TensorFlow import workaround not working with tensorflow-gpu if CUDA is not installed

2018-07-06 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2805: - Summary: [Python] TensorFlow import workaround not working with tensorflow-gpu if CUDA is not installed Key: ARROW-2805 URL: https://issues.apache.org/jira/browse/ARROW-2805

[jira] [Created] (ARROW-2803) [C++] Put hashing function into src/arrow/util

2018-07-06 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2803: - Summary: [C++] Put hashing function into src/arrow/util Key: ARROW-2803 URL: https://issues.apache.org/jira/browse/ARROW-2803 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2794) [Plasma] Add Delete method for multiple objects

2018-07-04 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2794: - Summary: [Plasma] Add Delete method for multiple objects Key: ARROW-2794 URL: https://issues.apache.org/jira/browse/ARROW-2794 Project: Apache Arrow Issue

[jira] [Created] (ARROW-2788) [Plasma] Defining Delete semantics

2018-07-03 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2788: - Summary: [Plasma] Defining Delete semantics Key: ARROW-2788 URL: https://issues.apache.org/jira/browse/ARROW-2788 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2758) [Plasma] Use Scope enum in Plasma

2018-06-27 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2758: - Summary: [Plasma] Use Scope enum in Plasma Key: ARROW-2758 URL: https://issues.apache.org/jira/browse/ARROW-2758 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2757) [Plasma] Huge pages test failing

2018-06-27 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2757: - Summary: [Plasma] Huge pages test failing Key: ARROW-2757 URL: https://issues.apache.org/jira/browse/ARROW-2757 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2737) [Plasma] Integrate TensorFlow Op with arrow packaging scripts

2018-06-24 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2737: - Summary: [Plasma] Integrate TensorFlow Op with arrow packaging scripts Key: ARROW-2737 URL: https://issues.apache.org/jira/browse/ARROW-2737 Project: Apache Arrow

Re: Gandiva Initiative

2018-06-22 Thread Philipp Moritz
This is really exciting, thanks a lot for sharing! In case anybody wants to try this out from Python, I wrote up some Cython bindings (very limited so far, but they can already be used to construct some computation graphs and do some benchmarks): https://github.com/apache/arrow/pull/2153 They

[jira] [Created] (ARROW-2629) [Plasma] Iterator invalidation for pending_notifications_

2018-05-22 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2629: - Summary: [Plasma] Iterator invalidation for pending_notifications_ Key: ARROW-2629 URL: https://issues.apache.org/jira/browse/ARROW-2629 Project: Apache Arrow

[jira] [Created] (ARROW-2612) [Plasma] Fix deprecated PLASMA_DEFAULT_RELEASE_DELAY

2018-05-17 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2612: - Summary: [Plasma] Fix deprecated PLASMA_DEFAULT_RELEASE_DELAY Key: ARROW-2612 URL: https://issues.apache.org/jira/browse/ARROW-2612 Project: Apache Arrow

[jira] [Created] (ARROW-2611) [Python] Python 2 integer serialization

2018-05-17 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2611: - Summary: [Python] Python 2 integer serialization Key: ARROW-2611 URL: https://issues.apache.org/jira/browse/ARROW-2611 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2595) [Plasma] operator[] creates entries in map

2018-05-16 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2595: - Summary: [Plasma] operator[] creates entries in map Key: ARROW-2595 URL: https://issues.apache.org/jira/browse/ARROW-2595 Project: Apache Arrow Issue Type

[jira] [Created] (ARROW-2577) [Plasma] Add ASV benchmarks

2018-05-13 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2577: - Summary: [Plasma] Add ASV benchmarks Key: ARROW-2577 URL: https://issues.apache.org/jira/browse/ARROW-2577 Project: Apache Arrow Issue Type: Improvement

[jira] [Created] (ARROW-2542) [Plasma] Refactor object notification code

2018-05-04 Thread Philipp Moritz (JIRA)
Philipp Moritz created ARROW-2542: - Summary: [Plasma] Refactor object notification code Key: ARROW-2542 URL: https://issues.apache.org/jira/browse/ARROW-2542 Project: Apache Arrow Issue Type

  1   2   3   >