Hi all, Please review the PR for enabling GPU support in pyarrow:
https://github.com/apache/arrow/pull/2536 Best regards, Pearu On Wed, Aug 29, 2018 at 6:45 PM, Antoine Pitrou <anto...@python.org> wrote: > > Hi Pearu, > > Your approach seems ok on the principle. Feel free to open a PR once > you have a basic API defined and exposed, so that we can iterate on that. > > Regards > > Antoine. > > > Le 29/08/2018 à 17:11, Pearu Peterson a écrit : > > Hi Wes, > > > > Thanks for the hints! > > > > I have created a very minimal GPU support to pyarrow, see > > > > > > https://github.com/apache/arrow/compare/master... > Quansight:pearu-cuda-pyarrow?expand=1 > > > > This includes: > > 1. introducing libarrow_gpu that is built into pyarrow.lib_gpu extension > > module, currently only CudaBuffer is exposed > > 2. updating cmake configuration files to detect libarrow_gpu.{so,a} > > libraries > > 3. updating setup.py with new options --with{,out}-arrow-gpu and > > PYARROW_BUILD_ARROW_GPU env variable support. > > > > Let me know if something in my approach needs to be changed. > > > > Regarding packaging: > > 1. FYI, there is a cudatoolkit-dev package in conda-forge: > > https://github.com/conda-forge/cudatoolkit-dev-feedstock > > > > 2. One approach is to introduce new conda packages arrow-cpp-gpu and > > pyarrow-gpu that are build using -DARROW_GPU and --with-arrow-gpu, > > respectively. > > > > Best regards, > > Pearu > > > > On Wed, Aug 29, 2018 at 12:32 AM, Wes McKinney <wesmck...@gmail.com> > wrote: > > > >> hi Pearu, > >> > >> Sounds good to me; I had wanted to do it already but got caught up > >> working on other things. You'll probably want to create > >> libarrow_gpu.pxd since this is an optional library. Otherwise it > >> should be reasonably straightforward. > >> > >> some relevant issues > >> > >> - https://issues.apache.org/jira/browse/ARROW-1424 > >> - https://issues.apache.org/jira/browse/ARROW-2512 > >> > >> If we wanted to ship Python wheels and conda packages with CUDA > >> support built in, what would we have to do (currently these binaries > >> are produced in Circle CI, Travis CI, and Appveyor)? > >> > >> - Wes > >> On Tue, Aug 28, 2018 at 4:34 PM Pearu Peterson > >> <pearu.peter...@quansight.com> wrote: > >>> > >>> Hi, > >>> > >>> Is there any interest in implementing CudaBuffer support in pyarrow? > Are > >>> there any obstacles that would prevent to do this? > >>> > >>> I would be willing to give it a try. > >>> > >>> As I understand, the entry point for this task would be to patch > >>> pyarrow/includes/libarrow.pxd . Any hints for what to take into account > >> are > >>> appreciated. > >>> > >>> Best regards, > >>> Pearu > >> > > >
