In cudf 0.18.0 (https://github.com/rapidsai/cudf/releases/tag/v0.18.0) you are looking at pyarrow 1.0.1 I believe ( https://github.com/rapidsai/cudf/blob/20778e5ddb7470845605acd7f879620eb25ff4ff/conda/environments/cudf_dev_cuda11.0.yml#L21 ) There are many API changes between 1.0.1 --> 6.0.0
On Thu, Mar 3, 2022 at 11:00 AM Cindy McMullen <cmcmul...@twitter.com> wrote: > Hi, Niranda - > > We generally use 'pip', since 'conda' is not configured in our env to > reach external repositories. We need 'pip' to install 'dgl' (Deep Graph > Library). > From pip: > > (base) jupyter@cmcmullen-usersims-gdl-cuda:~$ pip freeze | grep cudf > cudf==0.18.0a0+253.g53ed28e91c > cudf-kafka==0.18.0a0+253.g53ed28e91c > dask-cudf==0.18.0a0+253.g53ed28e91c > > -------- > From conda: > > conda list | grep cudf > cudf 0.18.0a210224 cuda_11.0_py37_g53ed28e91c_253 > rapidsai-nightly > cudf_kafka 0.18.0a210224 py37_g53ed28e91c_253 > rapidsai-nightly > dask-cudf 0.18.0a210224 py37_g53ed28e91c_253 > rapidsai-nightly > libcudf 0.18.0a210224 cuda11.0_g1544474166_254 > rapidsai-nightly > libcudf_kafka 0.18.0a210224 g1544474166_254 > rapidsai-nightly > ------- > > On Wed, Mar 2, 2022 at 8:10 PM Niranda Perera <niranda.per...@gmail.com> > wrote: > >> nvcc is the Nvidia c compiler. Can you try >> >> conda list | grep cudf >> >> On Wed, Mar 2, 2022, 19:55 Cindy McMullen <cmcmul...@twitter.com> wrote: >> >>> /usr/local/cuda/bin/nvcc --version >>> nvcc: NVIDIA (R) Cuda compiler driver >>> Copyright (c) 2005-2020 NVIDIA Corporation >>> Built on Thu_Jun_11_22:26:38_PDT_2020 >>> Cuda compilation tools, release 11.0, V11.0.194 >>> Build cuda_11.0_bu.TC445_37.28540450_0 >>> >>> >>> On Wed, Mar 2, 2022 at 5:54 PM Niranda Perera <niranda.per...@gmail.com> >>> wrote: >>> >>>> What's your cudf version? >>>> >>>> >>>> On Wed, Mar 2, 2022 at 7:44 PM Cindy McMullen <cmcmul...@twitter.com> >>>> wrote: >>>> >>>>> Same issue. >>>>> >>>>> >>>>> Installing collected packages: pyarrow >>>>> Attempting uninstall: pyarrow >>>>> Found existing installation: pyarrow 7.0.0 >>>>> Uninstalling pyarrow-7.0.0: >>>>> Successfully uninstalled pyarrow-7.0.0 >>>>> Successfully installed pyarrow-6.0.1 >>>>> >>>>> .... >>>>> >>>>> cudf/_lib/gpuarrow.pyx in init cudf._lib.gpuarrow() >>>>> AttributeError: module 'pyarrow.lib' has no attribute >>>>> '_CRecordBatchReader' >>>>> >>>>> >>>>> >>>>> On Wed, Mar 2, 2022 at 5:34 PM Niranda Perera < >>>>> niranda.per...@gmail.com> wrote: >>>>> >>>>>> Hi Cindy, >>>>>> I don't think cudf supports arrow 7.0.0. Even the master branch is >>>>>> going with 6.0.1 at the moment. >>>>>> >>>>>> https://github.com/rapidsai/cudf/blob/6bcfc104051f926f46467fc55c456a9b012fc4af/conda/environments/cudf_dev_cuda11.5.yml#L20 >>>>>> >>>>>> On Wed, Mar 2, 2022 at 7:21 PM Cindy McMullen <cmcmul...@twitter.com> >>>>>> wrote: >>>>>> >>>>>>> pyarrow-7.0.0 >>>>>>> >>>>>>> >>>>>>> /usr/local/cuda/bin/nvcc --version >>>>>>> nvcc: NVIDIA (R) Cuda compiler driver >>>>>>> Copyright (c) 2005-2020 NVIDIA Corporation >>>>>>> Built on Thu_Jun_11_22:26:38_PDT_2020 >>>>>>> Cuda compilation tools, release 11.0, V11.0.194 >>>>>>> Build cuda_11.0_bu.TC445_37.28540450_0 >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Wed, Mar 2, 2022 at 9:09 AM Keith Kraus <keith.j.kr...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hey Cindy, >>>>>>>> >>>>>>>> What versions of cuDF and PyArrow do you have installed? I've >>>>>>>> typically seen this pop up when there's a mismatched version. >>>>>>>> >>>>>>>> -Keith >>>>>>>> >>>>>>>> On Tue, Mar 1, 2022 at 8:28 PM Cindy McMullen < >>>>>>>> cmcmul...@twitter.com> wrote: >>>>>>>> >>>>>>>>> Hi - >>>>>>>>> >>>>>>>>> I'm trying to use DGL (Deep Graph Library) DGLDataset API with the >>>>>>>>> RAPIDS cuda DataFrame API. Am getting this error: >>>>>>>>> >>>>>>>>> module 'pyarrow.lib' has no attribute '_CRecordBatchReader' >>>>>>>>> >>>>>>>>> >>>>>>>>> Wonder if you see anything obvious in the stack trace that might help >>>>>>>>> me debug? >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Here's the full stack trace: >>>>>>>>> >>>>>>>>> >>>>>>>>> <ipython-input-5-81f2b1833437> in __init__(self) 10 """ 11 >>>>>>>>> def __init__(self):---> 12 super(UserSimsSingleFileDataset, >>>>>>>>> self).__init__(name='UserSimsDataset', verbose=False) 13 14 >>>>>>>>> # One quirk of DGLDataset is that process() and __len__ (load, >>>>>>>>> save) are called immediately after super(), >>>>>>>>> /opt/conda/lib/python3.7/site-packages/dgl/data/dgl_dataset.py in >>>>>>>>> __init__(self, name, url, raw_dir, save_dir, hash_key, force_reload, >>>>>>>>> verbose) 91 self._save_dir = save_dir 92 ---> 93 >>>>>>>>> self._load() 94 95 def download(self): >>>>>>>>> /opt/conda/lib/python3.7/site-packages/dgl/data/dgl_dataset.py in >>>>>>>>> _load(self) 176 if not load_flag: 177 >>>>>>>>> self._download()--> 178 self.process() 179 >>>>>>>>> self.save() 180 if self.verbose: >>>>>>>>> <ipython-input-5-81f2b1833437> in process(self) 20 21 >>>>>>>>> import gcsfs---> 22 import cudf 23 self.rows_per_batch = >>>>>>>>> 10000 24 gs = gcsfs.GCSFileSystem() >>>>>>>>> /opt/conda/lib/python3.7/site-packages/cudf/__init__.py in <module> >>>>>>>>> 9 import rmm 10 ---> 11 from cudf import core, datasets, >>>>>>>>> testing 12 from cudf._version import get_versions 13 from >>>>>>>>> cudf.api.extensions import ( >>>>>>>>> /opt/conda/lib/python3.7/site-packages/cudf/core/__init__.py in >>>>>>>>> <module> 1 # Copyright (c) 2018-2020, NVIDIA CORPORATION. 2 >>>>>>>>> ----> 3 from cudf.core import buffer, column, column_accessor, common >>>>>>>>> 4 from cudf.core.buffer import Buffer 5 from >>>>>>>>> cudf.core.dataframe import DataFrame, from_pandas, merge >>>>>>>>> /opt/conda/lib/python3.7/site-packages/cudf/core/column/__init__.py >>>>>>>>> in <module> 1 # Copyright (c) 2020-2021, NVIDIA CORPORATION. >>>>>>>>> 2 ----> 3 from cudf.core.column.categorical import CategoricalColumn >>>>>>>>> 4 from cudf.core.column.column import ( 5 ColumnBase, >>>>>>>>> /opt/conda/lib/python3.7/site-packages/cudf/core/column/categorical.py >>>>>>>>> in <module> 20 21 import cudf---> 22 from cudf import _lib >>>>>>>>> as libcudf 23 from cudf._lib.scalar import as_device_scalar >>>>>>>>> 24 from cudf._lib.transform import bools_to_mask >>>>>>>>> /opt/conda/lib/python3.7/site-packages/cudf/_lib/__init__.py in >>>>>>>>> <module> 2 import numpy as np 3 ----> 4 from . import ( >>>>>>>>> 5 avro, 6 binaryop, >>>>>>>>> cudf/_lib/gpuarrow.pyx in init cudf._lib.gpuarrow() >>>>>>>>> AttributeError: module 'pyarrow.lib' has no attribute >>>>>>>>> '_CRecordBatchReader' >>>>>>>>> >>>>>>>>> >>>>>> >>>>>> -- >>>>>> Niranda Perera >>>>>> https://niranda.dev/ >>>>>> @n1r44 <https://twitter.com/N1R44> >>>>>> >>>>>> >>>> >>>> -- >>>> Niranda Perera >>>> https://niranda.dev/ >>>> @n1r44 <https://twitter.com/N1R44> >>>> >>>> -- Niranda Perera https://niranda.dev/ @n1r44 <https://twitter.com/N1R44>
