Re: python module pyarrow.compute is not found
On Fri, Nov 27, 2020 at 10:42 AM Kirill Lykov wrote: > > ARROW_COMPUTE was enabled because ARROW_PYTHON was enabled (I double > checked). > And sanity check helped -- I should be in python folder, not in the root > folder of arrow repo, thanks! > > I wonder if there a way to detect if compute module is present by looking > into *.so files. > Because at our internal environment I cannot find _compute.cpython*.so and > import pyarrow.compute doesn't work although > python module is there (the arrow version there is 0.16), so I'm trying to > verify that it is not present. > "import pyarrow.compute" should always work with the current binary releases (2.0.0 and onward) of pyarrow and builds from source off of git master. We can't support old versions like 0.16.0 > > On Thu, Nov 26, 2020 at 6:47 PM Niklas Bivald > wrote: > > > Sanity check, you aren’t standing in a folder which has a folder named > > “pyarrow” in it from the build? > > > > Do you get the same error if you move to your home folder and install the > > wheel? > > > > On Thu, 26 Nov 2020 at 18:18, Antoine Pitrou wrote: > > > > > > > > Did you enable ARROW_COMPUTE=ON when building Arrow C++? > > > > > > > > > Le 26/11/2020 à 18:07, Kirill Lykov a écrit : > > > > I followed, as usual, the build procedure described there -- > > > > https://arrow.apache.org/docs/developers/python.html > > > > With `ARROW_PARQUET=OFF, ARROW_GANDIVA=ON` and `--inplace` option > > > > > > > > Traceback as the following: > > > > > > > > Python 3.8.5 (default, Jul 21 2020, 10:42:08) > > > > [Clang 11.0.0 (clang-1100.0.33.17)] on darwin > > > > Type "help", "copyright", "credits" or "license" for more information. > > > import pyarrow > > > import pyarrow.compute as pc > > > > Traceback (most recent call last): > > > > File "", line 1, in > > > > ModuleNotFoundError: No module named 'pyarrow.compute' > > > > > > > > On Thu, Nov 26, 2020 at 5:49 PM Antoine Pitrou > > > wrote: > > > > > > > >> > > > >> Hi Kirill, > > > >> > > > >> Can you post the full traceback? > > > >> > > > >> > > > >> Le 26/11/2020 à 17:21, Kirill Lykov a écrit : > > > >>> Hi, > > > >>> > > > >>> I've build arrow and python integration. > > > >>> There is file `pyarrow/_compute.cpython-38-darwin.so`. > > > >>> But when I try import pyarrow.compute I get error that the module is > > > not > > > >>> found. > > > >>> Do I miss some flags in the building procedure? It looks like it is > > > built > > > >>> by default from setup.py > > > >>> > > > >> > > > > > > > > > > > > > > > > -- > Best regards, > Kirill Lykov
Re: python module pyarrow.compute is not found
In my experience, if we turn on DARROW_PYTHON and DARROW_COMPUTE to ON and do build from source and then just build the wheel from the source would install the libarrow_python.so and within the site-packages for Pyarrow you can also get the compute*.so as well. If you used a virtual environment make sure you point to that Python when you're building from the source. With Regards, Vibhatha Abeykoon On Fri, Nov 27, 2020 at 11:41 AM Kirill Lykov wrote: > ARROW_COMPUTE was enabled because ARROW_PYTHON was enabled (I double > checked). > And sanity check helped -- I should be in python folder, not in the root > folder of arrow repo, thanks! > > I wonder if there a way to detect if compute module is present by looking > into *.so files. > Because at our internal environment I cannot find _compute.cpython*.so and > import pyarrow.compute doesn't work although > python module is there (the arrow version there is 0.16), so I'm trying to > verify that it is not present. > > > On Thu, Nov 26, 2020 at 6:47 PM Niklas Bivald > wrote: > > > Sanity check, you aren’t standing in a folder which has a folder named > > “pyarrow” in it from the build? > > > > Do you get the same error if you move to your home folder and install the > > wheel? > > > > On Thu, 26 Nov 2020 at 18:18, Antoine Pitrou wrote: > > > > > > > > Did you enable ARROW_COMPUTE=ON when building Arrow C++? > > > > > > > > > Le 26/11/2020 à 18:07, Kirill Lykov a écrit : > > > > I followed, as usual, the build procedure described there -- > > > > https://arrow.apache.org/docs/developers/python.html > > > > With `ARROW_PARQUET=OFF, ARROW_GANDIVA=ON` and `--inplace` option > > > > > > > > Traceback as the following: > > > > > > > > Python 3.8.5 (default, Jul 21 2020, 10:42:08) > > > > [Clang 11.0.0 (clang-1100.0.33.17)] on darwin > > > > Type "help", "copyright", "credits" or "license" for more > information. > > > import pyarrow > > > import pyarrow.compute as pc > > > > Traceback (most recent call last): > > > > File "", line 1, in > > > > ModuleNotFoundError: No module named 'pyarrow.compute' > > > > > > > > On Thu, Nov 26, 2020 at 5:49 PM Antoine Pitrou > > > wrote: > > > > > > > >> > > > >> Hi Kirill, > > > >> > > > >> Can you post the full traceback? > > > >> > > > >> > > > >> Le 26/11/2020 à 17:21, Kirill Lykov a écrit : > > > >>> Hi, > > > >>> > > > >>> I've build arrow and python integration. > > > >>> There is file `pyarrow/_compute.cpython-38-darwin.so`. > > > >>> But when I try import pyarrow.compute I get error that the module > is > > > not > > > >>> found. > > > >>> Do I miss some flags in the building procedure? It looks like it is > > > built > > > >>> by default from setup.py > > > >>> > > > >> > > > > > > > > > > > > > > > > -- > Best regards, > Kirill Lykov >
Re: python module pyarrow.compute is not found
ARROW_COMPUTE was enabled because ARROW_PYTHON was enabled (I double checked). And sanity check helped -- I should be in python folder, not in the root folder of arrow repo, thanks! I wonder if there a way to detect if compute module is present by looking into *.so files. Because at our internal environment I cannot find _compute.cpython*.so and import pyarrow.compute doesn't work although python module is there (the arrow version there is 0.16), so I'm trying to verify that it is not present. On Thu, Nov 26, 2020 at 6:47 PM Niklas Bivald wrote: > Sanity check, you aren’t standing in a folder which has a folder named > “pyarrow” in it from the build? > > Do you get the same error if you move to your home folder and install the > wheel? > > On Thu, 26 Nov 2020 at 18:18, Antoine Pitrou wrote: > > > > > Did you enable ARROW_COMPUTE=ON when building Arrow C++? > > > > > > Le 26/11/2020 à 18:07, Kirill Lykov a écrit : > > > I followed, as usual, the build procedure described there -- > > > https://arrow.apache.org/docs/developers/python.html > > > With `ARROW_PARQUET=OFF, ARROW_GANDIVA=ON` and `--inplace` option > > > > > > Traceback as the following: > > > > > > Python 3.8.5 (default, Jul 21 2020, 10:42:08) > > > [Clang 11.0.0 (clang-1100.0.33.17)] on darwin > > > Type "help", "copyright", "credits" or "license" for more information. > > import pyarrow > > import pyarrow.compute as pc > > > Traceback (most recent call last): > > > File "", line 1, in > > > ModuleNotFoundError: No module named 'pyarrow.compute' > > > > > > On Thu, Nov 26, 2020 at 5:49 PM Antoine Pitrou > > wrote: > > > > > >> > > >> Hi Kirill, > > >> > > >> Can you post the full traceback? > > >> > > >> > > >> Le 26/11/2020 à 17:21, Kirill Lykov a écrit : > > >>> Hi, > > >>> > > >>> I've build arrow and python integration. > > >>> There is file `pyarrow/_compute.cpython-38-darwin.so`. > > >>> But when I try import pyarrow.compute I get error that the module is > > not > > >>> found. > > >>> Do I miss some flags in the building procedure? It looks like it is > > built > > >>> by default from setup.py > > >>> > > >> > > > > > > > > > -- Best regards, Kirill Lykov
[NIGHTLY] Arrow Build Report for Job nightly-2020-11-27-0
Arrow Build Report for Job nightly-2020-11-27-0 All tasks: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0 Failed Tasks: - example-cpp-minimal-build-static-system-dependency: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-example-cpp-minimal-build-static-system-dependency - test-conda-python-3.7-spark-branch-3.0: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-test-conda-python-3.7-spark-branch-3.0 - test-conda-python-3.8-jpype: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-test-conda-python-3.8-jpype - test-ubuntu-18.04-docs: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-test-ubuntu-18.04-docs - wheel-win-cp36m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-wheel-win-cp36m - wheel-win-cp37m: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-wheel-win-cp37m - wheel-win-cp38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-wheel-win-cp38 Succeeded Tasks: - centos-6-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-centos-6-amd64 - centos-7-aarch64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-travis-centos-7-aarch64 - centos-7-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-centos-7-amd64 - centos-8-aarch64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-travis-centos-8-aarch64 - centos-8-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-centos-8-amd64 - conda-clean: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-clean - conda-linux-gcc-py36-aarch64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-drone-conda-linux-gcc-py36-aarch64 - conda-linux-gcc-py36-cpu-r36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-linux-gcc-py36-cpu-r36 - conda-linux-gcc-py36-cuda: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-linux-gcc-py36-cuda - conda-linux-gcc-py37-aarch64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-drone-conda-linux-gcc-py37-aarch64 - conda-linux-gcc-py37-cpu-r40: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-linux-gcc-py37-cpu-r40 - conda-linux-gcc-py37-cuda: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-linux-gcc-py37-cuda - conda-linux-gcc-py38-aarch64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-drone-conda-linux-gcc-py38-aarch64 - conda-linux-gcc-py38-cpu: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-linux-gcc-py38-cpu - conda-linux-gcc-py38-cuda: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-linux-gcc-py38-cuda - conda-osx-clang-py36-r36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-osx-clang-py36-r36 - conda-osx-clang-py37-r40: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-osx-clang-py37-r40 - conda-osx-clang-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-osx-clang-py38 - conda-win-vs2017-py36-r36: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-win-vs2017-py36-r36 - conda-win-vs2017-py37-r40: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-win-vs2017-py37-r40 - conda-win-vs2017-py38: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-azure-conda-win-vs2017-py38 - debian-buster-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-debian-buster-amd64 - debian-buster-arm64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-travis-debian-buster-arm64 - debian-stretch-amd64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-debian-stretch-amd64 - debian-stretch-arm64: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-travis-debian-stretch-arm64 - example-cpp-minimal-build-static: URL: https://github.com/ursa-labs/crossbow/branches/all?query=nightly-2020-11-27-0-github-example-cpp-minimal-build-static - gandiva-jar-osx: URL:
Re: Removing Python 3.5 support
Le 27/11/2020 à 06:07, Micah Kornfield a écrit : > Could we hold off for at least a few days, I'd like to run this by some > colleagues. Certainly. > Are there any options for maintaining a functioning CI for > Python 3.5? I don't know. Interested people would have to investigate :-) Regards Antoine.
