[jira] [Commented] (ARROW-8638) Arrow Cython API Usage Gives an error when calling CTable API Endpoints
[
https://issues.apache.org/jira/browse/ARROW-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096500#comment-17096500
]
Vibhatha Lakmal Abeykoon commented on ARROW-8638:
-
I tried the LD_LIBRARY_PATH approach and it worked fine. But I think, I need to
adopt a neat setup as you point out.
Thank you for this response.
I have another thing in mind. Think of an instance where the arrow is compiled
from source. In such cases is there a best practice that can be adopted.
> Arrow Cython API Usage Gives an error when calling CTable API Endpoints
> ---
>
> Key: ARROW-8638
> URL: https://issues.apache.org/jira/browse/ARROW-8638
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
>Affects Versions: 0.16.0
> Environment: Ubuntu 20.04 with Python 3.8.2
> RHEL7 with Python 3.6.8
>Reporter: Vibhatha Lakmal Abeykoon
>Priority: Blocker
> Fix For: 0.16.0
>
>
> I am working on using both Arrow C++ API and Cython API to support an
> application that I am developing. But here, I will add the issue I
> experienced when I am trying to follow the example,
> [https://arrow.apache.org/docs/python/extending.html]
> I am testing on Ubuntu 20.04 LTS
> Python version 3.8.2
> These are the steps I followed.
> # Create Virtualenv
> python3 -m venv ENVARROW
>
> 2. Activate ENV
> source ENVARROW/bin/activate
>
> 3. pip3 install pyarrow==0.16.0 cython numpy
>
> 4. Code block and Tools,
>
> +*example.pyx*+
>
>
> {code:java}
> from pyarrow.lib cimport *
> def get_array_length(obj):
> # Just an example function accessing both the pyarrow Cython API
> # and the Arrow C++ API
> cdef shared_ptr[CArray] arr = pyarrow_unwrap_array(obj)
> if arr.get() == NULL:
> raise TypeError("not an array")
> return arr.get().length()
> def get_table_info(obj):
> cdef shared_ptr[CTable] table = pyarrow_unwrap_table(obj)
> if table.get() == NULL:
> raise TypeError("not an table")
>
> return table.get().num_columns()
> {code}
>
>
> +*setup.py*+
>
>
> {code:java}
> from distutils.core import setup
> from Cython.Build import cythonize
> import os
> import numpy as np
> import pyarrow as pa
> ext_modules = cythonize("example.pyx")
> for ext in ext_modules:
> # The Numpy C headers are currently required
> ext.include_dirs.append(np.get_include())
> ext.include_dirs.append(pa.get_include())
> ext.libraries.extend(pa.get_libraries())
> ext.library_dirs.extend(pa.get_library_dirs())
> if os.name == 'posix':
> ext.extra_compile_args.append('-std=c++11')
> # Try uncommenting the following line on Linux
> # if you get weird linker errors or runtime crashes
> #ext.define_macros.append(("_GLIBCXX_USE_CXX11_ABI", "0"))
> setup(ext_modules=ext_modules)
> {code}
>
>
> +*arrow_array.py*+
>
> {code:java}
> import example
> import pyarrow as pa
> import numpy as np
> arr = pa.array([1,2,3,4,5])
> len = example.get_array_length(arr)
> print("Array length {} ".format(len))
> {code}
>
> +*arrow_table.py*+
>
> {code:java}
> import example
> import pyarrow as pa
> import numpy as np
> from pyarrow import csv
> fn = 'data.csv'
> table = csv.read_csv(fn)
> print(table)
> cols = example.get_table_info(table)
> print(cols)
>
> {code}
> +*data.csv*+
> {code:java}
> 1,2,3,4,5
> 6,7,8,9,10
> 11,12,13,14,15
> {code}
>
> +*Makefile*+
>
> {code:java}
> install:
> python3 setup.py build_ext --inplace
> clean:
> rm -R *.so build *.cpp
> {code}
>
> **When I try to run either of the python example scripts arrow_table.py or
> arrow_array.py,
> I get the following error.
>
> {code:java}
> File "arrow_array.py", line 1, in
> import example
> ImportError: libarrow.so.16: cannot open shared object file: No such file or
> directory
> {code}
>
>
> *Note: I also checked this on RHEL7 with Python 3.6.8, I got a similar
> response.*
>
>
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ARROW-8638) Arrow Cython API Usage Gives an error when calling CTable API Endpoints
[
https://issues.apache.org/jira/browse/ARROW-8638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17096493#comment-17096493
]
Uwe Korn commented on ARROW-8638:
-
You either need to extend the environment variable `LD_LIBRARY_PATH` to point
to the directory where `libarrow.so.16` resides or (a bit more complicated in
setup.py but the preferred approach) set the RPATH on the generated
`example.so` Python module to also include the directory where `libarrow.so.16`
reside, see turbodbc for an example:
https://github.com/blue-yonder/turbodbc/blob/8e2db0d0a26b620ad3e687e56a88fdab3117e09c/setup.py#L186-L189
> Arrow Cython API Usage Gives an error when calling CTable API Endpoints
> ---
>
> Key: ARROW-8638
> URL: https://issues.apache.org/jira/browse/ARROW-8638
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
>Affects Versions: 0.16.0
> Environment: Ubuntu 20.04 with Python 3.8.2
> RHEL7 with Python 3.6.8
>Reporter: Vibhatha Lakmal Abeykoon
>Priority: Blocker
> Fix For: 0.16.0
>
>
> I am working on using both Arrow C++ API and Cython API to support an
> application that I am developing. But here, I will add the issue I
> experienced when I am trying to follow the example,
> [https://arrow.apache.org/docs/python/extending.html]
> I am testing on Ubuntu 20.04 LTS
> Python version 3.8.2
> These are the steps I followed.
> # Create Virtualenv
> python3 -m venv ENVARROW
>
> 2. Activate ENV
> source ENVARROW/bin/activate
>
> 3. pip3 install pyarrow==0.16.0 cython numpy
>
> 4. Code block and Tools,
>
> +*example.pyx*+
>
>
> {code:java}
> from pyarrow.lib cimport *
> def get_array_length(obj):
> # Just an example function accessing both the pyarrow Cython API
> # and the Arrow C++ API
> cdef shared_ptr[CArray] arr = pyarrow_unwrap_array(obj)
> if arr.get() == NULL:
> raise TypeError("not an array")
> return arr.get().length()
> def get_table_info(obj):
> cdef shared_ptr[CTable] table = pyarrow_unwrap_table(obj)
> if table.get() == NULL:
> raise TypeError("not an table")
>
> return table.get().num_columns()
> {code}
>
>
> +*setup.py*+
>
>
> {code:java}
> from distutils.core import setup
> from Cython.Build import cythonize
> import os
> import numpy as np
> import pyarrow as pa
> ext_modules = cythonize("example.pyx")
> for ext in ext_modules:
> # The Numpy C headers are currently required
> ext.include_dirs.append(np.get_include())
> ext.include_dirs.append(pa.get_include())
> ext.libraries.extend(pa.get_libraries())
> ext.library_dirs.extend(pa.get_library_dirs())
> if os.name == 'posix':
> ext.extra_compile_args.append('-std=c++11')
> # Try uncommenting the following line on Linux
> # if you get weird linker errors or runtime crashes
> #ext.define_macros.append(("_GLIBCXX_USE_CXX11_ABI", "0"))
> setup(ext_modules=ext_modules)
> {code}
>
>
> +*arrow_array.py*+
>
> {code:java}
> import example
> import pyarrow as pa
> import numpy as np
> arr = pa.array([1,2,3,4,5])
> len = example.get_array_length(arr)
> print("Array length {} ".format(len))
> {code}
>
> +*arrow_table.py*+
>
> {code:java}
> import example
> import pyarrow as pa
> import numpy as np
> from pyarrow import csv
> fn = 'data.csv'
> table = csv.read_csv(fn)
> print(table)
> cols = example.get_table_info(table)
> print(cols)
>
> {code}
> +*data.csv*+
> {code:java}
> 1,2,3,4,5
> 6,7,8,9,10
> 11,12,13,14,15
> {code}
>
> +*Makefile*+
>
> {code:java}
> install:
> python3 setup.py build_ext --inplace
> clean:
> rm -R *.so build *.cpp
> {code}
>
> **When I try to run either of the python example scripts arrow_table.py or
> arrow_array.py,
> I get the following error.
>
> {code:java}
> File "arrow_array.py", line 1, in
> import example
> ImportError: libarrow.so.16: cannot open shared object file: No such file or
> directory
> {code}
>
>
> *Note: I also checked this on RHEL7 with Python 3.6.8, I got a similar
> response.*
>
>
>
>
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
