jorisvandenbossche commented on code in PR #38472: URL: https://github.com/apache/arrow/pull/38472#discussion_r1412356415
########## docs/source/python/dlpack.rst: ########## @@ -0,0 +1,82 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +.. _pyarrow-dlpack: + +The DLPack Protocol +=================== + +Producing side of the DLPack Protocol is implemented for ``pa.Array`` +and can be used to interchange data between PyArrow and other tensor +libraries. The data structures that are supported in the implementation +of the protocol are integer, unsigned integer and float arrays. The +protocol has no missing data support meaning PyArrow arrays with +validity mask can not be used to transfer data through the DLPack Review Comment: ```suggestion protocol has no missing data support meaning PyArrow arrays with missing values cannot be used to transfer data through the DLPack ``` ########## docs/source/python/dlpack.rst: ########## @@ -0,0 +1,82 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +.. _pyarrow-dlpack: + +The DLPack Protocol +=================== + +Producing side of the DLPack Protocol is implemented for ``pa.Array`` +and can be used to interchange data between PyArrow and other tensor +libraries. The data structures that are supported in the implementation +of the protocol are integer, unsigned integer and float arrays. The +protocol has no missing data support meaning PyArrow arrays with +validity mask can not be used to transfer data through the DLPack +protocol. Currently Arrow implementation of the protocol only supports +data on a CPU device. + +The DLPack Protocol is +`selected as the Python array API standard <https://data-apis.org/array-api/latest/design_topics/data_interchange.html?highlight=dlpack#dlpack-an-in-memory-tensor-structure>`_ Review Comment: ```suggestion `selected as the Python array API standard <https://data-apis.org/array-api/latest/design_topics/data_interchange.html#dlpack-an-in-memory-tensor-structure>`_ ``` ########## docs/source/python/dlpack.rst: ########## @@ -0,0 +1,82 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +.. _pyarrow-dlpack: + +The DLPack Protocol +=================== + +Producing side of the DLPack Protocol is implemented for ``pa.Array`` +and can be used to interchange data between PyArrow and other tensor +libraries. The data structures that are supported in the implementation +of the protocol are integer, unsigned integer and float arrays. The +protocol has no missing data support meaning PyArrow arrays with +validity mask can not be used to transfer data through the DLPack +protocol. Currently Arrow implementation of the protocol only supports +data on a CPU device. + +The DLPack Protocol is Review Comment: Some text suggestions: I would maybe first explain DLPack before saying what is implemented by pyarrow (so moving this paragraph first), and then within this paragraph I think you can also explain first what DLPack is and then mention it is selected by the consortium. ########## docs/source/python/dlpack.rst: ########## @@ -0,0 +1,82 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +.. _pyarrow-dlpack: + +The DLPack Protocol +=================== + +Producing side of the DLPack Protocol is implemented for ``pa.Array`` +and can be used to interchange data between PyArrow and other tensor +libraries. The data structures that are supported in the implementation +of the protocol are integer, unsigned integer and float arrays. The +protocol has no missing data support meaning PyArrow arrays with +validity mask can not be used to transfer data through the DLPack +protocol. Currently Arrow implementation of the protocol only supports +data on a CPU device. + +The DLPack Protocol is +`selected as the Python array API standard <https://data-apis.org/array-api/latest/design_topics/data_interchange.html?highlight=dlpack#dlpack-an-in-memory-tensor-structure>`_ +by the +`Consortium for Python Data API Standards <https://data-apis.org/>`_ +in order to enable device aware data interchange between array/tensor +libraries in the Python ecosystem. Being device aware allows exchange +of data on devices other than the CPU (e.g. GPU). See more about the standard +in the +`protocol documentation <https://data-apis.org/array-api/latest/index.html>`_ +and more about the DLPack in the +`Python Specification for DLPack <https://dmlc.github.io/dlpack/latest/python_spec.html#python-spec>`_. + +Data interchange syntax of the protocol includes + +1. ``from_dlpack(x)``: consuming an array object that implements a ``__dlpack__`å` method Review Comment: ```suggestion 1. ``from_dlpack(x)``: consuming an array object that implements a ``__dlpack__`` method ``` ########## docs/source/python/dlpack.rst: ########## @@ -0,0 +1,82 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + +.. _pyarrow-dlpack: + +The DLPack Protocol +=================== + +Producing side of the DLPack Protocol is implemented for ``pa.Array`` +and can be used to interchange data between PyArrow and other tensor +libraries. The data structures that are supported in the implementation +of the protocol are integer, unsigned integer and float arrays. The +protocol has no missing data support meaning PyArrow arrays with +validity mask can not be used to transfer data through the DLPack +protocol. Currently Arrow implementation of the protocol only supports +data on a CPU device. + +The DLPack Protocol is +`selected as the Python array API standard <https://data-apis.org/array-api/latest/design_topics/data_interchange.html?highlight=dlpack#dlpack-an-in-memory-tensor-structure>`_ +by the +`Consortium for Python Data API Standards <https://data-apis.org/>`_ +in order to enable device aware data interchange between array/tensor +libraries in the Python ecosystem. Being device aware allows exchange +of data on devices other than the CPU (e.g. GPU). See more about the standard +in the +`protocol documentation <https://data-apis.org/array-api/latest/index.html>`_ +and more about the DLPack in the +`Python Specification for DLPack <https://dmlc.github.io/dlpack/latest/python_spec.html#python-spec>`_. + +Data interchange syntax of the protocol includes + +1. ``from_dlpack(x)``: consuming an array object that implements a ``__dlpack__`å` method + and creating a new array while sharing the memory. + +2. ``__dlpack__(self, stream=None)`` and ``__dlpack_device__``: producing a PyCapsule with + the DLPack struct which is called from within ``from_dlpack(x)``. + +PyArrow implements the second part of the protocol (``__dlpack__(self, stream=None)`` and +``__dlpack_device__``). Review Comment: ```suggestion ``__dlpack_device__``), and can thus be consumed by libraries implementing ``from_dlpack``. ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
