pitrou commented on code in PR #37797: URL: https://github.com/apache/arrow/pull/37797#discussion_r1345616804
########## docs/source/format/CDataInterface/PyCapsuleInterface.rst: ########## @@ -0,0 +1,402 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +.. or more contributor license agreements. See the NOTICE file +.. distributed with this work for additional information +.. regarding copyright ownership. The ASF licenses this file +.. to you under the Apache License, Version 2.0 (the +.. "License"); you may not use this file except in compliance +.. with the License. You may obtain a copy of the License at + +.. http://www.apache.org/licenses/LICENSE-2.0 + +.. Unless required by applicable law or agreed to in writing, +.. software distributed under the License is distributed on an +.. "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +.. KIND, either express or implied. See the License for the +.. specific language governing permissions and limitations +.. under the License. + + +============================= +The Arrow PyCapsule Interface +============================= + +Rationale +========= + +The :ref:`C data interface <c-data-interface>` and +:ref:`C stream interface <c-stream-interface>` allow moving Arrow data between +different implementations of Arrow. However, these interfaces don't specify how +Python libraries should expose these structs to other libraries. Prior to this, +many libraries simply provided export to PyArrow data structures, using the +``_import_from_c`` and ``_export_from_c`` methods. However, this always required +PyArrow to be installed. + +This interface allows any library to export Arrow data structures to other +libraries that understand the same protocol. + +Goals +----- + +* Standardize the `PyCapsule`_ objects that represent ``ArrowSchema``, ``ArrowArray``, + and ``ArrowArrayStream``. +* Define standard methods to get these capsule objects, so any Python objects that + support export via Arrow can be recognized by other libraries. + + +Non-goals +--------- + +* Standardizing what public APIs should be used for import. This is left up to + individual libraries. + + +Comparison to DataFrame Interchange Protocol +-------------------------------------------- + +`The DataFrame Interchange Protocol <https://data-apis.org/dataframe-protocol/latest/>`_ +is another protocol in Python that allows for the sharing of data between libraries. +This protocol is complementary to the DataFrame Interchange Protocol. Many of +the objects that implement this protocol will also implement the DataFrame +Interchange Protocol. + +This protocol is specific to Arrow-based data structures, while the DataFrame +Interchange Protocol allows non-Arrow data frames and arrays to be shared as well. +Because of this, these PyCapsules can support Arrow-specific features such as +nested columns. + +This protocol is also much more minimal than the DataFrame Interchange Protocol. +It just handles data export, rather than defining accessors for details like +number of rows or columns. + +In summary, if you are implementing this protocol, you should also consider +implementing the DataFrame Interchange Protocol. + + +Comparison to __arrow_array__ protocol +-------------------------------------- + +The ``__arrow_array__`` protocol is a dunder method that defines how PyArrow Review Comment: Link to the relevant section in the Python docs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
