leofang commented on code in PR #34972:
URL: https://github.com/apache/arrow/pull/34972#discussion_r1169068200
##########
cpp/src/arrow/c/abi.h:
##########
@@ -106,6 +212,98 @@ struct ArrowArrayStream {
#endif // ARROW_C_STREAM_INTERFACE
+#ifndef ARROW_C_DEVICE_STREAM_INTERFACE
+#define ARROW_C_DEVICE_STREAM_INTERFACE
+
+/// \brief Equivalent to ArrowArrayStream, but for ArrowDeviceArrays.
+///
+/// This stream is intended to provide a stream of data on a single
+/// device, if a producer wants data to be produced on multiple devices
+/// then multiple streams should be provided. One per device.
+struct ArrowDeviceArrayStream {
+ /// \brief The device that this stream produces data on.
+ ///
+ /// All ArrowDeviceArrays that are produced by this
+ /// stream should have the same device_type as set
+ /// here. The device_type needs to be provided here
+ /// so that consumers can provide the correct type
+ /// of queue_ptr when calling get_next.
+ ArrowDeviceType device_type;
+
+ /// \brief Callback to get the stream schema
+ /// (will be the same for all arrays in the stream).
+ ///
+ /// If successful, the ArrowSchema must be released independantly from the
stream.
+ /// The schema should be accessible via CPU memory.
+ ///
+ /// \param[in] self The ArrowDeviceArrayStream object itself
+ /// \param[out] out C struct to export the schema to
+ /// \return 0 if successful, an `errno`-compatible error code otherwise.
+ int (*get_schema)(struct ArrowDeviceArrayStream* self, struct ArrowSchema*
out);
+
+ /// \brief Callback to get the device id for the next array.
+ ///
+ /// This is necessary so that the proper/correct stream pointer can be
provided
+ /// to get_next.
+ ///
+ /// The next call to `get_next` should provide an ArrowDeviceArray whose
+ /// device_id matches what is provided here, and whose device_type is the
+ /// same as the device_type member of this stream.
Review Comment:
> do you happen to know if there was any other discussion captured that
could be linked here regarding the decision to have a consumer hand a stream to
the producer
Sorry I wasn't able to respond promptly. Is the question still open?
In the case of CAI, [it is
required](https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#streams)
that someone handles the exporting stream's lifetime properly:
> Like data, CUDA streams also have a finite lifetime. It is therefore
required that a Producer exporting data on the interface with an associated
stream ensures that the exported stream’s lifetime is equal to or surpasses the
lifetime of the object from which the interface was exported.
and this was considered a burden when discussing the DLPack support. A few
libraries like Numba, for example, had to [hold the reference to the underlying
stream](https://numba.readthedocs.io/en/stable/cuda/cuda_array_interface.html#producing-arrays).
I believe this was the main concern for DLPack to place the requirement on the
consumer instead of the producer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
