vyasr commented on issue #38325:
URL: https://github.com/apache/arrow/issues/38325#issuecomment-2021744158
I like the `obj.__arrow_device_array__(requested_device=device)` API.
Producers should be responsible for handling the conversion and providing
suitable errors if necessary. I'm skeptical of making this conversion be "best
effort" though; the potential error modes are quite bad (e.g. segfaults) and
won't be very friendly even for implementers of a given consumer.
Deciding whether or not making a particular kind of copy is too expensive
seems out of scope for the protocol. Ultimately I'm not sure even producers
should be completely in the business of deciding what is too expensive.
Producers know best how to perform the copies and how expensive those copies
are in absolute terms, but ultimately only end users know the cost relative to
their overall workflow and whether the convenience is worthwhile. Producers
could be configurable in some way to determine whether to allow implicit
copies, but such configuration is out of scope of this spec. Consumers should
be able to make the desired request of a producer, allowing producers to choose
to honor that request or fail gracefully (e.g. cudf could decide not to allow
implicit D2H copies unless the user first does
`cudf.set_option("allow_implicit_arrow_copies", True)`; without that option
set, any implicit copy could raise an Exception).
> That said, the notion of schema requests pretty much implies that we allow
implicit data copies when doing a producer-side conversion.
@pitrou could you clarify what you mean by that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
