pitrou commented on issue #38325: URL: https://github.com/apache/arrow/issues/38325#issuecomment-2022190526
> > That said, the notion of schema requests pretty much implies that we allow implicit data copies when doing a producer-side conversion. > > @pitrou could you clarify what you mean by that? Well, quoting the spec: > In some cases, there might be multiple possible Arrow representations of the same data. For example, a library might have a single integer type, but Arrow has multiple integer types with different sizes and sign. If different representations of the same data can be returned, then it means that at least _some_ of these representations incur a copy. For example, if a library has a string type encoded as (length, ptr) string views, then regardless of whether the producer exposes them as a Arrow String, LargeString or StringView, it needs to convert the lengths into Arrow offsets, and to linearize the string data as a contiguous buffer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
