tustvold commented on issue #1176: URL: https://github.com/apache/arrow-rs/issues/1176#issuecomment-1436946435
In the interests of saving people from having to parse the entire backscroll, and to verify everyone is on the same page, here is what I have understood the consensus to be: 1. arrow-rs will be migrated to use strongly-typed buffer abstractions under ArrayData, i.e. `ArrayData` becomes an enumeration similar to the proposal in https://github.com/apache/arrow-rs/issues/1799 2. The weakly typed ArrayData APIs will be gradually deprecated and removed, in favour of new strongly-typed APIs 3. arrow2 will be donated to Apache Arrow 4. The arrow2 array abstractions will be combined with the buffer abstractions under ArrayData 5. The arrow-rs kernels and IO will be updated to use ArrayData instead of arrow-rs arrays, this will be done preserving backwards compatibility (e.g. impl AsRef<ArrayData>`) 6. The arrow2 IPC and FFI will be merged with the arrow-rs implementation This should mean that **users of arrow-rs by will not need to change anything immediately**, some functionality will gradually be deprecated and removed, but there will be no rewrite-the-world style releases and we will endeavor to minimise churn. Once the arrow2 array abstractions are incorporated (4.) **users of arrow2 will be able to gradually migrate over to the arrow-rs kernels and IO**, with this getting easier once the kernels are updated to use `ArrayData` instead of the arrow-rs arrays. It isn't clear to me yet what the maintenance story for the arrow2 kernels, IO, etc... is both during the transition period and afterward. I suspect they will be gradually deprecated, but this has only been implied and I'm wary of jumping to conclusions here :sweat_smile: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
