quinnj commented on PR #607: URL: https://github.com/apache/arrow-julia/pull/607#issuecomment-5592914331
@samtalki thanks for all the work on this stack, including splitting it up for review and working through the ownership, release, and validation details with @kou. I want to flag the overlap with #609 before moving ahead. I’m planning to merge that rewrite for Arrow.jl 3.0. It changes the underlying array and ownership model and includes C Data and C Stream support, so I expect that to be the implementation we build on. I realize you’ve put substantial time into getting this ready against the current code, and I apologize for the lack of communication until now on parallel efforts (I've been playing around w/ the major rewrite for a while now, but leading up to JuliaCon finally found the time to actually implement). You mentioned needing this for PowerIO.jl. If you’re interested, I’d welcome your feedback on whether the new C Data API covers that use case. There’s no expectation that you rework this whole stack against the new internals. My goal w/ the rewrite though was to make the C data model the core of everything since it's the more foundational layer, instead of building it on top of the, IMO, now dated architecture of the original Arrow.jl. It allows us to easily build the C data, stream, IPC, file formats, and normal Julia interop (Tables.jl, etc.) as small layers/adapters on top. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
