[
https://issues.apache.org/jira/browse/ARROW-9777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250140#comment-17250140
]
Neville Dipale commented on ARROW-9777:
---------------------------------------
Hi [~jorgecarleitao], that's my intention yes. We've fallen behind a bit, and
we might run a rist of hitting a compatibility issue in future, given that more
users are starting to use the Rust implementation (specifically IPC).
I've done some work already on the V5 JIRA, reads work well, but some writes
still create invalid outputs. If the next release is around Jan, I should be
able to get to it.
> [Rust] Implement IPC changes to catch up to 1.0.0 format
> --------------------------------------------------------
>
> Key: ARROW-9777
> URL: https://issues.apache.org/jira/browse/ARROW-9777
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Rust
> Affects Versions: 1.0.0
> Reporter: Neville Dipale
> Priority: Blocker
> Fix For: 3.0.0
>
>
> There are a number of IPC changes and features which the Rust implementation
> has fallen behind on. It's effectively using the legacy format that was
> released in 0.14.x.
> Some that I encountered are:
> * change padding from 4 bytes to 8 bytes (along with the padding algorithm)
> * add an IPC writer option to support the legacy format and updated format
> * add error handling for the different metadata versions, we should support
> v4+ so it's an oversight to not explicitly return errors if unsupported
> versions are read
> Some of the work already has Jiras open (e.g. body compression), I'll find
> them and mark them as related to this.
> I'm tight for spare time, but I'll try work on this before the next release
> (along with the Parquet writer)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
