FYI It looks like there is active work to change the Python [1] and R [2] implementations (built on C++) to write .arrow instead of .feather.
[1] https://issues.apache.org/jira/browse/ARROW-17089 [2] https://issues.apache.org/jira/browse/ARROW-17088 On Fri, Jul 22, 2022 at 11:16 AM Aldrin <akmon...@ucsc.edu.invalid> wrote: > sorry, I meant "...especially *for* the rust community if they are just > using IPC directly for file formats." > > Aldrin Montana > Computer Science PhD Student > UC Santa Cruz > > > On Fri, Jul 22, 2022 at 11:14 AM Aldrin <akmon...@ucsc.edu> wrote: > > > I always assumed IPC was when it was in memory, feather was when it was > > persisted as files. That distinction/indirection may be totally moot, > > though, especially if the rust community is just using IPC directly for > > file formats. > > > > In my mind, feather V1 came just before IPC and started using IPC since > > V2. I'm not sure if a feather V3 would ever diverge from IPC format or if > > feather adds anything that's more filesystem friendly (versus other > storage > > system interfaces) or makes filesystem performance more predictable. > > > > > > Aldrin Montana > > Computer Science PhD Student > > UC Santa Cruz > > > > > > On Fri, Jul 22, 2022 at 10:49 AM Andrew Lamb <al...@influxdata.com> > wrote: > > > >> For what it is worth, in the Rust Arrow community, I have only ever > heard > >> the format referred to as "IPC Format" , we haven't used the term > Feather > >> that I know of. > >> > >> On Fri, Jul 22, 2022 at 12:41 PM David Li <lidav...@apache.org> wrote: > >> > >> > FeatherV2 is the same thing as the IPC file format ( > >> > https://arrow.apache.org/faq/). This confusion keeps coming up, so: > how > >> > do we want to brand Arrow IPC? As "Feather", or as "Arrow IPC Files"? > It > >> > seems the latter, but even right now Arrow Dataset still writes files > >> with > >> > .feather extension (though there are PRs to update it). I don't think > >> I've > >> > seen a formal discussion/decision on whether to keep using Feather or > >> IPC > >> > as the name (though I likely missed it). > >> > > >> > On Fri, Jul 22, 2022, at 12:21, Andrew Lamb wrote: > >> > > I am not familiar with feather -- there are some Draft PRs to add > >> support > >> > > for reading [1] and writing [2] the IPC file format which may be > >> related > >> > to > >> > > your question > >> > > > >> > > [1] https://github.com/apache/arrow-datafusion/pull/1858 > >> > > [2] https://github.com/apache/arrow-datafusion/pull/1893 > >> > > > >> > > > >> > > On Fri, Jul 22, 2022 at 12:10 PM Ian Joiner <iajoiner...@gmail.com> > >> > wrote: > >> > > > >> > >> Hi, > >> > >> > >> > >> I'd like to ask about the level of Feather support in Datafusion. > >> Can we > >> > >> read and write Feather files? I don't really find any information > >> while > >> > >> searching through the repo or checking the docs. > >> > >> > >> > >> Thanks, > >> > >> Ian > >> > >> > >> > > >> > > >
