I think one source of ambiguity for Arrow files, at least for me, is whether they are just a string of messages concatenated or they are the files that contain the metadata footer.
On Tue, Aug 30, 2022 at 5:11 AM Dewey Dunnington <de...@voltrondata.com.invalid> wrote: > Ian has a very good point...I would be in favour of calling them "Arrow > files" wherever possible since there's no need to know or care what > interprocess communication is to use them! > > On Mon, Aug 29, 2022 at 6:50 PM Ian Cook <i...@ursacomputing.com> wrote: > > > +1 We should explicitly discourage further use of “Feather” to refer to > > Arrow IPC files. > > > > In this spirit of simplifying terminology: Does the “IPC” in the term > > “Arrow IPC files” serve a truly necessary purpose? Is there another type > of > > “Arrow file” that the “IPC” serves to disambiguate? If not, can we simply > > refer to these files as “Arrow files” in most places in the documentation > > and website? (In a few important places we should clarify that when we > say > > “Arrow file” we are referring to a file that uses the Arrow IPC file > > format.) > > > > Ian > > > > On Mon, Aug 29, 2022 at 17:33 Sutou Kouhei <k...@clear-code.com> wrote: > > > > > +1 for 1. > > > > > > Thanks, > > > -- > > > kou > > > > > > In <CAOYPqDCAib2wBKaKnRij9=__OsUJJghVq1UUTNibK2T0Np+=r...@mail.gmail.com > > > > > "Re: Usage of the name Feather?" on Mon, 29 Aug 2022 20:18:37 +0200, > > > Jorge Cardoso Leitão <jorgecarlei...@gmail.com> wrote: > > > > > > > I agree. > > > > > > > > I suspect that the most widely used API with "feather" is Pandas' > > > > read_feather. > > > > > > > > > > > > > > > > On Mon, 29 Aug 2022, 19:55 Weston Pace, <weston.p...@gmail.com> > wrote: > > > > > > > >> I agree as well. I think most lingering uses of the term "feather" > > > >> are in pyarrow and R however, so it might be good to hear from some > of > > > >> those maintainers. > > > >> > > > >> > > > >> > > > >> On Mon, Aug 29, 2022 at 9:35 AM Antoine Pitrou <anto...@python.org> > > > wrote: > > > >> > > > > >> > > > > >> > I agree with this as well. > > > >> > > > > >> > Regards > > > >> > > > > >> > Antoine. > > > >> > > > > >> > > > > >> > On Mon, 29 Aug 2022 11:29:45 -0400 > > > >> > Andrew Lamb <al...@influxdata.com> wrote: > > > >> > > In the rust implementation we use the term "Arrow IPC" and I > > support > > > >> your > > > >> > > option 1: > > > >> > > > > > >> > > > The name Feather V2 is deprecated. Only the extension ".arrow" > > > will > > > >> be > > > >> > > used for IPC files. > > > >> > > > > > >> > > Andrew > > > >> > > > > > >> > > On Mon, Aug 29, 2022 at 11:21 AM Matthew Topol > > > >> <m...@voltrondata.com.invalid> > > > >> > > wrote: > > > >> > > > > > >> > > > When I wrote "In-Memory Analytics with Apache Arrow" I > > definitely > > > >> > > > treated "Feather" as deprecated and mentioned it only in > passing > > > >> > > > specifically indicating "Arrow IPC" as the terminology to > use. I > > > only > > > >> > > > even mentioned "Feather" at all because there are still > methods > > in > > > >> > > > pyarrow that reference it by name. > > > >> > > > > > > >> > > > That's just my opinion though... > > > >> > > > > > > >> > > > On Mon, Aug 29 2022 at 11:08:53 AM -0400, David Li > > > >> > > > <lidav...@apache.org> wrote: > > > >> > > > > This has come up before, e.g. see [1] [2] [3]. > > > >> > > > > > > > >> > > > > I would say "Feather" is effectively deprecated and we are > > using > > > >> > > > > "Arrow IPC" now but I am not sure what others think. (From > > that > > > >> > > > > GitHub link, it seems to be mixed.) And ".arrow" is the > > official > > > >> > > > > extension now (since it is registered as part of our MIME > > type). > > > >> But > > > >> > > > > there's existing documentation and not everything has been > > > updated > > > >> to > > > >> > > > > be consistent (as you saw). > > > >> > > > > > > > >> > > > > [1]: > > > >> > > > > < > > > https://lists.apache.org/thread/0s6lgvd3g56ymd60vl5lgzhf4ro6hts5> > > > >> > > > > [2]: > > > >> > > > > < > > > https://arrow.apache.org/faq/#what-about-the-feather-file-format> > > > >> > > > > [3]: > > > >> > > > > < > > > >> > > > > > > >> > > > > > > https://stackoverflow.com/questions/67910612/arrow-ipc-vs-feather/67911190#67911190 > > > >> > > > > > > > >> > > > > > > > >> > > > > -David > > > >> > > > > > > > >> > > > > On Mon, Aug 29, 2022, at 10:50, 島 達也 wrote: > > > >> > > > >> Hi all. > > > >> > > > >> > > > >> > > > >> I know the documentation (mainly pyarrow documentation) > > > sometimes > > > >> > > > >> refers > > > >> > > > >> to IPC files as Feather files, but are there any > guidelines > > > for > > > >> > > > >> when to > > > >> > > > >> refer to an IPC file as a Feather file and when to refer > to > > > it as > > > >> > > > >> an IPC > > > >> > > > >> file? > > > >> > > > >> I believe that calling the same file an Arrow IPC file at > > > times > > > >> and > > > >> > > > >> a > > > >> > > > >> Feather file at other times is confusing to those > unfamiliar > > > with > > > >> > > > >> Apache > > > >> > > > >> Arrow (myself included). > > > >> > > > >> Surprisingly, these files may even have completely > different > > > >> > > > >> extensions, > > > >> > > > >> ".arrow" and ".feather", which are not similar. > > > >> > > > >> > > > >> > > > >> Perhaps there are several options for future use of the > name > > > >> > > > >> Feather, > > > >> > > > >> such as > > > >> > > > >> > > > >> > > > >> 1. The name Feather V2 is deprecated. Only the extension > > > >> ".arrow" > > > >> > > > >> will > > > >> > > > >> be used for IPC files. > > > >> > > > >> 2. In some contexts(?), IPC files are referred to as > > Feather; > > > >> only > > > >> > > > >> ".arrow" is used for the IPC file extension to clearly > > > >> > > > >> distinguish > > > >> > > > >> it from Feather V1's ".feather". > > > >> > > > >> 3. When an IPC file is called Feather by some rule, > > extension > > > >> > > > >> ".feather" is used, and when an IPC file is not called > > > >> Feather, > > > >> > > > >> extension ".arrow" is used. > > > >> > > > >> > > > >> > > > >> I mistakenly thought the current status was 2, but > according > > > to > > > >> the > > > >> > > > >> discussion in this PR > > > >> > > > >> (<https://github.com/apache/arrow/pull/13677>), > > > >> > > > >> apparently the current status seems 3. (However, there > seems > > > to > > > >> be > > > >> > > > >> no > > > >> > > > >> rule as to when an IPC file should be called a Feather) > > > >> > > > >> > > > >> > > > >> I am not very familiar with Arrow and this is my first > post > > to > > > >> this > > > >> > > > >> mailing list so I apologize if I have done something wrong > > or > > > >> > > > >> inappropriate. > > > >> > > > >> > > > >> > > > >> Best, > > > >> > > > >> SHIMA Tatsuya > > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > > > > > >
