kou commented on issue #99: URL: https://github.com/apache/arrow-js/issues/99#issuecomment-3082843298
You can use a file format that matches your use case. FYI: * Apache Arrow is a data format on memory https://arrow.apache.org/docs/format/Columnar.html * Apache Arrow standardizes 2 file formats: * Apache Arrow "file" format: https://arrow.apache.org/docs/format/Columnar.html#ipc-file-format * "Feather" version 2 is an alias of the "file" format. (We want to deprecate the "Feather" name".) * "Feather" version 1 is deprecated. It's not compatible Apache Arrow "file" format. * Apache Arrow "streaming" format: https://arrow.apache.org/docs/format/Columnar.html#ipc-streaming-format * In general, Apache Arrow format file is bigger than Apache Parquet format file * Because Apache Parquet format uses more aggressive compression * In general, Apache Arrow format file is faster than Apache Parquet format file * Because Apache Arrow format's parse cost is almost zero (by zero-copy feature) * Apache Arrow format and Apache Parquet format complement each other * Read data from Apache Parquet (from storage) can be represented as Apache Arrow data format on memory * See also: https://arrow.apache.org/faq/#what-is-the-difference-between-apache-arrow-and-apache-parquet -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
