From the official site http://arrow.apache.org/, Apache Arrow is used for Columnar In-Memory storage. I have two quick questions: 1. Does spark support Apache Arrow? 2. When dataframe is cached in memory, the data are saved in columnar in-memory style. What is the relationship between this feature and Apache Arrow,that is, when the data is in Apache Arrow format,does spark still need the effort to cache the dataframe in columnar in-memory?
- Does spark support Apache Arrow Todd
- Re: Does spark support Apache Arrow Sun Rui
- Re: Does spark support Apache Arrow Hyukjin Kwon