A simple (albeit somewhat limited) analogy would be a comparison to Parquet and HDFS.
Parquet is a way to store data in a columnar representation on disk. HDFS is a distributed way to read and write streams of data on physical media. Arrow is (amongst other things) a way to structure data in memory for high speed processing and deals with records, fields and datatypes. Tachyon is distributed way to read and write streams of data in-memory. There is a great opportunity to integrate Tachyon and Arrow as they are very complimentary. Being able to share Arrow representations through a shared in-memory storage system would be helpful to many applications. On Thu, Feb 18, 2016 at 11:45 AM, Sambit Tripathy (RBEI/EDS1) < [email protected]> wrote: > > Hi, > > I thank the community for this initiative. > > Just out of curiosity, is this similar to http://tachyon-project.org/ > (Tachyon)? > > > > Best regards > > Sambit Tripathy > > >
