Alexey Goncharuk created SPARK-9203:
---------------------------------------
Summary: Make filesystem pluggable in BlockStore and BlockManager
Key: SPARK-9203
URL: https://issues.apache.org/jira/browse/SPARK-9203
Project: Spark
Issue Type: Improvement
Reporter: Alexey Goncharuk
I was looking through the code in order to understand better how RDD is
persisted to Tachyon off-heap filesystem. It looks like that the Tachyon
filesystem is hard-coded and there is no way to switch to another in-memory
filesystem. I think it would be great if the implementation of the BlockManager
and BlockStore would be able to plug in another filesystem.
For example, Apache Ignite also has an implementation of in-memory filesystem
which can store data in on-heap and off-heap formats. It would be great if it
could integrate with Spark.
Apache Ignite In-Memory Filesystem:
https://ignite.incubator.apache.org/features/igfs.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
