Here is my understanding.
Spark used Tachyon as an off-heap solution for RDDs. In certain situations, it
would alleviate Garbage Collection or the RDDs.
Tungsten, Spark 2’s off-heap (columnar format) is much more efficient and used
as the default. Alluvio no longer makes sense for this use.
You can still use Tachyon/Alluxio to bring your files into Memory, which is
quicker for Spark to access than your DFS(HDFS or S3).
Alluxio actually supports a “Tiered Filesystem”, and automatically brings the
“hotter” files into the fastest storage (Memory, SSD). You can configure it
with Memory, SSD, and/or HDDs with the DFS as the persistent store, called
Hope this helps.
> On Sep 19, 2016, at 7:56 AM, aka.fe2s <aka.f...@gmail.com> wrote:
> Hi folks,
> What has happened with Tachyon / Alluxio in Spark 2? Doc doesn't mention it
> no longer.
> Oleksiy Dyagilev