where is ignite's resilience/fault-tolerance design documented? i can not find it. i would generally stay away from it if fault-tolerance is an afterthought.
On Mon, Jan 11, 2016 at 10:31 AM, RodrigoB <rodrigo.boav...@aspect.com> wrote: > Although I haven't work explicitly with either, they do seem to differ in > design and consequently in usage scenarios. > > Ignite is claimed to be a pure in-memory distributed database. > With Ignite, updating existing keys is something that is self-managed > comparing with Tachyon. In Tachyon once a value is created for a given key, > becomes immutable, so you either delete and insert again, or need to > manage/update the tachyon keys yourself. > Also, Tachyon's resilience design is based on the underlying file system > (typically hadoop), which means that if a node goes down, to recover the > lost data, it would need first to have been persisted on the corresponding > file partition. > With Ignite, there is no master dependency like with Tachyon, and my > understanding is that API calls will depend on master's availability in > Tachyon. I believe Ignite has some options for replication which would be > more aligned with the in-memory datastore. > > If you are looking for persisting some RDD's output into an in-memory store > and query it outside of Spark, on the paper Ignite sounds like a better > solution. > > Since you are asking about Ignite benefits that was the focus of my > response. Tachyon has its own benefits like the community support and the > Spark lineage persistency integration. If you are doing batch based > processing and want to persist fast Spark RDDs, Tachyon is your friend. > > Hope this helps. > > Tnks, > Rod > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Spark-on-Apache-Ingnite-tp25884p25933.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
