Val, Sounds reasonable to me. The fewer useless and potentially harmful features or “switches” we have in Ignite the clear it will be for the user how to use us in a right way.
+1 for the deprecation and further removal. — Denis > On Nov 30, 2017, at 3:07 PM, Valentin Kulichenko > <valentin.kuliche...@gmail.com> wrote: > > Igniters, > > Currently we claim to support IgniteRDD in two modes: standalone and > embedded. Standalone means there is a separately running Ignite cluster, > and Spark start client node(s) to interact with it. In embedded node > everything runs within Spark, including Ignite server nodes that are > started embedded into Spark executors. > > The latter case doesn't really work, mainly because the lifecycle of Spark > executors is not very predictable - Spark can start and stop them while > application is running. In case Ignite cluster is used to store data (which > is usually the case), this causes unnecessary rebalancing or even > unexpected data loss. > > I propose to deprecate and eventually discontinue the embedded mode. > Luckily, standalone mode is the default one, so we can simply print out a > clear warning if one switches to embedded mode, and also mention this in > the docs. > > Thoughts? If there are no objections, I will create a ticket and make the > change. > > -Val
