Valentin Kulichenko created IGNITE-7092:
-------------------------------------------
Summary: Deprecate embedded mode in IgniteRDD
Key: IGNITE-7092
URL: https://issues.apache.org/jira/browse/IGNITE-7092
Project: Ignite
Issue Type: Improvement
Components: spark
Affects Versions: 2.3
Reporter: Valentin Kulichenko
Assignee: Valentin Kulichenko
Fix For: 2.4
Currently we claim to support IgniteRDD in two modes: standalone and embedded.
Standalone means there is a separately running Ignite cluster, and Spark start
client node(s) to interact with it. In embedded node everything runs within
Spark, including Ignite server nodes that are started embedded into Spark
executors.
The latter case doesn't really work, mainly because the lifecycle of Spark
executors is not very predictable - Spark can start and stop them while
application is running. In case Ignite cluster is used to store data (which is
usually the case), this causes unnecessary rebalancing or even unexpected data
loss.
I propose to deprecate and eventually discontinue the embedded mode. Luckily,
standalone mode is the default one, so we can simply print out a clear warning
if one switches to embedded mode, and also mention this in the docs.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
