Not sure I got the picture of your setup, but the ignite cache should be started indecently of the application and not within the application.
Aside from that, can you please elaborate more on the problem you like to solve - maybe with pseudocode? I am not sure if the approach you have selected is the right one. > On 10 Feb 2017, at 04:23, Ranjit Sahu <[email protected]> wrote: > > Hi Guys, > > We are trying to use ignite key-value store inside the Spark cluster. What we > are to do is : > > 1) Load the data to a data frame in spark > 2) While doing transformation on the data frame, on each executor node we are > starting the ignite cache and load the data. > > > The issue what we are seeing is , when i start with 40 executor nodes, i can > see only 24+ nodes in ignite topology. There is nothing in the log with > exceptions and we are not seeing the full data available in the cache too. > > We were not able to use SharedRDD of ignite as we are kind of in a Nested RDD > situation and too avoid that we are trying to work with Ignite cache. > > Any clue what's going wrong here. > > Thanks, > Ranjit > >
