yes i am using it in embeded mode. Standalone mode we can go for, but additional hardware needed for that which will stay idle when we don't use it. That was the reason to try this embedded stuff. We build the cache on fly , which gets shut down with spark.
My question basically was, when i set rebalancing to NONE and have back up count as 3, how it behaves ? Will i still have all the data when one node stops cause of back up set to 3 ? On Thu, Feb 1, 2018 at 2:26 PM, Evgenii Zhuravlev <[email protected]> wrote: > > When due to some reason a task fails, the ignite node stops and when > the task re-starts, > Looks like you run in embedded mode. Do avoid too frequently node stopping > events, you need to run Ignite in standalone mode, in this case, the node > will run even if your task fails. > > Please let me know if I missed something. > > Evgenii > > 2018-02-01 11:44 GMT+03:00 Ranjit Sahu <[email protected]>: > >> I read the data which is in avro format using spark sql and load it to >> cache from spark program. I build the Ignite key-store inside spark >> executors. When due to some reason a task fails, the ignite node stops and >> when the task re-starts, >> the new node joins back.I see slowness from here onwards. I was thinking >> cause of rebalancing this is becoming slow. I can look at tuning the >> rebalancing too. Let me know if you have any suggestions. >> >> On Thu, Feb 1, 2018 at 1:54 PM, Evgenii Zhuravlev < >> [email protected]> wrote: >> >>> Ranjit, >>> >>> How do you load data to the cache? >>> >>> Evgenii >>> >>> 2018-02-01 11:18 GMT+03:00 Ranjit Sahu <[email protected]>: >>> >>>> Hi Val, >>>> >>>> Not always but out of 10, we see at least once the issue. Whats >>>> happening is when one node crashes\stops the new node joins . The loading >>>> process restarts but what ever was happening in few minutes (3-5) goes to >>>> 2-3 hours. >>>> >>>> Thanks, >>>> Ranjit >>>> >>>> On Wed, Jan 31, 2018 at 3:12 AM, vkulichenko < >>>> [email protected]> wrote: >>>> >>>>> Ranjit, >>>>> >>>>> Is it a really frequent event for node to crash in the middle of >>>>> loading >>>>> process? If so, then I think you should fix that instead of working >>>>> around >>>>> by disabling rebalancing. Such configuration definitely has a lot >>>>> drawbacks >>>>> and therefore can cause issues. >>>>> >>>>> -Val >>>>> >>>>> >>>>> >>>>> -- >>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >>>>> >>>> >>>> >>> >> >
