I can see two options here: - Between iteration 1 and iteration 2 some nodes were stopped. Perhaps some new nodes were started. Data on stopped nodes became unavailab.e - Cache key collisions between iterations 1 and 2 so that 80% keys are identical and only 20% are distinct the second time.
I expect it is the former. When you ask YARN to run 10 Ignite nodes, I guess it will start them on random machines, and not on the same ones every time. This will lead to different set of machines next time and lost data. I don't think you should be using persistence with YARN. In fact, /tmp in paths should give you the hint that you should not depend on availability of data between runs. Regards, -- Ilya Kasnacheev 2018-01-30 16:21 GMT+03:00 Raghav <[email protected]>: > Hi, > > 1) Load data to cache > > var cacheConf: CacheConfiguration[Long, Data] = new > CacheConfiguration[Long, Data]("DataCache") > cacheConf.setCacheMode(CacheMode.PARTITIONED) > cacheConf.setIndexedTypes(classOf[Long], classOf[Data]) > val cache = ignite.getOrCreateCache(cacheConf) > var dataMap = getDataMap() > cache.*putAll*(dataMap) > > There is no possibility of having duplicate keys as currentTimeInMillis > along with loop count is included. > > 2) Count Logic: > > val sql1 = "select * from DataCache" > val count = cache.*query*(new SqlFieldsQuery(sql1)).getAll.size() > > Used query instead of metrics. > > 3) Errors: No error in server as well as client logs. > > Also, checked for folder creation in ignite work > directory[IGNITE_WORKING_DIR] as per > https://cwiki.apache.org/confluence/display/IGNITE/ > Ignite+Persistent+Store+-+under+the+hood. > But no folders created for persistence. > > Thank you. > > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
