Re: Missing records Ignite cache size grows

[diopek]

Below is our implementation of LoadOnlyCacheStore inputIterator method.
As we tested many times there is no issue on uniqueness of generated cache
keys.
But still after loading high number of records into IgniteCache, records
count doesn't match, and few 10s of keys were missing out of million
records. Also if we use the Java cache populated just before serializing
this cache into IgniteCache, all number of records are matching.

In short, after several days of try-outs and debugging, we narrowed down the
root cause of issue to  LoadOnlyCacheStore base class. So, at this point, we
need Ignite core team support to resolve this bug.
Thanks


Yes, we checked uniqueness of key several times. Below is representative
snippet of our code, how we generate cache key inside inputIterator method. 

@Override 
protected Iterator<Tuple2&lt;Long,ArrayList&lt;MyDTO>>>*
inputIterator*(Object... args) throws CacheLoaderException { 
        Iterator<Tuple2&lt;Long,ArrayList&lt;MyDTO>>> iterator = null; 
       
ResultSetExtractor<ArrayList&lt;Tuple2&lt;Long,ArrayList&lt;MyDTO>>>>
extOrRepMapResultSetExtractor = new
ResultSetExtractor<ArrayList&lt;Tuple2&lt;Long,ArrayList&lt;MyDTO>>>>() { 
                @Override 
                public ArrayList<Tuple2&lt;Long,ArrayList&lt;MyDTO>>>
extractData(ResultSet rs) 
                                throws SQLException, DataAccessException { 
                        ArrayList<Tuple2&lt;Long,ArrayList&lt;MyDTO>>>
extOrRepList = new
ArrayList<Tuple2&lt;Long,ArrayList&lt;MyDTO>>>(startSize); 
                        *final AtomicLong entryCnt = new AtomicLong(0);*
                        while (rs.next()) {     
                                extOrRepList.add(new Tuple2<Long,
ArrayList&lt;MyDTO>>(*entryCnt.incrementAndGet()*,prevDTOList)); 
                        } 
                        return extOrRepList; 
                } 
        }; 

        jdbcTemplate.setFetchSize(SQL_FETCH_SIZE);      
        ArrayList<Tuple2&lt;Long,ArrayList&lt;MyDTO>>> extOrRepList = null; 
        extOrRepList = jdbcTemplate.query(sql,
extOrRepMapResultSetExtractor); 

        if (extOrRepList != null) { 
                iterator = extOrRepList.iterator(); 
        } 
        return iterator; 
        
}





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Missing-records-Ignite-cache-size-grows-tp10809p10966.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.