Re: Update all data in cache spend a long time
Thank you, I will try queryParallelism. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Update all data in cache spend a long time
Hello! 1. I don't think you should use LIMIT because it is very hard to make sure that all entries are updated exactly once. It is better to partition data by values of some field. 2. It is hard to say outright, did you try queryParallelism? Regards, -- Ilya Kasnacheev вт, 9 окт. 2018 г. в 16:27, Justin Ji : > Ilya - > > Thank for your reply and point out the problem may exist in my code, I will > try to split the operation into smaller ones. > > And I still have two problems: > > 1. Can I split the operation with LIMIT? > 2. Where is time spent in the operation? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Update all data in cache spend a long time
Ilya - Thank for your reply and point out the problem may exist in my code, I will try to split the operation into smaller ones. And I still have two problems: 1. Can I split the operation with LIMIT? 2. Where is time spent in the operation? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Update all data in cache spend a long time
Hello! Have you tried setting queryParallelism on this cache to some high value (such as your number of CPU cores)? Note that this query will likely cause lifting all data in cache to heap at once, which is probably not what you expect. My recommendation is also to try splitting this operation into smaller ones, see if they finish faster. Regards, -- Ilya Kasnacheev вт, 9 окт. 2018 г. в 15:47, Justin Ji : > Hi - > > I have a cache with more than 1 million records, when I update the whole > records in it, it spends a long time(almost 6 minutes). > > Here is the SQL: > "update " + IgniteTableKey.T_DEVICE_ONLINE_STATUS.getCode()+ " set > isOnline=0, mqttTime=" + System.currentTimeMillis() / 1000 + " where 1=1"; > > And the Cache configuration is : > > CacheConfiguration cacheCfg = new CacheConfiguration<>(); > cacheCfg.setName(cacheName); > cacheCfg.setCacheMode(CacheMode.PARTITIONED); > cacheCfg.setBackups(1); > cacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL); > > > cacheCfg.setCacheStoreFactory(FactoryBuilder.factoryOf(DeviceStatusCacheStore.class)); > cacheCfg.setWriteThrough(true); > cacheCfg.setWriteBehindEnabled(true); > cacheCfg.setReadThrough(true); > cacheCfg.setWriteBehindFlushThreadCount(4); > cacheCfg.setWriteBehindFlushFrequency(15 * 1000); > cacheCfg.setWriteBehindFlushSize(409600); > cacheCfg.setWriteBehindBatchSize(1024); > cacheCfg.setStoreKeepBinary(true); > cfg.setCacheConfiguration(cacheCfg); > > So I want to know is there a way that can speed up the execution? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Update all data in cache spend a long time
Hi - I have a cache with more than 1 million records, when I update the whole records in it, it spends a long time(almost 6 minutes). Here is the SQL: "update " + IgniteTableKey.T_DEVICE_ONLINE_STATUS.getCode()+ " set isOnline=0, mqttTime=" + System.currentTimeMillis() / 1000 + " where 1=1"; And the Cache configuration is : CacheConfiguration cacheCfg = new CacheConfiguration<>(); cacheCfg.setName(cacheName); cacheCfg.setCacheMode(CacheMode.PARTITIONED); cacheCfg.setBackups(1); cacheCfg.setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL); cacheCfg.setCacheStoreFactory(FactoryBuilder.factoryOf(DeviceStatusCacheStore.class)); cacheCfg.setWriteThrough(true); cacheCfg.setWriteBehindEnabled(true); cacheCfg.setReadThrough(true); cacheCfg.setWriteBehindFlushThreadCount(4); cacheCfg.setWriteBehindFlushFrequency(15 * 1000); cacheCfg.setWriteBehindFlushSize(409600); cacheCfg.setWriteBehindBatchSize(1024); cacheCfg.setStoreKeepBinary(true); cfg.setCacheConfiguration(cacheCfg); So I want to know is there a way that can speed up the execution? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/