Creation of each cache requires creation and initialization of
internal data structures that leads to increased pressure to GC. Could
you enable GC logs and look at result. I think you will find long GC
pauses.
In order to reduce memory consumption by created caches on creation
stage we can do the following:
- Decrease cache start size (see cacheConfiguration.setStartSize(),
default value is 1 500 000). It safe but can lead to some performance
penalty during dynamic increasing cache size;
- For atomic caches we can decrease size of deferred delete queue size
via JVM parameter (-DIGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE=<your
value>, default value is 200 000 entries per partition). It's static
configuration and can't be changed at runtime;
- Partitions number also affect memory consumption and cache creation
time. But it affect also data distribution so should be configured
carefully. Caches with default configuration uses
RendevousAffinityFunction ith 1024 partitions. This parameter can't be
changed after cache creation and start.
Does it work for you?
On Tue, Apr 11, 2017 at 12:53 PM, ctranxuan
<cedric.tranx...@streamdata.io> wrote:
> Hi,
> We are trying for test purposes to create lots of cache on a machine.
> Something like:
>
> for (int i = 0; i < 10000; i++) {
> IgniteCache<Long, Snapshot> cache;
> cache = ignite.<Long, Snapshot>getOrCreateCache("cache-" +
> i).withAsync();
> LOGGER.info("starting to read cache #" + cache.getName());
> }
>
> What we are noticing is that after ~200 caches created, the cache creation
> becomes slower and slower. For instance, when reaching the 2000th cache, it
> takes between 2-3 seconds. For instance, here some logs:
>
> 09:34:27.857 starting to read cache #cache-2087
> 09:34:29.621 starting to read cache #cache-2088
> 09:34:31.450 starting to read cache #cache-2089
> 09:34:33.127 starting to read cache #cache-2090
>
> That's may be a naive way of doing that and may be a naive question: but is
> it normal that the more cache is created, the more time the creation takes?
>
> The program is run on a 12GB RAM machine with 4 CPUs with a Java 8
> (1.8.0_121).
>
> Thanks in advance for the answers!
>
>
>
>
> --
> View this message in context:
> http://apache-ignite-users.70518.x6.nabble.com/Lots-of-cache-creation-become-slow-tp11875.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
