Hi, Denis! Previously, I answered Slava about implementation that I keep in mind, now it will be possible to add own warm-up strategy implementations. Which will be possible to implement in different ways.
At the moment, I suggest implementing one "Load all" strategy, which will be effective if persistent storage is less than RAM. 28.07.2020, 19:46, "Denis Mekhanikov" <dmekhani...@gmail.com>: > Kirill, > > That will be a great feature! Other popular databases already have it (e.g. > Postgres: https://www.postgresql.org/docs/11/pgprewarm.html), so it's good > that we're also going to have it in Ignite. > > What implementation of CacheWarmup interface do you have in mind? Will > there be some preconfigured implementation, and will users be able to > implement it themselves? > > Do you think it should be cache-based? I would say that a DataRegion-based > warm-up would come more naturally. Page IDs that are loaded into the data > region can be dumped periodically to disk and recovered on restarts. This > is more or less how it works in Postgres. > I'm afraid that if we make it cache-based, the implementation won't be that > obvious. We already have an API for warmup that appeared to be pretty much > impossible to apply in a useful way: > https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#preloadPartition-int- > Let's make sure that our new tool for warming up is actually useful. > > Denis > > вт, 28 июл. 2020 г. в 09:17, Zhenya Stanilovsky <arzamas...@mail.ru.invalid >> : > >> Looks like we need additional func for static caches, for >> example: warmup(List<CacheConfiguration> cconf) it would be helpful for >> spring too. >> >> > >> >------- Forwarded message ------- >> >From: "Вячеслав Коптилин" < slava.kopti...@gmail.com > >> >To: dev@ignite.apache.org >> >Cc: >> >Subject: Re: [DISCUSSION] Cache warmup >> >Date: Mon, 27 Jul 2020 16:47:48 +0300 >> > >> >Hello Kirill, >> > >> >Thanks a lot for driving this activity. If I am not mistaken, this >> >discussion relates to IEP-40. >> > >> >> I suggest adding a warmup phase after recovery here [1] after [2], >> before >> >discovery. >> >This means that the user's thread, which starts Ignite via >> >Ignition.start(), will wait for ana additional step - cache warm-up. >> >I think this fact has to be clearly mentioned in our documentation (at >> >Javadocat least) because this step can be time-consuming. >> > >> >> I suggest adding a new interface: >> >I would change it a bit. First of all, it would be nice to place this >> >interface to a public package and get rid of using GridCacheContext, >> >which is an internal class and it should not leak to the public API in any >> >case. >> >Perhaps, this parameter is not needed at all or we should add some public >> >abstraction instead of internal class. >> > >> >package org.apache.ignite.configuration; >> > >> >import org.apache.ignite.IgniteCheckedException; >> >import org.apache.ignite.lang.IgniteFuture; >> > >> >public interface CacheWarmupper { >> > /** >> > * Warmup cache. >> > * >> > * @param cachename Cache name. >> > * @return Future cache warmup. >> > * @throws IgniteCheckedException If failed. >> > */ >> > IgniteFuture<?> warmup(String cachename) throws >> >IgniteCheckedException; >> >} >> > >> >Thanks, >> >S. >> > >> >пн, 27 июл. 2020 г. в 15:03, ткаленко кирилл < tkalkir...@yandex.ru >: >> > >> >> Now, after restarting node, we have only cold caches, which at first >> >> requests to them will gradually load data from disks, which can slow >> down >> >> first calls to them. >> >> If node has more RAM than data on disk, then they can be loaded at start >> >> "warmup", thereby solving the issue of slowdowns during first calls to >> >> caches. >> >> >> >> I suggest adding a warmup phase after recovery here [1] after [2], >> before >> >> descovery. >> >> >> >> I suggest adding a new interface: >> >> >> >> package org.apache.ignite.internal.processors.cache; >> >> >> >> import org.apache.ignite.IgniteCheckedException; >> >> import org.apache.ignite.internal.IgniteInternalFuture; >> >> import org.jetbrains.annotations.Nullable; >> >> >> >> /** >> >> * Interface for warming up cache. >> >> */ >> >> public interface CacheWarmup { >> >> /** >> >> * Warmup cache. >> >> * >> >> * @param cacheCtx Cache context. >> >> * @return Future cache warmup. >> >> * @throws IgniteCheckedException if failed. >> >> */ >> >> @Nullable IgniteInternalFuture<?> process(GridCacheContext cacheCtx) >> >> throws IgniteCheckedException; >> >> } >> >> >> >> Which will allow to warm up caches in parallel and asynchronously. >> Warmup >> >> phase will end after all IgniteInternalFuture for all caches isDone. >> >> >> >> Also adding the ability to customize via methods: >> >> >> org.apache.ignite.configuration.IgniteConfiguration#setDefaultCacheWarmup >> >> org.apache.ignite.configuration.CacheConfiguration#setCacheWarmup >> >> >> >> Which will allow for each cache to set implementation of cache warming >> >> up, >> >> both for a specific cache, and for all if necessary. >> >> >> >> I suggest adding an implementation of SequentialWarmup that will use >> [3]. >> >> >> >> Questions, suggestions, comments? >> >> >> >> [1] - >> >> >> >> org.apache.ignite.internal.processors.cache.GridCacheProcessor.CacheRecoveryLifecycle#afterLogicalUpdatesApplied >> >> [2] - >> >> >> >> org.apache.ignite.internal.processors.cache.GridCacheProcessor.CacheRecoveryLifecycle#restorePartitionStates >> >> [3] - >> >> >> >> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManager.CacheDataStore#preload
