Re: Limiting the size of Persistent Store and clearing data on restart
On Sat, Sep 2, 2017 at 5:11 AM, userxwrote: > Hi all, > > Regarding question 2, let me put it this way. > > Like ignite has eviction policy for RAM described at > https://apacheignite.readme.io/docs/evictions > Is there an eviction policy for a persistent store ? Say for a use case > where in I cannot allocate a dedicated 1 or 2 tb space (like we do for > relational dbms), I would want to define an upper limit of say 300 GB and > if > $IGNITE_HOME/PersistentStore goes beyond the same same, evict the lru > cache. > > Evictions for RAM make sense because the data is still on disk and is not lost. Evictions from disk do not make sense for me, nor am I aware of other databases that have this feature. I believe it should be handled by users based on the application requirements. I think the Ignite community may agree to consider this feature as a possible enhancement. Can you provide an example of a similar feature you may have seen in another persistent store or a database? > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >
Re: Limiting the size of Persistent Store and clearing data on restart
Hi all, Regarding question 2, let me put it this way. Like ignite has eviction policy for RAM described at https://apacheignite.readme.io/docs/evictions Is there an eviction policy for a persistent store ? Say for a use case where in I cannot allocate a dedicated 1 or 2 tb space (like we do for relational dbms), I would want to define an upper limit of say 300 GB and if $IGNITE_HOME/PersistentStore goes beyond the same same, evict the lru cache. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Limiting the size of Persistent Store and clearing data on restart
Sorry, I didn't know that. I found a bunch of matches by "swapEnabled" keyword on master branch in the project, mostly in benchmarks configurations. These properties should probably be removed not to confuse readers. пт, 1 сент. 2017 г. в 19:23, Dmitriy Setrakyan: > On Fri, Sep 1, 2017 at 1:41 PM, Denis Mekhanikov > wrote: > >> Regarding your second question: looks like you don't actually need >> persistence. Its purpose is the opposite: to save cache data between >> restarts. >> If you use persistence to store more data than RAM available, then you >> can enable swap space: >> https://apacheignite.readme.io/v1.9/docs/off-heap-memory#section-swap-space >> > > Denis, starting with 2.1, Ignite does not have the swap space anymore, > since it never worked well and kept hurting the performance. We recommend > now that users do use persistence whenever the data size is bigger than the > available memory size. >
Re: Limiting the size of Persistent Store and clearing data on restart
On Tue, Aug 29, 2017 at 7:00 PM, userxwrote: > Hi All, > > 1) Like the conventional configuration of > https://apacheignite.readme.io/docs/memory-configuration# > section-memory-policies > where in we could limit the size of the memory, how do we define a > configuration which can limit the size of Persistent Store so that say out > of a 500GB HDisk, I don't want this Ignite related data to go beyond 300GB > ? > You would have to control your data on disk by yourself. Evicting from disk is very sensitive and use case specific, so it would be almost impossible to automate it. > > 2) Also, is there a configuration such that when Ignite data grid servers > are restarted (say some code change either on client or server side), > whatever was persisted before gets wiped out completely on all the > participating nodes and only fresh persisted data is there ? > How about calling destroyCache(...) on startup?
Re: Limiting the size of Persistent Store and clearing data on restart
On Fri, Sep 1, 2017 at 1:41 PM, Denis Mekhanikovwrote: > Regarding your second question: looks like you don't actually need > persistence. Its purpose is the opposite: to save cache data between > restarts. > If you use persistence to store more data than RAM available, then you can > enable swap space: https://apacheignite.readme.io/v1.9/docs/off-heap-mem > ory#section-swap-space > Denis, starting with 2.1, Ignite does not have the swap space anymore, since it never worked well and kept hurting the performance. We recommend now that users do use persistence whenever the data size is bigger than the available memory size.
Re: Limiting the size of Persistent Store and clearing data on restart
Regarding your second question: looks like you don't actually need persistence. Its purpose is the opposite: to save cache data between restarts. If you use persistence to store more data than RAM available, then you can enable swap space: https://apacheignite.readme.io/v1.9/docs/off-heap-memory#section-swap-space вт, 29 авг. 2017 г. в 20:06, ezhuravlev: > Hi, > > 1) There is no possibility to limit Persistent Store size right now, but > you > always can use separate disk volume with size 300GB for your case to use it > for Persistent Store. > > 2) Well it's not usual to use case, but after stopping node you can clear > data from work folder or just invoke removeAll on cache > > Evgenii > > > > -- > View this message in context: > http://apache-ignite-users.70518.x6.nabble.com/Limiting-the-size-of-Persistent-Store-and-clearing-data-on-restart-tp16507p16515.html > Sent from the Apache Ignite Users mailing list archive at Nabble.com. >
Re: Limiting the size of Persistent Store and clearing data on restart
Hi, 1) There is no possibility to limit Persistent Store size right now, but you always can use separate disk volume with size 300GB for your case to use it for Persistent Store. 2) Well it's not usual to use case, but after stopping node you can clear data from work folder or just invoke removeAll on cache Evgenii -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Limiting-the-size-of-Persistent-Store-and-clearing-data-on-restart-tp16507p16515.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Limiting the size of Persistent Store and clearing data on restart
Hi All, 1) Like the conventional configuration of https://apacheignite.readme.io/docs/memory-configuration#section-memory-policies where in we could limit the size of the memory, how do we define a configuration which can limit the size of Persistent Store so that say out of a 500GB HDisk, I don't want this Ignite related data to go beyond 300GB ? 2) Also, is there a configuration such that when Ignite data grid servers are restarted (say some code change either on client or server side), whatever was persisted before gets wiped out completely on all the participating nodes and only fresh persisted data is there ? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Limiting-the-size-of-Persistent-Store-and-clearing-data-on-restart-tp16507.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.