Ignite pages its data. So if you access a record, it will transparency be copied from disk to memory. It doesn’t proactively go to disk and pull in records you might need.
> On 19 Jan 2021, at 16:18, Ryan Trollip <ryanonthebe...@gmail.com> wrote: > > Stephen > > It seems the problem is how ignite deals with this was not clear to me. To > make sure I understand this correctly: > Ignite with native persistence on, automatically overflows pages to disk > using the least recently used policy - LRU. (which is great) > Pages could be a partial row in a table. But that's ok because a SQL query > that includes that will pull from RAM and disk automatically under the > covers. i.e. it's all auto-magically done for us. > > What is still not clear is, as branches are deleted or we scale-up servers > and more memory is then made available, will it rotate back into memory from > disk what was rotated out? assuming reverse LRU order? > > Thanks!! > Ryan > > > On Tue, Jan 19, 2021 at 7:28 AM Stephen Darlington > <stephen.darling...@gridgain.com <mailto:stephen.darling...@gridgain.com>> > wrote: > As long as new branches — to use your analogy — are in memory, why does it > matter that a few others are too? The least recently used branches will > automatically (LRU) be purged from memory if space is needed for new branches. > > In fact, if you’re worried about available memory, a time based eviction > policy won’t work. What if you expect to have 100 branches and size your > cluster appropriately. Suddenly, 1000 branches are created. Boom! > > With a space-based eviction policy — as is the default with Ignite native > persistence — that works just fine. > > You could create a cache with an eviction policy. When the records are > deleted after a week, you can have a process listening to delete events and > copy the record to a different cache in a small data region. > > So what you’re asking for is possible, it’s just more complicated and less > effective than the alternative. > >> On 19 Jan 2021, at 14:04, Ryan Trollip <ryanonthebe...@gmail.com >> <mailto:ryanonthebe...@gmail.com>> wrote: >> >> Stephen >> >> Let's use an analogy of projects in source control. Let's say we have a very >> active community of developers, they are creating 100 new branches a day. >> Each branch has a few thousand objects and associated properties etc. but >> these developers don't clean up by deleting branches. >> We want new branches to be cached in memory and available high performance >> read and write, but older branches to go to disk to save on memory hardware >> needs, since many are abandoned. >> The policy could read something like this: Branches that have not been >> accessed in 1 week, move to disk. On branch access, if on disk, move back to >> RAM. >> >> Thanks >> Ryan >> >> On Tue, Jan 19, 2021 at 2:33 AM Stephen Darlington >> <stephen.darling...@gridgain.com <mailto:stephen.darling...@gridgain.com>> >> wrote: >> I guess I’m still not clear why you need to explicitly remove them from >> memory. >> >> By virtue of using native persistence, they’re already on disk. If you load >> new data, the old entries will eventually be flushed from memory (but remain >> on disk). What do you gain by removing entries from memory at a specific >> time? >> >> Regards, >> Stephen >> >> > On 19 Jan 2021, at 06:02, Naveen <naveen.band...@gmail.com >> > <mailto:naveen.band...@gmail.com>> wrote: >> > >> > Hi Stephen >> > >> > on the same mail chain, we also data like OTP (one time passwds) which are >> > no relevant after a while, but we dont want to expire or delete them, just >> > get them flushed to disk, like wise we do have other requirements where >> > data >> > is very relevant only for a certain duration, later on its not important. >> > Thats the whole idea of exploring eviction policies >> > >> > Naveen >> > >> > >> > >> > -- >> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >> > <http://apache-ignite-users.70518.x6.nabble.com/> >> >> > >
