Alexey, Let me understand this part because it’s still not crystal clear to me:
> Now as per the > eviction mechanism, it is both per-page and per-entry: first, we choose a > page which fits most for eviction, however, we cannot simply erase the page > because quite a lot of data structures are referring to that page. To deal > with it, we read keys that the chosen page contains and then clear > individual cache entries, which in turn will clear up all necessary links. However will all the keys-values from a chosen page be removed eventually during the eviction phase? If it’s true then it’s still a page based eviction - we select 5 random oldest pages and purge all the key-values from them. — Denis > On Feb 4, 2018, at 12:05 PM, Alexey Goncharuk <alexey.goncha...@gmail.com> > wrote: > > Guys, > > To clarify the questions, I would like to reiterate over the mechanics of > evictions and then suggest a renaming that should resolve such things in > the future. > > First of all, Lucas is right - eviction policy only makes sense when native > persistence is disabled because what it does is actually freeing up memory > when a user hits the memory limit. The only way to do this is to destroy > inserted data because there is no other way to free memory. Now as per the > eviction mechanism, it is both per-page and per-entry: first, we choose a > page which fits most for eviction, however, we cannot simply erase the page > because quite a lot of data structures are referring to that page. To deal > with it, we read keys that the chosen page contains and then clear > individual cache entries, which in turn will clear up all necessary links. > If there are no concurrent updates, the page becomes empty and will be > reused for other user data. This is exactly what is explained on the wiki > page (at least in my reading of the page). > > Second, at this point, I would rename page management mechanism with > enabled persistence from 'eviction' to 'page replacement' or 'page > swapping', because it does not destroy any data, but rather replaces a > chosen buffer in memory from one page to another. The content of neither > pages does not change during page replacement. This mechanism is unlikely > to be selected by a user because the effectiveness of page replacements > heavily depends on internal data structures and may change from version to > version, and may even be adaptive depending on the load pattern. > > Hope this resolves the confusion. > > --AG > > 2018-02-03 1:03 GMT+03:00 Denis Magda <dma...@apache.org>: > >> Dmitriy, >> >> I’m surprised to hear that Random-LRU works at the entry level when Ignite >> persistence is off. Frankly, this section confuses a lot: >> https://cwiki.apache.org/confluence/display/IGNITE/ >> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory- >> underthehood-Entryeviction <https://cwiki.apache.org/ >> confluence/display/IGNITE/Ignite+Durable+Memory+-+under+ >> the+hood#IgniteDurableMemory-underthehood-Entryeviction> >> >> While it was assumed that the entry-based eviction works only for on-heap >> caches: >> https://apacheignite.readme.io/docs/evictions < >> https://apacheignite.readme.io/docs/evictions> >> >> Alex G., please chime in and clarify the misunderstanding. >> >> — >> Denis >> >>> On Feb 2, 2018, at 4:01 AM, Dmitry Pavlov <dpavlov....@gmail.com> wrote: >>> >>> Hi Val, >>> >>> I think it is quite accurate because eviction in case PDS enabled or not >>> has quite different purposes. >>> >>> 1. Let us consider PDS enabled and page eviction occurs. First of all it >> is >>> page based eviction, but not entry-based. Actually data is not removed, >> but >>> only written to disk. We can address this page later by ID. >>> PDS eviction is primarily the replacement of pages from memory to page on >>> disk. This eviction policy should ensure the maximum performance of the >>> solution in the future. There is no data removal from grid in this case. >>> And Ignite does not allow users to configure the policy. If benchmarks >> show >>> that a change in policy results in increased performance, then we can >>> switch policy. >>> >>> 2. Entry-based eviction (if there is no persistence is available) is >>> slithly different. User will need to reload data into grid in case entry >> is >>> evicted but required in cache. In that case Ignite provides selection of >>> policies. >>> >>> Sincerely, >>> Dmirtiy Pavlov >>> >>> >>> >>> чт, 1 февр. 2018 г. в 22:24, Valentin Kulichenko < >>> valentin.kuliche...@gmail.com>: >>> >>>> Guys, >>>> >>>> Thanks for responses. But I still fail to understand how it affects a >> user. >>>> >>>> Are you saying that with persistence enabled Random-LRU is always used >>>> regardless of what is specified in pageEvictionMode, while in in-memory >>>> only scenario we can also utilize Random-2-LRU for eviction? Is this >>>> accurate? Are there any other differences? >>>> >>>> -Val >>>> >>>> On Thu, Feb 1, 2018 at 5:01 AM, Dmitry Pavlov <dpavlov....@gmail.com> >>>> wrote: >>>> >>>>> Hi Vyacheslav, >>>>> >>>>> Yes, this is right, but for now PDS page based eviction uses >> Random-LRU, >>>>> not Random-2-LRU. Something may be changed in future Ignite releases, >> but >>>>> now Random-LRU is used. I am going to research if there is some room >> for >>>>> improvements in this aspect (e.g. IGNITE-7507 recenly found). >>>>> >>>>> If clean page is evicted, nothing really happends. If dirty page is >>>> evicted >>>>> from region during checkpointing process run, page is stored in file >>>> before >>>>> eviction >>>>> >>>>> Some info about this can be found in >>>>> https://cwiki.apache.org/confluence/display/IGNITE/ >>>>> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory- >>>>> underthehood-Pagebasedeviction >>>>> >>>>> Sincerely, >>>>> Dmitriy Pavlov >>>>> >>>>> чт, 1 февр. 2018 г. в 9:53, Vyacheslav Daradur <daradu...@gmail.com>: >>>>> >>>>>> Hi Valentin, >>>>>> >>>>>> As far as I know, when persistence is enabled and Ignite can't >>>>>> allocate new page-memory in the memory segment, then Ignite >>>>>> automatically evict some data from RAM to PDS using Random-2-LRU >>>>>> eviction algorithm. And there are no mechanisms to evict entries out >>>>>> when PDS is enabled. >>>>>> >>>>>> I'll be glad if I am not right and someone will correct me because, as >>>>>> Ignite user, I can't use PDS only as RAM disk replication for the use >>>>>> case in my project. >>>>>> >>>>>> >>>>>> On Thu, Feb 1, 2018 at 3:53 AM, Valentin Kulichenko >>>>>> <valentin.kuliche...@gmail.com> wrote: >>>>>>> Folks, >>>>>>> >>>>>>> On "Eviction Policies" documentation page [1] we have the following >>>>>> callout: >>>>>>> >>>>>>>> Configured eviction policy has no effect if Ignite persistence is >>>>>> enabled >>>>>>>> Note that if Ignite Persistence is enabled, then the page-based >>>>>> evictions >>>>>>> have no effect because the oldest pages will be purged from memory >>>>>>> automatically. >>>>>>> >>>>>>> This really confuses me. Why is there a difference in how data is >>>>> evicted >>>>>>> from memory depending on having disk enabled or not? And how does >>>>>> eviction >>>>>>> work with persistence enabled then? >>>>>>> >>>>>>> Does anyone can clarify? >>>>>>> >>>>>>> [1] https://apacheignite.readme.io/docs/evictions >>>>>>> >>>>>>> -Val >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Best Regards, Vyacheslav D. >>>>>> >>>>> >>>> >> >>