Alexey,

Let me understand this part because it’s still not crystal clear to me:

> Now as per the
> eviction mechanism, it is both per-page and per-entry: first, we choose a
> page which fits most for eviction, however, we cannot simply erase the page
> because quite a lot of data structures are referring to that page. To deal
> with it, we read keys that the chosen page contains and then clear
> individual cache entries, which in turn will clear up all necessary links.


However will all the keys-values from a chosen page be removed eventually 
during the eviction phase? If it’s true then it’s still a page based eviction - 
we select 5 random oldest pages and purge all the key-values from them.

—
Denis 

> On Feb 4, 2018, at 12:05 PM, Alexey Goncharuk <alexey.goncha...@gmail.com> 
> wrote:
> 
> Guys,
> 
> To clarify the questions, I would like to reiterate over the mechanics of
> evictions and then suggest a renaming that should resolve such things in
> the future.
> 
> First of all, Lucas is right - eviction policy only makes sense when native
> persistence is disabled because what it does is actually freeing up memory
> when a user hits the memory limit. The only way to do this is to destroy
> inserted data because there is no other way to free memory. Now as per the
> eviction mechanism, it is both per-page and per-entry: first, we choose a
> page which fits most for eviction, however, we cannot simply erase the page
> because quite a lot of data structures are referring to that page. To deal
> with it, we read keys that the chosen page contains and then clear
> individual cache entries, which in turn will clear up all necessary links.
> If there are no concurrent updates, the page becomes empty and will be
> reused for other user data. This is exactly what is explained on the wiki
> page (at least in my reading of the page).
> 
> Second, at this point, I would rename page management mechanism with
> enabled persistence from 'eviction' to 'page replacement' or 'page
> swapping', because it does not destroy any data, but rather replaces a
> chosen buffer in memory from one page to another. The content of neither
> pages does not change during page replacement. This mechanism is unlikely
> to be selected by a user because the effectiveness of page replacements
> heavily depends on internal data structures and may change from version to
> version, and may even be adaptive depending on the load pattern.
> 
> Hope this resolves the confusion.
> 
> --AG
> 
> 2018-02-03 1:03 GMT+03:00 Denis Magda <dma...@apache.org>:
> 
>> Dmitriy,
>> 
>> I’m surprised to hear that Random-LRU works at the entry level when Ignite
>> persistence is off. Frankly, this section confuses a lot:
>> https://cwiki.apache.org/confluence/display/IGNITE/
>> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-
>> underthehood-Entryeviction <https://cwiki.apache.org/
>> confluence/display/IGNITE/Ignite+Durable+Memory+-+under+
>> the+hood#IgniteDurableMemory-underthehood-Entryeviction>
>> 
>> While it was assumed that the entry-based eviction works only for on-heap
>> caches:
>> https://apacheignite.readme.io/docs/evictions <
>> https://apacheignite.readme.io/docs/evictions>
>> 
>> Alex G., please chime in and clarify the misunderstanding.
>> 
>> —
>> Denis
>> 
>>> On Feb 2, 2018, at 4:01 AM, Dmitry Pavlov <dpavlov....@gmail.com> wrote:
>>> 
>>> Hi Val,
>>> 
>>> I think it is quite accurate because eviction in case PDS enabled or not
>>> has quite different purposes.
>>> 
>>> 1. Let us consider PDS enabled and page eviction occurs. First of all it
>> is
>>> page based eviction, but not entry-based. Actually data is not removed,
>> but
>>> only written to disk. We can address this page later by ID.
>>> PDS eviction is primarily the replacement of pages from memory to page on
>>> disk. This eviction policy should ensure the maximum performance of the
>>> solution in the future. There is no data removal from grid in this case.
>>> And Ignite does not allow users to configure the policy. If benchmarks
>> show
>>> that a change in policy results in increased performance, then we can
>>> switch policy.
>>> 
>>> 2. Entry-based eviction (if there is no persistence is available) is
>>> slithly different. User will need to reload data into grid in case entry
>> is
>>> evicted but required in cache. In that case Ignite provides selection of
>>> policies.
>>> 
>>> Sincerely,
>>> Dmirtiy Pavlov
>>> 
>>> 
>>> 
>>> чт, 1 февр. 2018 г. в 22:24, Valentin Kulichenko <
>>> valentin.kuliche...@gmail.com>:
>>> 
>>>> Guys,
>>>> 
>>>> Thanks for responses. But I still fail to understand how it affects a
>> user.
>>>> 
>>>> Are you saying that with persistence enabled Random-LRU is always used
>>>> regardless of what is specified in pageEvictionMode, while in in-memory
>>>> only scenario we can also utilize Random-2-LRU for eviction? Is this
>>>> accurate? Are there any other differences?
>>>> 
>>>> -Val
>>>> 
>>>> On Thu, Feb 1, 2018 at 5:01 AM, Dmitry Pavlov <dpavlov....@gmail.com>
>>>> wrote:
>>>> 
>>>>> Hi Vyacheslav,
>>>>> 
>>>>> Yes, this is right, but for now PDS page based eviction uses
>> Random-LRU,
>>>>> not Random-2-LRU. Something may be changed in future Ignite releases,
>> but
>>>>> now Random-LRU is used. I am going to research if there is some room
>> for
>>>>> improvements in this aspect (e.g. IGNITE-7507 recenly found).
>>>>> 
>>>>> If clean page is evicted, nothing really happends. If dirty page is
>>>> evicted
>>>>> from region during checkpointing process run, page is stored in file
>>>> before
>>>>> eviction
>>>>> 
>>>>> Some info about this can be found in
>>>>> https://cwiki.apache.org/confluence/display/IGNITE/
>>>>> Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-
>>>>> underthehood-Pagebasedeviction
>>>>> 
>>>>> Sincerely,
>>>>> Dmitriy Pavlov
>>>>> 
>>>>> чт, 1 февр. 2018 г. в 9:53, Vyacheslav Daradur <daradu...@gmail.com>:
>>>>> 
>>>>>> Hi Valentin,
>>>>>> 
>>>>>> As far as I know, when persistence is enabled and Ignite can't
>>>>>> allocate new page-memory in the memory segment, then Ignite
>>>>>> automatically evict some data from RAM to PDS using Random-2-LRU
>>>>>> eviction algorithm. And there are no mechanisms to evict entries out
>>>>>> when PDS is enabled.
>>>>>> 
>>>>>> I'll be glad if I am not right and someone will correct me because, as
>>>>>> Ignite user, I can't use PDS only as RAM disk replication for the use
>>>>>> case in my project.
>>>>>> 
>>>>>> 
>>>>>> On Thu, Feb 1, 2018 at 3:53 AM, Valentin Kulichenko
>>>>>> <valentin.kuliche...@gmail.com> wrote:
>>>>>>> Folks,
>>>>>>> 
>>>>>>> On "Eviction Policies" documentation page [1] we have the following
>>>>>> callout:
>>>>>>> 
>>>>>>>> Configured eviction policy has no effect if Ignite persistence is
>>>>>> enabled
>>>>>>>> Note that if Ignite Persistence is enabled, then the page-based
>>>>>> evictions
>>>>>>> have no effect because the oldest pages will be purged from memory
>>>>>>> automatically.
>>>>>>> 
>>>>>>> This really confuses me. Why is there a difference in how data is
>>>>> evicted
>>>>>>> from memory depending on having disk enabled or not? And how does
>>>>>> eviction
>>>>>>> work with persistence enabled then?
>>>>>>> 
>>>>>>> Does anyone can clarify?
>>>>>>> 
>>>>>>> [1] https://apacheignite.readme.io/docs/evictions
>>>>>>> 
>>>>>>> -Val
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Best Regards, Vyacheslav D.
>>>>>> 
>>>>> 
>>>> 
>> 
>> 

Reply via email to