Guys,
+1 for renaming "eviction" for PDS mode.
I'd choose "page replacement" over "page swapping" for two reasons:
1) We already have "replace" metrics in DataRegionMetrics. The less we
change existing namings, the better.
2) "Page swapping" term is common in operating systems, and that's why
we'll confuse our "page swapping" (which is essential of Ignite native
persistence) with actual OS page swapping (which usually signals about
ineffective hardware resources usage and is better to be avoided) in
many discussions later.
Best Regards,
Ivan Rakov
On 04.02.2018 23:42, Dmitry Pavlov wrote:
Hi Alex,
Thank you for explanation. Do you mind if I copy this explanation to wiki
page?
Strongly +1 for new term for page replacement mechanism for PDS mode. I
like 'page swapping' because it has analogue in operating systems.
New term will also identify feature by class name PageEviction... and
PageSwap... will point to class purpose.
Sincerely,
Dmitriy Pavlov
вс, 4 февр. 2018 г. в 23:05, Alexey Goncharuk <alexey.goncha...@gmail.com>:
Guys,
To clarify the questions, I would like to reiterate over the mechanics of
evictions and then suggest a renaming that should resolve such things in
the future.
First of all, Lucas is right - eviction policy only makes sense when native
persistence is disabled because what it does is actually freeing up memory
when a user hits the memory limit. The only way to do this is to destroy
inserted data because there is no other way to free memory. Now as per the
eviction mechanism, it is both per-page and per-entry: first, we choose a
page which fits most for eviction, however, we cannot simply erase the page
because quite a lot of data structures are referring to that page. To deal
with it, we read keys that the chosen page contains and then clear
individual cache entries, which in turn will clear up all necessary links.
If there are no concurrent updates, the page becomes empty and will be
reused for other user data. This is exactly what is explained on the wiki
page (at least in my reading of the page).
Second, at this point, I would rename page management mechanism with
enabled persistence from 'eviction' to 'page replacement' or 'page
swapping', because it does not destroy any data, but rather replaces a
chosen buffer in memory from one page to another. The content of neither
pages does not change during page replacement. This mechanism is unlikely
to be selected by a user because the effectiveness of page replacements
heavily depends on internal data structures and may change from version to
version, and may even be adaptive depending on the load pattern.
Hope this resolves the confusion.
--AG
2018-02-03 1:03 GMT+03:00 Denis Magda <dma...@apache.org>:
Dmitriy,
I’m surprised to hear that Random-LRU works at the entry level when
Ignite
persistence is off. Frankly, this section confuses a lot:
https://cwiki.apache.org/confluence/display/IGNITE/
Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-
underthehood-Entryeviction <https://cwiki.apache.org/
confluence/display/IGNITE/Ignite+Durable+Memory+-+under+
the+hood#IgniteDurableMemory-underthehood-Entryeviction>
While it was assumed that the entry-based eviction works only for on-heap
caches:
https://apacheignite.readme.io/docs/evictions <
https://apacheignite.readme.io/docs/evictions>
Alex G., please chime in and clarify the misunderstanding.
—
Denis
On Feb 2, 2018, at 4:01 AM, Dmitry Pavlov <dpavlov....@gmail.com>
wrote:
Hi Val,
I think it is quite accurate because eviction in case PDS enabled or
not
has quite different purposes.
1. Let us consider PDS enabled and page eviction occurs. First of all
it
is
page based eviction, but not entry-based. Actually data is not removed,
but
only written to disk. We can address this page later by ID.
PDS eviction is primarily the replacement of pages from memory to page
on
disk. This eviction policy should ensure the maximum performance of the
solution in the future. There is no data removal from grid in this
case.
And Ignite does not allow users to configure the policy. If benchmarks
show
that a change in policy results in increased performance, then we can
switch policy.
2. Entry-based eviction (if there is no persistence is available) is
slithly different. User will need to reload data into grid in case
entry
is
evicted but required in cache. In that case Ignite provides selection
of
policies.
Sincerely,
Dmirtiy Pavlov
чт, 1 февр. 2018 г. в 22:24, Valentin Kulichenko <
valentin.kuliche...@gmail.com>:
Guys,
Thanks for responses. But I still fail to understand how it affects a
user.
Are you saying that with persistence enabled Random-LRU is always used
regardless of what is specified in pageEvictionMode, while in
in-memory
only scenario we can also utilize Random-2-LRU for eviction? Is this
accurate? Are there any other differences?
-Val
On Thu, Feb 1, 2018 at 5:01 AM, Dmitry Pavlov <dpavlov....@gmail.com>
wrote:
Hi Vyacheslav,
Yes, this is right, but for now PDS page based eviction uses
Random-LRU,
not Random-2-LRU. Something may be changed in future Ignite releases,
but
now Random-LRU is used. I am going to research if there is some room
for
improvements in this aspect (e.g. IGNITE-7507 recenly found).
If clean page is evicted, nothing really happends. If dirty page is
evicted
from region during checkpointing process run, page is stored in file
before
eviction
Some info about this can be found in
https://cwiki.apache.org/confluence/display/IGNITE/
Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-
underthehood-Pagebasedeviction
Sincerely,
Dmitriy Pavlov
чт, 1 февр. 2018 г. в 9:53, Vyacheslav Daradur <daradu...@gmail.com
:
Hi Valentin,
As far as I know, when persistence is enabled and Ignite can't
allocate new page-memory in the memory segment, then Ignite
automatically evict some data from RAM to PDS using Random-2-LRU
eviction algorithm. And there are no mechanisms to evict entries out
when PDS is enabled.
I'll be glad if I am not right and someone will correct me because,
as
Ignite user, I can't use PDS only as RAM disk replication for the
use
case in my project.
On Thu, Feb 1, 2018 at 3:53 AM, Valentin Kulichenko
<valentin.kuliche...@gmail.com> wrote:
Folks,
On "Eviction Policies" documentation page [1] we have the following
callout:
Configured eviction policy has no effect if Ignite persistence is
enabled
Note that if Ignite Persistence is enabled, then the page-based
evictions
have no effect because the oldest pages will be purged from memory
automatically.
This really confuses me. Why is there a difference in how data is
evicted
from memory depending on having disk enabled or not? And how does
eviction
work with persistence enabled then?
Does anyone can clarify?
[1] https://apacheignite.readme.io/docs/evictions
-Val
--
Best Regards, Vyacheslav D.
