[jira] [Commented] (IGNITE-12069) Create cache shared preloader
[
https://issues.apache.org/jira/browse/IGNITE-12069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907230#comment-16907230
]
Ivan Pavlukhin commented on IGNITE-12069:
-
[~xtern], thank you for a clarification.
> Create cache shared preloader
> -
>
> Key: IGNITE-12069
> URL: https://issues.apache.org/jira/browse/IGNITE-12069
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Maxim Muzafarov
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: iep-28
>
> {{CacheSharedPreloader}} must do the following:
> # build the map of partitions and corresponding supplier nodes from which
> partitions will be loaded [1];
> # switch cache data storage to {{no-op}} and back to original (HWM must be
> fixed here for the needs of historical rebalance) under the checkpoint and
> keep the partition update counter for each partition [1];
> # run async the eviction indexes for the list of collected partitions (API
> must be provided by IGNITE-11075) [2];
> # send a request message to each node one by one with the list of partitions
> to load [2];
> # wait for files received (listening for the transmission handler) [2];
> # run rebuild indexes async over the receiving partitions (API must be
> provided by IGNITE-11075) [2];
> # run historical rebalance from LWM to HWM collected above (LWM can be read
> from the received file meta page) [1];
> The points marked with the label {{[1]}} must be done prior to {{[2]}}.
>
> NOTE. The following things need to be checked:
> # Rebalancing of MVCC cache groups;
> # How LWM and HWM will be set for the historical rebalance;
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-12069) Create cache shared preloader
[
https://issues.apache.org/jira/browse/IGNITE-12069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907220#comment-16907220
]
Pavel Pereslegin commented on IGNITE-12069:
---
Hello, [~Pavlukhin].
Firstly, the naming is not yet final and may be inaccurate.
"Shared" is because, currently, one instance of preloader manages the
rebalancing of a single cache group.
And due to some time limitations (partition snapshot creation on supplier
should be done on checkpoint), the "p2p" preloader must manage the rebalancing
process of all participating cache groups.
_> What entities do share it?_
All cache groups "share" single preloader.
> Create cache shared preloader
> -
>
> Key: IGNITE-12069
> URL: https://issues.apache.org/jira/browse/IGNITE-12069
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Maxim Muzafarov
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: iep-28
>
> {{CacheSharedPreloader}} must do the following:
> # build the map of partitions and corresponding supplier nodes from which
> partitions will be loaded [1];
> # switch cache data storage to {{no-op}} and back to original (HWM must be
> fixed here for the needs of historical rebalance) under the checkpoint and
> keep the partition update counter for each partition [1];
> # run async the eviction indexes for the list of collected partitions (API
> must be provided by IGNITE-11075) [2];
> # send a request message to each node one by one with the list of partitions
> to load [2];
> # wait for files received (listening for the transmission handler) [2];
> # run rebuild indexes async over the receiving partitions (API must be
> provided by IGNITE-11075) [2];
> # run historical rebalance from LWM to HWM collected above (LWM can be read
> from the received file meta page) [1];
> The points marked with the label {{[1]}} must be done prior to {{[2]}}.
>
> NOTE. The following things need to be checked:
> # Rebalancing of MVCC cache groups;
> # How LWM and HWM will be set for the historical rebalance;
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-12069) Create cache shared preloader
[
https://issues.apache.org/jira/browse/IGNITE-12069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907196#comment-16907196
]
Ivan Pavlukhin commented on IGNITE-12069:
-
[~Mmuzaf], [~xtern], just for my understanding, could you please elaborate why
it is _shared_ preloader? What entities do share it?
> Create cache shared preloader
> -
>
> Key: IGNITE-12069
> URL: https://issues.apache.org/jira/browse/IGNITE-12069
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Maxim Muzafarov
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: iep-28
>
> {{CacheSharedPreloader}} must do the following:
> # build the map of partitions and corresponding supplier nodes from which
> partitions will be loaded [1];
> # witching cache data storage to {{no-op}} and back to original (HWM must be
> fixed here for the needs of historical rebalance) under the checkpoint and
> keep the partition update counter for each partition [1];
> # run async the eviction indexes for the list of collected partitions (API
> must be provided by IGNITE-11075) [2];
> # send a request message to each node one by one with the list of partitions
> to load [2];
> # listening for the transmission handler to receive files [2];
> # run rebuild indexes async over the receiving partitions (API must be
> provided by IGNITE-11075) [2];
> # run historical rebalance from LWM to HWM collected above (LWM can be read
> from the received file meta page) [1];
> The points marked with the label {{[1]}} must be done prior to {{[2]}}.
>
> NOTE. Check the following things:
> # Rebalancing of MVCC cache groups;
> # How LWM and HWM will be set for the historical rebalance;
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (IGNITE-12069) Create cache shared preloader
[
https://issues.apache.org/jira/browse/IGNITE-12069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907125#comment-16907125
]
Maxim Muzafarov commented on IGNITE-12069:
--
[~xtern],
As discussed with you privately, I've created and assigned to you this issue
with the previously discussed problem.
> Create cache shared preloader
> -
>
> Key: IGNITE-12069
> URL: https://issues.apache.org/jira/browse/IGNITE-12069
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Maxim Muzafarov
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: iep-28
>
> {{CacheSharedPreloader}} must do the following:
> # build the map of partitions and corresponding supplier nodes from which
> partitions will be loaded [1];
> # witching cache data storage to {{no-op}} and back to original (HWM must be
> fixed here for the needs of historical rebalance) under the checkpoint and
> keep the partition update counter for each partition [1];
> # run async the eviction indexes for the list of collected partitions (API
> must be provided by IGNITE-11075) [2];
> # send a request message to each node one by one with the list of partitions
> to load [2];
> # listening for the transmission handler to receive files [2];
> # run rebuild indexes async over the receiving partitions (API must be
> provided by IGNITE-11075) [2];
> # run historical rebalance from LWM to HWM collected above (LWM can be read
> from the received file meta page) [1];
> The points marked with the label {{[1]}} must be done prior to {{[2]}}.
>
> NOTE. Check the following things:
> # Rebalancing of MVCC cache groups;
> # How LWM and HWM will be set for the historical rebalance;
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
