[jira] [Assigned] (IGNITE-8017) Disable WAL during initial preloading
[ https://issues.apache.org/jira/browse/IGNITE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Lantukh reassigned IGNITE-8017: Assignee: Ilya Lantukh > Disable WAL during initial preloading > - > > Key: IGNITE-8017 > URL: https://issues.apache.org/jira/browse/IGNITE-8017 > Project: Ignite > Issue Type: Improvement >Reporter: Ilya Lantukh >Assignee: Ilya Lantukh >Priority: Major > > While handling SupplyMessage, node handles each supplied data entry > separately, which causes a WAL record for each entry to be written. It > significantly limits preloading speed, especially with WALMode == FSYNC - it > will perform fsync for every entry! > We can improve rebalancing speed and reduce pressure on disk by disabling WAL > until all data is loaded. The disadvantage of this approach is that data > might get corrupted if node crashes - but node that crashed during preloading > has to clear all it's data anyway. However, it is important to distinguish > situations when new node joined cluster or added to baseline topology (and > doesn't hold any data) and when additional partitions got assigned to node > after baseline topology changed (in this case node has to keep all data in > consistent state). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (IGNITE-8017) Disable WAL during initial preloading
[ https://issues.apache.org/jira/browse/IGNITE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ilya Lantukh reassigned IGNITE-8017: Assignee: (was: Ilya Lantukh) > Disable WAL during initial preloading > - > > Key: IGNITE-8017 > URL: https://issues.apache.org/jira/browse/IGNITE-8017 > Project: Ignite > Issue Type: Improvement >Reporter: Ilya Lantukh >Priority: Major > > While handling SupplyMessage, node handles each supplied data entry > separately, which causes a WAL record for each entry to be written. It > significantly limits preloading speed, especially with WALMode == FSYNC - it > will perform fsync for every entry! > We can improve rebalancing speed and reduce pressure on disk by disabling WAL > until all data is loaded. The disadvantage of this approach is that data > might get corrupted if node crashes - but node that crashed during preloading > has to clear all it's data anyway. However, it is important to distinguish > situations when new node joined cluster or added to baseline topology (and > doesn't hold any data) and when additional partitions got assigned to node > after baseline topology changed (in this case node has to keep all data in > consistent state). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
