[jira] [Updated] (IGNITE-11074) Implement catch-up temporary WAL
[ https://issues.apache.org/jira/browse/IGNITE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11074: - Description: This is an addition for the process of rebalancing caches via partition files for the place where historical rebalance is used by design (see the IEP-28 confluence page). While the demander node is in the partition file transmission state it must save all cache entries corresponding to the moving partition into a new temporary WAL storage. These entries will be applied later one by one on the received cache partition file. All asynchronous operations will be enrolled to the end of temporary WAL storage during storage reads until it becomes fully read. The file-based FIFO approach assumes to be used by this process. The new write-ahead-log manager for writing temporary records must support to: * Unlimited number of WAL-files to store temporary data records; * Iterating over stored data records during an asynchronous writer thread inserts new records; * WAL-per-partition approach needs to be used; * Write operations to temporary WAL storage must have higher priority over reading operations; was: While the demander node is in the partition file transmission state it must save all cache entries corresponding to the moving partition into a new temporary WAL storage. These entries will be applied later one by one on the received cache partition file. All asynchronous operations will be enrolled to the end of temporary WAL storage during storage reads until it becomes fully read. The file-based FIFO approach assumes to be used by this process. The new write-ahead-log manager for writing temporary records must support to: * Unlimited number of wal-files to store temporary data records; * Iterating over stored data records during an asynchronous writer thread inserts new records; * WAL-per-partiton approach needs to be used; * Write operations to temporary WAL storage must have higher priority over reading operations; > Implement catch-up temporary WAL > > > Key: IGNITE-11074 > URL: https://issues.apache.org/jira/browse/IGNITE-11074 > Project: Ignite > Issue Type: Improvement >Reporter: Maxim Muzafarov >Priority: Major > Labels: iep-28 > > This is an addition for the process of rebalancing caches via partition files > for the place where historical rebalance is used by design (see the IEP-28 > confluence page). > While the demander node is in the partition file transmission state it must > save all cache entries corresponding to the moving partition into a new > temporary WAL storage. These entries will be applied later one by one on the > received cache partition file. All asynchronous operations will be enrolled > to the end of temporary WAL storage during storage reads until it becomes > fully read. The file-based FIFO approach assumes to be used by this process. > The new write-ahead-log manager for writing temporary records must support to: > * Unlimited number of WAL-files to store temporary data records; > * Iterating over stored data records during an asynchronous writer thread > inserts new records; > * WAL-per-partition approach needs to be used; > * Write operations to temporary WAL storage must have higher priority over > reading operations; -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (IGNITE-11074) Implement catch-up temporary WAL
[ https://issues.apache.org/jira/browse/IGNITE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11074: - Issue Type: Improvement (was: Sub-task) Parent: (was: IGNITE-8020) > Implement catch-up temporary WAL > > > Key: IGNITE-11074 > URL: https://issues.apache.org/jira/browse/IGNITE-11074 > Project: Ignite > Issue Type: Improvement >Reporter: Maxim Muzafarov >Priority: Major > Labels: iep-28 > > While the demander node is in the partition file transmission state it must > save all cache entries corresponding to the moving partition into a new > temporary WAL storage. These entries will be applied later one by one on the > received cache partition file. All asynchronous operations will be enrolled > to the end of temporary WAL storage during storage reads until it becomes > fully read. The file-based FIFO approach assumes to be used by this process. > The new write-ahead-log manager for writing temporary records must support to: > * Unlimited number of wal-files to store temporary data records; > * Iterating over stored data records during an asynchronous writer thread > inserts new records; > * WAL-per-partiton approach needs to be used; > * Write operations to temporary WAL storage must have higher priority over > reading operations; -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (IGNITE-11074) Implement catch-up temporary WAL
[ https://issues.apache.org/jira/browse/IGNITE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11074: - Description: While the demander node is in the partition file transmission state it must save all cache entries corresponding to the moving partition into a new temporary WAL storage. These entries will be applied later one by one on the received cache partition file. All asynchronous operations will be enrolled to the end of temporary WAL storage during storage reads until it becomes fully read. The file-based FIFO approach assumes to be used by this process. The new write-ahead-log manager for writing temporary records must support to: * Unlimited number of wal-files to store temporary data records; * Iterating over stored data records during an asynchronous writer thread inserts new records; * WAL-per-partiton approach needs to be used; * Write operations to temporary WAL storage must have higher priority over reading operations; was: While the demander node is in the partition file transmission state it must save all cache entries corresponding to the moving partition into a new temporary WAL storage. These entries will be applied later one by one on the received cache partition file. All asynchronous operations will be enrolled to the end of temporary WAL storage during storage reads until it becomes fully read. The file-based FIFO approach assumes to be used by this process. The new write-ahead-log manager for writing temporary records must support to: Unlimited number of wal-files to store temporary data records; Iterating over stored data records during an asynchronous writer thread inserts new records; WAL-per-partiton approach needs to be used; Write operations to temporary WAL storage must have higher priority over reading operations; > Implement catch-up temporary WAL > > > Key: IGNITE-11074 > URL: https://issues.apache.org/jira/browse/IGNITE-11074 > Project: Ignite > Issue Type: Sub-task >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: iep-28 > > While the demander node is in the partition file transmission state it must > save all cache entries corresponding to the moving partition into a new > temporary WAL storage. These entries will be applied later one by one on the > received cache partition file. All asynchronous operations will be enrolled > to the end of temporary WAL storage during storage reads until it becomes > fully read. The file-based FIFO approach assumes to be used by this process. > The new write-ahead-log manager for writing temporary records must support to: > * Unlimited number of wal-files to store temporary data records; > * Iterating over stored data records during an asynchronous writer thread > inserts new records; > * WAL-per-partiton approach needs to be used; > * Write operations to temporary WAL storage must have higher priority over > reading operations; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-11074) Implement catch-up temporary WAL
[ https://issues.apache.org/jira/browse/IGNITE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-11074: - Ignite Flags: (was: Docs Required) > Implement catch-up temporary WAL > > > Key: IGNITE-11074 > URL: https://issues.apache.org/jira/browse/IGNITE-11074 > Project: Ignite > Issue Type: Sub-task >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: iep-28 > > While the demander node is in the partition file transmission state it must > save all cache entries corresponding to the moving partition into a new > temporary WAL storage. These entries will be applied later one by one on the > received cache partition file. All asynchronous operations will be enrolled > to the end of temporary WAL storage during storage reads until it becomes > fully read. The file-based FIFO approach assumes to be used by this process. > The new write-ahead-log manager for writing temporary records must support to: > Unlimited number of wal-files to store temporary data records; > Iterating over stored data records during an asynchronous writer thread > inserts new records; > WAL-per-partiton approach needs to be used; > Write operations to temporary WAL storage must have higher priority over > reading operations; -- This message was sent by Atlassian JIRA (v7.6.3#76005)
