[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-7384: Fix Version/s: (was: 2.10) > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Labels: rebalance > > Currently MVCC doesn't support historical (delta) rebalance. > The main difficulty is that MVCC writes changes on tx active phase while > partition update version, aka update counter, is being applied on tx finish. > This means we cannot start iteration over WAL right from the pointer where > the update counter updated, but should include updates, which the transaction > that updated the counter did. > Currently proposed approach: > * Maintain a list of active TXs with update counter (UC) which was actual at > the time before TX did its first update (on per partition basis) > * on each checkpoint save two counters - update counter (UC) and back > counter (BC) which is earliest UC mapped to a tx from active list at > checkpoint time. > * during local restore move UC and BC forward as far as possible. > * send BC instead of update counter in demand message. > * start iteration from a first checkpoint having UC less or equal received BC > See [linked dev list > thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] > for details -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Plekhanov updated IGNITE-7384: -- Fix Version/s: (was: 2.9) 2.10 > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Labels: rebalance > Fix For: 2.10 > > > Currently MVCC doesn't support historical (delta) rebalance. > The main difficulty is that MVCC writes changes on tx active phase while > partition update version, aka update counter, is being applied on tx finish. > This means we cannot start iteration over WAL right from the pointer where > the update counter updated, but should include updates, which the transaction > that updated the counter did. > Currently proposed approach: > * Maintain a list of active TXs with update counter (UC) which was actual at > the time before TX did its first update (on per partition basis) > * on each checkpoint save two counters - update counter (UC) and back > counter (BC) which is earliest UC mapped to a tx from active list at > checkpoint time. > * during local restore move UC and BC forward as far as possible. > * send BC instead of update counter in demand message. > * start iteration from a first checkpoint having UC less or equal received BC > See [linked dev list > thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] > for details -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-7384: Fix Version/s: (was: 2.8) 2.9 > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Labels: rebalance > Fix For: 2.9 > > > Currently MVCC doesn't support historical (delta) rebalance. > The main difficulty is that MVCC writes changes on tx active phase while > partition update version, aka update counter, is being applied on tx finish. > This means we cannot start iteration over WAL right from the pointer where > the update counter updated, but should include updates, which the transaction > that updated the counter did. > Currently proposed approach: > * Maintain a list of active TXs with update counter (UC) which was actual at > the time before TX did its first update (on per partition basis) > * on each checkpoint save two counters - update counter (UC) and back > counter (BC) which is earliest UC mapped to a tx from active list at > checkpoint time. > * during local restore move UC and BC forward as far as possible. > * send BC instead of update counter in demand message. > * start iteration from a first checkpoint having UC less or equal received BC > See [linked dev list > thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] > for details -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-7384: Labels: rebalance (was: ) > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Labels: rebalance > Fix For: 2.8 > > > Currently MVCC doesn't support historical (delta) rebalance. > The main difficulty is that MVCC writes changes on tx active phase while > partition update version, aka update counter, is being applied on tx finish. > This means we cannot start iteration over WAL right from the pointer where > the update counter updated, but should include updates, which the transaction > that updated the counter did. > Currently proposed approach: > * Maintain a list of active TXs with update counter (UC) which was actual at > the time before TX did its first update (on per partition basis) > * on each checkpoint save two counters - update counter (UC) and back > counter (BC) which is earliest UC mapped to a tx from active list at > checkpoint time. > * during local restore move UC and BC forward as far as possible. > * send BC instead of update counter in demand message. > * start iteration from a first checkpoint having UC less or equal received BC > See [linked dev list > thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] > for details -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Seliverstov updated IGNITE-7384: - Description: Currently MVCC doesn't support historical (delta) rebalance. The main difficulty is that MVCC writes changes on tx active phase while partition update version, aka update counter, is being applied on tx finish. This means we cannot start iteration over WAL right from the pointer where the update counter updated, but should include updates, which the transaction that updated the counter did. Currently proposed approach: * Maintain a list of active TXs with update counter (UC) which was actual at the time before TX did its first update (on per partition basis) * on each checkpoint save two counters - update counter (UC) and back counter (BC) which is earliest UC mapped to a tx from active list at checkpoint time. * during local restore move UC and BC forward as far as possible. * send BC instead of update counter in demand message. * start iteration from a first checkpoint having UC less or equal received BC See [linked dev list thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] for details was: Currently MVCC doesn't support historical (delta) rebalance. The main difficulty is that MVCC writes changes on tx active phase while partition update version, aka update counter, is being applied on tx finish. This means we cannot start iteration over WAL right from the pointer where the update counter updated, but should include updates, which the transaction that updated the counter did. Currently proposed approach: Maintain a list of active TXs with update counter (UC) which was actual at the time before TX did its first update (on per partition basis) on each checkpoint save two counters - update counter (UC) and back counter (BC) which is earliest UC mapped to a tx from active list at checkpoint time. during local restore move UC and BC forward as far as possible. send BC instead of update counter in demand message. start iteration from a first checkpoint having UC less or equal received BC See [linked dev list thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] for details > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Fix For: 2.8 > > > Currently MVCC doesn't support historical (delta) rebalance. > The main difficulty is that MVCC writes changes on tx active phase while > partition update version, aka update counter, is being applied on tx finish. > This means we cannot start iteration over WAL right from the pointer where > the update counter updated, but should include updates, which the transaction > that updated the counter did. > Currently proposed approach: > * Maintain a list of active TXs with update counter (UC) which was actual at > the time before TX did its first update (on per partition basis) > * on each checkpoint save two counters - update counter (UC) and back > counter (BC) which is earliest UC mapped to a tx from active list at > checkpoint time. > * during local restore move UC and BC forward as far as possible. > * send BC instead of update counter in demand message. > * start iteration from a first checkpoint having UC less or equal received BC > See [linked dev list > thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] > for details -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Seliverstov updated IGNITE-7384: - Description: Currently MVCC doesn't support historical (delta) rebalance. The main difficulty is that MVCC writes changes on tx active phase while partition update version, aka update counter, is being applied on tx finish. This means we cannot start iteration over WAL right from the pointer where the update counter updated, but should include updates, which the transaction that updated the counter did. Currently proposed approach: Maintain a list of active TXs with update counter (UC) which was actual at the time before TX did its first update (on per partition basis) on each checkpoint save two counters - update counter (UC) and back counter (BC) which is earliest UC mapped to a tx from active list at checkpoint time. during local restore move UC and BC forward as far as possible. send BC instead of update counter in demand message. start iteration from a first checkpoint having UC less or equal received BC See [linked dev list thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] for details was: Currently MVCC doesn't support historical (delta) rebalance. The main difficulty is that MVCC writes changes on tx active phase while partition update version, aka update counter, is being applied on tx finish. This means we cannot start iteration over WAL right from the pointer where the update counter updated, but should include updates, which the transaction that updated the counter did. Currently proposed approach: Maintain a list of active TXs with update counter (UC) which was actual at the time before TX did its first update (on per partition basis) on each checkpoint save two counters - update counter (UC) and back counter (BC) which is earliest UC mapped to a tx from active list at checkpoint time. during local restore move UC and BC forward as far as possible. send BC instead of update counter in demand message. start iteration from a first checkpoint having UC less or equal received BC See linked dev list thread for details > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Fix For: 2.8 > > > Currently MVCC doesn't support historical (delta) rebalance. > The main difficulty is that MVCC writes changes on tx active phase while > partition update version, aka update counter, is being applied on tx finish. > This means we cannot start iteration over WAL right from the pointer where > the update counter updated, but should include updates, which the transaction > that updated the counter did. > Currently proposed approach: > Maintain a list of active TXs with update counter (UC) which was actual at > the time before TX did its first update (on per partition basis) > on each checkpoint save two counters - update counter (UC) and back counter > (BC) which is earliest UC mapped to a tx from active list at checkpoint time. > during local restore move UC and BC forward as far as possible. > send BC instead of update counter in demand message. > start iteration from a first checkpoint having UC less or equal received BC > See [linked dev list > thread|http://apache-ignite-developers.2346864.n4.nabble.com/Historical-rebalance-td38380.html] > for details -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Seliverstov updated IGNITE-7384: - Description: Currently MVCC doesn't support historical (delta) rebalance. The main difficulty is that MVCC writes changes on tx active phase while partition update version, aka update counter, is being applied on tx finish. This means we cannot start iteration over WAL right from the pointer where the update counter updated, but should include updates, which the transaction that updated the counter did. Currently proposed approach: Maintain a list of active TXs with update counter (UC) which was actual at the time before TX did its first update (on per partition basis) on each checkpoint save two counters - update counter (UC) and back counter (BC) which is earliest UC mapped to a tx from active list at checkpoint time. during local restore move UC and BC forward as far as possible. send BC instead of update counter in demand message. start iteration from a first checkpoint having UC less or equal received BC See linked dev list thread for details was: In case a node returns to topology it requests a delta instead of full partition, WAL-based iterator is used there ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) WAL-based iterator doesn't contain MVCC versions which causes issues. > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Fix For: 2.8 > > > Currently MVCC doesn't support historical (delta) rebalance. > The main difficulty is that MVCC writes changes on tx active phase while > partition update version, aka update counter, is being applied on tx finish. > This means we cannot start iteration over WAL right from the pointer where > the update counter updated, but should include updates, which the transaction > that updated the counter did. > Currently proposed approach: > Maintain a list of active TXs with update counter (UC) which was actual at > the time before TX did its first update (on per partition basis) > on each checkpoint save two counters - update counter (UC) and back counter > (BC) which is earliest UC mapped to a tx from active list at checkpoint time. > during local restore move UC and BC forward as far as possible. > send BC instead of update counter in demand message. > start iteration from a first checkpoint having UC less or equal received BC > See linked dev list thread for details -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-7384: Fix Version/s: (was: 2.7) 2.8 > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Fix For: 2.8 > > > In case a node returns to topology it requests a delta instead of full > partition, WAL-based iterator is used there > ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) > WAL-based iterator doesn't contain MVCC versions which causes issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-7384: Issue Type: Task (was: Bug) > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Task > Components: mvcc >Reporter: Igor Seliverstov >Priority: Major > Fix For: 2.7 > > > In case a node returns to topology it requests a delta instead of full > partition, WAL-based iterator is used there > ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) > WAL-based iterator doesn't contain MVCC versions which causes issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-7384: Fix Version/s: 2.7 > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Bug > Components: mvcc >Reporter: Igor Seliverstov >Assignee: Roman Kondakov >Priority: Major > Fix For: 2.7 > > > In case a node returns to topology it requests a delta instead of full > partition, WAL-based iterator is used there > ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) > WAL-based iterator doesn't contain MVCC versions which causes issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Ozerov updated IGNITE-7384: Component/s: mvcc > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Bug > Components: mvcc >Reporter: Igor Seliverstov >Assignee: Roman Kondakov >Priority: Major > > In case a node returns to topology it requests a delta instead of full > partition, WAL-based iterator is used there > ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) > WAL-based iterator doesn't contain MVCC versions which causes issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-7384: --- Description: In case a node returns to topology it requests a delta instead of full partition, WAL-based iterator is used there ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) WAL-based iterator doesn't contain MVCC versions which causes issues. was: In case a node returns to topology it equests a delta instead of full partition, WAL-based iterator is used there ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) WAL-based iterator doesn't contain MVCC versions which causes issues. > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Bug >Reporter: Igor Seliverstov >Assignee: Roman Kondakov >Priority: Major > > In case a node returns to topology it requests a delta instead of full > partition, WAL-based iterator is used there > ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) > WAL-based iterator doesn't contain MVCC versions which causes issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (IGNITE-7384) MVCC TX: Support historical rebalance
[ https://issues.apache.org/jira/browse/IGNITE-7384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roman Kondakov updated IGNITE-7384: --- Summary: MVCC TX: Support historical rebalance (was: MVCC Mvcc versions may be lost in case of rebalanse with persistence enabled) > MVCC TX: Support historical rebalance > - > > Key: IGNITE-7384 > URL: https://issues.apache.org/jira/browse/IGNITE-7384 > Project: Ignite > Issue Type: Bug >Reporter: Igor Seliverstov >Assignee: Roman Kondakov >Priority: Major > > In case a node returns to topology it equests a delta instead of full > partition, WAL-based iterator is used there > ({{o.a.i.i.processors.cache.persistence.GridCacheOffheapManager#rebalanceIterator}}) > WAL-based iterator doesn't contain MVCC versions which causes issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)