[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357098#comment-17357098 ] Kirill Tkalenko commented on IGNITE-8719: - [~amashenkov] Please make code review. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 5h > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[
https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357095#comment-17357095
]
Ignite TC Bot commented on IGNITE-8719:
---
{panel:title=Branch: [pull/9090/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform .NET (Inspections)*{color} [[tests 0 Failure on metric
|https://ci.ignite.apache.org/viewLog.html?buildId=6029650]]
{panel}
{panel:title=Branch: [pull/9090/head] Base: [master] : New Tests
(8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS (Indexing){color} [[tests
8|https://ci.ignite.apache.org/viewLog.html?buildId=6034392]]
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage -
PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6029135buildTypeId=IgniteTests24Java8_RunAll]
> Index left partially built if a node crashes during index create or rebuild
> ---
>
> Key: IGNITE-8719
> URL: https://issues.apache.org/jira/browse/IGNITE-8719
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Alexey Goncharuk
>Assignee: Kirill Tkalenko
>Priority: Critical
> Fix For: 2.11
>
> Attachments: IndexRebuildAfterNodeCrashTest.java,
> IndexRebuildingTest.java
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Currently, we do not have any state associated with the index tree. Consider
> the following scenario:
> 1) Start node, put some data
> 2) start CREATE INDEX operation
> 3) Wait for a checkpoint and stop node before index create finished
> 4) Restart node
> Since the checkpoint finished, the new index tree will be persisted to the
> disk, but not all data will be present in the index.
> We should somehow store information about initializing index tree and mark it
> valid only after all data is indexed. The state should be persisted as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354836#comment-17354836 ] Kirill Tkalenko commented on IGNITE-8719: - [~jooger] Please make code review. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[
https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354498#comment-17354498
]
Ignite TC Bot commented on IGNITE-8719:
---
{panel:title=Branch: [pull/9090/head] Base: [master] : Possible Blockers
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform .NET (Inspections)*{color} [[tests 0 Failure on metric
|https://ci.ignite.apache.org/viewLog.html?buildId=6029650]]
{panel}
{panel:title=Branch: [pull/9090/head] Base: [master] : New Tests
(8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS (Indexing){color} [[tests
8|https://ci.ignite.apache.org/viewLog.html?buildId=6029102]]
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage -
PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6029135buildTypeId=IgniteTests24Java8_RunAll]
> Index left partially built if a node crashes during index create or rebuild
> ---
>
> Key: IGNITE-8719
> URL: https://issues.apache.org/jira/browse/IGNITE-8719
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Alexey Goncharuk
>Assignee: Kirill Tkalenko
>Priority: Critical
> Fix For: 2.11
>
> Attachments: IndexRebuildAfterNodeCrashTest.java,
> IndexRebuildingTest.java
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Currently, we do not have any state associated with the index tree. Consider
> the following scenario:
> 1) Start node, put some data
> 2) start CREATE INDEX operation
> 3) Wait for a checkpoint and stop node before index create finished
> 4) Restart node
> Since the checkpoint finished, the new index tree will be persisted to the
> disk, but not all data will be present in the index.
> We should somehow store information about initializing index tree and mark it
> valid only after all data is indexed. The state should be persisted as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345889#comment-17345889 ] Stanilovsky Evgeny commented on IGNITE-8719: [~ktkale...@gridgain.com] plz check one more comment from my side, overall looks good. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 3h 50m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[
https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17344961#comment-17344961
]
Ignite TC Bot commented on IGNITE-8719:
---
{panel:title=Branch: [pull/9090/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9090/head] Base: [master] : New Tests
(8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS (Indexing){color} [[tests
8|https://ci.ignite.apache.org/viewLog.html?buildId=6007815]]
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage -
PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6007848buildTypeId=IgniteTests24Java8_RunAll]
> Index left partially built if a node crashes during index create or rebuild
> ---
>
> Key: IGNITE-8719
> URL: https://issues.apache.org/jira/browse/IGNITE-8719
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Alexey Goncharuk
>Assignee: Kirill Tkalenko
>Priority: Critical
> Fix For: 2.11
>
> Attachments: IndexRebuildAfterNodeCrashTest.java,
> IndexRebuildingTest.java
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> Currently, we do not have any state associated with the index tree. Consider
> the following scenario:
> 1) Start node, put some data
> 2) start CREATE INDEX operation
> 3) Wait for a checkpoint and stop node before index create finished
> 4) Restart node
> Since the checkpoint finished, the new index tree will be persisted to the
> disk, but not all data will be present in the index.
> We should somehow store information about initializing index tree and mark it
> valid only after all data is indexed. The state should be persisted as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343715#comment-17343715 ] Kirill Tkalenko commented on IGNITE-8719: - [~zstan] Please make code review. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 2.5h > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[
https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343714#comment-17343714
]
Ignite TC Bot commented on IGNITE-8719:
---
{panel:title=Branch: [pull/9090/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9090/head] Base: [master] : New Tests
(8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS (Indexing){color} [[tests
8|https://ci.ignite.apache.org/viewLog.html?buildId=6004413]]
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage -
PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6004446buildTypeId=IgniteTests24Java8_RunAll]
> Index left partially built if a node crashes during index create or rebuild
> ---
>
> Key: IGNITE-8719
> URL: https://issues.apache.org/jira/browse/IGNITE-8719
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Alexey Goncharuk
>Assignee: Kirill Tkalenko
>Priority: Critical
> Fix For: 2.11
>
> Attachments: IndexRebuildAfterNodeCrashTest.java,
> IndexRebuildingTest.java
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Currently, we do not have any state associated with the index tree. Consider
> the following scenario:
> 1) Start node, put some data
> 2) start CREATE INDEX operation
> 3) Wait for a checkpoint and stop node before index create finished
> 4) Restart node
> Since the checkpoint finished, the new index tree will be persisted to the
> disk, but not all data will be present in the index.
> We should somehow store information about initializing index tree and mark it
> valid only after all data is indexed. The state should be persisted as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17343198#comment-17343198 ] Stanilovsky Evgeny commented on IGNITE-8719: [~ktkale...@gridgain.com] plz check my comments, if you ok with my proposals i would like to review it once more after fixes, thanks ! > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[
https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342487#comment-17342487
]
Ignite TC Bot commented on IGNITE-8719:
---
{panel:title=Branch: [pull/9090/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9090/head] Base: [master] : New Tests
(8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS (Indexing){color} [[tests
8|https://ci.ignite.apache.org/viewLog.html?buildId=6001403]]
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage -
PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5999042buildTypeId=IgniteTests24Java8_RunAll]
> Index left partially built if a node crashes during index create or rebuild
> ---
>
> Key: IGNITE-8719
> URL: https://issues.apache.org/jira/browse/IGNITE-8719
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Alexey Goncharuk
>Assignee: Kirill Tkalenko
>Priority: Critical
> Fix For: 2.11
>
> Attachments: IndexRebuildAfterNodeCrashTest.java,
> IndexRebuildingTest.java
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, we do not have any state associated with the index tree. Consider
> the following scenario:
> 1) Start node, put some data
> 2) start CREATE INDEX operation
> 3) Wait for a checkpoint and stop node before index create finished
> 4) Restart node
> Since the checkpoint finished, the new index tree will be persisted to the
> disk, but not all data will be present in the index.
> We should somehow store information about initializing index tree and mark it
> valid only after all data is indexed. The state should be persisted as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[
https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17341353#comment-17341353
]
Ignite TC Bot commented on IGNITE-8719:
---
{panel:title=Branch: [pull/9090/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9090/head] Base: [master] : New Tests
(7)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}PDS (Indexing){color} [[tests
7|https://ci.ignite.apache.org/viewLog.html?buildId=5999055]]
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5999042buildTypeId=IgniteTests24Java8_RunAll]
> Index left partially built if a node crashes during index create or rebuild
> ---
>
> Key: IGNITE-8719
> URL: https://issues.apache.org/jira/browse/IGNITE-8719
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Reporter: Alexey Goncharuk
>Assignee: Kirill Tkalenko
>Priority: Critical
> Fix For: 2.11
>
> Attachments: IndexRebuildAfterNodeCrashTest.java,
> IndexRebuildingTest.java
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently, we do not have any state associated with the index tree. Consider
> the following scenario:
> 1) Start node, put some data
> 2) start CREATE INDEX operation
> 3) Wait for a checkpoint and stop node before index create finished
> 4) Restart node
> Since the checkpoint finished, the new index tree will be persisted to the
> disk, but not all data will be present in the index.
> We should somehow store information about initializing index tree and mark it
> valid only after all data is indexed. The state should be persisted as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17243203#comment-17243203 ] Pavel Vinokurov commented on IGNITE-8719: - The issue still could be reproduced in both cases when an index is creating or rebuilding. Considering impact of this issue I suppose it could be fixed before implementation of IEP-28. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Priority: Critical > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943412#comment-16943412 ] Maxim Muzafarov commented on IGNITE-8719: - Folks, This is a known issue and will be solved as a part of [IEP-28: Index rebuild|[https://cwiki.apache.org/confluence/display/IGNITE/IEP-28%3A+Cluster+peer-2-peer+balancing]] index rebuild procedure. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Priority: Critical > Fix For: 2.8 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16670124#comment-16670124 ] ASF GitHub Bot commented on IGNITE-8719: GitHub user antonovsergey93 opened a pull request: https://github.com/apache/ignite/pull/5223 IGNITE-8719 Store information about starting of index rebuild. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8719 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5223.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5223 commit cfbd9ba1cc7c31fbe43bdd2e00bad81c8396a464 Author: Sergey Antonov Date: 2018-10-30T09:04:33Z IGNITE-8719 Try reproduce problem. commit b0c5d2740f9aa1814b6809ac7ad11b09d5d53250 Author: Sergey Antonov Date: 2018-10-31T13:34:11Z IGNITE-8719 First version of fix. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Sergey Antonov >Priority: Critical > Fix For: 2.8 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16667382#comment-16667382 ] Sergey Antonov commented on IGNITE-8719: [~agoncharuk] I wrote test by your scena [^IndexRebuildAfterNodeCrashTest.java] rio. But the problem wasn't reproduced. What I did wrong? [^IndexRebuildAfterNodeCrashTest.java] > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Sergey Antonov >Priority: Critical > Fix For: 2.8 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16655040#comment-16655040 ] Sergey Antonov commented on IGNITE-8719: [~cyberdemon], Looks like the issue is abandoned but looks serious. Would you mind if I start working on it? > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.8 > > Attachments: IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623655#comment-16623655 ] Dmitriy Sorokin commented on IGNITE-8719: - [~NIzhikov], A chance is exists, but if is not critical, fix version can be moved to 2.8. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > Attachments: IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623549#comment-16623549 ] Nikolay Izhikov commented on IGNITE-8719: - [~cyberdemon] Do we have a chance to resolve this ticket until the code freeze of 2.7? > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > Attachments: IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554095#comment-16554095 ] Stanislav Lukyanov commented on IGNITE-8719: There is a similar issue with an empty index tree being persisted. I can reproduce it by shutting node down forcibly (kill -9 or taskkill /f), but I guess the key is in shutting down in the middle of checkpoint or something like that. The reproducer is attached: [^IndexRebuildingTest.java]. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > Attachments: IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549465#comment-16549465 ] Dmitriy Sorokin commented on IGNITE-8719: - [~agoncharuk], I wrote the reproducer working by your scheme, but so far no confirmation for case when partially built index can be used. Look at my patch [#4380|https://github.com/apache/ignite/pull/4380], please, and tell me your opinion. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549150#comment-16549150 ] Stanilovsky Evgeny commented on IGNITE-8719: also, index creation can be crashed independently, for example with Error during parallel index create/rebuild. IgniteChackedException: maximum of retries 1000 reached. at org.apache.ignite.internal.processors.cache.persistence.tree.BplusTree$Get.checkLockRetry(BPlusTree.java:2565) > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
