[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986096#comment-16986096
]
Maxim Muzafarov commented on IGNITE-9720:
-
Folks,
I've cancelled the patch. PR is outdated.
Please, feel free to set the PA status again when the issue will be ready for
review.
> Initialize partition free lists lazily
> --
>
> Key: IGNITE-9720
> URL: https://issues.apache.org/jira/browse/IGNITE-9720
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Goncharuk
>Assignee: Semen Boikov
>Priority: Major
> Labels: performance
> Fix For: 2.8
>
>
> When persistence is enabled, partition free lists metadata may take quite a
> lot of pages.
> This results in a very long start time because
> {{GridCacheOffheapManager.GridCacheDataStore#init0}} will read all metadata
> for free list in each partition on exchange start (this is done in the
> {{CacheFreeListImpl}} constructor)
> We should only read required information on exchange and defer actual free
> list initialization to the first access.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16946902#comment-16946902
]
Maxim Muzafarov commented on IGNITE-9720:
-
[~agoncharuk] [~jokser] [~sboikov]
Folks, any updates here? Can we proceed with this patch?
> Initialize partition free lists lazily
> --
>
> Key: IGNITE-9720
> URL: https://issues.apache.org/jira/browse/IGNITE-9720
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Goncharuk
>Assignee: Semen Boikov
>Priority: Major
> Labels: performance
> Fix For: 2.8
>
>
> When persistence is enabled, partition free lists metadata may take quite a
> lot of pages.
> This results in a very long start time because
> {{GridCacheOffheapManager.GridCacheDataStore#init0}} will read all metadata
> for free list in each partition on exchange start (this is done in the
> {{CacheFreeListImpl}} constructor)
> We should only read required information on exchange and defer actual free
> list initialization to the first access.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877639#comment-16877639
]
Ignite TC Bot commented on IGNITE-9720:
---
{panel:title=-- Run :: All: No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4252138buildTypeId=IgniteTests24Java8_RunAll]
> Initialize partition free lists lazily
> --
>
> Key: IGNITE-9720
> URL: https://issues.apache.org/jira/browse/IGNITE-9720
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Goncharuk
>Assignee: Semen Boikov
>Priority: Major
> Labels: performance
> Fix For: 2.8
>
>
> When persistence is enabled, partition free lists metadata may take quite a
> lot of pages.
> This results in a very long start time because
> {{GridCacheOffheapManager.GridCacheDataStore#init0}} will read all metadata
> for free list in each partition on exchange start (this is done in the
> {{CacheFreeListImpl}} constructor)
> We should only read required information on exchange and defer actual free
> list initialization to the first access.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805123#comment-16805123
]
Alexey Goncharuk commented on IGNITE-9720:
--
Multiple test fails and JVM crashes need to be investigated.
> Initialize partition free lists lazily
> --
>
> Key: IGNITE-9720
> URL: https://issues.apache.org/jira/browse/IGNITE-9720
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Goncharuk
>Assignee: Semen Boikov
>Priority: Major
> Labels: performance
> Fix For: 2.8
>
>
> When persistence is enabled, partition free lists metadata may take quite a
> lot of pages.
> This results in a very long start time because
> {{GridCacheOffheapManager.GridCacheDataStore#init0}} will read all metadata
> for free list in each partition on exchange start (this is done in the
> {{CacheFreeListImpl}} constructor)
> We should only read required information on exchange and defer actual free
> list initialization to the first access.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16805122#comment-16805122
]
Ignite TC Bot commented on IGNITE-9720:
---
{panel:title=-- Run :: All: Possible
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}MVCC PDS 2{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=3384183]]
* IgnitePdsMvccTestSuite2: CheckpointFreeListTest.testFreeListRestoredCorrectly
- 0,0% fails in last 354 master runs.
{color:#d04437}MVCC PDS 4{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=3384187]]
* IgnitePdsMvccTestSuite4: FreeListLazyInitializationTest.initializationError
{color:#d04437}PDS 2{color} [[tests
2|https://ci.ignite.apache.org/viewLog.html?buildId=3384189]]
* IgnitePdsTestSuite2: CheckpointFreeListTest.testFreeListRestoredCorrectly -
0,0% fails in last 393 master runs.
{color:#d04437}PDS 4{color} [[tests
31|https://ci.ignite.apache.org/viewLog.html?buildId=3384193]]
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionAtomicPrimaryAsync - 0,0%
fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionAtomicNodeFilteredAsync -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testLocalPreloadPartitionPrimary - 0,0% fails in
last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalBackupSync -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testLocalPreloadPartitionBackup - 0,0% fails in
last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionAtomicBackupAsync - 0,0%
fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalBackupAsync -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalClientSync -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4: FreeListLazyInitializationTest.initializationError
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalPrimaryAsync -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalNodeFilteredSync
- 0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalPrimarySync -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadLocalTransactionalSync - 0,0% fails in
last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalClientAsyncMvcc
- 0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalClientSyncMvcc -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalPrimarySyncMvcc
- 0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionAtomicNodeFilteredSync - 0,0%
fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalBackupSyncMvcc -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testLocalPreloadPartitionBackupMvcc - 0,0% fails
in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalNodeFilteredSyncMvcc
- 0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testLocalPreloadPartitionPrimaryMvcc - 0,0% fails
in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionAtomicClientSync - 0,0% fails
in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalPrimaryAsyncMvcc
- 0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalNodeFilteredAsyncMvcc
- 0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalClientAsync -
0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionTransactionalBackupAsyncMvcc
- 0,0% fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionAtomicClientAsync - 0,0%
fails in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadLocalTransactionalAsync - 0,0% fails
in last 395 master runs.
* IgnitePdsTestSuite4:
IgnitePdsPartitionPreloadTest.testPreloadPartitionAtomicBackupSync - 0,0% fails
in last 395 master runs.
* IgnitePdsTestSuite4:
[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704676#comment-16704676
]
Alexey Goncharuk commented on IGNITE-9720:
--
Added an assert to make sure that lazy list initialization does not break
checkpoint lock invariant.
> Initialize partition free lists lazily
> --
>
> Key: IGNITE-9720
> URL: https://issues.apache.org/jira/browse/IGNITE-9720
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Goncharuk
>Assignee: Semen Boikov
>Priority: Major
> Labels: performance
> Fix For: 2.8
>
>
> When persistence is enabled, partition free lists metadata may take quite a
> lot of pages.
> This results in a very long start time because
> {{GridCacheOffheapManager.GridCacheDataStore#init0}} will read all metadata
> for free list in each partition on exchange start (this is done in the
> {{CacheFreeListImpl}} constructor)
> We should only read required information on exchange and defer actual free
> list initialization to the first access.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16704630#comment-16704630
]
Ignite TC Bot commented on IGNITE-9720:
---
{panel:title=-- Run :: All: No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=2416015buildTypeId=IgniteTests24Java8_RunAll]
> Initialize partition free lists lazily
> --
>
> Key: IGNITE-9720
> URL: https://issues.apache.org/jira/browse/IGNITE-9720
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Goncharuk
>Assignee: Semen Boikov
>Priority: Major
> Labels: performance
> Fix For: 2.8
>
>
> When persistence is enabled, partition free lists metadata may take quite a
> lot of pages.
> This results in a very long start time because
> {{GridCacheOffheapManager.GridCacheDataStore#init0}} will read all metadata
> for free list in each partition on exchange start (this is done in the
> {{CacheFreeListImpl}} constructor)
> We should only read required information on exchange and defer actual free
> list initialization to the first access.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (IGNITE-9720) Initialize partition free lists lazily
[
https://issues.apache.org/jira/browse/IGNITE-9720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671265#comment-16671265
]
ASF GitHub Bot commented on IGNITE-9720:
GitHub user agoncharuk opened a pull request:
https://github.com/apache/ignite/pull/5231
IGNITE-9720 Lazy free lists
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gridgain/apache-ignite ignite-9720
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/ignite/pull/5231.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5231
commit 057f4fdc7ede35779e98b09a7e9c7a7844f3d70e
Author: sboikov
Date: 2018-10-19T12:39:22Z
ignite-9720
commit c6b0905111563473bcd4aabf53a1e620f62af190
Author: sboikov
Date: 2018-10-22T10:33:01Z
ignite-9720
commit 28a76b3bb60759f5d812e5333c1ab429b88b88e3
Author: sboikov
Date: 2018-10-22T10:40:33Z
ignite-9720
commit a6cd973a2b80d3bb1b6ead420b4d9d7d2afc9084
Author: sboikov
Date: 2018-10-22T15:15:20Z
Merge remote-tracking branch 'remotes/origin/master' into ignite-9720
commit 144ca212edf6dfcbd93feca5bc4c1244fb4edc5e
Author: sboikov
Date: 2018-10-23T07:56:08Z
ignite-9720
commit 9c330467687078438b3cd89527c9a4e0cb55af4b
Author: sboikov
Date: 2018-10-23T09:32:26Z
Attempt to make ttl tests more robust.
commit 23d4f72176febac0be9c97ed020d00ffa81461e6
Author: sboikov
Date: 2018-10-23T12:27:29Z
ignite-9720
commit d98bfd955d3188c1685be42e842e806347ab0bb7
Author: sboikov
Date: 2018-10-23T12:27:56Z
Merge remote-tracking branch 'remotes/origin/master' into ignite-9720
commit 2a52875facdb5e8e20b65f0e4a742be394145099
Author: sboikov
Date: 2018-10-24T06:05:30Z
Merge remote-tracking branch 'remotes/origin/master' into ignite-9720
commit 8464bd1976641bd35ac0de20d01e270d47726012
Author: sboikov
Date: 2018-10-24T06:17:54Z
ignite-9720
commit 36d3169fa2b6aec10e8fb19e15741c9b7675bdb5
Author: Alexey Goncharuk
Date: 2018-11-01T08:31:40Z
Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/ignite
into ignite-9720
Conflicts:
modules/core/src/test/java/org/apache/ignite/testsuites/IgnitePdsTestSuite4.java
> Initialize partition free lists lazily
> --
>
> Key: IGNITE-9720
> URL: https://issues.apache.org/jira/browse/IGNITE-9720
> Project: Ignite
> Issue Type: Task
>Reporter: Alexey Goncharuk
>Assignee: Semen Boikov
>Priority: Major
> Labels: performance
> Fix For: 2.8
>
>
> When persistence is enabled, partition free lists metadata may take quite a
> lot of pages.
> This results in a very long start time because
> {{GridCacheOffheapManager.GridCacheDataStore#init0}} will read all metadata
> for free list in each partition on exchange start (this is done in the
> {{CacheFreeListImpl}} constructor)
> We should only read required information on exchange and defer actual free
> list initialization to the first access.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
