[
https://issues.apache.org/jira/browse/IGNITE-18721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17689953#comment-17689953
]
Ignite TC Bot commented on IGNITE-18721:
----------------------------------------
{panel:title=Branch: [pull/10533/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10533/head] Base: [master] : New Tests
(4)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#00008b}PDS (Indexing){color} [[tests
4|https://ci2.ignite.apache.org/viewLog.html?buildId=7053185]]
* {color:#013220}IgnitePdsWithIndexingTestSuite:
WalDisabledDuringIndexRecreateTest.testDisabled[cacheGroups=true] -
PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
WalDisabledDuringIndexRecreateTest.testRestartInCaseOfFailure[cacheGroups=false]
- PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
WalDisabledDuringIndexRecreateTest.testDisabled[cacheGroups=false] -
PASSED{color}
* {color:#013220}IgnitePdsWithIndexingTestSuite:
WalDisabledDuringIndexRecreateTest.testRestartInCaseOfFailure[cacheGroups=true]
- PASSED{color}
{panel}
[TeamCity *--> Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7051268&buildTypeId=IgniteTests24Java8_RunAll]
> Improve WAL writing during index rebuilding
> -------------------------------------------
>
> Key: IGNITE-18721
> URL: https://issues.apache.org/jira/browse/IGNITE-18721
> Project: Ignite
> Issue Type: Improvement
> Affects Versions: 2.14
> Reporter: Ilya Shishkov
> Assignee: Nikolay Izhikov
> Priority: Major
> Labels: ise
> Attachments: WalDuringIndexRebuildTest.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently, Ignite writes WAL segments during index rebuilding. For large
> index files this process can last for a several hours and can lead to very
> intensive of WAL segments. It seems, that we could optimize or even turn off
> WAL segments writing during index rebuild process in order to minimize index
> rebuilding time.
> In attached test ([^WalDuringIndexRebuildTest.patch]) rebuilding process
> generates 37-38 segments of a 64MB size more than 2GB for an 1,3GB index file.
> In my environment, index rebuild for {{LOG_ONLY}} estimates 28-33 seconds,
> for {{NONE}}: 11-14 seconds.
> Also, there are 2 possible ways now to force index rebuilding:
> # Force index rebuild by a {{control.sh}} (which leads to
> {{IgniteCacheDatabaseSharedManager#forceRebuildIndexes}} calls).
> # Manually delete index.bin files.
> Both rebuilding processes generates different WALRecords in different
> proportions, as you can see in a tables below (results derived from the
> attached test).
> *Summary for force index rebuild:*
> ||Record type ||Cnt||
> |BTREE_PAGE_REPLACE |5045341|
> |PAGE_RECORD |389204|
> |CHECKPOINT_RECORD |4|
> |INIT_NEW_PAGE_RECORD |1|
> |DATA_PAGE_SET_FREE_LIST_PAGE |2|
> |PAGES_LIST_ADD_PAGE |1|
> |METASTORE_DATA_RECORD |1|
> *Summary for index.bin remove:*
> ||Record type ||Count||
> |PAGE_RECORD |386487|
> |BTREE_PAGE_INSERT |5075253|
> |BTREE_FIX_COUNT |20216|
>
> |BTREE_EXISTING_PAGE_SPLIT |351907|
> |CHECKPOINT_RECORD |5|
> |INIT_NEW_PAGE_RECORD |133|
> |DATA_PAGE_INSERT_FRAGMENT_RECORD |2|
> |DATA_PAGE_SET_FREE_LIST_PAGE |3|
> |BTREE_META_PAGE_ADD_ROOT |103|
> |BTREE_INIT_NEW_ROOT |154|
> |BTREE_PAGE_RECYCLE |2|
> |PAGES_LIST_ADD_PAGE |4|
> |PAGES_LIST_REMOVE_PAGE |1|
> |METASTORE_DATA_RECORD |3|
> |BTREE_META_PAGE_INIT_ROOT_V3 |52|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
