[
https://issues.apache.org/jira/browse/IGNITE-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819105#comment-16819105
]
Dmitriy Govorukhin commented on IGNITE-11641:
---------------------------------------------
[~mstepachev] Thanks for the review. I fixed your comments.
> Server node copies a lot of WAL files in WAL archive after restart
> ------------------------------------------------------------------
>
> Key: IGNITE-11641
> URL: https://issues.apache.org/jira/browse/IGNITE-11641
> Project: Ignite
> Issue Type: Bug
> Reporter: Dmitriy Govorukhin
> Assignee: Dmitriy Govorukhin
> Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Pre-condition: PDS is enabled, wal_path and wal_archive_path are set in
> config file.
> 1. Cluster is up and running. Some data uploaded into caches.
> 2. Start load to generate a lot of files in wal archive (more than files in
> wal directory).
> 3. Stop some node and delete all files from wal archive.
> 4. Start node.
> In this case node copies WAL files from WAL dir into wal archive dir again
> and again until the amount of files will be the same it was in wal archive
> before stop.
> Here is information from server node log
> {code}
> 10:10:17,054][INFO][main][GridCacheDatabaseSharedManager] Restoring partition
> state for local groups.
> [10:10:18,522][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal]
> [10:10:18,523][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=1, segIdx=1,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal]
> [10:10:20,631][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal]
> [10:10:20,632][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=2, segIdx=2,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal]
> [10:10:23,276][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal]
> [10:10:23,276][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=3, segIdx=3,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal]
> [10:10:23,995][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal]
> [10:10:23,996][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=4, segIdx=4,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal]
> [10:10:24,644][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal]
> [10:10:24,645][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=5, segIdx=5,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal]
> [10:10:25,301][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal]
> [10:10:25,301][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=6, segIdx=6,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal]
> [10:10:26,043][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal]
> [10:10:26,044][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=7, segIdx=7,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal]
> [10:10:26,768][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal]
> [10:10:26,769][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=8, segIdx=8,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal]
> [10:10:27,600][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal]
> [10:10:27,601][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=9, segIdx=9,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal]
> [10:10:28,315][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal]
> [10:10:28,316][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=10, segIdx=0,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000010.wal]
> [10:10:29,092][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000010.wal]
> [10:10:29,092][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=11, segIdx=1,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000011.wal]
> [10:10:29,908][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000011.wal]
> [10:10:29,908][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=12, segIdx=2,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000012.wal]
> [10:10:30,716][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000012.wal]
> [10:10:30,716][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=13, segIdx=3,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000013.wal]
> [10:10:31,402][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000013.wal]
> [10:10:31,403][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=14, segIdx=4,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000014.wal]
> [10:10:32,052][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000014.wal]
> [10:10:32,053][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Starting to copy WAL segment [absIdx=15, segIdx=5,
> origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
>
> dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000015.wal]
> [10:10:32,679][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
> Copied file
> [src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
>
> dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000015.wal]
> [10:10:33,986][INFO][main][GridCacheDatabaseSharedManager] Finished restoring
> partition state for local groups [groupsProcessed39partitionsProcessed=15056,
> time=16928ms]
> [10:10:34,269][INFO][main][TcpDiscoverySpi] Connection check threshold is
> calculated: 2000
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
