Dmitriy Govorukhin created IGNITE-11641:
-------------------------------------------
Summary: Server node copies a lot of WAL files in WAL archive
after restart
Key: IGNITE-11641
URL: https://issues.apache.org/jira/browse/IGNITE-11641
Project: Ignite
Issue Type: Bug
Reporter: Dmitriy Govorukhin
Pre-condition: PDS is enabled, wal_path and wal_archive_path are set in config
file.
1. Cluster is up and running. Some data uploaded into caches.
2. Start load to generate a lot of files in wal archive (more than files in wal
directory).
3. Stop some node and delete all files from wal archive.
4. Start node.
In this case node copies WAL files from WAL dir into wal archive dir again and
again until the amount of files will be the same it was in wal archive before
stop.
Here is information from server node log
{code}
10:10:17,054][INFO][main][GridCacheDatabaseSharedManager] Restoring partition
state for local groups.
[10:10:18,522][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal]
[10:10:18,523][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=1, segIdx=1,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal]
[10:10:20,631][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal]
[10:10:20,632][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=2, segIdx=2,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal]
[10:10:23,276][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal]
[10:10:23,276][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=3, segIdx=3,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal]
[10:10:23,995][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal]
[10:10:23,996][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=4, segIdx=4,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal]
[10:10:24,644][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal]
[10:10:24,645][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=5, segIdx=5,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal]
[10:10:25,301][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal]
[10:10:25,301][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=6, segIdx=6,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal]
[10:10:26,043][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000006.wal]
[10:10:26,044][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=7, segIdx=7,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal]
[10:10:26,768][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000007.wal]
[10:10:26,769][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=8, segIdx=8,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal]
[10:10:27,600][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000008.wal]
[10:10:27,601][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=9, segIdx=9,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal]
[10:10:28,315][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000009.wal]
[10:10:28,316][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=10, segIdx=0,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000010.wal]
[10:10:29,092][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000000.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000010.wal]
[10:10:29,092][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=11, segIdx=1,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000011.wal]
[10:10:29,908][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000001.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000011.wal]
[10:10:29,908][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=12, segIdx=2,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000012.wal]
[10:10:30,716][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000002.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000012.wal]
[10:10:30,716][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=13, segIdx=3,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000013.wal]
[10:10:31,402][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000003.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000013.wal]
[10:10:31,403][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=14, segIdx=4,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000014.wal]
[10:10:32,052][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000004.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000014.wal]
[10:10:32,053][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Starting to copy WAL segment [absIdx=15, segIdx=5,
origFile=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
dstFile=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000015.wal]
[10:10:32,679][INFO][wal-file-archiver%null-#64][FileWriteAheadLogManager]
Copied file
[src=/storage/ssd/avolkov/wal/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000005.wal,
dst=/storage/ssd/avolkov/wal_archive/node00-83c9db32-fee5-4f3e-8a1c-559221817759/0000000000000015.wal]
[10:10:33,986][INFO][main][GridCacheDatabaseSharedManager] Finished restoring
partition state for local groups [groupsProcessed39partitionsProcessed=15056,
time=16928ms]
[10:10:34,269][INFO][main][TcpDiscoverySpi] Connection check threshold is
calculated: 2000
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
