Hi folks,
We've upgraded 2 of our filesystems recently to 2.10.5 (from IEEL 3.1-
lustre 2.7.x) and I've noticed that the one with DNE doesn't seem to
be cleaning / reading changelogs in robinhood since then
(i'd only been doing manual scans since the upgrade - today I enabled
changelogs via changelog_register on each MDT)
rbh-report -a
Changelog stats for MDT0000:
last_read record: rec_id=25431, rec_time=2018/12/14
08:40:49.587248, step_time=2018/12/14 08:41:39.443366
last_pushed record: rec_id=25407, rec_time=2018/12/14
08:40:44.699018, step_time=2018/12/14 08:41:39.442628
last_committed record: rec_id=28414, rec_time=2018/12/14
08:50:31.019852, step_time=2018/12/14 08:51:26.367623
last_cleared record: rec_id=25407, rec_time=2018/12/14
08:40:44.699018, step_time=2018/12/14 08:41:39.450362
Changelog stats per type (MDT0000):
type total (diff) (rate)
CREAT: 771
MKDIR: 157
UNLNK: 768
RMDIR: 157
CLOSE: 12414 (+2309) (2.57/sec)
TRUNC: 11173 (+2308) (2.56/sec)
Changelog stats for MDT0001:
last_read record: rec_id=15480, rec_time=2018/12/14
08:23:16.923499, step_time=2018/12/14 08:24:07.599741
and thats all it has for astrofs-MDT0001
Both are set up with the same mask:
root@hpc-xcat1 ~]# xdsh astrofs-mds[1,2],pgfs-mds2 lctl get_param
mdd.*.changelog_mask | xdshbak -c
HOSTS -------------------------------------------------------------------------
astrofs-mds1
-------------------------------------------------------------------------------
mdd.astrofs-MDT0000.changelog_mask=
MARK CREAT MKDIR HLINK SLINK MKNOD UNLNK RMDIR RENME RNMTO OPEN CLOSE
LYOUT TRUNC SATTR XATTR HSM MTIME CTIME MIGRT
HOSTS -------------------------------------------------------------------------
astrofs-mds2
-------------------------------------------------------------------------------
mdd.astrofs-MDT0001.changelog_mask=
MARK CREAT MKDIR HLINK SLINK MKNOD UNLNK RMDIR RENME RNMTO OPEN CLOSE
LYOUT TRUNC SATTR XATTR HSM MTIME CTIME MIGRT
HOSTS -------------------------------------------------------------------------
pgfs-mds2
-------------------------------------------------------------------------------
mdd.pgfs-MDT0000.changelog_mask=
MARK CREAT MKDIR HLINK SLINK MKNOD UNLNK RMDIR RENME RNMTO OPEN CLOSE
LYOUT TRUNC SATTR XATTR HSM MTIME CTIME MIGRT
and the changelogs seem to be racking up:
[root@hpc-xcat1 ~]# xdsh astrofs-mds[1,2] cat
/proc/fs/lustre/mdd/*/changelog_users
astrofs-mds2: current index: 23220
astrofs-mds2: ID index
astrofs-mds2: cl1 0
astrofs-mds1: current index: 31839
astrofs-mds1: ID index
astrofs-mds1: cl1 31839
and a quick read of the astrofs-MDT0001 changelog looks like it's full
of CREAT,CLOSE,UNLINK as expected
magnus-1:~ # lfs changelog astrofs-MDT0001 | head
1 01CREAT 23:49:09.486538354 2018.12.13 0x0 t=[0xa4000dd40:0xb144:0x0]
p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
2 11CLOSE 23:49:09.487007753 2018.12.13 0xc3 t=[0xa4000dd40:0xb144:0x0]
3 06UNLNK 23:49:09.487335599 2018.12.13 0x1 t=[0xa4000dd40:0xb144:0x0]
p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
4 01CREAT 23:49:09.487814447 2018.12.13 0x0 t=[0xa4000dd40:0xb145:0x0]
p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
5 11CLOSE 23:49:09.487986846 2018.12.13 0xc3 t=[0xa4000dd40:0xb145:0x0]
6 06UNLNK 23:49:09.488221218 2018.12.13 0x1 t=[0xa4000dd40:0xb145:0x0]
p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
7 01CREAT 23:49:09.488591157 2018.12.13 0x0 t=[0xa4000dd40:0xb146:0x0]
p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
8 11CLOSE 23:49:09.488745392 2018.12.13 0xc3 t=[0xa4000dd40:0xb146:0x0]
9 06UNLNK 23:49:09.488970060 2018.12.13 0x1 t=[0xa4000dd40:0xb146:0x0]
p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
10 01CREAT 23:49:09.489334721 2018.12.13 0x0
t=[0xa4000dd40:0xb147:0x0] p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
magnus-1:~ # lfs changelog astrofs-MDT0001 | tail
23211 06UNLNK 00:51:54.340464800 2018.12.14 0x1
t=[0xa4000dd40:0xcf7c:0x0] p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
23212 01CREAT 00:51:54.341178127 2018.12.14 0x0
t=[0xa4000dd40:0xcf7d:0x0] p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
23213 11CLOSE 00:51:54.341328738 2018.12.14 0xc3 t=[0xa4000dd40:0xcf7d:0x0]
23214 06UNLNK 00:51:54.341530488 2018.12.14 0x1
t=[0xa4000dd40:0xcf7d:0x0] p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
23215 01CREAT 00:51:54.341881777 2018.12.14 0x0
t=[0xa4000dd40:0xcf7e:0x0] p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
23216 11CLOSE 00:51:54.342020836 2018.12.14 0xc3 t=[0xa4000dd40:0xcf7e:0x0]
23217 06UNLNK 00:51:54.342211583 2018.12.14 0x1
t=[0xa4000dd40:0xcf7e:0x0] p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
23218 01CREAT 00:51:54.342805330 2018.12.14 0x0
t=[0xa4000dd40:0xcf7f:0x0] p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
23219 11CLOSE 00:51:54.342941539 2018.12.14 0xc3 t=[0xa4000dd40:0xcf7f:0x0]
23220 06UNLNK 00:51:54.343132968 2018.12.14 0x1
t=[0xa4000dd40:0xcf7f:0x0] p=[0xa4000d488:0x6a8b:0x0]
AAVS1_SKALA2_embedded_element_02_rev0_100_100.bof.lock
My configuration is as follows:
ChangeLog {
MDT {
mdt_name = "MDT0000";
reader_id = "cl1";
}
MDT {
mdt_name = "MDT0001";
reader_id = "cl1";
}
}
Anyone have any ideas?
Andrew
_______________________________________________
robinhood-support mailing list
robinhood-support@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/robinhood-support
