Hello,
I’m completely new to Robinhood, so forgive my ignorance, but I’m having a
problem with some of the stats being reported by robinhood. For example, our
Lustre filesystem is mounted on /scratch and there is a user directory
/scratch/mkg52 that has well over 40GB of data in it. However, when I run
`rbh-report -u mkg52`, i get a response telling me that there is far less. Here
is the relevant output:
[root@tillit ~]# rbh-report -u mkg52
Using config file '/etc/robinhood.d/lustre.conf'.
user, type, count, volume, spc_used, avg_size
mkg52, dir, 15, 60.00 KB, 60.00 KB, 4.00 KB
mkg52, file, 327, 399.32 MB, 400.20 MB, 1.22 MB
Now when I run du you can see I have much more than 400MB of data being used
[mkg52@wind /scratch/mkg52 ]$ du -smh *
217M archive
108K bigdata_snd73
4.0K build_wine.sh
4.0K cd833.out
1.5G CESM
578M cesm1_2_2
68K clr96_bigdata
4.0K clr96_lazy.out
4.0K clr96_lazy.txt
4.0K clr96_output.out
16K cmdstan_testing
1.4G drivers
[root@tillit ~]# rbh-du -s -H /scratch/mkg52
Using config file '/etc/robinhood.d/lustre.conf'.
4.4M total
I am very confused by this output. There are other user folders where this
information is completely accurate, but specifically /scratch/mkg52 is missing
a bunch of information.
Here is the output of the errors in readlog:
[root@tillit ~]# robinhood --readlog --once -L stderr
Using config file '/etc/robinhood.d/lustre.conf'.
2016/12/15 16:07:13 [45806/1] CheckFS | '/scratch' matches mount point
'/scratch', type=lustre, fs=mds1.ib@o2ib:mds2.ib@o2ib:/blizzard
2016/12/15 16:07:13 [45806/1] EntryProc | No fileclass defined in
configuration, disabling fileclass matching.
2016/12/15 16:07:13 [45806/1] llapi | warning: llapi_changelog_start() called
w/o CHANGELOG_FLAG_JOBID
2016/12/15 16:07:13 [45806/2] ChangeLog | LU-1331 is fixed in this version of
Lustre.
2016/12/15 16:07:13 [45806/2] ChangeLog | Changelog reader thread terminating
2016/12/15 16:07:13 [45806/1] Main | Event Processing finished
2016/12/15 16:07:13 [45806/1] EntryProc | Pipeline successfully flushed
2016/12/15 16:07:13 [45806/1] STATS | ==== EntryProcessor Pipeline Stats ===
2016/12/15 16:07:13 [45806/1] STATS | Idle threads: 0
2016/12/15 16:07:13 [45806/1] STATS | Id constraints count: 0 (hash
min=0/max=0/avg=0.0)
2016/12/15 16:07:13 [45806/1] STATS | Name constraints count: 0 (hash
min=0/max=0/avg=0.0)
2016/12/15 16:07:13 [45806/1] STATS | Stage | Wait | Curr | Done |
Total | ms/op |
2016/12/15 16:07:13 [45806/1] STATS | 0: GET_FID | 0 | 0 | 0 |
0 | 0.00 |
2016/12/15 16:07:13 [45806/1] STATS | 1: GET_INFO_DB | 0 | 0 | 0 |
248 | 0.37 |
2016/12/15 16:07:13 [45806/1] STATS | 2: GET_INFO_FS | 0 | 0 | 0 |
148 | 1.60 |
2016/12/15 16:07:13 [45806/1] STATS | 3: PRE_APPLY | 0 | 0 | 0 |
242 | 0.00 |
2016/12/15 16:07:13 [45806/1] STATS | 4: DB_APPLY | 0 | 0 | 0 |
242 | 0.78 | 11.57% batched (avg batch size: 2.0)
2016/12/15 16:07:13 [45806/1] STATS | 5: CHGLOG_CLR | 0 | 0 | 0 |
248 | 0.01 |
2016/12/15 16:07:13 [45806/1] STATS | 6: RM_OLD_ENTRIES | 0 | 0 | 0 |
0 | 0.00 |
2016/12/15 16:07:13 [45806/1] STATS | DB ops: get=192/ins=54/upd=89/rm=99
2016/12/15 16:07:13 [45806/1] STATS | ChangeLog reader #0:
2016/12/15 16:07:13 [45806/1] STATS | fs_name = blizzard
2016/12/15 16:07:13 [45806/1] STATS | mdt_name = MDT0000
2016/12/15 16:07:13 [45806/1] STATS | reader_id = cl1
2016/12/15 16:07:13 [45806/1] STATS | records read = 252
2016/12/15 16:07:13 [45806/1] STATS | interesting records = 160
2016/12/15 16:07:13 [45806/1] STATS | suppressed records = 92
2016/12/15 16:07:13 [45806/1] STATS | records pending = 0
2016/12/15 16:07:13 [45806/1] STATS | last received = 2016/12/15
16:07:13
2016/12/15 16:07:13 [45806/1] STATS | last read record time = 2016/12/15
16:07:13.445247
2016/12/15 16:07:13 [45806/1] STATS | last read record id = 16147371
2016/12/15 16:07:13 [45806/1] STATS | last pushed record id = 16147371
2016/12/15 16:07:13 [45806/1] STATS | last committed record id = 16147371
2016/12/15 16:07:13 [45806/1] STATS | last cleared record id = 16147371
2016/12/15 16:07:13 [45806/1] STATS | status = almost idle
2016/12/15 16:07:13 [45806/1] STATS | ChangeLog stats:
2016/12/15 16:07:13 [45806/1] STATS | MARK: 0, CREAT: 55, MKDIR: 0, HLINK:
0, SLINK: 0, MKNOD: 0, UNLNK: 12, RMDIR: 0, RENME: 45
2016/12/15 16:07:13 [45806/1] STATS | RNMTO: 0, OPEN: 0, CLOSE: 99, LYOUT:
0, TRUNC: 40, SATTR: 1, XATTR: 0, HSM: 0, MTIME: 0
2016/12/15 16:07:13 [45806/1] STATS | CTIME: 0, ATIME: 0, MIGRT: 0
Here is my lustre.conf file:
General {
fs_path = "/scratch";
# filesystem type, as displayed by 'mount' (e.g. ext4, xfs, lustre, ...)
fs_type = lustre;
}
Log {
log_file = "/var/log/robinhood/robinhood.log";
report_file = "/var/log/robinhood/robinhood_actions.log";
alert_file = "/var/log/robinhood/robinhood_alerts.log";
}
ListManager {
MySQL {
server = localhost;
db = robinhood_lustre;
user = robinhood;
password_file = /etc/robinhood.d/.dbpassword;
}
}
# Lustre 2.x only
ChangeLog {
MDT {
mdt_name = "MDT0000";
reader_id = "cl1";
}
}
Any help would be greatly appreciated!
Thank you,
Mike
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
robinhood-support mailing list
robinhood-support@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/robinhood-support
