Hi Thomas,
Apologies for not replying sooner. From running the command I can see the rmdir
policy is failing but not the reason why - any thoughts?
Cheers, Doug
[xctrh00:~]$ sudo robinhood -f test --run=rmdir --target
file:/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN -l full
-L stderr
Using config file '/etc/robinhood.d/test.conf'.
2018/04/09 11:52:41 [43268/1] CfgLoader | Loading global config
2018/04/09 11:52:41 [43268/1] CfgLoader | Loading logs config
2018/04/09 11:52:41 [43268/1] CfgLoader | Loading updt params config
2018/04/09 11:52:41 [43268/1] CfgLoader | Loading list manager config
2018/04/09 11:52:41 [43268/1] CfgLoader | Loading policies config
2018/04/09 11:52:41 [43268/1] mod_mgt | Module 'librbh_mod_common.so': optional
symbol 'mod_get_status_manager' not found:
/usr/lib64/robinhood/librbh_mod_common.so: undefined symbol:
mod_get_status_manager
2018/04/09 11:52:41 [43268/1] mod_mgt | Successfully loaded module 'common'
2018/04/09 11:52:41 [43268/1] CfgLoader | Loading status managers config
2018/04/09 11:52:41 [43268/1] CfgLoader | Loading policy run config
2018/04/09 11:52:41 @xctrh00[43268/1] CheckFS | Root mountpoint is NOT allowed
for matching /work, type=rootfs, fs=rootfs
2018/04/09 11:52:41 @xctrh00[43268/1] CheckFS | Root mountpoint is NOT allowed
for matching /work, type=xfs, fs=/dev/mapper/vgopsys-root
2018/04/09 11:52:41 @xctrh00[43268/1] CheckFS | /work is under mountpoint
/work, type=lustre, fs=10.xxx@o2ib:10.xxx@o2ib:/snx11106
2018/04/09 11:52:41 @xctrh00[43268/1] CheckFS | '/work' matches mount point
'/work', type=lustre, fs=10.xxx@o2ib:10.xxx@o2ib:/snx11106
2018/04/09 11:52:41 @xctrh00[43268/1] FSInfo | fs_key:
hash(fsname=snx11106)=C8E82E31
2018/04/09 11:52:41 @xctrh00[43268/1] Main | Signal handler thread started
successfully
2018/04/09 11:52:41 @xctrh00[43268/2] SigHdlr | Signals SIGTERM and SIGINT
(daemon shutdown) are ready to be used
2018/04/09 11:52:41 @xctrh00[43268/2] SigHdlr | Signal SIGHUP (config
reloading) is ready to be used
2018/04/09 11:52:41 @xctrh00[43268/2] SigHdlr | Signal SIGUSR1 (stats dump) is
ready to be used
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Logged on to database
'robinhood_work' successfully
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Checking database schema
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SELECT value FROM
VARS WHERE varname='VersionFunctionSet'
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Functions version is up-to-date
(1.6)
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SELECT value FROM
VARS WHERE varname='VersionTriggerSet'
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Triggers version is up-to-date
(1.4)
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW COLUMNS FROM
VARS
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | VARS.varname field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | VARS.value field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW COLUMNS FROM
ENTRIES
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.id field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.uid field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | uid field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | uid field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.gid field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | gid field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | gid field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.size field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | size field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | size field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.blocks field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | blocks field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | blocks field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.creation_time field
name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | creation_time field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.last_access field name
OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | last_access field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.last_mod field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | last_mod field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.last_mdchange field
name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | last_mdchange field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.type field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | type field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | type field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.mode field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | mode field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.nlink field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | nlink field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.md_update field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | md_update field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.invalid field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | invalid field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.fileclass field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | fileclass field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ENTRIES.class_update field name
OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | class_update field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW COLUMNS FROM
NAMES
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | NAMES.id field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | NAMES.pkn field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | NAMES.parent_id field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | NAMES.name field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | NAMES.path_update field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW COLUMNS FROM
ANNEX_INFO
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ANNEX_INFO.id field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ANNEX_INFO.link field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | link field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW FUNCTION STATUS
WHERE DB='robinhood_work' AND NAME='sz_range'
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Function sz_range exists
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW COLUMNS FROM
ACCT_STAT
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.uid field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | uid field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | uid field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.gid field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | gid field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | gid field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.type field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | type field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | type field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.size field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | size field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | size field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.blocks field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | blocks field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | blocks field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.count field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz0 field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz1 field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz32 field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz1K field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz32K field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz1M field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz32M field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz1G field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz32G field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | ACCT_STAT.sz1T field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW COLUMNS FROM
STRIPE_INFO
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_INFO.id field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_INFO.validator field
name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_INFO.stripe_count field
name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_INFO.stripe_size field
name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_INFO.pool_name field
name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW COLUMNS FROM
STRIPE_ITEMS
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_ITEMS.id field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_ITEMS.stripe_index field
name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_ITEMS.ostidx field name
OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | STRIPE_ITEMS.details field name
OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW COLUMNS FROM
SOFT_RM
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.id field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.fullpath field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | fullpath field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.uid field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | uid field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | uid field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.gid field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | gid field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | gid field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.size field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | size field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | size field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.blocks field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | blocks field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | blocks field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.last_access field name
OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | last_access field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.last_mod field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | last_mod field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.last_mdchange field
name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | last_mdchange field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.type field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | type field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | type field default OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.mode field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | mode field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.nlink field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | nlink field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SOFT_RM.rm_time field name OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | rm_time field type OK
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SELECT
EVENT_OBJECT_TABLE FROM INFORMATION_SCHEMA.TRIGGERS WHERE
TRIGGER_SCHEMA='robinhood_work'AND TRIGGER_NAME='ACCT_ENTRY_INSERT'
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Trigger ACCT_ENTRY_INSERT
exists and is defined on ENTRIES
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SELECT
EVENT_OBJECT_TABLE FROM INFORMATION_SCHEMA.TRIGGERS WHERE
TRIGGER_SCHEMA='robinhood_work'AND TRIGGER_NAME='ACCT_ENTRY_DELETE'
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Trigger ACCT_ENTRY_DELETE
exists and is defined on ENTRIES
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SELECT
EVENT_OBJECT_TABLE FROM INFORMATION_SCHEMA.TRIGGERS WHERE
TRIGGER_SCHEMA='robinhood_work'AND TRIGGER_NAME='ACCT_ENTRY_UPDATE'
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Trigger ACCT_ENTRY_UPDATE
exists and is defined on ENTRIES
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW FUNCTION STATUS
WHERE DB='robinhood_work' AND NAME='one_path'
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Function one_path exists
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SHOW FUNCTION STATUS
WHERE DB='robinhood_work' AND NAME='this_path'
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Function this_path exists
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Database connection closed
2018/04/09 11:52:41 @xctrh00[43268/1] Main | ListManager successfully
initialized
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Logged on to database
'robinhood_work' successfully
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SET SESSION
TRANSACTION ISOLATION LEVEL READ COMMITTED
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SELECT value FROM
VARS WHERE varname='FS_Path'
2018/04/09 11:52:41 @xctrh00[43268/1] CheckFS | /work matches database content.
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | SQL query: SELECT value FROM
VARS WHERE varname='RootId'
2018/04/09 11:52:41 @xctrh00[43268/1] CheckFS | Root id '0x200000007:0x1:0x0'
matches previous value.
2018/04/09 11:52:41 @xctrh00[43268/1] ListMgr | Database connection closed
2018/04/09 11:52:41 @xctrh00[43268/1] rmdir | Notice: no 'default' policy rule
is defined. Unmatched entries will be ignored.
2018/04/09 11:52:41 @xctrh00[43268/1] Main | Policy rmdir successfully
initialized
2018/04/09 11:52:41 @xctrh00[43268/3] ListMgr | Logged on to database
'robinhood_work' successfully
2018/04/09 11:52:41 @xctrh00[43268/3] ListMgr | SQL query: SET SESSION
TRANSACTION ISOLATION LEVEL READ COMMITTED
2018/04/09 11:52:41 @xctrh00[43268/4] ListMgr | Logged on to database
'robinhood_work' successfully
2018/04/09 11:52:41 @xctrh00[43268/4] ListMgr | SQL query: SET SESSION
TRANSACTION ISOLATION LEVEL READ COMMITTED
2018/04/09 11:52:41 @xctrh00[43268/5] ListMgr | Logged on to database
'robinhood_work' successfully
2018/04/09 11:52:41 @xctrh00[43268/5] ListMgr | SQL query: SET SESSION
TRANSACTION ISOLATION LEVEL READ COMMITTED
2018/04/09 11:52:41 @xctrh00[43268/6] ListMgr | Logged on to database
'robinhood_work' successfully
2018/04/09 11:52:41 @xctrh00[43268/6] ListMgr | SQL query: SET SESSION
TRANSACTION ISOLATION LEVEL READ COMMITTED
2018/04/09 11:52:41 @xctrh00[43268/5] Main | Statistics thread started
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | Logged on to database
'robinhood_work' successfully
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | Logged on to database
'robinhood_work' successfully
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: SET SESSION
TRANSACTION ISOLATION LEVEL READ COMMITTED
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: SET SESSION
TRANSACTION ISOLATION LEVEL READ COMMITTED
2018/04/09 11:52:41 @xctrh00[43268/7] rmdir | Checking policy rules for entry
'/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN'
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: INSERT INTO VARS
(varname,value) VALUES ('rmdir_start_current','1523274761') ON DUPLICATE KEY
UPDATE value='1523274761'
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: INSERT INTO VARS
(varname,value) VALUES ('rmdir_trigger_current','manual run, target: entry
\'/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN\'') ON
DUPLICATE KEY UPDATE value='manual run, target: entry
\'/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN\''
2018/04/09 11:52:41 @xctrh00[43268/7] rmdir | target: none
2018/04/09 11:52:41 @xctrh00[43268/7] rmdir | done: none
2018/04/09 11:52:41 @xctrh00[43268/7] rmdir | Error running policy on entry
'/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN'. 0
successful actions; volume: 0; 0 entries skipped; 0 errors.
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: DELETE FROM VARS
WHERE varname ='rmdir_start_current'
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: DELETE FROM VARS
WHERE varname ='rmdir_trigger_current'
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: INSERT INTO VARS
(varname,value) VALUES ('rmdir_start','0') ON DUPLICATE KEY UPDATE value='0'
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: INSERT INTO VARS
(varname,value) VALUES ('rmdir_end','1523274761') ON DUPLICATE KEY UPDATE
value='1523274761'
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: INSERT INTO VARS
(varname,value) VALUES ('rmdir_trigger','manual run, target: entry
\'/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN\'') ON
DUPLICATE KEY UPDATE value='manual run, target: entry
\'/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN\''
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | SQL query: INSERT INTO VARS
(varname,value) VALUES ('rmdir_status','Fatal error running policy after 0
successful actions, volume: 0; 0 entries skipped; 0 errors') ON DUPLICATE KEY
UPDATE value='Fatal error running policy after 0 successful actions, volume: 0;
0 entries skipped; 0 errors'
2018/04/09 11:52:41 @xctrh00[43268/7] ListMgr | Database connection closed
2018/04/09 11:52:41 @xctrh00[43268/7] rmdir | Policy run terminated with status
-2
2018/04/09 11:52:41 @xctrh00[43268/1] Main | rmdir: policy run terminated (rc =
0).
2018/04/09 11:52:41 @xctrh00[43268/1] Main | All tasks done! Exiting.
From: Thomas LEIBOVICI (mail perso) [mailto:thomasleibov...@free.fr]
Sent: 29 March 2018 20:02
To: Hayman, Doug <doug.hay...@metoffice.gov.uk>; LEIBOVICI Thomas
<thomas.leibov...@cea.fr>; robinhood-support@lists.sourceforge.net
Subject: Re: [robinhood-support] rmdir policy
Indeed, it looks it should match.
What you can run for troubleshooting is:
robinhood --run=rmdir --target
file:/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN<file://work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN>
-l full -L stderr
It will trace everything about its decision to remove this entry or not.
Hayman, Doug wrote :
Hi Thomas,
Thanks for your prompt reply.
[xctrh00:~]$ sudo rbh-report -f test.conf -e
/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN
id : [0x200010227:0x158:0x0]
parent_id : [0x2000101fa:0xa5ab:0x0]
name : tmp.gjmMLApNvN
path updt : 2018/03/29 08:55:43
path :
/work/.lustre_home/research/working/d03/itwl/jtmp/tmp.gjmMLApNvN
depth : 6
dircount : 0
user : itwl
group : mo_users
size : 4.00 KB
spc_used : 4.00 KB
creation : 2017/07/25 09:11:05
last_access : 2018/01/18 14:06:50
last_mod : 2017/07/25 09:11:05
last_mdchange : 2017/07/25 09:11:05
type : dir
mode : rwxr-xr-x
nlink : 2
md updt : 2018/03/29 08:55:43
invalid : no
fileclass : working_empty_dir
class updt : 2018/03/29 08:55:43
Cheers, Doug
From: LEIBOVICI Thomas [mailto:thomas.leibov...@cea.fr]
Sent: 29 March 2018 08:21
To: Hayman, Doug
<doug.hay...@metoffice.gov.uk><mailto:doug.hay...@metoffice.gov.uk>;
robinhood-support@lists.sourceforge.net<mailto:robinhood-support@lists.sourceforge.net>
Subject: Re: [robinhood-support] rmdir policy
Hello Doug,
At first sight, your configuration looks good.
Could you run "rbh-report -e" on an entry that should be eligible to rmdir and
is not processed by the policy?
Thanks
Thomas
On 03/28/18 12:14, Hayman, Doug wrote:
Hello,
I have been using Robinhood 3.1-1 to cleanup old files from our scratch file
systems but when adding a rmdir policy to remove empty directories it is not
working. Scanning the file system with a fresh database populates the target
fileclass as expected and the database entry-info for the directories seems to
fit my policy deletion rules. However, when running the policy, it completes
successfully but without removing any directories. This is how I am running the
policy:
[xctrh00:~]$ sudo robinhood -f test --run=rmdir --once
When running the policy with log verbosity level DEBUG or FULL the directories
within the target fileclass are not in the output.
I have appended the configuration file below - any help with this would be
appreciated.
Many thanks
Doug
[xctrh00:~]$ cat /etc/robinhood.d/test.conf
General {
fs_path = "/work";
fs_type = lustre;
}
#### policy definitions ####
# include template policy definitions for legacy TMPFS flavor
%include "includes/tmpfs.inc"
# include template policy definitions for removing directories
%include "includes/rmdir.inc"
#### fileclass definitions ####
FileClass working_empty_dir {
definition { type == "directory" and dircount == 0 and tree ==
"/work/.lustre_home/research/working/d0*/*/jtmp/*" }
}
FileClass scratch_empty_dir {
definition { type == "directory" and dircount == 0 and tree ==
"/work/.lustre_multi/research/scratch/d0*/*/*" }
}
FileClass research_home {
definition { type == "file" and tree ==
"/work/.lustre_home/research/home/d0*/*" }
}
FileClass research_working {
definition { type == "file" and tree ==
"/work/.lustre_home/research/working/d0*/*" }
}
FileClass research_data {
definition { type == "file" and tree ==
"/work/.lustre_multi/research/data/d0*/*" }
}
FileClass research_scratch {
definition { type == "file" and tree ==
"/work/.lustre_multi/research/scratch/d0*/*" }
}
FileClass largedir {
definition { type == "directory" and dircount > 10000 }
}
FileClass hpc_admins {
definition { group == "crayadm" or group == "super_unix" }
}
#### Deleting old unused files ####
cleanup_rules {
# don't purge entries matching the following conditions
ignore { owner == "root" or type == "symlink" or type == "fifo" or type ==
"socket" }
# don't purge files from the following fileclasses
ignore_fileclass = hpc_admins;
ignore_fileclass = research_home;
ignore_fileclass = research_data;
#ignore_fileclass = working_empty_dir;
ignore_fileclass = scratch_empty_dir;
# cleanup policy for files older than 30 days
rule clean_files {
target_fileclass = research_working;
target_fileclass = research_scratch;
condition { last_access > 30d }
}
}
# trigger cleanup policy every 24 hours
cleanup_trigger {
trigger_on = periodic;
check_interval = 24h;
}
############# rmdir rules ############
# specifies no default sort order should be applied to the policy
#rmdir_parameters {
# lru_sort_attr = none;
#}
# trigger rmdir policy every 24 hours
rmdir_trigger {
trigger_on = periodic;
check_interval = 24h;
}
rmdir_rules {
# don't remove directories matching the following conditions
ignore { owner == "root" or depth < 5 }
# don't remove directories from the following fileclasses
ignore_fileclass = hpc_admins;
ignore_fileclass = research_home;
#ignore_fileclass = research_working;
ignore_fileclass = research_data;
ignore_fileclass = research_scratch;
ignore_fileclass = scratch_empty_dir;
# remove empty directories after 30 days
rule rmdir_empty {
target_fileclass = working_empty_dir;
action = common.rmdir;
condition { last_mod > 30d }
}
}
#### end of policy rules #####
Log {
debug_level = EVENT;
log_file = "/var/log/robinhood/work_fs.log";
report_file = "/var/log/robinhood/reports.log";
alert_file = "/var/log/robinhood/alerts.log";
stats_interval = 5min;
log_hostname = TRUE;
}
ListManager {
commit_behavior = autocommit;
MySQL {
server = localhost;
db = robinhood_work;
user = robinhood;
password_file = /etc/robinhood.d/.dbpassword;
engine = InnoDB;
}
}
EntryProcessor {
nb_threads = 32;
STAGE_GET_FID_threads_max = 24;
STAGE_GET_INFO_DB_threads_max = 24;
STAGE_GET_INFO_FS_threads_max = 24;
STAGE_PRE_APPLY_threads_max = 24;
STAGE_DB_APPLY_threads_max = 24;
max_pending_operations = 100000;
max_batch_size = 1;
}
FS_Scan {
scan_interval = 1d;
nb_threads_scan = 8;
scan_op_timeout = 1h;
exit_on_timeout = TRUE;
spooler_check_interval = 1min;
Ignore {
# ignore large directories
tree == "/work/home/crayadm/lfsck-1M"
or
tree ==
"/work/.lustre_home/research/home/d03/xxxx/xxxx-2017/836597/data"
}
}
Changelog {
MDT {
mdt_name = "MDT0000";
reader_id = "cl2";
}
MDT {
mdt_name = "MDT0001";
reader_id = "cl1";
}
MDT {
mdt_name = "MDT0002";
reader_id = "cl1";
}
batch_ack_count = 1024;
force_polling = ON;
polling_interval = 1s;
queue_max_size = 1000;
queue_max_age = 5s;
queue_check_interval = 1s;
}
[xctrh00:~]$ cat /etc/robinhood.d/includes/rmdir.inc
# This rmdir policy applies to all directories (empty or non-empty)
# This allow applying policies to both empty and non empty directories
# in a single policy run.
# - Explicitely add conditions "dircount == 0" in targeted fileclass
# to apply action to empty directories only.
# - Explicitely set action = cmd("rm -fr {fullpath}") in the policy rules
# to recursively remove non-empty directories.
define_policy rmdir {
scope {type == directory}
status_manager = none;
default_action = common.rmdir;
default_lru_sort_attr = last_mod;
}
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
robinhood-support mailing list
robinhood-support@lists.sourceforge.net<mailto:robinhood-support@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/robinhood-support
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
robinhood-support mailing list
robinhood-support@lists.sourceforge.net<mailto:robinhood-support@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/robinhood-support
[https://ipmcdn.avast.com/images/icons/icon-envelope-tick-round-orange-animated-no-repeat-v1.gif]<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
Garanti sans virus.
www.avast.com<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
robinhood-support mailing list
robinhood-support@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/robinhood-support
