Hi Arnau, Which module system are you using?
At Sheffield we use LMOD with our newest cluster (which has extensive guidance here: https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html ) as well as used Environment Modules on our older retired cluster. On the newer cluster in the SitePackage.lua: local function logmsg(logTbl) > -- Print to syslog with generic header > -- All the elements in the table logTbl are > -- added in order. Expect format: > -- logTbl[#logTbl+1] = {'log_key', 'log_value'} > > local user = os.getenv("USER") > local microArch = os.getenv("MICROARCH") > local jobId = os.getenv("SLURM_JOB_ID") > local partition = os.getenv("SLURM_JOB_PARTITION") > local currentTime = os.date("!%Y%m%dT%H%M%S") > > if isempty(microArch) then > microArch = "UNKNOWN" > end > > if isempty(jobId) then > jobId = "-1" > end > > if isempty(partition) then > partition = "UNKNOWN" > end > > if isempty(user) then > user = "UNKNOWN" > end > > local fullHost = capture("hostname -f"):gsub("%s", "") > local i, j, clusterName, first, second > i,j, first, second, clusterName = > fullHost:find("([^.]*)%.([^.]*)%.([^.]*)%.") -- Sadly this is NOT regex! > (ノಠ益ಠ)ノ彡┻━┻ Check the Lua pattern matching docs... ┬─┬ノ( º _ ºノ) > > local msg = string.format("USER=%s CLUSTER=%s PARTITION=%s > MICROARCH=%s OPERATION=%s MHOST=%s JOBID=%s TIMESTAMP=%s", user, > clusterName, partition, microArch, "LOAD", fullHost, jobId, currentTime) > > for _, val in ipairs(logTbl) do > msg = msg .. string.format(", %s=%q", val[1], val[2] or "") > end > > lmod_system_execute("logger -t ModuleUsageTracking -p local0.info " .. > msg) -- > end > > local function load_hook(t) > if (mode() ~= "load") then return end > > -- the module at the top of the FrameStack is the one the user > requested https://lmod.readthedocs.io/en/latest/170_hooks.html#hook-tips > local frameStk = FrameStk:singleton() > local userLoad = (frameStk:atTop()) and "yes" or "no" > > -- the arg t is a table: > -- t.modFullName: the module full name: (i.e: gcc/4.7.2) > -- t.fn: The file name: (i.e > /apps/modulefiles/Core/gcc/4.7.2.lua) > > local logTbl = {} > logTbl[#logTbl+1] = {"USERLOAD", userLoad} > logTbl[#logTbl+1] = {"MODULE", t.modFullName} > logTbl[#logTbl+1] = {"MODULEPATH", t.fn} > > logmsg(logTbl) > > end > > hook.register("load", load_hook) On the older (now retired) cluster TCL script this looks like: ## Log the usage of the module files > if { [info exists ::env(JOB_SCRIPT) ] } { > if { $::env(JOB_SCRIPT)=="INTERACTIVE" } { > set sge_q qsh > } else { > set sge_q batch > } > } else { > set sge_q qrsh > } > > # Note that in qrsh sessions the scheduler is not able to set environment > # variables (e.g. JOB_ID) but it can in qrshx, qsh and qsub sessions. > if { [info exists ::env(JOB_ID) ] } { > set jid $::env(JOB_ID) > } else { > set jid -1 > } > > set minfo [module-info name] > set mmode [module-info mode] > exec logger -p local1.debug "modulefile: USER=$tcl_platform(user) > OPERATION=$mmode MODULE=$minfo SGEQ=$sge_q JOB_ID=$jid" tl;dr for both being, we stick it in the logs for both then pipe it where we need. Snipped this all very quickly so apologies for any mistakes! Thanks, James. Dr James Moore Senior Research Platforms Engineer Research Platforms Research and Innovation IT IT-Services The University of Sheffield p: 0114 222 3110 ext: 23110 On Wed, 3 Apr 2024 at 10:31, Arnau <[email protected]> wrote: > Hi all, > > I'd like to know if there's a tool to track modules usage. Our modules are > installed in a shared NFS space and, IIRC, there was some way to track when > a module was loaded. > > We're now upgrading to Rocky9 from SL7 and some old modules are broken, so > we would like to understand which ones need to be fixed before using them > in the new OS. > > TIA, > Arnau >
