Also a caution on this... you may want to retain the file's modified time in something like purge.modified... so you can also re-calc for files where purge.modified != file modified time. Else you may purge something too early.
Alec On Fri, Jan 29, 2021 at 6:53 PM Alec <[email protected]> wrote: > Based on the problem you have. > > I would write an mmfind / mmxarg command that sets a custom attr such as > puge.after, have a ksh/perl/php script that simply makes the necessary > calculations using all the tricks it has... Skip files that already have > the attribute set, or are too new to bother having the attribute. > > Then use a single purge policy to query all files that have a purge.after > set to the appropriate datestamp. > > You could get way more concise with this mechanism and have a much simpler > process. > > Alec > > On Fri, Jan 29, 2021 at 4:32 PM Owen Morgan < > [email protected]> wrote: > >> Hi all, >> >> Sorry I appear to have missed a load of replies and screwed up the >> threading thing when looking online... not used to this email group thing! >> Might look at the slack option! >> >> Just wanted to clarify my general issue a bit: >> >> So the methodology I've started to implement is per department policy >> files where all rules related to managing a specific teams assets are all >> in one policy file and then I have fine control over when and how each >> departments rule run, when, and potentially (if it mattered) what order etc. >> >> >> So team a want me to manage two folders where in folder 1a all files >> older than 4 week days of age are deleted, and in filder 1b all files older >> than 8 week days are deleted. >> >> They now want me to manage a different set of two folders with two >> different "thresholds" for how old they need to be in week days before they >> delete (ie. I now need additional rules for folders 2a and 2b). >> >> >> The issue is for each scenario there is a different 'offset' required >> depending on the day of the week the policy is run to maintian the number >> of weekdays required (the 'threshold' is always in weekdays, so intervening >> weekends need to be added to take them into account). >> >> For instance when run on a Monday, if the threshold were 4 weekdays of >> age, I need to be deleting files that were created on the previous Tuesday. >> Which is 6 days (ie 4 days + 2 weekend days). If the threshold was 8 week >> days the threhold in terms of the policy would be 12 (ie 8 plus 2x 2 >> weekend days). >> >> >> The only way I was able to work this out in the sql like policy file was >> to split the week days into groups where the offset would be the same (so >> for 4 week days, Monday through Thursday share the offset of 2 - which then >> has to be added to the 4 for the desired result) and then a separate rule >> for the Friday. >> >> >> However for every addition of a different threshold I have to write all >> new groups to match the days etc.. so the policy ends up with 6 rules but >> 150 lines of definition macros.... >> >> >> I was trying to work out if there was a more concise way of, within the >> sql like framework, programmatically calculating the day offest the needs >> to be added to the threshold to allow a more generic function that could >> just automatically work it out.... >> >> >> The algorithm I have recently thought up is to effectively calculate the >> difference in weeks between the current run time and the desired deletion >> day and multiply it by 2. >> >> >> Psudocode it would be (threshold is the number of week days for the rule, >> offset is the number that needs to be added to account for the weekends >> between those dates): >> >> >> If current day of month - threshold = sunday, then add 1 to the threshold >> value (sundays are de oted as the week start so Saturday would represent >> the previous week). >> >> Offset = (difference between current week and week of (current day of >> month - threshold)) x 2 >> >> A worked example: >> >> Threshold = 11 week days >> Policy run on the 21st Jan which is the week 4 of 2021 >> >> 21st - 11 days = Sunday 10th >> >> Therefore need to add 1 to threshold to push the day into the previous >> week. New threshold is 12 >> >> Saturday 9th is in week 2 of 2021 so the offset is week 4 - week 2 = 2 >> (ie difference in weeks) x 2 which is 4. >> >> Add 4 to the original 11 to make 15. >> >> So for the policy running on the 21st Jan to delete only files older than >> 11 week days of age I need to set my rule to be >> >> Delete where ((Current_date - creation_time) >= interval '15' days >> >> >> Unfortunately, I'm now struggling to implement that algorithm..... it >> seems the SQL-ness is very limited and I cant declare variables to use or >> stuff.... its a shame as that algorithm is generic so only needs to be >> written once and you could have ad many unique rules as you want all with >> different thresholds etc... >> >> Is there another way to get the same results? >> >> I would prefer to stay in the bounds of the SQL policy rule setup as that >> is the framework I have created and started to implement.. >> >> Hope the above gives more clarity to what Im asking.... sorry if one of >> the previous rplies addresses this, if it does I clearly was confused by >> the response (I seriously feel like an amateur at this at the moment and am >> having to learn all these finer things as I go). >> >> Thanks in advance, >> >> Owen. >> >> Owen Morgan >> Data Wrangler >> Motion Picture Solutions Ltd >> T: >> E: *[email protected]* >> <[email protected]> | W: >> *motionpicturesolutions.com* <https://www.motionpicturesolutions.com/> >> A: Mission Hall, 9‑11 North End Road , London , W14 8ST >> Motion Picture Solutions Ltd is a company registered in England and Wales >> under number 5388229, VAT number 201330482 >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at spectrumscale.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >> >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
