Re: [gpfsug-discuss] Policy scan of symbolic links with contents?
Does that check the target of the symlink, or the path to the link itself? I think the OP was checking the target (or I misunderstood). -- Stephen Ulmer Sent from a mobile device; please excuse auto-correct silliness. > On Mar 8, 2021, at 3:34 PM, Jonathan Buzzard > wrote: > > On 08/03/2021 20:45, Jonathan Buzzard wrote: > > [SNIP] > >> So noting that you can write very SQL like statements something like the >> following should in theory do it >> RULE finddangling LIST dangle WHERE MISC_ATTRIBUTES='L' AND >> SUBSTR(PATH_NAME,0,4)='/fs1/' >> Note the above is not checked in any way shape or form for working. Even >> if you do have bind mounts of other GPFS file systems you just need a >> more complicated WHERE statement. > > Duh, of course as soon as I sent it, I realized there is a missing SHOW > > RULE finddangling LIST dangle SHOW(PATH_NAME) WHERE MISC_ATTRIBUTES='L' AND > SUBSTR(PATH_NAME,0,4)='/fs1/' > > You could replace the SUBSTR with a REGEX if you prefer > > > JAB. > > -- > Jonathan A. Buzzard Tel: +44141-5483420 > HPC System Administrator, ARCHIE-WeSt. > University of Strathclyde, John Anderson Building, Glasgow. G4 0NG > ___ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Policy scan of symbolic links with contents?
On 08/03/2021 20:45, Jonathan Buzzard wrote: [SNIP] So noting that you can write very SQL like statements something like the following should in theory do it RULE finddangling LIST dangle WHERE MISC_ATTRIBUTES='L' AND SUBSTR(PATH_NAME,0,4)='/fs1/' Note the above is not checked in any way shape or form for working. Even if you do have bind mounts of other GPFS file systems you just need a more complicated WHERE statement. Duh, of course as soon as I sent it, I realized there is a missing SHOW RULE finddangling LIST dangle SHOW(PATH_NAME) WHERE MISC_ATTRIBUTES='L' AND SUBSTR(PATH_NAME,0,4)='/fs1/' You could replace the SUBSTR with a REGEX if you prefer JAB. -- Jonathan A. Buzzard Tel: +44141-5483420 HPC System Administrator, ARCHIE-WeSt. University of Strathclyde, John Anderson Building, Glasgow. G4 0NG ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Policy scan of symbolic links with contents?
On 08/03/2021 16:07, Frederick Stock wrote: CAUTION: This email originated outside the University. Check before clicking links or attachments. Presumably the only feature that would help here is if policy could determine that the end location pointed to by a symbolic link is within the current file system. I am not aware of any such feature or attribute which policy could check so I think all you can do is run policy to find the symbolic links and then check each link to see if it points into the same file system. You might find the mmfind command useful for this purpose. I expect it would eliminate the need to create a policy to find the symbolic links. Unless you are using bind mounts if the symbolic link points outside the mount point of the file system it is not within the current file system. So noting that you can write very SQL like statements something like the following should in theory do it RULE finddangling LIST dangle WHERE MISC_ATTRIBUTES='L' AND SUBSTR(PATH_NAME,0,4)='/fs1/' Note the above is not checked in any way shape or form for working. Even if you do have bind mounts of other GPFS file systems you just need a more complicated WHERE statement. When doing policy engine stuff I find having that section of the GPFS manual printed out and bound, along with an SQL book for reference is very helpful. JAB. -- Jonathan A. Buzzard Tel: +44141-5483420 HPC System Administrator, ARCHIE-WeSt. University of Strathclyde, John Anderson Building, Glasgow. G4 0NG ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Policy scan of symbolic links with contents?
Presumably the only feature that would help here is if policy could determine that the end location pointed to by a symbolic link is within the current file system. I am not aware of any such feature or attribute which policy could check so I think all you can do is run policy to find the symbolic links and then check each link to see if it points into the same file system. You might find the mmfind command useful for this purpose. I expect it would eliminate the need to create a policy to find the symbolic links. Fred___Fred Stock | Spectrum Scale Development Advocacy | 720-430-8821sto...@us.ibm.com - Original message -From: "Oesterlin, Robert" Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: gpfsug main discussion list Cc:Subject: [EXTERNAL] Re: [gpfsug-discuss] Policy scan of symbolic links with contents?Date: Mon, Mar 8, 2021 10:35 AM Well - the case here is that the file system has, let’s say, 100M files. Some percentage of these are sym-links to a location that’s not in this file system. I want a report of all these off file system links. However, not all of the sym-links off file system are of interest, just some of them. I can’t say for sure where in the file system they are (and I don’t care). Bob Oesterlin Sr Principal Storage Engineer, Nuance From: on behalf of Frederick Stock Reply-To: gpfsug main discussion list Date: Monday, March 8, 2021 at 9:29 AMTo: "gpfsug-discuss@spectrumscale.org" Cc: "gpfsug-discuss@spectrumscale.org" Subject: [EXTERNAL] Re: [gpfsug-discuss] Policy scan of symbolic links with contents? CAUTION: This Email is from an EXTERNAL source. Ensure you trust this sender before clicking on any links or attachments. Could you use the PATHNAME LIKE statement to limit the location to the files of interest? Fred___Fred Stock | Spectrum Scale Development Advocacy | 720-430-8821sto...@us.ibm.com - Original message -From: "Oesterlin, Robert" Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: gpfsug main discussion list Cc:Subject: [EXTERNAL] [gpfsug-discuss] Policy scan of symbolic links with contents?Date: Mon, Mar 8, 2021 10:12 AM Looking to craft a policy scan that pulls out symbolic links to a particular destination. For instance: file1.py -> /fs1/patha/pathb/file1.py (I want to include these) file2.py -> /fs2/patha/pathb/file2.py (exclude these) The easy way would be to pull out all sym-links and just grep for the ones I want but was hoping for a more elegant solution… Bob Oesterlin Sr Principal Storage Engineer, Nuance ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttps://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwICAg=jf_iaSHvJObTbx-siA1ZOg=p_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw=i6m1zVXf4peZo0yo02IiRaQ_pUX95MN3wU53M0NiWcI=z-ibh2kAPHbehAsrGavNIg2AJdXmHkpUwy5YhZfUbpc= ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttps://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwICAg=jf_iaSHvJObTbx-siA1ZOg=p_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw=oRbtYApZcD6DP-VZN9gW7dTGSZU6I4r9b_Q-nb9Xc7k=SvdmxtC5aLPE7HL4nkPNlImv4pzfOCwcTkVBQsrAo-Q= ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Policy scan of symbolic links with contents?
Well - the case here is that the file system has, let’s say, 100M files. Some percentage of these are sym-links to a location that’s not in this file system. I want a report of all these off file system links. However, not all of the sym-links off file system are of interest, just some of them. I can’t say for sure where in the file system they are (and I don’t care). Bob Oesterlin Sr Principal Storage Engineer, Nuance From: on behalf of Frederick Stock Reply-To: gpfsug main discussion list Date: Monday, March 8, 2021 at 9:29 AM To: "gpfsug-discuss@spectrumscale.org" Cc: "gpfsug-discuss@spectrumscale.org" Subject: [EXTERNAL] Re: [gpfsug-discuss] Policy scan of symbolic links with contents? CAUTION: This Email is from an EXTERNAL source. Ensure you trust this sender before clicking on any links or attachments. Could you use the PATHNAME LIKE statement to limit the location to the files of interest? Fred ___ Fred Stock | Spectrum Scale Development Advocacy | 720-430-8821 sto...@us.ibm.com - Original message - From: "Oesterlin, Robert" Sent by: gpfsug-discuss-boun...@spectrumscale.org To: gpfsug main discussion list Cc: Subject: [EXTERNAL] [gpfsug-discuss] Policy scan of symbolic links with contents? Date: Mon, Mar 8, 2021 10:12 AM Looking to craft a policy scan that pulls out symbolic links to a particular destination. For instance: file1.py -> /fs1/patha/pathb/file1.py (I want to include these) file2.py -> /fs2/patha/pathb/file2.py (exclude these) The easy way would be to pull out all sym-links and just grep for the ones I want but was hoping for a more elegant solution… Bob Oesterlin Sr Principal Storage Engineer, Nuance ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwICAg=jf_iaSHvJObTbx-siA1ZOg=p_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw=i6m1zVXf4peZo0yo02IiRaQ_pUX95MN3wU53M0NiWcI=z-ibh2kAPHbehAsrGavNIg2AJdXmHkpUwy5YhZfUbpc= ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Policy scan of symbolic links with contents?
Could you use the PATHNAME LIKE statement to limit the location to the files of interest? Fred___Fred Stock | Spectrum Scale Development Advocacy | 720-430-8821sto...@us.ibm.com - Original message -From: "Oesterlin, Robert" Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: gpfsug main discussion list Cc:Subject: [EXTERNAL] [gpfsug-discuss] Policy scan of symbolic links with contents?Date: Mon, Mar 8, 2021 10:12 AM Looking to craft a policy scan that pulls out symbolic links to a particular destination. For instance: file1.py -> /fs1/patha/pathb/file1.py (I want to include these) file2.py -> /fs2/patha/pathb/file2.py (exclude these) The easy way would be to pull out all sym-links and just grep for the ones I want but was hoping for a more elegant solution… Bob Oesterlin Sr Principal Storage Engineer, Nuance ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttps://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwICAg=jf_iaSHvJObTbx-siA1ZOg=p_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw=i6m1zVXf4peZo0yo02IiRaQ_pUX95MN3wU53M0NiWcI=z-ibh2kAPHbehAsrGavNIg2AJdXmHkpUwy5YhZfUbpc= ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
[gpfsug-discuss] Policy scan of symbolic links with contents?
Looking to craft a policy scan that pulls out symbolic links to a particular destination. For instance: file1.py -> /fs1/patha/pathb/file1.py (I want to include these) file2.py -> /fs2/patha/pathb/file2.py (exclude these) The easy way would be to pull out all sym-links and just grep for the ones I want but was hoping for a more elegant solution… Bob Oesterlin Sr Principal Storage Engineer, Nuance ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
