Re: [gpfsug-discuss] GPFS de duplication
Do file clones meet the workflow requirement? That is, can you control from whence the second (and further) copies are made? -- Stephen > On May 20, 2021, at 9:01 AM, Andrew Beattie wrote: > > Dave, > > Spectrum Scale does not support de-duplication, it does support compression. > You can however use block storage that supports over subscription / thin > provisioning / deduplication for data only NSD’s, we do not recommend them > for metadata. > > In your scenario is user B planning on making changes to the data which is > why you need a copy? > > I know of customers that do this regularly with block storage such as the IBM > Flashsystem product family In conjunction with IBM Spectrum Copy Data > Management. But I don’t believe CDM supports file based storage. > > Regards, > > Andrew Beattie > Technical Sales Specialist > Storage for Data and AI > IBM Australia and New Zealand > P. +61 421 337 927 > E. abeat...@au1.ibm.com > >>> On 20 May 2021, at 22:58, Dave Bond wrote: >>> >> >> >> >> Hello >> >> As part of a project I am doing I am looking if there are any de duplication >> options for GPFS? I see there is no native de dupe for the filesystem. The >> scenario would be user A creates a file or folder and user B takes a copy >> within the same filesystem, though separate independent filesets. The >> intention would be to store 1 copy.So I was wondering >> >> 1) Is this is planned to be implemented into GPFS in the future? >> 2) Is anyone is using any other solutions that have good GPFS integration? >> >> Dave > > ___ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] FAL audit events not always triggered
Hello Dave, This seems like unexpected behavior, especially the part where you need to disable/re-enable FAL to see the records.To answer your question Is it intended for the audit mechanism to be a soft audit? i.e the audit is best effort, and non blocking for file access. >>> file audit logging is indeed best effort. We buffer the records in memory and write them out at a later time, therefore non blocking for file accesses. You can verify if any records have been dropped with the "mmdiag --eventproducer" command. This command is per node, and will show if audit messages were dropped in the "Msg failed: " field. Even though audit is enabled on your NSD node, the client nodes where the file accesses happen do the buffering/writing. So make sure to runn the mmdiag --eventproducer command from the node where the file accesses are happening. Thanks, Luis I. Teran Software DeveloperScalable Data InterfacesIBM Spectrum Scaleluis.i.te...@ibm.com+1-520 799 2346 ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] GPFS de duplication
Dave, Spectrum Scale does not support de-duplication, it does support compression. You can however use block storage that supports over subscription / thin provisioning / deduplication for data only NSD’s, we do not recommend them for metadata. In your scenario is user B planning on making changes to the data which is why you need a copy? I know of customers that do this regularly with block storage such as the IBM Flashsystem product family In conjunction with IBM Spectrum Copy Data Management. But I don’t believe CDM supports file based storage. Regards, Andrew Beattie Technical Sales Specialist Storage for Data and AI IBM Australia and New Zealand P. +61 421 337 927 E. abeat...@au1.ibm.com > On 20 May 2021, at 22:58, Dave Bond wrote: > > > This Message Is From an External Sender > This message came from outside your organization. > > > Hello > > As part of a project I am doing I am looking if there are any de duplication > options for GPFS? I see there is no native de dupe for the filesystem. The > scenario would be user A creates a file or folder and user B takes a copy > within the same filesystem, though separate independent filesets. The > intention would be to store 1 copy.So I was wondering > > 1) Is this is planned to be implemented into GPFS in the future? > 2) Is anyone is using any other solutions that have good GPFS integration? > > Dave ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
[gpfsug-discuss] GPFS de duplication
Hello As part of a project I am doing I am looking if there are any de duplication options for GPFS? I see there is no native de dupe for the filesystem. The scenario would be user A creates a file or folder and user B takes a copy within the same filesystem, though separate independent filesets. The intention would be to store 1 copy.So I was wondering 1) Is this is planned to be implemented into GPFS in the future? 2) Is anyone is using any other solutions that have good GPFS integration? Dave ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
[gpfsug-discuss] FAL audit events not always triggered
Hello I am currently testing out the FAL and watch folders auditing mechanism in 5.1.0. I have noticed that audit events such as file access or creation are not always recorded in the FAL log on the filesystem. It would seem necessary to disable and re enable FAL auditing for the same file access to be recorded. I have only triggered this condition twice so far, but it has set in my mind a few questions. 1) Have others seen this? If so what is the trigger, as I do not yet have a reproducer. 2) Is it intended for the audit mechanism to be a soft audit? i.e the audit is best effort, and non blocking for file access. This is using a single NSD and a single remote cluster doing the file writing. FAL is running on the NSD node. Regards Dave ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
