Re: Files not rebinding as expected
Hi Eric, The folder was backed up on 13/7 under management class with longest retention (BA7Y_L) since no DIRMC was configured. During backup on 14/7 folder was already removed from the server so the object was only inactivated. Rebinding of MC only occurs for existing files/folders. You most probably added DIRMC option *after* 14/July. If an existing folder was backed up with new DIRMC (BA14_L) option, also all previous versions of the folder were rebound from BA7Y_L to BA14_L. If you delete such folder, its inactive version will be stored under BA14_L. Martin Janosik -Original Message- From: ADSM: Dist Stor Manager On Behalf Of Loon, Eric van (ITOP DI) - KLM Sent: pondelok 8. augusta 2022 14:40 To: ADSM-L@VM.MARIST.EDU Subject: [EXTERNAL] Re: [ADSM-L] Files not rebinding as expected Hi Andy, I changed the includes accordingly, so the clopset now looks like this: DIRMC0 No BA14_L INCLEXCL 1 No include.backup d:\...\* BA14_L INCLEXCL 2 No include.backup c:\...\* BA14_L INCLEXCL 3 No INCLUDE.BACKUP d:\MSP\...\* BA7Y_L INCLEXCL 4 No INCLUDE.BACKUP d:\FTPDATA\sapdw\...\* BA7Y_L INCLEXCL 5 No INCLUDE.BACKUP d:\FTPDATA\workathome\...\* BA7Y_L The includes 1 to 5 seem to be working OK now, the only strange thing I can't explain is why several (but certainly not all) inactive directories on the C: drive are now bound to BA7Y_L too: > select * from backups where node_name='YS0023SA' and > object_id=3771909693 NODE_NAME: YS0023SA FILESPACE_NAME: \\ys0023sa\c$ FILESPACE_ID: 2 STATE: INACTIVE_VERSION TYPE: DIR HL_NAME: \PROGRAMDATA\MICROSOFT\WINDOWS\WER\REPORTQUEUE\ LL_NAME: NONCRITICAL_MICROSOFTEDGEUPD_CD9A261DDE7F7CA3A975FAD1C617B3157C4755__D3238B06 OBJECT_ID: 3771909693 BACKUP_DATE: 2022-07-13 23:19:26.00 DEACTIVATE_DATE: 2022-07-14 22:45:18.00 OWNER: CLASS_NAME: BA7Y_L ACTUAL_SIZE: 659 Thanks for your help! Kind regards, Eric van Loon Air France/KLM Core Infra -Original Message- From: ADSM: Dist Stor Manager On Behalf Of Andrew Raibeck Sent: woensdag 3 augustus 2022 22:05 To: ADSM-L@VM.MARIST.EDU Subject: Re: Files not rebinding as expected Hi Erik, Looking at your option set INCLEXCL statements, at least part of the binding problem looks to be the sequence of statements: INCLEXCL 0 Yes INCLUDE.BACKUP d:\MSP\...\*.* BA7Y_L INCLEXCL 1 Yes INCLUDE.BACKUP d:\FTPDATA\sapdw\...\*.* BA7Y_L INCLEXCL 2 Yes INCLUDE.BACKUP d:\FTPDATA\workathome\...\*.* BA7Y_L INCLEXCL 3 Yes include.backup d:\...\*.* BA14_L INCLEXCL 4 Yes include.backup c:\...\*.* BA14_L (note: The FORCE option is ignored for INCLEXCL.) The above include-exclude statements are processed from the bottom up, the same as if you put them in your dsm.opt file. For example, a file such as "D:\FTPDATA\sapdw\somefile.txt" matches the pattern "d:\...\*.*", so it is bound to BA14_L. You can fix this by resequencing the INCLEXCL options in the client option set. For example: update clientopt mpsserver inclexcl 0 5 update clientopt mpsserver inclexcl 1 6 update clientopt mpsserver inclexcl 2 7 Thus the new sequence is: include.backup d:\...\*.* BA14_L include.backup c:\...\*.* BA14_L INCLUDE.BACKUP d:\MSP\...\*.* BA7Y_L INCLUDE.BACKUP d:\FTPDATA\sapdw\...\*.* BA7Y_L INCLUDE.BACKUP d:\FTPDATA\workathome\...\*.* BA7Y_L Note that if you run "dsmc query inclexcl", the include-exclude list is displayed in inverted order (don't ask me why...), so patterns closer to the top of the displayed list are matched first. One more observation: The "*.*" pattern means "match any file name with a dot ( . ) character in the name. Thus files with no extension, though not so common on Windows, do not match the "*.*" pattern. In general, it is best to use "*" to refer to any file name... unless you really mean to include only files that have a dot in the name. To augment this comment from Josh-Daniel Davis: > * Directories always are bound to the longest retention management class > unless you use DIRMC to override it. Specifically, it is the management class with the longest RETONLY setting. If more than one management class has the same (highest) RETONLY setting, then the management class name that is last in ascending sort order is the default DIRMC. For example, if you have two management classes named DISK and TAPE, and they both have the same highest RETONLY setting of all the management classes, then the default DIRMC is TAPE. You probably are not doing this, but I will mention it anyway, for completeness: When
Re: Files not rebinding as expected
Hi Erik, Looking at your option set INCLEXCL statements, at least part of the binding problem looks to be the sequence of statements: INCLEXCL 0 Yes INCLUDE.BACKUP d:\MSP\...\*.* BA7Y_L INCLEXCL 1 Yes INCLUDE.BACKUP d:\FTPDATA\sapdw\...\*.* BA7Y_L INCLEXCL 2 Yes INCLUDE.BACKUP d:\FTPDATA\workathome\...\*.* BA7Y_L INCLEXCL 3 Yes include.backup d:\...\*.* BA14_L INCLEXCL 4 Yes include.backup c:\...\*.* BA14_L (note: The FORCE option is ignored for INCLEXCL.) The above include-exclude statements are processed from the bottom up, the same as if you put them in your dsm.opt file. For example, a file such as "D:\FTPDATA\sapdw\somefile.txt" matches the pattern "d:\...\*.*", so it is bound to BA14_L. You can fix this by resequencing the INCLEXCL options in the client option set. For example: update clientopt mpsserver inclexcl 0 5 update clientopt mpsserver inclexcl 1 6 update clientopt mpsserver inclexcl 2 7 Thus the new sequence is: include.backup d:\...\*.* BA14_L include.backup c:\...\*.* BA14_L INCLUDE.BACKUP d:\MSP\...\*.* BA7Y_L INCLUDE.BACKUP d:\FTPDATA\sapdw\...\*.* BA7Y_L INCLUDE.BACKUP d:\FTPDATA\workathome\...\*.* BA7Y_L Note that if you run "dsmc query inclexcl", the include-exclude list is displayed in inverted order (don't ask me why...), so patterns closer to the top of the displayed list are matched first. One more observation: The "*.*" pattern means "match any file name with a dot ( . ) character in the name. Thus files with no extension, though not so common on Windows, do not match the "*.*" pattern. In general, it is best to use "*" to refer to any file name... unless you really mean to include only files that have a dot in the name. To augment this comment from Josh-Daniel Davis: > * Directories always are bound to the longest retention management class > unless you use DIRMC to override it. Specifically, it is the management class with the longest RETONLY setting. If more than one management class has the same (highest) RETONLY setting, then the management class name that is last in ascending sort order is the default DIRMC. For example, if you have two management classes named DISK and TAPE, and they both have the same highest RETONLY setting of all the management classes, then the default DIRMC is TAPE. You probably are not doing this, but I will mention it anyway, for completeness: When you do incremental backups, do NOT do this: dsmc incremental c:\* -subdir=yes Instead, use the idiomatic file system specification for full incremental: dsmc incremental c: Regards, Andy Andrew Raibeck IBM Spectrum Protect Level 3 IBM Systems, Storage stor...@us.ibm.com IBM -Original Message- From: ADSM: Dist Stor Manager On Behalf Of Josh-Daniel Davis Sent: Wednesday, 3 August, 2022 14:57 To: ADSM-L@VM.MARIST.EDU Subject: [EXTERNAL] Re: Files not rebinding as expected For windows systems, the filespaces are not stored as C: and D:. They are stored as UNC names such as \\hostname\c$. The drawback here is that if the hostname ever got updated, you could have C: backed up two or more times. Q OCC will tell you what to expect. When you run an incremental, it will not update filespaces which do not exist at the time of the incremental. That includes mismatched UNC names. As to the binding: * UPD NODE will not change the management class assigned to any backup objects. * If an object belongs to a management class that exists in both policy domains, then EXPIRE INV will work as expected, using the new retention plan. * If an object belongs to a management class that does not exist in the target policy domain's ACTIVE policyset, then EXPIRE will process as DEFAULT, and the next incremental will rebind the objects to the default management class. * If the default management class is missing a copygroup, then the rendition grace period will be used. * Directories always are bound to the longest retention management class unless you use DIRMC to override it. * If client-side inclexcl is used, that can override server-side depending on settings. * If the node does not have the client option set assigned, it will not be used. * ARCHIVE data is never rebound. If the target is missing, then you get DEFAULT. If default is missing you get the grace period. * TOCDEST is used for nas (NDMP) and , snapmirror, and ndmp backups. If this does not explain it, then it would help to get Q OCC and also a few records out of the backups table for examples to what you're seeing. Also, the default line wrap is difficult to read. Please post -tabdelimit or data that is in stanza format (one field per line) for readability. Please do not omit ACTIVE when doing Q CO. If you did not, then did you forgot to ACTIVATE POLICYSET? With friendly Regards, Josh-Daniel S. Davis On Wed, Aug 3, 2022 at 3:28 AM Loon,
Re: Files not rebinding as expected
For windows systems, the filespaces are not stored as C: and D:. They are stored as UNC names such as \\hostname\c$. The drawback here is that if the hostname ever got updated, you could have C: backed up two or more times. Q OCC will tell you what to expect. When you run an incremental, it will not update filespaces which do not exist at the time of the incremental. That includes mismatched UNC names. As to the binding: * UPD NODE will not change the management class assigned to any backup objects. * If an object belongs to a management class that exists in both policy domains, then EXPIRE INV will work as expected, using the new retention plan. * If an object belongs to a management class that does not exist in the target policy domain's ACTIVE policyset, then EXPIRE will process as DEFAULT, and the next incremental will rebind the objects to the default management class. * If the default management class is missing a copygroup, then the rendition grace period will be used. * Directories always are bound to the longest retention management class unless you use DIRMC to override it. * If client-side inclexcl is used, that can override server-side depending on settings. * If the node does not have the client option set assigned, it will not be used. * ARCHIVE data is never rebound. If the target is missing, then you get DEFAULT. If default is missing you get the grace period. * TOCDEST is used for nas (NDMP) and , snapmirror, and ndmp backups. If this does not explain it, then it would help to get Q OCC and also a few records out of the backups table for examples to what you're seeing. Also, the default line wrap is difficult to read. Please post -tabdelimit or data that is in stanza format (one field per line) for readability. Please do not omit ACTIVE when doing Q CO. If you did not, then did you forgot to ACTIVATE POLICYSET? With friendly Regards, Josh-Daniel S. Davis On Wed, Aug 3, 2022 at 3:28 AM Loon, Eric van (ITOP DI) - KLM < eric-van.l...@klm.com> wrote: > Hi Martin, > > The strange thing is that I also see active files still bound to the > no-more-existing-in-new-domain management class... That should not be > possible, I think. We are running full incremental backups only. > I think I will open a case for it, I'll keep you all posted on the outcome. > > Kind regards, > Eric van Loon > Air France/KLM Core Infra > > -Original Message- > From: ADSM: Dist Stor Manager On Behalf Of Martin > Janosik > Sent: dinsdag 2 augustus 2022 13:18 > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Files not rebinding as expected > > Hi Eric, > > from my observation updating node to new policy domain is instantaneous no > matter how many files are stored under the node. There is no way that > during this instant operation files are re-bound from > no-more-existing-in-new-domain management class to new domain's default > management class. That would explains question #3. > > Also: > > if a user changes the binding in an include statement, then instead of > > running a full incremental backup, only selective or > > incremental-by-date backups are performed, then only active versions > > will be rebound, while inactive versions will remain bound to the > original management class. > > If you run anything else than full incremental (in this case `dsmc incr C: > D:`) then only newly backed up files are stored under new management class > > Expire inventory goes through all inactive files and if their MC matches > with the MC defined in the policy domain, it expires data accordingly. > If there is no matching MC present, objects will expire based on default > management class (or grace retention period). > > Is the destination by any chance a directory pool? I *think* that deduped > objects (=those referenced by multiple backup versions) are not necessarily > re-bound. > > Martin Janosik > > -Original Message- > From: ADSM: Dist Stor Manager On Behalf Of Loon, > Eric van (ITOP DI) - KLM > Sent: utorok 2. augusta 2022 12:42 > To: ADSM-L@VM.MARIST.EDU > Subject: [EXTERNAL] Re: [ADSM-L] Files not rebinding as expected > > Hi Skylar, > > Yes, this node is backed up every day... > > Kind regards, > Eric van Loon > Core Infra > > -Original Message- > From: ADSM: Dist Stor Manager On Behalf Of Skylar > Thompson > Sent: maandag 1 augustus 2022 17:43 > To: ADSM-L@VM.MARIST.EDU > Subject: Re: Files not rebinding as expected > > Hi Eric, > > Have you run an incremental backup yet? You'll need to do that in order to > get active versions rebound to the management classes in the nodes > include/exclude rules. > > Inactive versions will stay bound to their old management class but, in > the event of a domain change, I think will use either a manageme
Re: Files not rebinding as expected
Hi Eric, from my observation updating node to new policy domain is instantaneous no matter how many files are stored under the node. There is no way that during this instant operation files are re-bound from no-more-existing-in-new-domain management class to new domain's default management class. That would explains question #3. Also: > if a user changes the binding in an include statement, then instead of > running a full incremental backup, only selective or incremental-by-date > backups > are performed, then only active versions will be rebound, while inactive > versions > will remain bound to the original management class. If you run anything else than full incremental (in this case `dsmc incr C: D:`) then only newly backed up files are stored under new management class Expire inventory goes through all inactive files and if their MC matches with the MC defined in the policy domain, it expires data accordingly. If there is no matching MC present, objects will expire based on default management class (or grace retention period). Is the destination by any chance a directory pool? I *think* that deduped objects (=those referenced by multiple backup versions) are not necessarily re-bound. Martin Janosik -Original Message- From: ADSM: Dist Stor Manager On Behalf Of Loon, Eric van (ITOP DI) - KLM Sent: utorok 2. augusta 2022 12:42 To: ADSM-L@VM.MARIST.EDU Subject: [EXTERNAL] Re: [ADSM-L] Files not rebinding as expected Hi Skylar, Yes, this node is backed up every day... Kind regards, Eric van Loon Core Infra -Original Message- From: ADSM: Dist Stor Manager On Behalf Of Skylar Thompson Sent: maandag 1 augustus 2022 17:43 To: ADSM-L@VM.MARIST.EDU Subject: Re: Files not rebinding as expected Hi Eric, Have you run an incremental backup yet? You'll need to do that in order to get active versions rebound to the management classes in the nodes include/exclude rules. Inactive versions will stay bound to their old management class but, in the event of a domain change, I think will use either a management class with the same name in the new domain/policyset, or the domain default management class if that management class doesn't exist. I'm not sure how that would be reported with SQL or QUERY BACKUP, though. On Mon, Aug 01, 2022 at 11:51:08AM +, Loon, Eric van (ITOP DI) - KLM wrote: > Hi everybody, > > I have a node called YS0023SA, which uses domain MPS_7_YEARS and client > option set MPSSERVER. I listed all policy details down below. > I issued a select * from backups for this node and for some reason, all files > are either bound to mgmtclass DEFAULT, bound to BA14_L (which is the > default), or to TOC, which doesn't even exist in this domain (it does exist > in the node's previous domain). > Can anybody explain: > > 1. Why are files using both DEFAULT and BA14_L? > 2. Why is the include.backup for the other mgmtclass BA7Y_L in the > cloptset not working? > 3. Why are some files (active and inactive files) still bound to a > management class from the previous domain? > Thanks for any help in advance! > > Kind regards, > Eric van Loon > > > > Domain MPS_7_YEARS: > Policy Domain Name: MPS_7_YEARS > Activated Policy Set: MPS_7_YEARS > Activation Date/Time: 19-07-2022 12:04:05 >Days Since Activation: 13 > Activated Default Mgmt Class: BA14_L > Number of Registered Nodes: 1 > Description: Domain for MPS server Backup > Retention (Grace Period): 14 Archive Retention (Grace Period): 14 > Last Update by (administrator): >Last Update Date/Time: 19-07-2022 12:04:05 > Managing profile: > Changes Pending: No >Active Data Pool List: > Domain Type: Standard > > Policy set MPS_7_YEARS: > PolicyPolicyDefault Description Last Update by Last > Upda- Managing profile Changes > DomainSet Name Mgmt (administrator) te > Date/T- Pending > NameClass ime > Name > - - - --- > -- -- > MPS_7_YE- ACTIVEBA14_LPolicy set for MPS > 19-07-202- No > ARS server 2 > 11:46:- > > 26 > > MGMTClasses: > PolicyPolicyMgmt DefaultDescription > DomainSet Name Class Mgmt Clas- > NameName
Re: Files not rebinding as expected
Hi Eric, Have you run an incremental backup yet? You'll need to do that in order to get active versions rebound to the management classes in the nodes include/exclude rules. Inactive versions will stay bound to their old management class but, in the event of a domain change, I think will use either a management class with the same name in the new domain/policyset, or the domain default management class if that management class doesn't exist. I'm not sure how that would be reported with SQL or QUERY BACKUP, though. On Mon, Aug 01, 2022 at 11:51:08AM +, Loon, Eric van (ITOP DI) - KLM wrote: > Hi everybody, > > I have a node called YS0023SA, which uses domain MPS_7_YEARS and client > option set MPSSERVER. I listed all policy details down below. > I issued a select * from backups for this node and for some reason, all files > are either bound to mgmtclass DEFAULT, bound to BA14_L (which is the > default), or to TOC, which doesn't even exist in this domain (it does exist > in the node's previous domain). > Can anybody explain: > > 1. Why are files using both DEFAULT and BA14_L? > 2. Why is the include.backup for the other mgmtclass BA7Y_L in the > cloptset not working? > 3. Why are some files (active and inactive files) still bound to a > management class from the previous domain? > Thanks for any help in advance! > > Kind regards, > Eric van Loon > > > > Domain MPS_7_YEARS: > Policy Domain Name: MPS_7_YEARS > Activated Policy Set: MPS_7_YEARS > Activation Date/Time: 19-07-2022 12:04:05 >Days Since Activation: 13 > Activated Default Mgmt Class: BA14_L > Number of Registered Nodes: 1 > Description: Domain for MPS server > Backup Retention (Grace Period): 14 > Archive Retention (Grace Period): 14 > Last Update by (administrator): >Last Update Date/Time: 19-07-2022 12:04:05 > Managing profile: > Changes Pending: No >Active Data Pool List: > Domain Type: Standard > > Policy set MPS_7_YEARS: > PolicyPolicyDefault Description Last Update by Last > Upda- Managing profile Changes > DomainSet Name Mgmt (administrator) te > Date/T- Pending > NameClass ime > Name > - - - --- > -- -- > MPS_7_YE- ACTIVEBA14_LPolicy set for MPS > 19-07-202- No > ARS server 2 > 11:46:- >26 > > MGMTClasses: > PolicyPolicyMgmt DefaultDescription > DomainSet Name Class Mgmt Clas- > NameName s ? > - - - -- > > MPS_7_YE- MPS_7_YE- BA14_LYesManagement Class for > ARS ARSMPS server 14 days > backup > MPS_7_YE- MPS_7_YE- BA7Y_LNo Management Class for > ARS ARSMPS server 7 years > backup > > Cloptset: > OptionsetDescription Last Update by > Managing profileReplica Option > (administrator) >Set > ----- > --- > MPSSERVEROptionset for the MPS >No > server > > OptionSequenc- Use Optio- Option Value > e number n Set >Value > (FORCE) > - -- > > DIRMC5Yes BA14_L > INCLEXCL 0Yes INCLUDE.BACKUP > d:\MSP\...\*.* BA7Y_L > INCLEXCL 1Yes INCLUDE.BACKUP > d:\FTPDATA\sapdw\...\*.* BA7Y_L > INCLEXCL 2Yes INCLUDE.BACKUP > d:\FTPDATA\workathome\...\*.* BA7Y_L > INCLEXCL 3Yes include.backup > d:\...\*.* BA14_L > INCLEXCL 4
