Hi Eric,
Thanks for pointing out that doc issue. I am looking into getting that
resolved. It is correct that the EXCLUDE (without the .DIR) and INCLUDE operate
on file objects only, not directories.
I agree that the approach you are using is probably the easiest if the backup
of undesired directory objects is only an annoyance. The alternative I
mentioned might be more viable if there were a huge number, say, tens of
thousands or more, of directories you did not want to back up, or if traversing
the directories you did not want to back up was too time consuming.
Best regards,
Andy
Andrew Raibeck
IBM Storage Protect Level 3
stor...@us.ibm.com
From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> on behalf of Loon, Eric
van (ITOP DI) - KLM <eric-van.l...@klm.com>
Date: Friday, March 8, 2024 at 09:50
To: ADSM-L@VM.MARIST.EDU <ADSM-L@VM.MARIST.EDU>
Subject: [EXTERNAL] Re: Fiddling with exclude statement
Hi Andy,
I have been able to get the job done through the following lines in the
cloptset:
Use Option Set Value (FORCE): No
Option Value: EXCLUDE '/tech/splunk/.../*'
Option: INCLEXCL
Sequence number: 105
Use Option Set Value (FORCE): No
Option Value: INCLUDE '/tech/splunk/etc/.../*'
Option: INCLEXCL
Sequence number: 106
Use Option Set Value (FORCE): No
Option Value: INCLUDE '/tech/splunk/var/.../*'
Option: INCLEXCL
Sequence number: 107
Use Option Set Value (FORCE): No
Option Value: EXCLUDE.DIR '/tech/splunk/var/run'
This still causes the subdirectories to be backed up in /tech/splunk (except
for /tech/splunk/var/run and everything below), but I think I can live with
that. I was put on the wrong foot by one of the examples in the manual:
https://www.ibm.com/docs/en/storage-protect/8.1.21?topic=options-examples-using-wildcards-include-exclude-patterns.
In table 1 it states:
EXCLUDE /.../Documents/.../*
INCLUDE "/home/aleko/Documents/
Current"
Exclude all files and directories under any Documents directory that might
exist, except for the Current file of user aleko.
In the example above, all files are excluded, NOT the subdirectories. So, I
guess the manual is wrong here?
Thanks for your work-around. I like the idea, but the only downside would be
that the restore command syntax changes, which will most likely be very
confusing for the end user.
Kind regards,
Eric van Loon
Air France/KLM Core Infra
________________________________
From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> on behalf of Andrew
Raibeck <stor...@us.ibm.com>
Sent: Friday, March 8, 2024 13:35
To: ADSM-L@VM.MARIST.EDU <ADSM-L@VM.MARIST.EDU>
Subject: Re: Fiddling with exclude statement
Hi Eric,
Based on your previous response to the exclude.dir suggestion, I suppose you
have a scenario like this:
* Exclude all other files underneath /tech/splunk,
* but directory objects are still processed.
exclude /tech/splunk/.../*
* Back up everything in /tech/splunk/keep1 and /tech/splunk/keep2
include /tech/splunk/keep1/.../*
include /tech/splunk/keep2/.../*
In this case, you cannot avoid scanning and backing up all subdirectories of
/tech/splunk.
Another alternative is to consider using the VIRTUALMOUNTPOINT
(https://www.ibm.com/docs/en/storage-protect/8.1.21?topic=reference-virtualmountpoint)
option in dsm.sys, like this:
...
virtualmountpoint /tech/splunk
virtualmountpoint /tech/splunk/keep1
virtualmountpoint /tech/splunk/keep2
exclude.fs /tech/splunk
...
The virtual mount points are effectively treated like separate file systems.
The exclude.fs then prevents backup of everything in /tech/splunk that is NOT
part of the other virtual mount points.
When using the command line client for queries and restores, you might need to
put curly braces '{' and '}' around the file system name to avoid ambiguity in
file specifications. For example, if you want to search for a file named
myfile.txt, but you are not sure in which directory the file resides, then you
will have to do this:
dsmc query {/tech/splunk/
Also, if you want to restore everything you backed up under /tech/splunk, you
cannot do this:
dsmc restore /tech/splunk/ -subdir=yes
Instead, you need to do this:
dsmc restore {/tech/splunk/keep1} -subdir=yes
dsmc restore {/tech/splunk/keep2} -subdir=yes
I suggest trying this concept in a test environment first, to see if it is
feasible for you.
Regards,
Andy
Andrew Raibeck
IBM Storage Protect Level 3
stor...@us.ibm.com
-----Original Message-----
From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> On Behalf Of Loon, Eric
van (ITOP DI) - KLM
Sent: Tuesday, 27 February, 2024 08:55
To: ADSM-L@VM.MARIST.EDU
Subject: [EXTERNAL] Re: Fiddling with exclude statement
Hi Skylar,
Yes, we have a DIRMC set in the client option file. I removed it, just to check
whether it makes any difference, but the subdirectories are still being backed
up, with the only difference that they are bound to the longest retention
mgmtclass ofcourse.
Thanks for your help.
Kind regards,
Eric van Loon
Air France/KLM Core Infra
________________________________
From: ADSM: Dist Stor Manager <ADSM-L@VM.MARIST.EDU> on behalf of Skylar
Thompson <skyl...@uw.edu>
Sent: Monday, February 26, 2024 17:07
To: ADSM-L@VM.MARIST.EDU <ADSM-L@VM.MARIST.EDU>
Subject: Re: Fiddling with exclude statement
Do you have DIRMC set? My experience is that supersedes any normal exclude
rules, though would only backup the directories and not their contents. I think
exclude.dir would work to exclude that path and avoid the directories being
backed up, though.
On Mon, Feb 26, 2024 at 03:59:47PM +0000, Loon, Eric van (ITOP DI) - KLM wrote:
> Hi everybody,
>
> I'm trying to exclude a directory called /tech/splunk (including all
> subdirectories) through a client option set. The exclude on the client seem
> to be processed OK:
>
> [root@hostname ~]# dsmc q inclexcl|grep splunk
> Exclude All /tech/splunk/.../* Server
>
> But when I run an incremental on /tech, I do see subdirectories being backed
> up:
>
> Incremental backup of volume '/tech'
> Directory--> 1,024 /tech/splunk/etc [Sent]
> Directory--> 1,024 /tech/splunk/var [Sent]
> Directory--> 1,024 /tech/splunk/var/run [Sent]
> Successful incremental backup of '/tech'
>
> I don't understand why, what am I doing wrong here? Thanks for any help in
> advance!
>
>
> Kind regards,
>
> Eric van Loon
>
> Air France/KLM Core Infra
> ********************************************************
> For information, services and offers, please visit our web site:
> http://www.klm.com <http://www.klm.com > . This e-mail and any attachment may
> contain confidential and privileged material intended for the addressee only.
> If you are not the addressee, you are notified that no part of the e-mail or
> any attachment may be disclosed, copied or distributed, and that any other
> action related to this e-mail or attachment is strictly prohibited, and may
> be unlawful. If you have received this e-mail by error, please notify the
> sender immediately by return e-mail, and delete this message.
>
> Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its
> employees shall not be liable for the incorrect or incomplete transmission of
> this e-mail or any attachments, nor responsible for any delay in receipt.
> Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal
> Dutch Airlines) is registered in Amstelveen, The Netherlands, with
> registered number 33014286
> ********************************************************
--
-- Skylar Thompson (skyl...@u.washington.edu)
-- Genome Sciences Department (UW Medicine), System Administrator
-- Foege Building S046, (206)-685-7354
-- Pronouns: He/Him/His
********************************************************
For information, services and offers, please visit our web site:
http://www.klm.com . This e-mail and any attachment may contain confidential
and privileged material intended for the addressee only. If you are not the
addressee, you are notified that no part of the e-mail or any attachment may be
disclosed, copied or distributed, and that any other action related to this
e-mail or attachment is strictly prohibited, and may be unlawful. If you have
received this e-mail by error, please notify the sender immediately by return
e-mail, and delete this message.
Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its
employees shall not be liable for the incorrect or incomplete transmission of
this e-mail or any attachments, nor responsible for any delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch
Airlines) is registered in Amstelveen, The Netherlands, with registered number
33014286
********************************************************
********************************************************
For information, services and offers, please visit our web site:
http://www.klm.com . This e-mail and any attachment may contain confidential
and privileged material intended for the addressee only. If you are not the
addressee, you are notified that no part of the e-mail or any attachment may be
disclosed, copied or distributed, and that any other action related to this
e-mail or attachment is strictly prohibited, and may be unlawful. If you have
received this e-mail by error, please notify the sender immediately by return
e-mail, and delete this message.
Koninklijke Luchtvaart Maatschappij NV (KLM), its subsidiaries and/or its
employees shall not be liable for the incorrect or incomplete transmission of
this e-mail or any attachments, nor responsible for any delay in receipt.
Koninklijke Luchtvaart Maatschappij N.V. (also known as KLM Royal Dutch
Airlines) is registered in Amstelveen, The Netherlands, with registered number
33014286
********************************************************
