Hi Mark,
Just a follow up to your suggestion few months ago.
I finally got to a point where I do 2 independent backups of the same path to 2 servers, and they are pretty even, finishing within 4 hours each, when
serialized.
I now just would like to use one mmbackup instance to 2 servers at the same time, with the --tsm-servers option, however it's not being
accepted/recognized (see below).
So, what is the proper syntax for this option?
Thanks
Jaime
# /usr/lpp/mmfs/bin/mmbackup /gpfs/fs1/home -N tapenode3-ib ‐‐tsm‐servers TAPENODE3,TAPENODE4 -s /dev/shm --tsm-errorlog $tmpDir/home-tsm-errorlog
--scope inodespace -v -a 8 -L 2
mmbackup: Incorrect extra argument: ‐‐tsm‐servers
Usage:
mmbackup {Device | Directory} [-t {full | incremental}]
[-N {Node[,Node...] | NodeFile | NodeClass}]
[-g GlobalWorkDirectory] [-s LocalWorkDirectory]
[-S SnapshotName] [-f] [-q] [-v] [-d]
[-a IscanThreads] [-n DirThreadLevel]
[-m ExecThreads | [[--expire-threads ExpireThreads]
[--backup-threads BackupThreads]]]
[-B MaxFiles | [[--max-backup-count MaxBackupCount]
[--max-expire-count MaxExpireCount]]]
[--max-backup-size MaxBackupSize] [--qos QosClass] [--quote |
--noquote]
[--rebuild] [--scope {filesystem | inodespace}]
[--backup-migrated | --skip-migrated] [--tsm-servers
TSMServer[,TSMServer...]]
[--tsm-errorlog TSMErrorLogFile] [-L n] [-P PolicyFile]
Changing the order of the options/arguments makes no difference.
Even when I explicitly specify only one server, mmbackup still doesn't seem to
recognize the ‐‐tsm‐servers option (it thinks it's some kind of argument):
# /usr/lpp/mmfs/bin/mmbackup /gpfs/fs1/home -N tapenode3-ib ‐‐tsm‐servers TAPENODE3 -s /dev/shm --tsm-errorlog $tmpDir/home-tsm-errorlog --scope
inodespace -v -a 8 -L 2
mmbackup: Incorrect extra argument: ‐‐tsm‐servers
Usage:
mmbackup {Device | Directory} [-t {full | incremental}]
[-N {Node[,Node...] | NodeFile | NodeClass}]
[-g GlobalWorkDirectory] [-s LocalWorkDirectory]
[-S SnapshotName] [-f] [-q] [-v] [-d]
[-a IscanThreads] [-n DirThreadLevel]
[-m ExecThreads | [[--expire-threads ExpireThreads]
[--backup-threads BackupThreads]]]
[-B MaxFiles | [[--max-backup-count MaxBackupCount]
[--max-expire-count MaxExpireCount]]]
[--max-backup-size MaxBackupSize] [--qos QosClass] [--quote |
--noquote]
[--rebuild] [--scope {filesystem | inodespace}]
[--backup-migrated | --skip-migrated] [--tsm-servers
TSMServer[,TSMServer...]]
[--tsm-errorlog TSMErrorLogFile] [-L n] [-P PolicyFile]
I defined the 2 servers stanzas as follows:
# cat dsm.sys
SERVERNAME TAPENODE3
SCHEDMODE PROMPTED
ERRORLOGRETENTION 0 D
TCPSERVERADDRESS 10.20.205.51
NODENAME home
COMMMETHOD TCPIP
TCPPort 1500
PASSWORDACCESS GENERATE
TXNBYTELIMIT 1048576
SERVERNAME TAPENODE4
SCHEDMODE PROMPTED
ERRORLOGRETENTION 0 D
TCPSERVERADDRESS 192.168.94.128
NODENAME home
COMMMETHOD TCPIP
TCPPort 1500
PASSWORDACCESS GENERATE
TXNBYTELIMIT 1048576
TCPBuffsize 512
On 2019-11-03 8:56 p.m., Jaime Pinto wrote:
On 11/3/2019 20:24:35, Marc A Kaplan wrote:
Please show us the 2 or 3 mmbackup commands that you would like to run
concurrently.
Hey Marc,
They would be pretty similar, with the only different being the target TSM server, determined by sourcing a different dsmenv1(2 or 3) prior to the
start of each instance, each with its own dsm.sys (3 wrappers).
(source dsmenv1; /usr/lpp/mmfs/bin/mmbackup /gpfs/fs1/home -N tapenode3-ib -s /dev/shm --tsm-errorlog $tmpDir/home-tsm-errorlog -g
/gpfs/fs1/home/.mmbackupCfg1 --scope inodespace -v -a 8 -L 2)
(source dsmenv3; /usr/lpp/mmfs/bin/mmbackup /gpfs/fs1/home -N tapenode3-ib -s /dev/shm --tsm-errorlog $tmpDir/home-tsm-errorlog -g
/gpfs/fs1/home/.mmbackupCfg2 --scope inodespace -v -a 8 -L 2)
(source dsmenv3; /usr/lpp/mmfs/bin/mmbackup /gpfs/fs1/home -N tapenode3-ib -s /dev/shm --tsm-errorlog $tmpDir/home-tsm-errorlog -g
/gpfs/fs1/home/.mmbackupCfg3 --scope inodespace -v -a 8 -L 2)
I was playing with the -L (to control the policy), but you bring up a very good point I had not experimented with, such as a single traverse for
multiple target servers. It may be just what I need. I'll try this next.
Thank you very much,
Jaime
Peeking into the script, I find:
if [[ $scope == "inode-space" ]]
then
deviceSuffix="${deviceName}.${filesetName}"
else
deviceSuffix="${deviceName}"
I believe mmbackup is designed to allow concurrent backup of different
independent filesets within the same filesystem, Or different filesystems...
And a single mmbackup instance can drive several TSM servers, which can be
named with an option or in the dsm.sys file:
# --tsm-servers TSMserver[,TSMserver...]
# List of TSM servers to use instead of the servers in the dsm.sys file.
Inactive hide details for Jaime Pinto ---11/01/2019 07:40:47 PM---How can I force secondary processes to use the folder instrucJaime Pinto
---11/01/2019 07:40:47 PM---How can I force secondary processes to use the folder instructed by the -g option? I started a mmbac
From: Jaime Pinto <[email protected]>
To: gpfsug main discussion list <[email protected]>
Date: 11/01/2019 07:40 PM
Subject: [EXTERNAL] [gpfsug-discuss] mmbackup ‐g GlobalWorkDirectory not being
followed
Sent by: [email protected]
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
How can I force secondary processes to use the folder instructed by the -g
option?
I started a mmbackup with ‐g /gpfs/fs1/home/.mmbackupCfg1 and another with ‐g /gpfs/fs1/home/.mmbackupCfg2 (and another with ‐g
/gpfs/fs1/home/.mmbackupCfg3 ...)
However I'm still seeing transient files being worked into a "/gpfs/fs1/home/.mmbackupCfg" folder (created by magic !!!). This absolutely can not
happen, since it's mixing up workfiles from multiple mmbackup instances for different target TSM servers.
See below the "-f /gpfs/fs1/home/.mmbackupCfg/prepFiles" created by
mmapplypolicy (forked by mmbackup):
DEBUGtsbackup33: /usr/lpp/mmfs/bin/mmapplypolicy "/gpfs/fs1/home" -g /gpfs/fs1/home/.mmbackupCfg2 -N tapenode3-ib -s /dev/shm -L 2 --qos maintenance
-a 8 -P /var/mmfs/mmbackup/.mmbackupRules.fs1.home -I prepare -f /gpfs/fs1/home/.mmbackupCfg/prepFiles --irule0 --sort-buffer-size=5% --scope
inodespace
Basically, I don't want a "/gpfs/fs1/home/.mmbackupCfg" folder to ever exist. Otherwise I'll be forced to serialize these backups, to avoid the
different mmbackup instances tripping over each other. The serializing is very undesirable.
Thanks
Jaime
************************************
TELL US ABOUT YOUR SUCCESS STORIES
http://www.scinethpc.ca/testimonials
************************************
---
Jaime Pinto - Storage Analyst
SciNet HPC Consortium - Compute/Calcul Canada
www.scinet.utoronto.ca - www.computecanada.ca
University of Toronto
661 University Ave. (MaRS), Suite 1140
Toronto, ON, M5G1M1
P: 416-978-2755
C: 416-505-1477
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
