Spectrum Protect backup (under the hood of mmbackup) and Spectrum
Protect for Space Management (HSM) can be combined on the same data.
There are some valuable integration topics between the products that
can reduce the overall network traffic if using backup and HSM on the
same files. With the combination of the products you have the ability
to free file system space from cold data and migrate them out to tape
and to have several versions of frequently used files in backup in the
same file system.
Greetings, Dominic.
______________________________________________________________________________________________________________
Dominic Mueller-Wicke | IBM Spectrum Protect Development | Technical
Lead | +49 7034 64 32794 | [email protected]
Vorsitzende des Aufsichtsrats: Martina Koederitz; Geschäftsführung:
Dirk Wittkopp
Sitz der Gesellschaft: Böblingen; Registergericht: Amtsgericht
Stuttgart, HRB 243294
Inactive hide details for gpfsug-discuss-request---11.04.2016
17:11:55---Send gpfsug-discuss mailing list submissions to
gpfsugpfsug-discuss-request---11.04.2016 17:11:55---Send
gpfsug-discuss mailing list submissions to
[email protected]
From: [email protected]
To: [email protected]
Date: 11.04.2016 17:11
Subject: gpfsug-discuss Digest, Vol 51, Issue 9
Sent by: [email protected]
------------------------------------------------------------------------
Send gpfsug-discuss mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of gpfsug-discuss digest..."
Today's Topics:
1. backup and disaster recovery solutions (Damir Krstic)
2. Re: backup and disaster recovery solutions (Jaime Pinto)
3. Re: backup and disaster recovery solutions (Jonathan Buzzard)
4. Re: backup and disaster recovery solutions (Marc A Kaplan)
----- Message from Damir Krstic <[email protected]> on Mon, 11
Apr 2016 13:15:30 +0000 -----
*To:*
gpfsug main discussion list <[email protected]>
*Subject:*
[gpfsug-discuss] backup and disaster recovery solutions
We have implemented 1.5PB ESS solution recently in our HPC
environment. Today we are kicking of backup and disaster recovery
discussions so I was wondering what everyone else is using for their
backup?
In our old storage environment we simply rsync-ed home and software
directories and projects were not backed up.
With ESS we are looking for more of a GPFS based backup solution -
something to tape possibly and also something that will have life
cycle feature - so if the file is not touched for number of days, it's
moved to a tape (something like LTFS).
Thanks in advance.
DAmir
----- Message from Jaime Pinto <[email protected]> on Mon, 11
Apr 2016 10:34:54 -0400 -----
*To:*
gpfsug main discussion list <[email protected]>, Damir
Krstic <[email protected]>
*Subject:*
Re: [gpfsug-discuss] backup and disaster recovery solutions
Do you want backups or periodic frozen snapshots of the file system?
Backups can entail some level of version control, so that you or
end-users can get files back on certain points in time, in case of
accidental deletions. Besides 1.5PB is a lot of material, so you may
not want to take full snapshots that often. In that case, a
combination of daily incremental backups using TSM with GPFS's
mmbackup can be a good option. TSM also does a very good job at
controlling how material is distributed across multiple tapes, and
that is something that requires a lot of micro-management if you want
a home grown solution of rsync+LTFS.
On the other hand, you could use gpfs built-in tools such a
mmapplypolicy to identify candidates for incremental backup, and send
them to LTFS. Just more micro management, and you may have to come up
with your own tool to let end-users restore their stuff, or you'll
have to act on their behalf.
Jaime
Quoting Damir Krstic <[email protected]>:
> We have implemented 1.5PB ESS solution recently in our HPC environment.
> Today we are kicking of backup and disaster recovery discussions so
I was
> wondering what everyone else is using for their backup?
>
> In our old storage environment we simply rsync-ed home and software
> directories and projects were not backed up.
>
> With ESS we are looking for more of a GPFS based backup solution -
> something to tape possibly and also something that will have life cycle
> feature - so if the file is not touched for number of days, it's
moved to a
> tape (something like LTFS).
>
> Thanks in advance.
>
> DAmir
>
************************************
TELL US ABOUT YOUR SUCCESS STORIES
http://www.scinethpc.ca/testimonials
************************************
---
Jaime Pinto
SciNet HPC Consortium - Compute/Calcul Canada
www.scinet.utoronto.ca - www.computecanada.org
University of Toronto
256 McCaul Street, Room 235
Toronto, ON, M5T1W5
P: 416-978-2755
C: 416-505-1477
----------------------------------------------------------------
This message was sent using IMP at SciNet Consortium, University of
Toronto.
----- Message from Jonathan Buzzard <[email protected]> on Mon,
11 Apr 2016 16:02:45 +0100 -----
*To:*
[email protected]
*Subject:*
Re: [gpfsug-discuss] backup and disaster recovery solutions
On Mon, 2016-04-11 at 10:34 -0400, Jaime Pinto wrote:
> Do you want backups or periodic frozen snapshots of the file system?
>
> Backups can entail some level of version control, so that you or
> end-users can get files back on certain points in time, in case of
> accidental deletions. Besides 1.5PB is a lot of material, so you may
> not want to take full snapshots that often. In that case, a
> combination of daily incremental backups using TSM with GPFS's
> mmbackup can be a good option. TSM also does a very good job at
> controlling how material is distributed across multiple tapes, and
> that is something that requires a lot of micro-management if you want
> a home grown solution of rsync+LTFS.
Is there any other viable option other than TSM for backing up 1.5PB of
data? All other backup software does not handle this at all well.
> On the other hand, you could use gpfs built-in tools such a
> mmapplypolicy to identify candidates for incremental backup, and send
> them to LTFS. Just more micro management, and you may have to come up
> with your own tool to let end-users restore their stuff, or you'll
> have to act on their behalf.
>
I was not aware of a way of letting end users restore their stuff from
*backup* for any of the major backup software while respecting the file
system level security of the original file system. If you let the end
user have access to the backup they can restore any file to any location
which is generally not a good idea.
I do have a concept of creating a read only Fuse mounted file system
from a TSM point in time synthetic backup, and then using the shadow
copy feature of Samba to enable restores using the "Previous Versions"
feature of windows file manager.
I got as far as getting a directory tree you could browse through but
then had an enforced change of jobs and don't have access to a TSM
server any more to continue development.
Note if anyone from IBM is listening that would be a super cool feature.
JAB.
--
Jonathan A. Buzzard Email: jonathan (at) buzzard.me.uk
Fife, United Kingdom.
----- Message from "Marc A Kaplan" <[email protected]> on Mon, 11
Apr 2016 11:11:24 -0400 -----
*To:*
gpfsug main discussion list <[email protected]>
*Subject:*
Re: [gpfsug-discuss] backup and disaster recovery solutions
Since you write "so if the file is not touched for number of days,
it's moved to a tape" -
that is what we call the HSM feature. This is additional function
beyond backup. IBM has two implementations.
(1) TSM/HSM now called IBM Spectrum Protect.
_http://www-03.ibm.com/software/products/en/spectrum-protect-for-space-management_
(2) HPSS _http://www.hpss-collaboration.org/_
The GPFS (Spectrum Scale File System) policy feature supports both, so
that mmapplypolicy and GPFS policy rules can be used to perform
accelerated metadata scans to identify which files should be migrated.
Also, GPFS supports on-demand recall (on application reads) of data
from long term storage (tape) to GPFS storage (disk or SSD). See also
DMAPI.
Marc A Kaplan
From: Damir Krstic <[email protected]>
To: gpfsug main discussion list <[email protected]>
Date: 04/11/2016 09:16 AM
Subject: [gpfsug-discuss] backup and disaster recovery solutions
Sent by: [email protected]
------------------------------------------------------------------------
We have implemented 1.5PB ESS solution recently in our HPC
environment. Today we are kicking of backup and disaster recovery
discussions so I was wondering what everyone else is using for their
backup?
In our old storage environment we simply rsync-ed home and software
directories and projects were not backed up.
With ESS we are looking for more of a GPFS based backup solution -
something to tape possibly and also something that will have life
cycle feature - so if the file is not touched for number of days, it's
moved to a tape (something like LTFS).
Thanks in advance.
DAmir _______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org_
__http://gpfsug.org/mailman/listinfo/gpfsug-discuss_
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
