Upendra,
On Tue, Nov 23, 2010 at 02:34:17PM -0500, upengan78 wrote:
> Thanks for the advise.
>
> I understand you have asked me some questions and that is actually making me
> think of more points to be considered while sizing and make correct
> decisions. Some of the points that you have mentioned I did not really think
> of them when I started. Thanks for pointing them out.
Pretty much that is what I do. I ask or re-ask refined questions
and often lead conversations in that mode.
> One thing I am confused about and want to confirm do I really need to use
> spanning in Virtual tapes ?
My assumption was that with 50 Gig of data you'd be backing up
DLEs that where more than 4 gig. Since they clearly fit on DVD
at that size you either don't use, will not need, spanning or
you are allowing your dump/tar mechanism to roll your dumps onto
secondary volumes.
Er, vocabulary. DLEs traditionally had to fit on a tape volume.
DLE-size <= backup_media_size
This causes problems when your backup segments are larger than
your media, tape spanning was invented to handle this, it allowed
the larger dumps to span across your output media.
I guess in the earliest sense of it, dumps where unix partitions
and they where small relative to physical tape. With increases
in disk size and the advent of raid the partitions are allowed
to become 'large' relative to the tape media.
I don't know how you do your backups now, you may have included
that information in your details but I have not absorbed it.
If you have many small partitions to backup you can treat them
each as a DLE and DUMP them to vtape. If you have larger partitions
and have been TAR'ing segments then you can tell amanda that you
are backing up a segment of the partition with TAR, actually gtar
in most cases and continue to dump these small DLEs to your vtapes.
You may, with the vtapes want to restructure your sementation of
the data so that you have fewer but larger DLEs and increase the
size of the vtapes accordingly.
> This document here mentions that one doesn't need to configure 'runtapes'
> variable with vtapes but looking at amdump o/p in previous email I think I
> must mention 'runtape'
With jukeboxes, vtapes (where you can think of your vtapes as being
in a virtual-jukebox) or attended backups where you can have an
operator swap media for you, you can have more than one run-tape
volumes written per amdump run.
You must have some mechanism now to dump your data to multiple
DVD each night, so in effect your backup schema utilizes a
runtapes value greater than 1.
> Now back to the questions.
>
> @Charles
> * Are you planning to back up vtapes to DVD? If not, I'd say, don't worry
> about the 4 G size and pick something that suits your data set.
>
> No I am not planning on that. I think I better make tapesize larger like 40G
> or so
>
> @Brian
> *You talk in terms of 'new data' each day, do you touch existing
> files ?
>
> It will be modification of exsiting files as well as addition of directories
> and more files and may involve removal of directories/files.
Traditional unix dump is the default for amdump, non-level 0 dumps
utilize the well known level 1, 2... dumps. A level one will select
files that have been modified since the most recent level 0, so any
files that are either new or modified since the most recent lower
level number will be selected.
You have to expect in this case to backup more data than you are
adding to your system, but also remember that its segregated by
the DLE which in effect partitions your data.
> *How many DLEs, disklist entries, will you have, what are their
> relative sizes ?
>
> Only 1 entry for a amanda-network client server. Size is about 50G and grows
> by 100M-200M a day or data size sometimes stays as it is for 2-3 days.
So, what mechanism do you use to back it up ?
You are using either Tar or dump and allowing it to span output volumes ?
> *Did you plan on using the VTape pool for anything else ?
> No but I think I did not mention that actual filesystem size of RAID5 storage
> mounted is 300G and I only created 20 tapes of 4.x G size on it. I don't plan
> to use the remaining space for anything else.
If you are backing up 50 Gig at one time you need to either have a
50 Gig vtape or you need to tell amanda it can span the output data
across multiple vtape volumes. You need to enable tape spanning as
well as set runtapes accordingly.
If its practical I'd think in terms of dividing your data into
different DLE, different backup partitions, perhaps by selecting
a set of top level directories on the raid partition and treating
them as separate partitions and telling amanda to dump with gtar
rather than dump (which only dumps in partition level segments).
There are many examples of this in the amanda wiki and on the
internet in general, I can send you my own working examples.
> *How large are your DLEs ? Are they actually all small enough to expect level
> 0 dumps to fit on 4 gig volumes ?
>
> I have only 1 DLE in disklist and DUMP 0 will be of about 50G therefore won't
> fit on a single 4GB volume(vtape) however Ithought Amanda will span it
> automatically for Virtual tapes.
Only if you enable tape spanning. I believe its off by default.
> *Ah - how many DVD do you typically write per day now ?
> *Is that all level 0 or a mix of 0 and non-0 dumps ?
>
> No I don't write DVDs actually, it is just named DVD as per the wiki example.
> I have tried taking manual dumps using amdump and they fail as in my earlier
> email :( . Once it is confirmed amdump works manually I will let Amanda do
> this automatically.
Sorry - I thought amanda was replacing an actual DVD writer.
Amanda has the ability to dump multiple DLEs concurrently, it
does this by running the dump part (whether dump or gtar) on
the amanda client (client may be on the amanda server) and
putting the DLE output to an amanda work-area.
When each DLE finishes its re-written to your output devices
(tape or vtape, or writable DVD) and removed from the spool
area.
Amanda will not overallocate the work area, exceed a specified
maximum number of dumps per client or exceed a max number of
dumps overall.
The writing to the work area allows multiple concurrent dumps
that run more quickly overall than a single threaded series
of dumps of the DLEs, and since the write to take is from a
single source file (well, no but more later) tape time to write
each DLE is very short compared to dump time.
If you divide your data someone into 5 - 10 gig pieces then
you can dump more than one at a time and shorten wall clock
time of the overall data dump.
This isn't necessary, but often useful. You can set amanda to
dump single threaded, and it will always do do if you lack a
work area, amanda will operate from dump to tape directly but
you lose any parallelism in dumping.
You can config amanda to dump a single DLE to tape, er vtape,
perhaps forcing the level 0 to the weekend, and allow it or
instruct it to dump incrementals on week days so that your
single large dump doesn't run into business hours the next
day. But this doesn't take advantage of any of amanda's features.
> *Are you manually intermixing the levels now to even out dump
> times across your week ?
>
> No I don't plan on doing that. I am only trying to do amdump for testing if a
> Backup job works or no.
>
>
> *What is the structure of your vtape pool ?
>
> It is a directory with 1 slot and 20 vtaps. This directory resides on a
> sotrage mount which itself is RAID5 configured.
>
> size of /blah is 300GB
>
> lrwxrwxrwx 1 amanda sys 50 Nov 23 12:32 data ->
> /blah/amanda/vtapes/test/slots/slot2
> -rw------- 1 amanda sys 15 Nov 23 13:13 info
> drwxr-xr-x 2 amanda sys 512 Nov 23 11:07 slot1
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot10
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot11
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot12
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot13
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot14
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot15
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot16
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot17
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot18
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot19
> drwxr-xr-x 2 amanda sys 512 Nov 23 13:13 slot2
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot20
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot3
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot4
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot5
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot6
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot7
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot8
> drwxr-xr-x 2 amanda sys 512 Nov 23 10:27 slot9
>
> cat tapelist
>
> 0 TEST-20 reuse
> 0 TEST-19 reuse
> 0 TEST-18 reuse
> 0 TEST-17 reuse
> 0 TEST-16 reuse
> 0 TEST-15 reuse
> 0 TEST-14 reuse
> 0 TEST-13 reuse
> 0 TEST-12 reuse
> 0 TEST-11 reuse
> 0 TEST-10 reuse
> 0 TEST-9 reuse
> 0 TEST-8 reuse
> 0 TEST-7 reuse
> 0 TEST-6 reuse
> 0 TEST-5 reuse
> 0 TEST-4 reuse
> 0 TEST-3 reuse
> 0 TEST-2 reuse
> 0 TEST-1 reuse
>
> *Do you need to think in terms of off-site storage for
> disaster recovery ?
>
> No I don't think I will be doing that. Don't want any redundancy
> for now because there is already RAID5 on DAS.
I've gone to RAID 6 where possible.
> *Are you worried about data recovery if you lose the vtape
> Nope, Mount is RAID 5 so don't think it will be a big problem
> pool as well as files/drives in your data pool ?
No fear of fire damage in your computer room ?
No risk of a water pipe bursting ?
thanks,
Brian
> +----------------------------------------------------------------------
> |This was sent by [email protected] via Backup Central.
> |Forward SPAM to [email protected].
> +----------------------------------------------------------------------
>
>
---
Brian R Cuttler [email protected]
Computer Systems Support (v) 518 486-1697
Wadsworth Center (f) 518 473-6384
NYS Department of Health Help Desk 518 473-0773
IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure. It
is intended only for the addressee. If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments. Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.
