Re: [Bacula-users] Backup in disk AND tape

[Chris Wilkinson]

I have this working now. The script is below, as per Bill's script with an
extention. It is called from a Runscript as "Command =
/full-path/run-copy-job.sh copy-job-name". There is a copy job defined for
each backup job. The $1 parameter passes the copy job name to the script.

#!/bin/bash
#usage run-copy-job.sh job-name
# Pipe the run command to bconsole
# 
echo "run yes job=$1" | bconsole

This all works fine. The disadvantage is that the copy job hard codes the
level to be copied so the same copy job (in my case full) runs irrespective
of the job being incr/diff/full. For "selection type=PoolUncopiedJobs" this
is only a minor issue since already copied jobs are skipped.

Chris

On Tue, 9 Apr 2024, 15:12 Bill Arlofski,  wrote:

> On 4/9/24 6:53 AM, Chris Wilkinson wrote:
> > Regarding the suggestion to put a Runafter block in the job to run the
> copy job at the end, that doesn't seem to be allowed.
> > Run job=xx commands are not permitted in a Runscript as I just found
> out. It gives a not allowed command error.
> >
> > Perhaps there is another way to accomplish this?
> >
>
> Hello Chris,
>
> Yes, convert that run command to a simple script like:
>
> /opt/bacula/script/run_catalog-copy.sh:
> 8<
> #!/bin/bash
>
> # Pipe the run command to bconsole
> # 
> echo "run yes job=catalog-copy" | bconsole
> 8<
>
> Now, that is the most basic it needs to be, but you can add other things
> to it. ie: error checking, command line options,
> etc. Although in your use case it does not seem necessary to complicate
> things. :)
>
>
> Then, just replace the
> 8<
> Console = "run job=catalog-copy yes"
> 8<
>
> ...line in your RunScript with:
>
> 8<
> `Command = /opt/bacula/script/ru
> n_catalog-copy.sh`
> 8<
>
> And you should be OK.
>
> Make sure your `catalog-copy` job has the same Priority (11) as your
> Catalog job, otherwise you will end up in a dead-lock
> where the Copy job waits for the Catalog job to finish, and the Copy job
> is waiting for the catalog-copy job (which will
> never start) to finish.
>
>
> Hope this helps,
> Bill
>
> --
> Bill Arlofski
> w...@protonmail.com
>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup in disk AND tape

[Bill Arlofski via Bacula-users]

On 4/9/24 6:53 AM, Chris Wilkinson wrote:
Regarding the suggestion to put a Runafter block in the job to run the copy job at the end, that doesn't seem to be allowed. 
Run job=xx commands are not permitted in a Runscript as I just found out. It gives a not allowed command error.


Perhaps there is another way to accomplish this?



Hello Chris,

Yes, convert that run command to a simple script like:

/opt/bacula/script/run_catalog-copy.sh:
8<
#!/bin/bash

# Pipe the run command to bconsole
# 
echo "run yes job=catalog-copy" | bconsole
8<

Now, that is the most basic it needs to be, but you can add other things to it. ie: error checking, command line options, 
etc. Although in your use case it does not seem necessary to complicate things. :)



Then, just replace the
8<
Console = "run job=catalog-copy yes"
8<

...line in your RunScript with:

8<
`Command = /opt/bacula/script/ru
n_catalog-copy.sh`
8<

And you should be OK.

Make sure your `catalog-copy` job has the same Priority (11) as your Catalog job, otherwise you will end up in a dead-lock 
where the Copy job waits for the Catalog job to finish, and the Copy job is waiting for the catalog-copy job (which will 
never start) to finish.



Hope this helps,
Bill

--
Bill Arlofski
w...@protonmail.com



signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup in disk AND tape

[Chris Wilkinson]

Regarding the suggestion to put a Runafter block in the job to run the copy
job at the end, that doesn't seem to be allowed. Run job=xx commands are
not permitted in a Runscript as I just found out. It gives a not allowed
command error.

Perhaps there is another way to accomplish this?

-Chris-

On Wed, 3 Apr 2024, 00:00 Bill Arlofski via Bacula-users, <
bacula-users@lists.sourceforge.net> wrote:

> On 4/2/24 12:01 PM, Roberto Greiner wrote:
> > Hi,
> >
> > I've installed Bacula recently in a server with a 7TB RAID5 storage, and
> > a LTO-6 tape unit.
> >
> > I have configured 9 remote servers (most Linux, one Windows) to have the
> > backup made in this server in the disk storage, and I'm finish to
> > understand how to do the tape backup. Now, I have a question about
> > making the backup into both destinations.
> >
> > I have the following setup for JobsDef:
> >
> > JobDefs {
> > Name = "DefaultJob"
> > Type = Backup
> > Level = Incremental
> > Client = bacula2-fd
> > FileSet = "Full Set"
> > Schedule = "WeeklyCycle"
> > Storage = FileAligned
> > Messages = Standard
> > Pool = File
> > SpoolAttributes = yes
> > Priority = 10
> > Write Bootstrap = "/opt/bacula/working/%c.bsr"
> > }
> >
> > Then I added a server to have the backup, let's say (it's a linux,
> > despite the name):
> >
> > Job {
> > Name = "AD"
> > JobDefs = "DefaultJob"
> > Client = ad-fd
> > FileSet = "etc"
> > }
> >
> > This will, obviously go to the dedup-disk storage. The question is, how
> > should I add the tape setup? Is there a way to add a couple of lines to
> > the job definition above so that the backup goes to both systems? Should
> > I create a separate job definition for the tape backup? Some other way I
> > didn't consider?
> >
> > Thanks,
> >
> > Roberto
> >
> >
> > PS: The storage definitions for the disk and tape destinations:
> >
> > Storage {
> > Name = FileAligned
> > Address = bacula2
> > SDPort = 9103
> > Password = ""
> > Device = Aligned-Disk
> > Media Type = File1
> > }
> >
> > Storage {
> > Name = Fita
> > Address = bacula2
> > SDPort = 9103
> > Password = ""
> > Device = Ultrium
> > Media Type = LTO
> > }
>
> Hello Marcos,
>
> With Bacula, there are almost always 10+ different ways to accomplish
> things, and/or to even think about them.
>
> For example, you can override the Pool, Level, and Storage in a Schedule...
>
> So, with this in mind, you might set your job to run Incs each weekday to
> disk, and then set the Fulls to run to tape on the
> weekend. (just one idea)
>
> Another option is to use Copy jobs. With Copy jobs, you can run your Incs
> and Fulls to disk, then you can run a Copy job to
> copy your Incs, Fulls, or both to tape during normal working hours because
> Copy jobs do not make use of any Clients, so
> business productivity will not be affected on your server(s).
>
> In your case, I would probably go with a Copy job. This way, you have your
> backups on disk for fast restores when needed, and
> you have the same data copied to new jobids onto tape - maybe with longer
> retention periods, for example.
>
> Also have a look at the `SelectionType = PoolUncopiedJobs` feature for
> Copy jobs. This is a nice, handy "shortcut" to make
> sure that each of your jobs in some Pool is copied once, and only once to
> tape.
>
> In this case, you can have two Copy jobs configured, one looking at your
> Full disk pool and one looking at your Inc disk pool
> and copying jobs that have not been copied.
>
> OR, you can have one copy job running on a schedule where the Pool is
> overridden at two different times of the day to copy
> from the Full disk pool, and then also from the Inc disk pool.
>
> OR... (lol I said 10, so I am working towards that number, and I am
> getting close :) ... You can have your normal backup jobs
> include a `RunScript {RunsWhen = after}` section which triggers an
> immediate copy of the job to tape as soon as it is completed.
>
> So, I would start with a look at Copy jobs and see where that goes. :)
>
> Feel free to ask more questions once you have taken a look at Copy jobs.
>
>
> Hope this helps,
> Bill
>
> --
> Bill Arlofski
> w...@protonmail.com
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup in disk AND tape

[Roberto Greiner]

Em 02/04/2024 19:58, Bill Arlofski via Bacula-users escreveu:

On 4/2/24 12:01 PM, Roberto Greiner wrote:

Hi,

I've installed Bacula recently in a server with a 7TB RAID5 storage, and
a LTO-6 tape unit.

I have configured 9 remote servers (most Linux, one Windows) to have the
backup made in this server in the disk storage, and I'm finish to
understand how to do the tape backup. Now, I have a question about
making the backup into both destinations.

I have the following setup for JobsDef:

JobDefs {
    Name = "DefaultJob"
    Type = Backup
    Level = Incremental
    Client = bacula2-fd
    FileSet = "Full Set"
    Schedule = "WeeklyCycle"
    Storage = FileAligned
    Messages = Standard
    Pool = File
    SpoolAttributes = yes
    Priority = 10
    Write Bootstrap = "/opt/bacula/working/%c.bsr"
}

Then I added a server to have the backup, let's say (it's a linux,
despite the name):

Job {
    Name = "AD"
    JobDefs = "DefaultJob"
    Client = ad-fd
    FileSet = "etc"
}

This will, obviously go to the dedup-disk storage. The question is, how
should I add the tape setup? Is there a way to add a couple of lines to
the job definition above so that the backup goes to both systems? Should
I create a separate job definition for the tape backup? Some other way I
didn't consider?

Thanks,

Roberto


PS: The storage definitions for the disk and tape destinations:

Storage {
    Name = FileAligned
    Address = bacula2
    SDPort = 9103
    Password = ""
    Device = Aligned-Disk
    Media Type = File1
}

Storage {
    Name = Fita
    Address = bacula2
    SDPort = 9103
    Password = ""
    Device = Ultrium
    Media Type = LTO
}


Hello Marcos,

With Bacula, there are almost always 10+ different ways to accomplish 
things, and/or to even think about them.


For example, you can override the Pool, Level, and Storage in a 
Schedule...


So, with this in mind, you might set your job to run Incs each weekday 
to disk, and then set the Fulls to run to tape on the weekend. (just 
one idea)


Another option is to use Copy jobs. With Copy jobs, you can run your 
Incs and Fulls to disk, then you can run a Copy job to copy your Incs, 
Fulls, or both to tape during normal working hours because Copy jobs 
do not make use of any Clients, so business productivity will not be 
affected on your server(s).


In your case, I would probably go with a Copy job. This way, you have 
your backups on disk for fast restores when needed, and you have the 
same data copied to new jobids onto tape - maybe with longer retention 
periods, for example.


Also have a look at the `SelectionType = PoolUncopiedJobs` feature for 
Copy jobs. This is a nice, handy "shortcut" to make sure that each of 
your jobs in some Pool is copied once, and only once to tape.


In this case, you can have two Copy jobs configured, one looking at 
your Full disk pool and one looking at your Inc disk pool and copying 
jobs that have not been copied.


OR, you can have one copy job running on a schedule where the Pool is 
overridden at two different times of the day to copy from the Full 
disk pool, and then also from the Inc disk pool.


OR... (lol I said 10, so I am working towards that number, and I am 
getting close :) ... You can have your normal backup jobs include a 
`RunScript {RunsWhen = after}` section which triggers an immediate 
copy of the job to tape as soon as it is completed.


So, I would start with a look at Copy jobs and see where that goes. :)

Feel free to ask more questions once you have taken a look at Copy jobs.


Hope this helps,
Bill


Yes, this helps A LOT. I will study the copy job option. It really seems 
perfect for my scenario.


Tks

Roberto


--
-
Marcos Roberto Greiner

   Os otimistas acham que estamos no melhor dos mundos
Os pessimistas tem medo de que isto seja verdade
 James Branch Cabell
  -



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup in disk AND tape

[Bill Arlofski via Bacula-users]

On 4/2/24 12:01 PM, Roberto Greiner wrote:

Hi,

I've installed Bacula recently in a server with a 7TB RAID5 storage, and
a LTO-6 tape unit.

I have configured 9 remote servers (most Linux, one Windows) to have the
backup made in this server in the disk storage, and I'm finish to
understand how to do the tape backup. Now, I have a question about
making the backup into both destinations.

I have the following setup for JobsDef:

JobDefs {
    Name = "DefaultJob"
    Type = Backup
    Level = Incremental
    Client = bacula2-fd
    FileSet = "Full Set"
    Schedule = "WeeklyCycle"
    Storage = FileAligned
    Messages = Standard
    Pool = File
    SpoolAttributes = yes
    Priority = 10
    Write Bootstrap = "/opt/bacula/working/%c.bsr"
}

Then I added a server to have the backup, let's say (it's a linux,
despite the name):

Job {
    Name = "AD"
    JobDefs = "DefaultJob"
    Client = ad-fd
    FileSet = "etc"
}

This will, obviously go to the dedup-disk storage. The question is, how
should I add the tape setup? Is there a way to add a couple of lines to
the job definition above so that the backup goes to both systems? Should
I create a separate job definition for the tape backup? Some other way I
didn't consider?

Thanks,

Roberto


PS: The storage definitions for the disk and tape destinations:

Storage {
    Name = FileAligned
    Address = bacula2
    SDPort = 9103
    Password = ""
    Device = Aligned-Disk
    Media Type = File1
}

Storage {
    Name = Fita
    Address = bacula2
    SDPort = 9103
    Password = ""
    Device = Ultrium
    Media Type = LTO
}


Hello Marcos,

With Bacula, there are almost always 10+ different ways to accomplish things, 
and/or to even think about them.

For example, you can override the Pool, Level, and Storage in a Schedule...

So, with this in mind, you might set your job to run Incs each weekday to disk, and then set the Fulls to run to tape on the 
weekend. (just one idea)


Another option is to use Copy jobs. With Copy jobs, you can run your Incs and Fulls to disk, then you can run a Copy job to 
copy your Incs, Fulls, or both to tape during normal working hours because Copy jobs do not make use of any Clients, so 
business productivity will not be affected on your server(s).


In your case, I would probably go with a Copy job. This way, you have your backups on disk for fast restores when needed, and 
you have the same data copied to new jobids onto tape - maybe with longer retention periods, for example.


Also have a look at the `SelectionType = PoolUncopiedJobs` feature for Copy jobs. This is a nice, handy "shortcut" to make 
sure that each of your jobs in some Pool is copied once, and only once to tape.


In this case, you can have two Copy jobs configured, one looking at your Full disk pool and one looking at your Inc disk pool 
and copying jobs that have not been copied.


OR, you can have one copy job running on a schedule where the Pool is overridden at two different times of the day to copy 
from the Full disk pool, and then also from the Inc disk pool.


OR... (lol I said 10, so I am working towards that number, and I am getting close :) ... You can have your normal backup jobs 
include a `RunScript {RunsWhen = after}` section which triggers an immediate copy of the job to tape as soon as it is completed.


So, I would start with a look at Copy jobs and see where that goes. :)

Feel free to ask more questions once you have taken a look at Copy jobs.


Hope this helps,
Bill

--
Bill Arlofski
w...@protonmail.com



signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Backup in disk AND tape

[Roberto Greiner]

Hi,

I've installed Bacula recently in a server with a 7TB RAID5 storage, and 
a LTO-6 tape unit.


I have configured 9 remote servers (most Linux, one Windows) to have the 
backup made in this server in the disk storage, and I'm finish to 
understand how to do the tape backup. Now, I have a question about 
making the backup into both destinations.


I have the following setup for JobsDef:

JobDefs {
  Name = "DefaultJob"
  Type = Backup
  Level = Incremental
  Client = bacula2-fd
  FileSet = "Full Set"
  Schedule = "WeeklyCycle"
  Storage = FileAligned
  Messages = Standard
  Pool = File
  SpoolAttributes = yes
  Priority = 10
  Write Bootstrap = "/opt/bacula/working/%c.bsr"
}

Then I added a server to have the backup, let's say (it's a linux, 
despite the name):


Job {
  Name = "AD"
  JobDefs = "DefaultJob"
  Client = ad-fd
  FileSet = "etc"
}

This will, obviously go to the dedup-disk storage. The question is, how 
should I add the tape setup? Is there a way to add a couple of lines to 
the job definition above so that the backup goes to both systems? Should 
I create a separate job definition for the tape backup? Some other way I 
didn't consider?


Thanks,

Roberto


PS: The storage definitions for the disk and tape destinations:

Storage {
  Name = FileAligned
  Address = bacula2
  SDPort = 9103
  Password = ""
  Device = Aligned-Disk
  Media Type = File1
}

Storage {
  Name = Fita
  Address = bacula2
  SDPort = 9103
  Password = ""
  Device = Ultrium
  Media Type = LTO
}




--
-
Marcos Roberto Greiner

   Os otimistas acham que estamos no melhor dos mundos
Os pessimistas tem medo de que isto seja verdade
 James Branch Cabell
  -



___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[Christopher Manly]

Matthias Kellermann wrote:
 Hi everyone,

 I want to use Bacula to backup my files to disk AND to tape at the same
 time. The backup on disk should be available for 3 months or so while
 the backup on tape should not be overwritten before 12 months.

 Is this possible with Bacula? How do I accomplish this without the need
 to run the backup job twice?

   
I've been wrestling with this one myself.  In my case, my primary backup 
is a NAS device that's NFS mounted.  I've got my fulls, differentials, 
and incrementals each going into their own pool/directory on the NAS.  I 
already plan to migrate last month's backups off the NAS onto tape for 
long-term storage.  However, I'd also like to make a copy of this 
month's full backups onto a tape to take off-site.  (Then I sync the 
incrementals to a portable hard drive and get the whole ball of yarn in 
a single duffle bag, and the portable drive doesn't have to have the 
full capacity of the NAS.)

I've tried bcopy and failed.

I've tried just tarring the disk volumes, but it's hard to predict the 
end of tape and get efficient tape usage.

So, I think I'm going to do some recursive baculating.  (Is that a 
word?  :-)  I think I'm going to set up a job in bacula that will back 
up the full files to tape in another pool.

Here's my thinking: if my NAS takes a total header, I'll need to restore 
from tape onto disk, and then from disk to the client machines.  But if 
it's that much of a DR situation, I can live with that.  Since pulling 
from tape onto disk is really only a first step, I don't need to bscan 
the tape...I can just brute force bextract it onto my shiny new NAS or 
whatever I'm using to be my new disk store.

If anyone sees any nasty flaws in my logic there, please let me know.  
I'll try to share when I figure out whether it'll really work or not.

Something else that just occurred to me that I might do at some point is 
to build a VMware image with a linux or Solaris build (I'm in a Solaris 
shop, but linux might be more universally useful) with a bacula install 
set up and running.  Burn that image onto a DVD along with VMware 
player, and in a true DR situation I could set up a virtual Bacula 
server on just about any laptop or PC I could lay hands on to start my 
recoveries (assuming I had access to my backup media from it).  But 
that's a bit further off for me.



-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[Matthias Kellermann]

Hi,

thanks for your answer, Arno.

Arno Lehmann schrieb:
 Hi,
 
 No, that's not possible.
 
 There are a number of workarounds possible, but the ideal thing is 
 currently not possible. It's discussed, though, and I think anyone 
 (relevant :-) thinks such athing would be useful.

Which workaround is possible at the moment to get the files to tape AND
to disk? I did not find an suitable answer when searching the lists.

Would be nice to have the files on disk for some period to get a file
back quickly ;)

Matthias

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[Eric Böse-Wolf]

Hi,

 Would be nice to have the files on disk for some period to get a file
 back quickly ;)

You could backup to disk and migrate the files, when you
don't need them quickly any more, to tape.

Eric


pgpcRbpKfGt1m.pgp
Description: PGP signature
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[Arno Lehmann]

Hi,

24.09.2007 11:09,, Eric Böse-Wolf wrote::
 Hi,
 
 Would be nice to have the files on disk for some period to get a file
 back quickly ;)
 
 You could backup to disk and migrate the files, when you
 don't need them quickly any more, to tape.

Or run two jobs, one to disk, one to tape. The backups will not be 
absolutely identical, and your clients will be loaded twice, though.

You could also write backups to disk with Bacula and store the volume 
files on tape using an extra script (or Bacula job). Restoring will 
require at least one extra step then.

Arno

 Eric
 
 
 
 
 -
 This SF.net email is sponsored by: Microsoft
 Defy all challenges. Microsoft(R) Visual Studio 2005.
 http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
 
 
 
 
 ___
 Bacula-users mailing list
 Bacula-users@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bacula-users

-- 
Arno Lehmann
IT-Service Lehmann
www.its-lehmann.de

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[Eric Böse-Wolf]

Tom Allison [EMAIL PROTECTED] writes:

 Unless you have a carousel, does anyone really use tapes anymore?
 I'm speaking of small networks, but it seems to me that hard drives
 are much cheaper than tapes for the $$/GB ratio.

That's really a good question. Yes hard drives have a better money/GB
ratio, but cheap hard drive keep your data safe only for 3 - 4 years for
sure (maybe longer) and some tapes (DLT, LTO) are specified to hold your
data for 15 - 30 Years (if the tape is not constantly in use, so for
archiving purposes).

So if you have to do a revision proof backup according to the laws of
some nations, you need to use something, that holds your data long and
here tape comes in I think.

Also tapes are safe to problems in electric circuits or electric supply,
viruses, trojans and hacker attacks. You can take them offsite easily.

I think it depends on the problem you want to solve.

Disk backups for data that need only be kept a few weeks and easily
accessable or home office backups, tape for revision proof long term
archiving.

mfg

Eric

PS.: I'm a noob to this so maybe some grown ups correct me, if I stated
something wrong :-)


pgp95mn0OFMno.pgp
Description: PGP signature
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[John Drescher]

 That's really a good question. Yes hard drives have a better money/GB
 ratio,
After you get over the cost of the tape drive LTO2 and better are
cheaper than hard disks.

 but cheap hard drive keep your data safe only for 3 - 4 years for
 sure (maybe longer) and some tapes (DLT, LTO) are specified to hold your
 data for 15 - 30 Years (if the tape is not constantly in use, so for
 archiving purposes).

On top of that I have several other reasons why tape is better for
backups. We have 10TB of data online (linux software raid 5 and 6)
which represents between 1/2 and 2/3 of our data but we do not in any
way consider this as a backup. What happens if the file system
corrupts (I have seen this happen) and 1/2 of your data is lost? Hard
drives use power and require extras (servers/cages) that make the cost
of them a lot more than the price of a single drive. And they do not
scale anywhere near as well as tape. And you have to replace them
every 3 to 5 years or fear that you will loose your data. To avoid
some of these problems you could store the drives on a shelf (in a
temp / humidity controlled environment), however there is a big risk
here that the drive will not spin when you install it 10 years down
the line making the data on the disk very expensive to recover.

John

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[Bob Hetzel]

The reality is that if you really need reliable data for 10 years you're 
probably stuck with technology like paper, optical media (choose wisely 
as many of these formats are gone too), or online hard drive space that 
you'll be continually checking and carrying along with each upgrade (and 
backing up with all your regular fulls).  All have their own major 
drawbacks.  The benefit to online hard drive space is that new data 
needs grow so fast that in many cases it's not that much more expensive 
to keep the old stuff around--for instance... 20 years ago 100GB of data 
was not available in one storage system.  10 years ago it was a lot but 
quite pricey.  Now it's about the smallest hard drive you can buy new.

The odds that you can locate a working tape drive of any current (2007 
hardware) type and adapters to plug it into 10 years from now (2017 
hardware) aren't good the way things are moving.  Not only will you have 
to worry about hardware--is PCI still going to go the way of the ISA bus 
by then?--but drivers for old adapters on the OS after the next OS are 
quite possibly going to be a problem--there's tons of useless adapters 
out there now where manufacturers went out of business before updating 
their NT 4.0 drivers to work with XP, Server 2000, and Server 2003, so 
even if you save both the tape drive and the adapter who's to say the 
adapter will have a spot to plug it in and a driver you can load.

With regard to 30 years I can almost guarantee problems with just about 
any electronic removable media.  While it's true that you can probably 
find a 9-track mainframe style tape reader to read 30 year old data 
tapes on many current computer systems, the market does not seem to be 
maintaining that trend for the current storage stuff--things are moving 
just too fast.  That's been driven by IBM's mainframe dominance over the 
last 30 years--corporations have been migrating off IBM mainframe 
hardware right and left in favor of hardware from companies that may or 
may not still be in existence 10 years from now.

In summary... backup software is extremely important for disaster 
recovery but should not be considered for long term (5+ years, possibly 
even less depending on what you need it for) storage needs in my humble 
opinion.

 Message: 22 Date: Mon, 24 Sep 2007 08:06:29 -0400 From: John Drescher 
 [EMAIL PROTECTED] 
 
  but cheap hard drive keep your data safe only for 3 - 4 years for
  sure (maybe longer) and some tapes (DLT, LTO) are specified to hold your
  data for 15 - 30 Years (if the tape is not constantly in use, so for
  archiving purposes).
 
 On top of that I have several other reasons why tape is better for
 backups. We have 10TB of data online (linux software raid 5 and 6)
 which represents between 1/2 and 2/3 of our data but we do not in any
 way consider this as a backup. What happens if the file system
 corrupts (I have seen this happen) and 1/2 of your data is lost? Hard
 drives use power and require extras (servers/cages) that make the cost
 of them a lot more than the price of a single drive. And they do not
 scale anywhere near as well as tape. And you have to replace them
 every 3 to 5 years or fear that you will loose your data. To avoid
 some of these problems you could store the drives on a shelf (in a
 temp / humidity controlled environment), however there is a big risk
 here that the drive will not spin when you install it 10 years down
 the line making the data on the disk very expensive to recover.
 
 John


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[Alan Brown]

On Mon, 24 Sep 2007, Bob Hetzel wrote:

 In summary... backup software is extremely important for disaster
 recovery but should not be considered for long term (5+ years, possibly
 even less depending on what you need it for) storage needs in my humble
 opinion.

Anyone considering backups for archival purposes MUST have periodic 
rereading/migrations setup - in order to verify the media's integrity as 
much as the drives - and ensure that when new technology is obtained, 
enough overlap is left to allow old data to be moved to the new media.

Technology such as LTO is guaranteed read/write for one generation 
backwards and read-only for two. LTO4 cannot read LTO1, LTO5 will not be 
able to read LTO2. This makes the update! pressure at least a little 
less extreme...

I would not trust hard drives for long term cold storage. My experience is 
that they tend to fail to spin up after a few years.


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[mark . bergman]

In the message dated: Mon, 24 Sep 2007 11:04:00 EDT,
The pithy ruminations from Bob Hetzel on 
Re: [Bacula-users] Backup to disk AND tape were:
= The reality is that if you really need reliable data for 10 years you're 
= probably stuck with technology like paper, optical media (choose wisely 

Paper? True, it's about the best archival media that we've got...however, 
the information density is too low to be practical for most data formats. I've 
yet to see a usable way of rapidly and reliably converting arbitrary data to
--and from--machine readable symbols on paper. I guess that the 
Data Matrix encoding from Siemens has sufficient density (theoretically storing 
about 37GB per US Letter page, but practically many orders of magnitude 
smaller)/

However, I'd argue that the issues you raise with reading tapes--having the
hardware and software to access the data, rather than deterioration of the media
itself--would also be a major problem for any high density storage printed on
paper. I think that microfilm would be a better answer, with a very simple
binary-data-to-text encoding (UUENCODE, for example).


= as many of these formats are gone too), or online hard drive space that 
= you'll be continually checking and carrying along with each upgrade (and 
= backing up with all your regular fulls).  All have their own major 
= drawbacks.  The benefit to online hard drive space is that new data 
= needs grow so fast that in many cases it's not that much more expensive 
= to keep the old stuff around--for instance... 20 years ago 100GB of data 
= was not available in one storage system.  10 years ago it was a lot but 
= quite pricey.  Now it's about the smallest hard drive you can buy new.

While that does reflect the change in drive space per unit or per cost over 
time, that's not in and of itself an advantage of hard drives. If I purchase a 
drive today and put data on it with the intent of keeping that data for 10 
years, I've still got to be able to access the drive that I just purchased, 
even if 5 years from now new drives are cheaper and have higher densities. The 
same issues with bus connections, drivers, etc. still apply. We're already 
seeing this with machines that don't support PATA drives, and with pulling data 
off older SCSI drives.

= 
= The odds that you can locate a working tape drive of any current (2007 
= hardware) type and adapters to plug it into 10 years from now (2017 
= hardware) aren't good the way things are moving.  Not only will you have 

It depends on what level of technology you are considering.

= to worry about hardware--is PCI still going to go the way of the ISA bus 
= by then?--but drivers for old adapters on the OS after the next OS are 
= quite possibly going to be a problem--there's tons of useless adapters 
= out there now where manufacturers went out of business before updating 
= their NT 4.0 drivers to work with XP, Server 2000, and Server 2003, so 
= even if you save both the tape drive and the adapter who's to say the 
= adapter will have a spot to plug it in and a driver you can load.

Absolutely, if you're dealing with low-end (small office/consumer) hardware. 
For this discussion, I'd immediately rule out any device that has it's own bus 
adapter and drivers.

I'm very, very confident that a 4Gb/s fibre-attached LTO3 or LTO4 or AIT5 drive
purchased today will be usable on a SAN in 10 years. Of course, that might be a
64GB/s SAN, and I might not be able to purchase any more LTO4 media, and I may
need to keep the drive that I purchase in 5 years in order to read the tapes I
write today, and my current drive might not be able to write to my 10 year old
LTO4 media...but I believe that I'd be able to read the data. For example, AIT
manufactureres have a very firm commitment to generational compatability. AIT5
(released in 2006), for example, is read/write compatible with media back to
AIT3 (released in 2001). I believe that the standard says that AITn will be RW
compatible with AITn-1, and read compatible with AITn-2. 


= 
= With regard to 30 years I can almost guarantee problems with just about 
= any electronic removable media.  While it's true that you can probably 


Yep.


[SNIP!]


= 
= In summary... backup software is extremely important for disaster 
= recovery but should not be considered for long term (5+ years, possibly 
= even less depending on what you need it for) storage needs in my humble 
= opinion.
= 

[SNIP!]

I agree, in that backup is not entirely the same thing as archive.

Mark



Mark Bergman  [EMAIL PROTECTED]
System Administrator
Section of Biomedical Image Analysis 215-662-7310
Department of Radiology,   University of Pennsylvania

http://pgpkeys.pca.dfn.de:11371/pks/lookup?search=mark.bergman%40.uphs.upenn.edu


-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005

Re: [Bacula-users] Backup to disk AND tape

[Chris Hoogendyk]

Bob Hetzel wrote:
 The reality is that if you really need reliable data for 10 years you're 
 probably stuck with technology like paper, optical media (choose wisely 
 as many of these formats are gone too), or online hard drive space that 
 you'll be continually checking and carrying along with each upgrade (and 
 backing up with all your regular fulls).  All have their own major 
 drawbacks.  The benefit to online hard drive space is that new data 
 needs grow so fast that in many cases it's not that much more expensive 
 to keep the old stuff around--for instance... 20 years ago 100GB of data 
 was not available in one storage system.  10 years ago it was a lot but 
 quite pricey.  Now it's about the smallest hard drive you can buy new.

 The odds that you can locate a working tape drive of any current (2007 
 hardware) type and adapters to plug it into 10 years from now (2017 
 hardware) aren't good the way things are moving.  Not only will you have 
 to worry about hardware--is PCI still going to go the way of the ISA bus 
 by then?--but drivers for old adapters on the OS after the next OS are 
 quite possibly going to be a problem--there's tons of useless adapters 
 out there now where manufacturers went out of business before updating 
 their NT 4.0 drivers to work with XP, Server 2000, and Server 2003, so 
 even if you save both the tape drive and the adapter who's to say the 
 adapter will have a spot to plug it in and a driver you can load.

 With regard to 30 years I can almost guarantee problems with just about 
 any electronic removable media.  While it's true that you can probably 
 find a 9-track mainframe style tape reader to read 30 year old data 
 tapes on many current computer systems, the market does not seem to be 
 maintaining that trend for the current storage stuff--things are moving 
 just too fast.  That's been driven by IBM's mainframe dominance over the 
 last 30 years--corporations have been migrating off IBM mainframe 
 hardware right and left in favor of hardware from companies that may or 
 may not still be in existence 10 years from now.

 In summary... backup software is extremely important for disaster 
 recovery but should not be considered for long term (5+ years, possibly 
 even less depending on what you need it for) storage needs in my humble 
 opinion.

   
 Message: 22 Date: Mon, 24 Sep 2007 08:06:29 -0400 From: John Drescher 
 [EMAIL PROTECTED] 

 
 but cheap hard drive keep your data safe only for 3 - 4 years for
 sure (maybe longer) and some tapes (DLT, LTO) are specified to hold your
 data for 15 - 30 Years (if the tape is not constantly in use, so for
 archiving purposes).
 
 On top of that I have several other reasons why tape is better for
 backups. We have 10TB of data online (linux software raid 5 and 6)
 which represents between 1/2 and 2/3 of our data but we do not in any
 way consider this as a backup. What happens if the file system
 corrupts (I have seen this happen) and 1/2 of your data is lost? Hard
 drives use power and require extras (servers/cages) that make the cost
 of them a lot more than the price of a single drive. And they do not
 scale anywhere near as well as tape. And you have to replace them
 every 3 to 5 years or fear that you will loose your data. To avoid
 some of these problems you could store the drives on a shelf (in a
 temp / humidity controlled environment), however there is a big risk
 here that the drive will not spin when you install it 10 years down
 the line making the data on the disk very expensive to recover.


All of which means that any really serious long term backup and archive 
strategy becomes fairly complex and needs some real planning and ongoing 
attention. You need to have plans for checking the readability of your 
archival media regularly, plans for redundancy in your archival storage, 
and plans for migrating archives to newer media as the older media 
starts to become risky or unavailable.

Unfortunately, few think in these terms. The monks of the middle ages 
copied and preserved books for many hundreds of years. Librarians think 
in terms of long term archives, rare book collections, etc. Now, in the 
digital age, Libraries can't afford to maintain their extensive 
periodical collections. It becomes the responsibility of commercial 
publishing houses to maintain the digital archives, and Libraries pay 
subscriptions for access. If a publishing house goes out of business, we 
no longer have the widely distributed redundant paper archives in 
Libraries all over the country or world.

In the raging free market economy in the U.S., businesses come and go. 
Departments and staff in large corporations come and go. The 5 year 
business plan is thought to be thinking way ahead. Sorry, can't afford 
wasting money worrying about what might happen after that. If we can 
just churn enough business and make enough money, then that will 
inherently solve whatever problems we might face then.

In the 

[Bacula-users] Backup to disk AND tape

[Matthias Kellermann]

Hi everyone,

I want to use Bacula to backup my files to disk AND to tape at the same
time. The backup on disk should be available for 3 months or so while
the backup on tape should not be overwritten before 12 months.

Is this possible with Bacula? How do I accomplish this without the need
to run the backup job twice?

Matthias

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk AND tape

[Arno Lehmann]

Hi,

21.09.2007 15:27,, Matthias Kellermann wrote::
 Hi everyone,
 
 I want to use Bacula to backup my files to disk AND to tape at the same
 time. The backup on disk should be available for 3 months or so while
 the backup on tape should not be overwritten before 12 months.
 
 Is this possible with Bacula? How do I accomplish this without the need
 to run the backup job twice?

No, that's not possible.

There are a number of workarounds possible, but the ideal thing is 
currently not possible. It's discussed, though, and I think anyone 
(relevant :-) thinks such athing would be useful.

In other words, it will most probably be implemented some day.
(If you want to know more, search for the term copy pool and SD 
mux in this lest and the -devel one.)

Before that happens, I assume that job copying will be possible. This 
is still in the Kern thinks about it-phase, but again, the need and 
usefulness of such a feature is not seriously disputed.

Currently, as far as I know, Kern is not working on these features, 
though. If you want to see them as soon as possible, you'd have to 
offer some help developing them, I guess.

Arno

 Matthias
 
 -
 This SF.net email is sponsored by: Microsoft
 Defy all challenges. Microsoft(R) Visual Studio 2005.
 http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
 ___
 Bacula-users mailing list
 Bacula-users@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bacula-users

-- 
Arno Lehmann
IT-Service Lehmann
www.its-lehmann.de

-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk then tape

[Alan Brown]

On Mon, 1 May 2006, Bill Moran wrote:


I backup ~20Tb to LTO2. The tapes run almost as fast as most of the
disk arrays (MSA 1000 and Nexsan Atabeast)


I guess I didn't make my point.

The tapes are several orders of magnitude slower than disk.


Then get faster tapes.

If things are that critical you can afford a LTO3 or SAIT2 changer.

 The time it takes to get them back onsite alone is too slow for most 
recovery scenerios.


Hence the necessity for job cloning - AND a decent data firesafe.


Sveral people have requested simultaneous backups for the same reason.
Asking for D-D-T just obscures what you really want.


Can you please define these terms, then.  I'm fuzzy on what they mean:
1) simultaneous backups: ?


Job cloning - data is dumped to 2 sets of tape at the same time (or disk 
and tape simultaneously)



2) D-D-T: ?


disk to disk to tape (which isn't done for the reasons you want it done)


Not at all, but 1Tb is a small amount of disk these days.


I suppose.  We don't expect it to stay that small, and we've got a limited
amount of time to shake the bugs out of the system before our backup
needs balloon.


Then you DEFINITELY need faster tape.


In any case, the functionality you need can probably be achieved
using bcopy on the backup sets - and given the small dataset you have, it
wouldn't take long.


You mean the utterly undocumented bcopy?  Or is there some other bcopy that
you're referring to?


That's the one.


I did do some experiments with bcopy, and wasn't happy with the results.
If I rememeber correctly, it required me to write a script that:
1) Knew which on-disk volume it needed to duplicate.
2) Was able to generate a label for the tape _prior_ to running bcopy.

While that's certainly possible, it's still far from ready for general
consumption.  Quite frankly, I'm in the dark on how to do #1 reliably,
and #2 requires btape.  Starts to seem rather klunky.


It is, but it is an interim workaround.

Your other alternative is to backup twice, to 2 different pools.


Before the end of the year, we will require redundancy that can survive
major catastropies that are geographically localized.  IOW: if Pittsburgh
were to get hit with a nuclear warhead, our customers would expect us
to fail over to a second site.  Thus we need truely offsite backups.


No.

If you have that kind of demand then you need fully replicated 
geographically dispersed filesystems. Bringing things online from backups 
will take too long.




Very cool.  However, it wouldn't suit our needs.  As I mentioned, the unit
would _literally_ have to be nuclear warhead-proof.  Since nobody makes
such a unit, our DRP must include geographically dispersed redundancy.


The internet was designed (originally(*)) to withstand nuclear attack and 
route around the damage.


If you need that kind of durability then relying on a tape backup system 
is at least an order of magnitude below your _true_ requirements and will 
leave you caught short if things do go bang.(**)



AB



(*) There has been so much consolidation into major backbones that I 
suspect the 'net would catastrophically fail in the event of even a minor 
problem, witness the severe degradation which occurs with localalised 
communications outages around major nexus points.



(**) there are far nastier things which can be done with fissile material 
than to make it go bang.





---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk then tape

[Bill Moran]

On Tue, 2 May 2006 11:18:53 +0100 (BST)
Alan Brown [EMAIL PROTECTED] wrote:

 On Mon, 1 May 2006, Bill Moran wrote:
 
  I backup ~20Tb to LTO2. The tapes run almost as fast as most of the
  disk arrays (MSA 1000 and Nexsan Atabeast)
 
  I guess I didn't make my point.
 
  The tapes are several orders of magnitude slower than disk.
 
 Then get faster tapes.

I appears as if I'm still not communicating.

The actual speed of the tapes themselves is acceptable.

The time it takes to get the tapes back on site is too long.  Since
we are _required_ to have offsite backups, and we also have a logical
_need_ for onsite backups, we would benefit from what you are calling
job cloning.

 If things are that critical you can afford a LTO3 or SAIT2 changer.

Already there.

   The time it takes to get them back onsite alone is too slow for most 
  recovery scenerios.
 
 Hence the necessity for job cloning - AND a decent data firesafe.

What does a firesafe have to do with this?

  Sveral people have requested simultaneous backups for the same reason.
  Asking for D-D-T just obscures what you really want.
 
  Can you please define these terms, then.  I'm fuzzy on what they mean:
  1) simultaneous backups: ?
 
 Job cloning - data is dumped to 2 sets of tape at the same time (or disk 
 and tape simultaneously)

That's what we want.  That fits our needs.

  2) D-D-T: ?
 
 disk to disk to tape (which isn't done for the reasons you want it done)

Huh?  I'm not following either your explanation of the term, nor your
assertion as to why I would want to use it.

  Not at all, but 1Tb is a small amount of disk these days.
 
  I suppose.  We don't expect it to stay that small, and we've got a limited
  amount of time to shake the bugs out of the system before our backup
  needs balloon.
 
 Then you DEFINITELY need faster tape.

No.  As previously described, the _tapes_ are fast enough, getting them
back from the offsite storage facility is too slow.
Our need for offsite storage is different than our on-disk backups.
The offsite backups are for archival and legal requirements.  Our
on-disk backups are for handling 99% of the data-loss incidents.

[snip]
  I did do some experiments with bcopy, and wasn't happy with the results.
  If I rememeber correctly, it required me to write a script that:
  1) Knew which on-disk volume it needed to duplicate.
  2) Was able to generate a label for the tape _prior_ to running bcopy.
 
  While that's certainly possible, it's still far from ready for general
  consumption.  Quite frankly, I'm in the dark on how to do #1 reliably,
  and #2 requires btape.  Starts to seem rather klunky.
 
 It is, but it is an interim workaround.
 
 Your other alternative is to backup twice, to 2 different pools.

Which is what we're doing now.

  Before the end of the year, we will require redundancy that can survive
  major catastropies that are geographically localized.  IOW: if Pittsburgh
  were to get hit with a nuclear warhead, our customers would expect us
  to fail over to a second site.  Thus we need truely offsite backups.
 
 No.

Yes.

 If you have that kind of demand then you need fully replicated 
 geographically dispersed filesystems. Bringing things online from backups 
 will take too long.

You are correct.  But simply implementing geographically redundant
filesystems would not meet all our needs.  In some ways it would be
overkill and prohibitively expensive.  Additionally, redundancy
_never_ replaces backup.  I can have 1000 redundant systems dispersed
across the entire galaxy, and if a user accidentally deletes an
important record, I'll still need to go to backup to recover it,
since the redundant systems will faithfully delete it from all mirrors.

Here's the upshot:
1) We will have multiple locations that can fail over in near real
   time via database replication.
2) Each of these locations is redundant so that simple hardware failures
   are failed over transparently.
3) Each location keeps on-disk backups on a raid array for the purpose
   of restoring corrupt data.
4) One site will be used to generate backup tapes that will be taken
   offsite for legal and archival purposes.
5) Additionally, we have supporting systems used by staff, that are
   _not_ required to fail over like the product is.  However, the data
   in these is still important, just the recovery can be slower.  So:
 a) Some systems are locally redundant (such as DNS, LDAP)
 b) Data is backed up nightly to local RAID
 c) Data is backed up offsite weekly for legal/archival purposes.

The upshot is that the product is expected to be reliable, period.  It
has to survive considerable man-made or natural disasters.  Our DRP
reflects that.
The supporting stuff: email, financial records, and other business
stuff needs to be reliable, backed up, and archived - but the
requirements are less.  We can afford _some_ data loss in this area
(about a week) and we can afford some time to recover from a
catastrophe.

If we apply the business 

RE: [Bacula-users] Backup to disk then tape

[James Harper]

 _never_ replaces backup.  I can have 1000 redundant systems dispersed
 across the entire galaxy, and if a user accidentally deletes an
 important record, I'll still need to go to backup to recover it,
 since the redundant systems will faithfully delete it from all
mirrors.

There are filesystem based solutions to this particular problem, (and
I'm not talking about the 'recycle bin' :), where all changes to files
are kept in the filesystem for a specified period, and you can say to
the filesystem I want to see this file or this directory as it was at
2:37pm last Tuesday, just before my wife dropped by with my 2 year old
son and he started banging away on the keyboard while I wasn't
watching.

There are also application based solutions, eg CVS, although that
requires a bit more discipline and/or mucking around.

I'm not suggesting that this in any way reduces the need for an offsite
backup, but it can relieve some of the need to go and fetch last
Monday's tape from the offsite location to restore a single file.

James



---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid0709bid3057dat1642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk then tape

[Bill Moran]

On Tue, 2 May 2006 23:38:42 +1000
James Harper [EMAIL PROTECTED] wrote:

  _never_ replaces backup.  I can have 1000 redundant systems dispersed
  across the entire galaxy, and if a user accidentally deletes an
  important record, I'll still need to go to backup to recover it,
  since the redundant systems will faithfully delete it from all
 mirrors.
 
 There are filesystem based solutions to this particular problem, (and
 I'm not talking about the 'recycle bin' :), where all changes to files
 are kept in the filesystem for a specified period, and you can say to
 the filesystem I want to see this file or this directory as it was at
 2:37pm last Tuesday, just before my wife dropped by with my 2 year old
 son and he started banging away on the keyboard while I wasn't
 watching.
 
 There are also application based solutions, eg CVS, although that
 requires a bit more discipline and/or mucking around.
 
 I'm not suggesting that this in any way reduces the need for an offsite
 backup, but it can relieve some of the need to go and fetch last
 Monday's tape from the offsite location to restore a single file.

It's a good point, and part of our plan.  Using PostgreSQL transaction
log rotation will give us point-in-time recovery of the database.  We're
already using subversion to track a number of things.

As you noted, it doesn't replace the need for backups.  For one thing,
lawyers and governments stipulate certain things, and saying we don't
need that because we have this instead doesn't fly with them.

-- 
Bill Moran
Collaborative Fusion Inc.


IMPORTANT: This message contains confidential information and is
intended only for the individual named. If the reader of this
message is not an intended recipient (or the individual
responsible for the delivery of this message to an intended
recipient), please be advised that any re-use, dissemination,
distribution or copying of this message is prohibited. Please
notify the sender immediately by e-mail if you have received
this e-mail by mistake and delete this e-mail from your system.
E-mail transmission cannot be guaranteed to be secure or
error-free as information could be intercepted, corrupted, lost,
destroyed, arrive late or incomplete, or contain viruses. The
sender therefore does not accept liability for any errors or
omissions in the contents of this message, which arise as a
result of e-mail transmission.



---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk then tape

[Alan Brown]

On Tue, 2 May 2006, Bill Moran wrote:


 The time it takes to get them back onsite alone is too slow for most
recovery scenerios.


Hence the necessity for job cloning - AND a decent data firesafe.


What does a firesafe have to do with this?


Data backed up to disk can still be erased.

A New Zealand ISP got very publically humiliated in 1999 when a 14yo 
script kiddie did exactly this to their webservers - first he erased the 
backup disks and then he erased the data spools themselves.


Even without malicious intent, it's not difficult to come up with a number 
of Ooops causes for mounted filesystems getting trashed - including 
those being used as backup volumes.



2) D-D-T: ?


disk to disk to tape (which isn't done for the reasons you want it done)


Huh?  I'm not following either your explanation of the term, nor your
assertion as to why I would want to use it.


You were originally asking for a way of going backing up from disks, to a 
backup disk and then from there to tape. That's dist to disk to tape.



Then you DEFINITELY need faster tape.


No.  As previously described, the _tapes_ are fast enough, getting them
back from the offsite storage facility is too slow.


Treat archival/offsite sets differently to your onsite needs.

Local disk backup is fast, but it's vulnerable.

Local tape backup is marginally slower, but it's a lot harder to erase.

Either way is only suitable for file recovery or system rebuilds.

Offsite backup is only ever suitable for historic snapshot reebuilds.


Our need for offsite storage is different than our on-disk backups.
The offsite backups are for archival and legal requirements.  Our
on-disk backups are for handling 99% of the data-loss incidents.


It's the 1% which will kill you. Your need for a separate datacentre to be 
up and running quickly. Moves you out of the class of a simple backup 
system and into the realm of requiring replicated storage systems - which 
doesn't cost much (if anything) more for geographical failover abilities 
than it does for snapshotting with an eye to being up and running in a few 
hours - restoring off the tapes will take you at least a day for 1Tb of 
data.



Your other alternative is to backup twice, to 2 different pools.


Which is what we're doing now.


Bacula can run both backups simultaneously.


If you have that kind of demand then you need fully replicated
geographically dispersed filesystems. Bringing things online from backups
will take too long.


You are correct.  But simply implementing geographically redundant
filesystems would not meet all our needs.  In some ways it would be
overkill and prohibitively expensive.  Additionally, redundancy
_never_ replaces backup.  I can have 1000 redundant systems dispersed
across the entire galaxy, and if a user accidentally deletes an
important record, I'll still need to go to backup to recover it,
since the redundant systems will faithfully delete it from all mirrors.


Correct, hence why replicated systems need backup.

However once you are at that level, the issue raises that using backup 
disks themselves opens up a simlar erasure possibility, which in turn 
means that you need local backup tapes. Disks can be used too, but they 
shouldn't be the only line of local restoration.



The upshot is that the product is expected to be reliable, period.  It
has to survive considerable man-made or natural disasters.  Our DRP
reflects that.






The supporting stuff: email, financial records, and other business
stuff needs to be reliable, backed up, and archived - but the
requirements are less.  We can afford _some_ data loss in this area
(about a week) and we can afford some time to recover from a
catastrophe.


You'd be surprised, in many ways.

Email is only email until it goes down, at which point it becomes a 
business-critical application - and that's only the tip of the iceberg.


Your business stuff needs to be treated as importantly as the other 
items, else things get difficult, quickly.



If we apply the business DRP to our product, the product will be
unacceptable.  If we apply the product DRP to the business data,
we'll incur significant expenses that are unwarranted.


I've seen that argument used many times by IT admins, until things go 
wrong and they're told in no uncertain terms that it just aint so.



Additionally, redundancy and backups serve two different purposes.
Trying to use backup to solve redundancy problems will not work and
vice versa.


I never said it would, however your actual need becomes more vieweable 
the more you actually reveal.


Until this response you hadn't stated you already had any replication in 
place at all and it seemed you were relying on offsite backups as a 
failover mechanism.



The internet was designed (originally(*)) to withstand nuclear attack and
route around the damage.


Yeah, and the constitution of the US was supposed to protect individuals
from unreasonable taxation, but look how that ended up.


Do people 

[Bacula-users] Backup to disk then tape

[John Gerritse]

Hi all,

I work in a ICT company and we are looking to replace Amanda. I figured 
Bacula would be te best solution, but we hit a snag.


A feature my boss feels is paramount to a back-up package is write the 
data to disk, then to tape. Bacula does that in a way, but data 
spooling does only that; spooling. I need it to write to disk, then 
copy or migrate the data to tape. According to the documentation, that 
is a feature planned for the future. Unfortunately, I need it now.


I've considered simply backing data up to disk volumes, then copy that 
to tape by other means, but I'm puzzled how a restore would work in that 
sort of situation.


Is there a workaround someone has successfully used in the field, on 
production servers? Hating to have to reinvent a proverbial wheel, I 
figured I'd ask the community. I really love Bacula and I would love to 
use it in stead of giving Amanda a version bump and a fresh install on 
our shiny new backup server.


So in short; I want backups on disk, then copy them to tape (with 
Bacula, other backup software, commandline script, whatever). We run a 
schedule much like the one explained in the manual (a daily, weekly, 
monthly tape usage example).


Thanks in advance!


 John Gerritse
 Sogeti Nederland B.V.
 http://www.sogeti.nl

 ''~``
( o o )
+--.oooO--(_)--Oooo.--+
|   This email was brought to you by LINUX|
|.oooO|
|(   )   Oooo.|
+-\ ((   )+
   \_)) /
 (_/
We don't make mistakes, just happy little accidents.


Disclaimer:
This message contains information that may be privileged or confidential and is 
the property of Sogeti Nederland B.V. or its Group members. It is intended only 
for the person to whom it is addressed. If you are not the intended recipient, 
you are not authorized to read, print, retain, copy, disseminate, distribute, 
or use this message or any part thereof. If you receive this message in error, 
please notify the sender immediately and delete all copies of this message.



---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk then tape

[Kern Sibbald]

On Monday 01 May 2006 09:11, John Gerritse wrote:
 Hi all,

 I work in a ICT company and we are looking to replace Amanda. I figured
 Bacula would be te best solution, but we hit a snag.

 A feature my boss feels is paramount to a back-up package is write the
 data to disk, then to tape. Bacula does that in a way, but data
 spooling does only that; spooling. I need it to write to disk, then
 copy or migrate the data to tape. According to the documentation, that
 is a feature planned for the future. Unfortunately, I need it now.

 I've considered simply backing data up to disk volumes, then copy that
 to tape by other means, but I'm puzzled how a restore would work in that
 sort of situation.

 Is there a workaround someone has successfully used in the field, on
 production servers? Hating to have to reinvent a proverbial wheel, I
 figured I'd ask the community. I really love Bacula and I would love to
 use it in stead of giving Amanda a version bump and a fresh install on
 our shiny new backup server.

 So in short; I want backups on disk, then copy them to tape (with
 Bacula, other backup software, commandline script, whatever). We run a
 schedule much like the one explained in the manual (a daily, weekly,
 monthly tape usage example).

 Thanks in advance!

In Bacula terminology what you want is called Migration.  I am currently 
working on it and expected to have it mostly completed by now, however I am 
running a bit behind (see the email I will send later today for more 
details).  The basics of the feature are  working in the development code, 
but it will be several months before it is complete and ready for release.  I 
had originally planed the release for June-July, but it is now looking like 
August ...



   John Gerritse
   Sogeti Nederland B.V.
   http://www.sogeti.nl

   ''~``
  ( o o )
 +--.oooO--(_)--Oooo.--+

 |   This email was brought to you by LINUX|
 |.oooO|
 |(   )   Oooo.|

 +-\ ((   )+
 \_)) /
   (_/
 We don't make mistakes, just happy little accidents.


 Disclaimer:
 This message contains information that may be privileged or confidential
 and is the property of Sogeti Nederland B.V. or its Group members. It is
 intended only for the person to whom it is addressed. If you are not the
 intended recipient, you are not authorized to read, print, retain, copy,
 disseminate, distribute, or use this message or any part thereof. If you
 receive this message in error, please notify the sender immediately and
 delete all copies of this message.



 ---
 Using Tomcat but need to do more? Need to support web services, security?
 Get stuff done quickly with pre-integrated technology to make your job
 easier Download IBM WebSphere Application Server v.1.0.1 based on Apache
 Geronimo
 http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
 ___
 Bacula-users mailing list
 Bacula-users@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bacula-users

-- 
Best regards,

Kern

  (
  /\
  V_V


---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk then tape

[Kern Sibbald]

On Monday 01 May 2006 10:11, John Gerritse wrote:
 Great! Thanks for the quick reply.

 I'm looking forward to the new release that will include migration.
 Is there a workaround, so we can use Bacula anyway until the new feature
   is implemented?

I don't think so, but it depends on what you are trying to do. It would be 
best to ask the list if they have any suggestions. My setup is rather simple 
so I don't use this kind of feature.


 Kind regards,

 John

 Kern Sibbald wrote:
  On Monday 01 May 2006 09:11, John Gerritse wrote:
 Hi all,
 
 I work in a ICT company and we are looking to replace Amanda. I figured
 Bacula would be te best solution, but we hit a snag.
 
 A feature my boss feels is paramount to a back-up package is write the
 data to disk, then to tape. Bacula does that in a way, but data
 spooling does only that; spooling. I need it to write to disk, then
 copy or migrate the data to tape. According to the documentation, that
 is a feature planned for the future. Unfortunately, I need it now.
 
 I've considered simply backing data up to disk volumes, then copy that
 to tape by other means, but I'm puzzled how a restore would work in that
 sort of situation.
 
 Is there a workaround someone has successfully used in the field, on
 production servers? Hating to have to reinvent a proverbial wheel, I
 figured I'd ask the community. I really love Bacula and I would love to
 use it in stead of giving Amanda a version bump and a fresh install on
 our shiny new backup server.
 
 So in short; I want backups on disk, then copy them to tape (with
 Bacula, other backup software, commandline script, whatever). We run a
 schedule much like the one explained in the manual (a daily, weekly,
 monthly tape usage example).
 
 Thanks in advance!
 
  In Bacula terminology what you want is called Migration.  I am currently
  working on it and expected to have it mostly completed by now, however I
  am running a bit behind (see the email I will send later today for more
  details).  The basics of the feature are  working in the development
  code, but it will be several months before it is complete and ready for
  release.  I had originally planed the release for June-July, but it is
  now looking like August ...
 
   John Gerritse
   Sogeti Nederland B.V.
   http://www.sogeti.nl
 
   ''~``
  ( o o )
 +--.oooO--(_)--Oooo.--+
 
 |   This email was brought to you by LINUX|
 |.oooO|
 |(   )   Oooo.|
 
 +-\ ((   )+
 \_)) /
   (_/
 We don't make mistakes, just happy little accidents.
 
 
 Disclaimer:
 This message contains information that may be privileged or confidential
 and is the property of Sogeti Nederland B.V. or its Group members. It is
 intended only for the person to whom it is addressed. If you are not the
 intended recipient, you are not authorized to read, print, retain, copy,
 disseminate, distribute, or use this message or any part thereof. If you
 receive this message in error, please notify the sender immediately and
 delete all copies of this message.
 
 
 
 ---
 Using Tomcat but need to do more? Need to support web services, security?
 Get stuff done quickly with pre-integrated technology to make your job
 easier Download IBM WebSphere Application Server v.1.0.1 based on Apache
 Geronimo
 http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
 ___
 Bacula-users mailing list
 Bacula-users@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/bacula-users

 Disclaimer:
 This message contains information that may be privileged or confidential
 and is the property of Sogeti Nederland B.V. or its Group members. It is
 intended only for the person to whom it is addressed. If you are not the
 intended recipient, you are not authorized to read, print, retain, copy,
 disseminate, distribute, or use this message or any part thereof. If you
 receive this message in error, please notify the sender immediately and
 delete all copies of this message.

-- 
Best regards,

Kern

  (
  /\
  V_V


---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup to disk then tape

[John Boris]

John,
We do that here (Backup to disk and then to tape). We have just started
using Bacula but we us it to backup to tape. I use Microlite's
BackupEDGE to backup my remote Servers to disk on my main backup server.
I have a network that is only used by me over night so differentials at
night are a piece of cake. We have a long time invested in BackupEDGE on
our SCO Servers. Then during the day I use Bacula to backup my Server
that holds the nightly backups. This works fine for us.  I am not a
Bacula Guru but I think it would be just a matter of setting up
different jobs at the right interval to do what you want. 

As for restores here we would first look to the on disk differentials
to see if the required file is there. In case of a disaster here I would
have to make a site visit to rebuild things so I would have a tape on
site of the Master, then once the system is back online I would just
restore the last differential. If I had to go to a previous volume that
was no longer on Disk I would locate that volume in the tape library and
restore that volume to disk and then restore that to my system.  Before
we went to this we only kept the last thirty days of data anyway so this
provides us a longer recovery trail. I have had users delete something
and then ask me 45 days later if I still have the file. But in those
cases it was usually just a report format that could be recreated in
less time than it takes to restore the file.


John J. Boris, Sr.
JEN-A-SyS Administrator
Archdiocese of Philadelphia

Remember! That light at the end of the tunnel
Just might be the headlight of an oncoming train!

 John Gerritse [EMAIL PROTECTED] 05/01/06 3:11 AM 
Hi all,

I work in a ICT company and we are looking to replace Amanda. I figured

Bacula would be te best solution, but we hit a snag.

A feature my boss feels is paramount to a back-up package is write the

data to disk, then to tape. Bacula does that in a way, but data 
spooling does only that; spooling. I need it to write to disk, then 
copy or migrate the data to tape. According to the documentation, that

is a feature planned for the future. Unfortunately, I need it now.

I've considered simply backing data up to disk volumes, then copy that

to tape by other means, but I'm puzzled how a restore would work in
that 
sort of situation.

Is there a workaround someone has successfully used in the field, on 
production servers? Hating to have to reinvent a proverbial wheel, I 
figured I'd ask the community. I really love Bacula and I would love to

use it in stead of giving Amanda a version bump and a fresh install on

our shiny new backup server.

So in short; I want backups on disk, then copy them to tape (with 
Bacula, other backup software, commandline script, whatever). We run a

schedule much like the one explained in the manual (a daily, weekly, 
monthly tape usage example).

Thanks in advance!


  John Gerritse
  Sogeti Nederland B.V.
  http://www.sogeti.nl 

  ''~``
 ( o o )
+--.oooO--(_)--Oooo.--+
|   This email was brought to you by LINUX|
|.oooO|
|(   )   Oooo.|
+-\ ((   )+
\_)) /
  (_/
We don't make mistakes, just happy little accidents.


Disclaimer:
This message contains information that may be privileged or
confidential and is the property of Sogeti Nederland B.V. or its Group
members. It is intended only for the person to whom it is addressed. If
you are not the intended recipient, you are not authorized to read,
print, retain, copy, disseminate, distribute, or use this message or any
part thereof. If you receive this message in error, please notify the
sender immediately and delete all copies of this message.



---
Using Tomcat but need to do more? Need to support web services,
security?
Get stuff done quickly with pre-integrated technology to make your job
easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache
Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net 
https://lists.sourceforge.net/lists/listinfo/bacula-users 





---
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users