Re: [Bacula-users] A device per pool

[Dimitri Maziuk]

On 09/08/2017 11:27 AM, Phil Stracchino wrote:
> On 09/08/17 11:28, Dimitri Maziuk wrote:
>>
>> We do what we must because we can,
>> for the good of all of us. ;)
> 
> But there's no use crying over every mistake
> You just keep on trying 'til you run out of cake

I think the answer to your question is why use multiple pools? If you
have a reason to use separate pools, you may have a reason to put them
on separate storages. Or not.

-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu



signature.asc
Description: OpenPGP digital signature
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] A device per pool

[Phil Stracchino]

On 09/08/17 11:28, Dimitri Maziuk wrote:
> 
> We do what we must because we can,
> for the good of all of us. ;)

But there's no use crying over every mistake
You just keep on trying 'til you run out of cake


-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] A device per pool

[Dimitri Maziuk]

On 2017-09-08 09:39, Dan Langille wrote:
On Sep 7, 2017, at 11:45 PM, Phil Stracchino 

Re: [Bacula-users] A device per pool

[Phil Stracchino]

On 09/08/17 10:39, Dan Langille wrote:
>> On Sep 7, 2017, at 11:45 PM, Phil Stracchino >
>> I'm curious.  What do you see as the advantage of this layout?

> ###
> 
> If the Pool is no longer required, it is easy to delete. Dividing data
> up this way
> it's always a good idea, because of the flexibility for future
> manipulation it provides. 
> 
> If these were all on separate devices, you'd get better concurrent
> throughput.
> 
> Full backups are usually bigger, incremental backups are usually
> smaller, so you 
> could just recordsize accordingly.
> 
> If you assigned each client to a different pool, deleting their backups
> when the
> client leaves is now a simple matter of deleting the appropriate ZFS
> datasets.
> 
> If you want to move a pool to a different bacula-sd, you move that data set.
> 
> ###

Makes sense.  :)


-- 
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] A device per pool

[Dan Langille]

> On Sep 7, 2017, at 11:45 PM, Phil Stracchino  wrote:
> 
> On 09/07/17 22:04, Dan Langille wrote:
>> I have recently moved to one Storage device per pool.
>> 
>> Why?
>> 
>> So each pool is on a different ZFS dataset.
> 
> 
> I'm curious.  What do you see as the advantage of this layout?


Background: I'm using ZFS on FreeBSD.  I like keeping application specific data
in its own ZFS dataset. It allows for an easy snapshot and backup solution.

Storage space was the original motivation. One zpool was at 80%,
the other is now at about 44%. I wanted to move some data from one zpool
to another.

When moving to another pool, the mount point changes, which meant creating
a new Storage device pointing at that mount point.

After moving the first Pool I figured: well, if it's this easy, why not do it 
for each Bacula Pool?

Back to your question: A similar question was asked on Google+: 

https://plus.google.com/+DanLangille/posts/K2zfNo1ii22 


ZFS can certainly and easily handle many files in the same directory. That 
wasn't
a basis for change.

My answer from there (with slight edits) appears below:

###

If the Pool is no longer required, it is easy to delete. Dividing data up this 
way
it's always a good idea, because of the flexibility for future manipulation it 
provides. 

If these were all on separate devices, you'd get better concurrent throughput.

Full backups are usually bigger, incremental backups are usually smaller, so 
you 
could just recordsize accordingly.

If you assigned each client to a different pool, deleting their backups when the
client leaves is now a simple matter of deleting the appropriate ZFS datasets.

If you want to move a pool to a different bacula-sd, you move that data set.

###

The above was typed very early in my day and I wasn't quite awake.

Perhaps the highlights are:

- keep like data in like datasets
- more datasets gives more flexibility
- because I can (I was at vBSDCon and had the time to do this)

Ideas? Suggestions?

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Bacula 9.0.3 tape pool definition reports incorrectly

[Hicks, Daniel CTR OSD DMEA]

Hello, 

 

I am using Bacula 9.0.3 on Centos7 and have the following issue when
reviewing storage from bat. I am using LTO-5 tapes that are 1.5TB raw and
3TB compressed. I believe that I have set the Bacula-dir.conf tape pool
definition correctly but when I look at bat after a single job wrote to
tape. That job consisting of 479MG in size but bat reports that 48% of the
tape is used. 

 

Please let me know if there is better way to define the tapes and if so
where and how. 

 

 

# Tape pool definition

Pool {

  Name = Tape

  Pool Type = Backup

  Recycle = yes

  AutoPrune = yes

  Volume Retention = 5 years

  Maximum Volume Bytes = 1500G

  Storage = HP-Overland-TL

}

 

Thanks

 

Daniel Hicks

Senior Systems Analyst

DMEA IT Support

 



smime.p7s
Description: S/MIME cryptographic signature
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] A device per pool

[Dan Langille]

> On Sep 8, 2017, at 6:47 AM, Martin Simmons  wrote:
> 
> That's a useful article.

Thank you.

> There is a minor caveat when copying volume files: the volume label contains
> the PoolName and MediaType.  This is minor because it only matters if you need
> to bscan the volume before it is recycled.

I hope nobody ever has to use bscan. (using bscan means something has gone 
wrong;
this is not a criticism of the tool).

I don't think we can easily change that label.  For those reading, here is an 
older document
I found via Google. 
http://www.bacula.org/2.4.x-manuals/en/developers/Storage_Media_Output_Format.html
 


As you say, it won't be relevant until bscan time.

-- 
Dan Langille - BSDCan / PGCon
d...@langille.org





> __Martin
> 
> 
> 
>> On Thu, 7 Sep 2017 22:04:35 -0400, Dan Langille said:
>> 
>> I have recently moved to one Storage device per pool.
>> 
>> Why?
>> 
>> So each pool is on a different ZFS dataset.
>> 
>> Once I figured out the process for the first one, the rest was mostly 
>> waiting for data to be copied.
>> 
>> I documented it here: 
>> https://dan.langille.org/2017/09/06/bacula-ran-out-of-space-moved-some-volumes-to-another-zpool/
>>  
>> 
>> 
>> The final zfs layout looks like this: 
>> https://gist.github.com/dlangille/78811ba69b2a14f6c1e63c2b4bafc99e 
>> 
>> 
>> Here is the rough overview:
>> 
>> 
>> # in bacula-dir.conf
>> 
>> Pool {
>>  Name = DiffFile
>>  Pool Type= Backup
>>  Recycle  = yes
>>  AutoPrune= yes
>>  Volume Retention = 6 weeks
>>  Storage  = bacula-sd-01-DiffFile
>>  Next Pool= Differentials
>> 
>>  Maximum Volume Bytes = 10G
>>  Maximum Volume Jobs  = 1
>>  Maximum Volumes  = 300
>> 
>>  LabelFormat = "DiffAuto-"
>> }
>> 
>> # Definiton of file storage device
>> Storage {
>>  Name   = bacula-sd-01-DiffFile
>>  Address= bacula-sd-01.int.unixathome.org# N.B. Use a 
>> fully qualified name here
>>  SDPort = 9103
>>  Password   = "[redacted]"
>> 
>>  AutoChanger = Yes
>> 
>>  Device = VirtualDisk-DiffFile
>>  Media Type = DiffFile
>> 
>>  Maximum Concurrent Jobs = 27
>> }
>> 
>> # in bacula-sd.conf
>> 
>> # why do I do an autochanger? Some AutoChangers have multiple drives.
>> # It's easier to make them all similar.
>> # an example with 10 virtual drives is here: 
>> https://gist.github.com/dlangille/ba787f6e6d838d2fbcc6ad9aad9afd55 
>> 
>> 
>> Autochanger {
>>  Name = VirtualDisk-DiffFile
>> 
>>  Changer Device  = /dev/null
>>  Changer Command = /dev/null
>> 
>>  Device  = vDrive-DiffFile-0
>> }
>> 
>> Device {
>>  Name   = vDrive-DiffFile-0
>>  Media Type = DiffFile
>>  Archive Device = /usr/local/bacula/volumes/DiffFile
>>  LabelMedia = yes
>>  Random Access  = yes
>>  AutomaticMount = yes
>>  RemovableMedia = no
>>  AlwaysOpen = no
>> 
>>  Autochanger= yes
>>  Drive Index= 0
>> 
>>  Maximum Concurrent Jobs = 1
>>  Volume Poll Interval= 15
>> }
>> 
>> 
>> 
>> 
>> Dan Langille - BSDCan / PGCon
>> d...@langille.org
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Bacula admin tool download

[Adam Weremczuk]

Hello,

I'm after BAT for Bacula community edition version 5.2.6.

The binary I've downloaded from SourceForge: 
https://sourceforge.net/projects/bacula/files/Win32_64/5.2.6/


only appears to offer bconsole.exe and bwx-console.exe

BaculaSystem 
https://www.baculasystems.com/bat-the-bacula-administration-tool


only appears to offer free trial sign up, even though a limited BAT 
version should be free for all.


I'm reluctant to download and install from other, potentially dodgy, 
sources.


Please advise.

Regards
Adam


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] A device per pool

[Martin Simmons]

That's a useful article.

There is a minor caveat when copying volume files: the volume label contains
the PoolName and MediaType.  This is minor because it only matters if you need
to bscan the volume before it is recycled.

__Martin



> On Thu, 7 Sep 2017 22:04:35 -0400, Dan Langille said:
> 
> I have recently moved to one Storage device per pool.
> 
> Why?
> 
> So each pool is on a different ZFS dataset.
> 
> Once I figured out the process for the first one, the rest was mostly waiting 
> for data to be copied.
> 
> I documented it here: 
> https://dan.langille.org/2017/09/06/bacula-ran-out-of-space-moved-some-volumes-to-another-zpool/
>  
> 
> 
> The final zfs layout looks like this: 
> https://gist.github.com/dlangille/78811ba69b2a14f6c1e63c2b4bafc99e 
> 
> 
> Here is the rough overview:
> 
> 
> # in bacula-dir.conf
> 
> Pool {
>   Name = DiffFile
>   Pool Type= Backup
>   Recycle  = yes
>   AutoPrune= yes
>   Volume Retention = 6 weeks
>   Storage  = bacula-sd-01-DiffFile
>   Next Pool= Differentials
> 
>   Maximum Volume Bytes = 10G
>   Maximum Volume Jobs  = 1
>   Maximum Volumes  = 300
> 
>   LabelFormat = "DiffAuto-"
> }
> 
> # Definiton of file storage device
> Storage {
>   Name   = bacula-sd-01-DiffFile
>   Address= bacula-sd-01.int.unixathome.org# N.B. Use a 
> fully qualified name here
>   SDPort = 9103
>   Password   = "[redacted]"
> 
>   AutoChanger = Yes
> 
>   Device = VirtualDisk-DiffFile
>   Media Type = DiffFile
> 
>   Maximum Concurrent Jobs = 27
> }
> 
> # in bacula-sd.conf
> 
> # why do I do an autochanger? Some AutoChangers have multiple drives.
> # It's easier to make them all similar.
> # an example with 10 virtual drives is here: 
> https://gist.github.com/dlangille/ba787f6e6d838d2fbcc6ad9aad9afd55 
> 
> 
> Autochanger {
>   Name = VirtualDisk-DiffFile
> 
>   Changer Device  = /dev/null
>   Changer Command = /dev/null
> 
>   Device  = vDrive-DiffFile-0
> }
> 
> Device {
>   Name   = vDrive-DiffFile-0
>   Media Type = DiffFile
>   Archive Device = /usr/local/bacula/volumes/DiffFile
>   LabelMedia = yes
>   Random Access  = yes
>   AutomaticMount = yes
>   RemovableMedia = no
>   AlwaysOpen = no
> 
>   Autochanger= yes
>   Drive Index= 0
> 
>   Maximum Concurrent Jobs = 1
>   Volume Poll Interval= 15
> }
> 
> 
> 
> 
> Dan Langille - BSDCan / PGCon
> d...@langille.org

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users