Re: [Bacula-users] can file retention be job rather than client based?

2018-04-11 Thread mark . bergman 
In the message dated: Wed, 11 Apr 2018 06:25:43 -0700,
The pithy ruminations from Stephen Thompson on 
[Re: [Bacula-users] can file retention be job rather than client based?] were:
=> 
=> Thanks Kern.
=> 
=> I think given the limited nature of his need, I may use a postrun script 

+1 for this request.

I've got what may be a similar environment -- one major Bacula "client"
(ie., a floating VIP/hostname pointing to a fileserver) with ~150 jobs,
ranging from KB to multi-TB. Varying retention periods per-job would be
a tremendous enhancement.

=> to simply wipe database records out of band.
=> 
=> Also if I did use multi-client definitions, I would need to use the same 
=> pool as they all go to the same monthly tapes.

Same here.

Mark


=> 
=> Stephen
=> 

-- 
Mark Bergman   voice: 215-746-4061  
 
mark.berg...@uphs.upenn.edu  fax: 215-614-0266
http://www.med.upenn.edu/cbica/
IT Technical Director, Center for Biomedical Image Computing and Analytics
Department of Radiology University of Pennsylvania

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula 9.x.x and Solaris 11.3 (and OpenSUSE)...

2018-04-11 Thread Dimitri Maziuk 

On 2018-04-11 03:52, Radosław Korzeniewski wrote:

I'm very surprised that your build is not working and you have to 
disable libtool. Unfortunate I never tried Solaris Compiler Suite with 
Bacula.


I haven't used solaris in forever but back when, libtool did not work 
with Sun's CC. Linker commands, options, and everything else libtool 
does was simply wrong for Studio toolchain, linking step failed every time.


Dima

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] can file retention be job rather than client based?

2018-04-11 Thread Dimitri Maziuk 

On 2018-04-11 01:59, Kern Sibbald wrote:

I have never used two Client definitions to backup the same machine, but 
in principle it would work fine.


The closest I have is a "cluster ip" floating between two machines, with 
3 client/job definitions: one for each physical machine (backing up /etc 
and crontabs) and one for the cluster ip (backing up DRBD filesystem 
mounted on the host that has cluster ip).


The only thing non-obvious to me was that bacula-fd running as

FileDaemon {
  name = nodea-fd
...
}

will happily answer to (bacula-dir)

Client {
  name = nodea-fd
  address = 1.2.3.4
  ...
}
Client {
  name = nodeb-fd
  address = 1.2.3.5
  ...
}
Client {
  name = cluster-fd
  address = 1.2.3.1
  ...
}
Job {
  name = cluster-drbd
  client = cluster-fd
  fileset = drbd
  ...
}

(I.e there is no FileDaemon named "cluster-fd" running anywhere but the 
Job and Client definitions for it work just fine.)


Dima

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] can file retention be job rather than client based?

2018-04-11 Thread Stephen Thompson 


Thanks Kern.

I think given the limited nature of his need, I may use a postrun script 
to simply wipe database records out of band.


Also if I did use multi-client definitions, I would need to use the same 
pool as they all go to the same monthly tapes.


Stephen


On 4/10/18 11:59 PM, Kern Sibbald wrote:

Hello Stephen,

What you are asking for, as you suspect, does not exist and implementing 
it would be a bit problematic because every Job would need to keep it's 
own retention period.  For one client, there can be any number of Jobs 
-- typically thousands.  Thus the catalog would grow faster (more data 
for the File table having the most records), and the complexity of 
pruning including the time to prune would probably explode -- probably 
thousands of times slower.


I have never used two Client definitions to backup the same machine, but 
in principle it would work fine.  If you name your Clients appropriately 
it might be easier to remember what was done.  E.g. 
Client1-Normal-Files, Client1-Archived-Files, ... Also, if you put clear 
comments on the resource definitions, it would help.  Note two things, 
if you go this route:


1. Be sure to define each of your two Client1-xxx with different Pools 
with different Volume retention periods
2. I would appreciate feedback on how this works -- especially 
operationally


Best regards,
Kern

PS: At the current time the Enterprise version of Bacula has a number of 
performance improvements that should significantly speed up the backups 
of 50+million files.  It does this at a small extra expense (size) of 
the catalog.


On 04/07/2018 06:21 AM, Stephen Thompson wrote:


I believe the answer is no, but as a happy bacula user for 10 years I 
am somewhat surprised at the lack of flexibility.


The scenarios is this:  A fileserver (1 client) with dozens of large 
(size-wise) filesystems (12 jobs), but a couple of those filesystems 
are large (filecount-wise).  We would really like to set different 
file retention periods on those high-filecount jobs (50+million), 
because they are forcing the Catalog to go beyond our size 
constraints. However, we also don't want to lose the file retention 
longevity of that client's other jobs (5 years).  The only hack I can 
think of is to define 2 clients for 1 actual host, but I'd rather not 
go down that route, because tracking jobs and associating them, 
especially over multiple years, will get that much more tricky.


Ideas?

thanks,
Stephen


--
Stephen Thompson   Berkeley Seismo Lab
step...@seismo.berkeley.edu215 McCone Hall
Office: 510.664.9177   University of California
Remote: 510.214.6506 (Tue) Berkeley, CA 94720-4760

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula 9.x.x and Solaris 11.3 (and OpenSUSE)...

2018-04-11 Thread Radosław Korzeniewski 
Hello,

I'm building Bacula 9.0.6 on Solaris 11 using gcc (I'm not sure if I used
gcc provided by Oracle or installed externally) and the following configure
(which forces 64bit compilation and debugging symbols):

CXXFLAGS="-m64 -g" CFLAGS="-m64 -g" LDFLAGS="-m64 -g" ./configure
--prefix=/export/home/radekk/bacula
--with-working-dir=/export/home/radekk/bacula/working
--with-pid-dir=/export/home/radekk/bacula/working
--with-subsys-dir=/export/home/radekk/bacula/subsys --with-postgresql
--enable-smartalloc --enable-lockmgr
--with-logdir=/export/home/radekk/bacula/log

I'm very surprised that your build is not working and you have to disable
libtool. Unfortunate I never tried Solaris Compiler Suite with Bacula. I
have to check if I have access to it.

best regards

2018-04-06 14:53 GMT+02:00 Gary R. Schmidt :

> On 06/04/2018 18:40, Daniel Heitepriem wrote:
>
>> Hi Gary,
>>
>> I just tried to compile 9.0.6 on a Solaris 11.3 box (but using GCC 5.4.0
>> and MySQL) with these options:
>>
>> CFLAGS="-g -m64" LDFLAGS="-m64" CXXFLAGS="-m64" ./configure \
>> --prefix=/opt/bacula \
>> --with-dir-user=bacula \
>> --with-dir-group=bacula \
>> --with-sd-user=bacula \
>> --with-sd-group=bacula \
>> --with-fd-user=root \
>> --with-fd-group=bacula \
>> --enable-smartalloc \
>> --sbindir=/opt/bacula/bin \
>> --sysconfdir=/opt/bacula/etc \
>> --with-subsys-dir=/opt/bacula/var \
>> --with-working-dir=/opt/bacula/var/bacula/working \
>> --with-pid-dir=/opt/bacula/var \
>> --with-logdir=/opt/bacula/var \
>> --with-archivedir=/opt/bacula/var \
>> --with-mysql=/opt/mysql \
>> --with-openssl
>>
>> and it's working fine so far. Your issue may be related to the
>> "disable-libtool" parameter you provided. Can you try to build it
>> without this disable parameter.
>>
>> Thank you Daniel, removing "--disable-libtool" allowed it to build, why
> whoever disabled it didn't make configure barf on it is something I can
> only wonder about.
>
> And in some further fiddling about I found that replacing "NO_ECHO = @"
> with "NO_ECHO =" in the makefiles makes it *much* easier to work out just
> what is going wrong.  Hiding information just makes life difficult.
>
>
> Cheers,
> GaryB-)
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>



-- 
Radosław Korzeniewski
rados...@korzeniewski.net
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] precedence of config parameters in bacula?

2018-04-11 Thread Radosław Korzeniewski 
Hello,

2018-04-10 13:13 GMT+02:00 Johannsen, Thorsten :

> Hello list,
>
> in my mail
>
>
> [Bacula-users] scheduled job different that manually initiated job? --WAS:
> Re: Backup problem with 1 of 2 Raspberry PI using Raspbian (Debian)
>
> one of the problems I was facing is colliding configuration parameters. I
> set "accudate=no" in the job config, but "accurate=yes" in the schedule.
>
>
> I noticed this a few times that it is possible to specify an option in
> multiple locations. (Though I cannot give another example at the moment)
>

There are a dozen of a such parameters. :)


>
> I am wondering if there is some kind of overview or documentation about
> the order of precedence of configuration parameters?
>
>
I think all is described in documentation in different places, i.e.

"17.5 The Schedule Resource
(...)
The Job-overrides permit overriding the Level, the Storage, the Messages,
and the Pool specifications provided in the Job resource.
(...)"

You have to carefully read it.

best regards
-- 
Radosław Korzeniewski
rados...@korzeniewski.net
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] can file retention be job rather than client based?

2018-04-11 Thread Kern Sibbald 

Hello Stephen,

What you are asking for, as you suspect, does not exist and implementing 
it would be a bit problematic because every Job would need to keep it's 
own retention period.  For one client, there can be any number of Jobs 
-- typically thousands.  Thus the catalog would grow faster (more data 
for the File table having the most records), and the complexity of 
pruning including the time to prune would probably explode -- probably 
thousands of times slower.


I have never used two Client definitions to backup the same machine, but 
in principle it would work fine.  If you name your Clients appropriately 
it might be easier to remember what was done.  E.g. 
Client1-Normal-Files, Client1-Archived-Files, ... Also, if you put clear 
comments on the resource definitions, it would help.  Note two things, 
if you go this route:


1. Be sure to define each of your two Client1-xxx with different Pools 
with different Volume retention periods

2. I would appreciate feedback on how this works -- especially operationally

Best regards,
Kern

PS: At the current time the Enterprise version of Bacula has a number of 
performance improvements that should significantly speed up the backups 
of 50+million files.  It does this at a small extra expense (size) of 
the catalog.


On 04/07/2018 06:21 AM, Stephen Thompson wrote:


I believe the answer is no, but as a happy bacula user for 10 years I 
am somewhat surprised at the lack of flexibility.


The scenarios is this:  A fileserver (1 client) with dozens of large 
(size-wise) filesystems (12 jobs), but a couple of those filesystems 
are large (filecount-wise).  We would really like to set different 
file retention periods on those high-filecount jobs (50+million), 
because they are forcing the Catalog to go beyond our size 
constraints. However, we also don't want to lose the file retention 
longevity of that client's other jobs (5 years).  The only hack I can 
think of is to define 2 clients for 1 actual host, but I'd rather not 
go down that route, because tracking jobs and associating them, 
especially over multiple years, will get that much more tricky.


Ideas?

thanks,
Stephen



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users