Re: [Bacula-users] File deamon not following configuration
bconsole doesn't seem to support this. * list media pool=File > out.txt Still prints them on screen. Wouter. On Fri, 2013-06-14 at 09:42 +0200, Radosław Korzeniewski wrote: > Hello, > > 2013/6/14 Wouter van Marle > OK just checked on this. > > When entering this command I get a list of a couple thousand > volumes. I > think I should invalidate/clear them all. Heading of the table > rolls > right off the screen so no idea what the numbers mean! > > > > Redirect it to the file and check again. :) > > > best regards > P.S. please do not toppost on this list. > -- > Radosław Korzeniewski > rados...@korzeniewski.net -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] File deamon not following configuration
Upon a bit further investigation, I find that at least now the 50M limit is gone; just the problem remains that old volume names are being reused instead of new volumes being created. Is there a way to just purge all the existing volumes from the database? Or blanket change them to no recycling? Wouter. On Wed, 2013-05-15 at 12:21 +0200, Radosław Korzeniewski wrote: > Hello, > > > 2013/5/15 Wouter van Marle > Hereby example of a message from a job. It seems to be > recycling a > volume, even though I have set "Recycle = no" in > bacula-dir.conf! > > > Show output of the bconsole command (the pool name is the pool > resource where your recycled volume is assigned, put below a valid > one): > * list media pool=File > > > Then check if recycle parameter is set to '0' or '1' for this > particular volume. If it is setup to '1' then you can update this flag > with "update" command. > > > best regards > -- > Radosław Korzeniewski > rados...@korzeniewski.net -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] File deamon not following configuration
OK just checked on this. When entering this command I get a list of a couple thousand volumes. I think I should invalidate/clear them all. Heading of the table rolls right off the screen so no idea what the numbers mean! Wouter. On Wed, 2013-05-15 at 12:21 +0200, Radosław Korzeniewski wrote: > Hello, > > > 2013/5/15 Wouter van Marle > Hereby example of a message from a job. It seems to be > recycling a > volume, even though I have set "Recycle = no" in > bacula-dir.conf! > > > Show output of the bconsole command (the pool name is the pool > resource where your recycled volume is assigned, put below a valid > one): > * list media pool=File > > > Then check if recycle parameter is set to '0' or '1' for this > particular volume. If it is setup to '1' then you can update this flag > with "update" command. > > > best regards > -- > Radosław Korzeniewski > rados...@korzeniewski.net -- This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] File deamon not following configuration
On Thu, 2013-05-09 at 12:46 +0200, Radosław Korzeniewski wrote:
> Hello,
>
> 2013/5/9 Wouter van Marle
> *show pool=File
> Pool: name=File PoolType=Backup
> use_cat=1 use_once=1 cat_files=1
> max_vols=0 auto_prune=1 VolRetention=1 year
> VolUse=0 secs recycle=0 LabelFormat=Squirrel-${JobName}
> CleaningPrefix=*None* LabelType=0
> RecyleOldest=0 PurgeOldest=0 ActionOnPurge=0
> MaxVolJobs=0 MaxVolFiles=0 MaxVolBytes=53687091200
>
>
> Maxvolbytes above is your volume "splitter" at 50MB chunks.
That number is 50GB already. Not 50 MB. So again the configuration the
console gives me is correct.
>
> MigTime=0 secs MigHiBytes=0 MigLoBytes=0
> JobRetention=0 secs FileRetention=4 years 8 months 23
> days 11
> hours 8 mins 40 secs
>
> (note: this LabelFormat is correct, it is what I expect to see
> as my
> file name! However the file name my volumes get is different.)
>
>
> If you'd like to get a single job on single volume which will get its
> name from $job name then you have to set maximum volume jobs to '1'.
> Then another job will get a new volume, hopefully with creation of new
> volume.
I don't have such a setting at all - where exactly do I have to set
this? It may be the issue as everything else seems to be correct.
Wouter.
>
>
> best regards
> --
> Radosław Korzeniewski
> rados...@korzeniewski.net
--
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and
their applications. This 200-page book is written by three acclaimed
leaders in the field. The early access version is available now.
Download your free book today! http://p.sf.net/sfu/neotech_d2d_may
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] File deamon not following configuration
I have done an update for the File and Default pools in bconsole, but the problem persists. Still not using the correct file names, and still cutting it off at 50M per chunk. Wouter. On Mon, 2013-05-06 at 11:49 +0200, Radosław Korzeniewski wrote: > Hello, > > 2013/5/6 Wouter van Marle > Hi, > > I'm having a big problem with my bacula configuration. > > It is supposed to create a new large archive every time a job > is run, > with a.o. the job name and start date and time in the name. > > Now some time ago I changed this scheme, made it small chunks > (50 MB), > many chunks for one archive. I didn't like it, turned it back > (or so I > thought) but Bacula is not only still archiving in 50 MB > chunks, it's > recycling names from back in 2012! > > > > You should check Pool configuration in bacula-dir.conf and be sure to > update pool resource with "update" command in bconsole. > > > best regards > -- > Radosław Korzeniewski > rados...@korzeniewski.net -- Learn Graph Databases - Download FREE O'Reilly Book "Graph Databases" is the definitive new guide to graph databases and their applications. This 200-page book is written by three acclaimed leaders in the field. The early access version is available now. Download your free book today! http://p.sf.net/sfu/neotech_d2d_may ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] File deamon not following configuration
Hi,
I'm having a big problem with my bacula configuration.
It is supposed to create a new large archive every time a job is run,
with a.o. the job name and start date and time in the name.
Now some time ago I changed this scheme, made it small chunks (50 MB),
many chunks for one archive. I didn't like it, turned it back (or so I
thought) but Bacula is not only still archiving in 50 MB chunks, it's
recycling names from back in 2012!
I've tried many things, I'm quite sure my configuration is fine now, but
it still doesn't work. Hereby bacula-dir.conf (stripped password for
posting):
#
# Default Bacula Director Configuration file
#
# The only thing that MUST be changed is to add one or more
# file or directory names in the Include directive of the
# FileSet resource.
#
# For Bacula release 5.0.2 (28 April 2010) -- debian squeeze/sid
#
# You might also want to change the default email address
# from root to your address. See the "mail" and "operator"
# directives in the Messages resource.
#
Director {# define myself
Name = acorn.squirrel-dir
DIRport = 9101# where we listen for UA connections
QueryFile = "/etc/bacula/scripts/query.sql"
WorkingDirectory = "/var/lib/bacula"
PidDirectory = "/var/run/bacula"
Maximum Concurrent Jobs = 1
Password = "password" # Console password
Messages = Daemon
DirAddress = 127.0.0.1
}
JobDefs {
Name = "DefaultJob"
Type = Backup
# accurate = yes
Level = Incremental
Client = acorn.squirrel-fd
# FileSet = "Full Set"
# Schedule = "WeeklyCycle"
Storage = File
Messages = Standard
Pool = File
Priority = 10
Write Bootstrap = "/var/lib/bacula/%c.bsr"
RunAfterJob = "/usr/local/bin/purge-old-archives.sh "\%l\" \"%n\""
RunAfterJob = "/usr/local/bin/backup_to_glacier \"%v\" \"%j\" \"%l\"
\"%s\""
}
# Specific job for the backup of the user's home dir.
Job {
Name = "Users"
JobDefs = "DefaultJob"
FileSet = "Home"
Schedule = "WeeklyCycleUsers"
# RunBeforeJob = "/usr/local/bin/purge-old-archives.sh "\%l\" \"%n\""
# RunAfterJob = "/usr/local/bin/backup_to_glacier \"%v\" \"%j\" \"%l\"
\"%s\""
}
# Specific job for the backup of the system data.
Job {
Name = "System"
JobDefs = "DefaultJob"
FileSet = "System"
Schedule = "WeeklyCycleSystem"
# RunAfterJob = "/usr/local/bin/backup_to_glacier \"%v\" \"%j\" \"%l\"
\"%s\""
}
# Backup the catalog database (after the nightly save)
Job {
Name = "BackupCatalog"
JobDefs = "DefaultJob"
Level = Full
FileSet="Catalog"
Schedule = "WeeklyCycleAfterBackup"
# This creates an ASCII copy of the catalog
# Arguments to make_catalog_backup.pl are:
# make_catalog_backup.pl
RunBeforeJob = "/etc/bacula/scripts/make_catalog_backup.pl MyCatalog"
# This deletes the copy of the catalog
RunAfterJob = "/etc/bacula/scripts/delete_catalog_backup"
# RunAfterJob = "/usr/local/bin/backup_to_glacier \"%v\" \"%j\" \"%l\"
\"%s\""
Priority = 11 # run after main backup
}
#
# Standard Restore template, to be changed by Console program
# Only one such job is needed for all Jobs/Clients/Storage ...
#
Job {
Name = "RestoreFiles"
Type = Restore
Client=acorn.squirrel-fd
FileSet="System"
Storage = File
Pool = Default
Messages = Standard
Where = /backup/bacula-restores
}
# Back up of the /home directory; with a bunch of excludes listed in
# a separate file.
FileSet {
Name = "Home"
Include {
Options {
compression = GZIP
signature = MD5
@/etc/bacula/excludes.home
exclude = yes
}
File = /home
@/etc/bacula/includes.home
Exclude Dir Containing = .baculaExclude
}
}
# Backup of the System files; with a bunch of excludes listed in
# a separate file.
FileSet {
Name = "System"
Include {
Options {
compression = GZIP
signature = MD5
@/etc/bacula/excludes.home
exclude = yes
}
File = /etc
File = /var
Exclude Dir Containing = .baculaExclude
}
}
# This is the backup of the catalog
FileSet {
Name = "Catalog"
Include {
Options {
signature = MD5
}
File = "/var/lib/bacula/bacula.sql"
}
}
#
# When to do the backups, full backup on first sunday of the month,
# differential (i.e. incremental since full) every other sunday,
# and incremental backups other days
Schedule {
Name = "WeeklyCycleUsers"
Run = Full 1st sun at 01:05
Run = Differential 2nd-5th sun at 04:05
Run = Incremental mon-sat at 04:05
}
Schedule {
Name = "WeeklyCycleSystem"
Run = Full 1st sun at 04:35
Run = Differential 2nd-5th sun at 04:35
Run = Incremental mon-sat at 04:35
}
# This schedule does the catalog. It starts after the WeeklyCycle
Schedule {
Name = "WeeklyCycleAfterBackup"
Run = Full sun-sat at 05:05
}
# Client (File Services) to backup
Client {
Name = acorn.squirrel-fd
Address = localhost
FDPort = 9102
Catalog = MyCatalog
Password = "pa
Re: [Bacula-users] Use modify time, not access time
> You should include the Job {} in question, and any JobDefs in use.
>
> I suspect you are doing accurate backups.
>
That I switched off already; may have been the cause. I always got
excessive incremental backup sizes, 500 MB while expecting 10-50 MB.
Issue at hand is that I moved a mailbox, then moved it back (original
location) and it's still being backuped. That's not good.
Wouter.
--
Monitor your physical, virtual and cloud infrastructure from a single
web console. Get in-depth insight into apps, servers, databases, vmware,
SAP, cloud infrastructure, etc. Download 30-day Free Trial.
Pricing starts from $795 for 25 servers or applications!
http://p.sf.net/sfu/zoho_dev2dev_nov
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Use modify time, not access time
On Fri, 2012-11-16 at 15:46 +1100, Gary R, Schmidt wrote: > On 16.11.2012 13:52, Wouter van Marle wrote: > [SNIP] > > Result: yesterday I moved a big mail folder with subfolders, causing > > the > You *moved* a directory, and you are surprised that the files in it are > being backed up? Sorry I was incomplete. Files were accidentally moved, and subsequently moved back to the exact original location. Wouter. > > Bacula has never seen these files in that location before, if it > *didn't* back them up that would be a total failure. > > Cheers, > GaryB-) > > > -- > Monitor your physical, virtual and cloud infrastructure from a single > web console. Get in-depth insight into apps, servers, databases, vmware, > SAP, cloud infrastructure, etc. Download 30-day Free Trial. > Pricing starts from $795 for 25 servers or applications! > http://p.sf.net/sfu/zoho_dev2dev_nov > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > > -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Use modify time, not access time
Hi, I have a big problem with Bacula and the backup of my Cyrus mail store: it is backing up way too much. When doing incremental it's supposed to look at modify time (which for many mails - one file per e-mail - is months or years past), but it appears to look at the access time instead. Result: yesterday I moved a big mail folder with subfolders, causing the access time but not the modify time of all contents to change, and now Bacula is backing up everything. Looking at the restore function in bconsole, I see the file time given is the access time, not the modify time. When doing `ls -la` I see the original modify time, long time ago. When doing `ls -lu` I see the access time, which is the time these mail boxes were moved (yesterday). And when doing `dir` in `bconsole` the time that I see is the access time; not the modify time! How to stop this unnecessary backup? It's annoying and wasting disk space. Wouter. -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Runafterjob issue
Related issues I just ran into improving my script. The RunAfterJob now reads (one line): RunAfterJob = "/bin/sh -c \"echo '/usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup %v \\\"%j (%l, since %s)\\\" ' | batch\"" Which results in these messages (two lines): > 04-Oct 21:43 acorn.squirrel-dir JobId 44: shell command: run AfterJob > "/bin/sh -c "echo '/usr/local/bin/glacier-cmd --logtostdout upload > Squirrel_backup \"Users.2012-10-04_21.43.17_03 (Incremental, since > 2012-10-04 21:38:58)\" ' | batch"" > 04-Oct 21:43 acorn.squirrel-dir JobId 44: AfterJob: > Users.2012-10-04_21.43.17_03: -c: line 0: unexpected EOF while looking for > matching `'' But when run from the command prompt, the command works just fine (one line): $ /bin/sh -c "echo '/usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup \"Users.2012-10-04_21.43.17_03 (Incremental, since 2012-10-04 21:38:58)\" ' | batch" And the %v translation does not work properly (the whole %-translation thing seems to be hit and miss, I had several more issues when trying to get some sample outputs and so). When I put '/backup/bacula/%v' in the script, it does NOT replace the %v with the actual file name. Without the path, it does the replacement. But I need the path for the command to actually work! It seems that going for a second script is the only way out here :-( And I'll think of some improvement of the docs. Adding an example will probably do the job, both by clarifying the issue and by giving it more focus. Wouter. On Thu, 2012-10-04 at 17:40 +0400, Konstantin Khomoutov wrote: > On Thu, 04 Oct 2012 20:44:33 +0800 > Wouter van Marle wrote: > > > > To not mess with writing the real shell script, consider embedding a > > > call to the shell to your script string, like this: > > > > > > RunAfterJob = "/bin/sh -c \"/that/cmd ... '%v' | batch\"" > > > > > > > > > That works, thanks for the quick reply! > > > > Would be nice to have this mentioned in the docs, if it's there I > > totally missed it. > > > > Writing a real script is of course easy, it's just messy and harder to > > see what's going on if I revisit this which may be years from now. > > http://bit.ly/O7z6SU reads this: > > "In addition, the command string is parsed then fed to the OS, which > means that the path will be searched to execute your specified command, > but there is no shell interpretation, as a consequence, if you invoke > complicated commands or want any shell features such as redirection or > piping, you must call a shell script and do it inside that script." > > I would say the wording is imperfect. > If you can propose a better one, feel free to file a feature request > with http://bugs.bacula.org/ > > -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Runafterjob issue
That works, thanks for the quick reply! Would be nice to have this mentioned in the docs, if it's there I totally missed it. Writing a real script is of course easy, it's just messy and harder to see what's going on if I revisit this which may be years from now. Wouter. > To not mess with writing the real shell script, consider embedding a > call to the shell to your script string, like this: > > RunAfterJob = "/bin/sh -c \"/that/cmd ... '%v' | batch\"" > > -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Runafterjob issue
Hi all, I'm trying to launch a runafterjob script to upload my just created archive to another server. As this can take long - a day or so, and other backup jobs are to be run (I have three jobs: two backup jobs of separate parts of the system plus the database backup), I intend to do this upload in the background, so I don't lock up Bacula. So what I did: RunAfterJob = "echo '/usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup %v' | batch" Running this command (replacing the %v with an actual file name of course) from the command line works fine. The process is scheduled by batch to run as soon as system load allows (usually instantly) and it's running in the background, sending output to user bacula which ultimately ends up in my mail box. But running from Bacula fails. In the result e-mailed to me when the job is finished I see this: 04-Oct 05:05 acorn.squirrel-dir JobId 34: AfterJob: /usr/local/bin/glacier-cmd --logtostdout upload Squirrel_backup Squirrel-BackupCatalog.2012-10-04_05.05.00_04 | batch The `echo` part is gone, and it didn't run. An obvious hack would be to create a shell script that is called by Bacula and that in turn starts the upload, but that shouldn't be necessary. Wouter. -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
