Re: [Bacula-users] Copy Job Configuration Question
On 4/6/24 10:53 AM, Chris Wilkinson wrote:
I am attempting to write a copy job to copy uncopied jobs from one SD to another. It seems that the client and fileset
directives are required or the syntax check will fail. The documentation (v9) is not explicit on this point.
Since the client is not involved in a copy job, it seems that these clauses are redundant. If they really are required, does
it matter what value they have, even a non-existent one?
This is what I have for the copy job:
Job {
Name = "catalog-copy"
Description = "copy of catalog"
Type = "Copy"
Level = "Full"
Messages = "Standard"
Storage = "dns-325-sd" #source storage
Pool = "catalog"
NextPool = "catalog-copy" #destination storage
Client = "catalog-fd" #why is this needed, what value?
Fileset = "Catalog" #why is this needed, what value?
Schedule = "sched_none"
SelectionPattern = "catalog" #copy only job names matching "catalog"
SelectionType = "PoolUncopiedJobs"
}
Many Thanks
Chris Wilkinson
Hello Chris,
The parser sees that it is parsing a Job resource, and then requires all the settings for a Job resource, and does not
distinguish a Backup type job from an Admin one, or Copy, or Verify etc. This had annoyed me also for some time, but I
suspect the developers will never want to spend time on making this distinction when parsing resources. :)
What I have been doing in my Bacula environments for many years is I create some "dummy/fake" resources and use them in
places where the parser requires them but they are clearly not needed/used.
The nice (OCD?) thing here is that in my Copy, Migration, Admin, Restore, etc job logs and summaries, it is clear that no
Fileset, or Client, or Storage, etc was really used.
The same is true when viewing Job listings in BWeb, Baculaum, Bacula-Web, Bacularis, or in my
https://github.com/waa/baculabackupreport script. ie: It is explicitly clear that a Copy/Migration Control job (for example)
in the list contacted no Client.
In each of my fake resources, I have just the bare minimum required to satisfy the parser fro that type of resource. I name
them all "None" (there is funny bug in my reporting script story about this - Python programmers will know straight away :)
and I use them in special jobs as mentioned above.
Fake Client for copy jobs, etc:
8<
Client {
Name = None
Address = localhost
Password = N/A
@/opt/comm-bacula/include/Clients-Defaults # Some required things for all
Clients like FDPort, Catalog are in here
}
8<
Fake Fileset for copy jobs, etc:
8<
Fileset {
Name = None
Include {
Options {
Signature = md5
}
}
}
8<
Fake Storage for admin jobs, etc
8<
Autochanger {
Name = None
Address = localhost
Enabled = no
Device = N/A
Password = N/A
Media Type = None
}
8<
Fake Pool for copy jobs, etc:
8<
Pool {
Name = None
PoolType = Backup
}
8<
Fake Schedule with no run times. This way I can implicitly see what are my "Manually
run" jobs:
8<
Schedule {
Name = Manual
}
8<
Hope this helps!
Bill
--
Bill Arlofski
w...@protonmail.com
signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] baculum + bacula 13.0.4 slow web interface
Unkdef, one important thing to know is that Bacularis is a friendly fork of baculum. It's very similar, and Marcin has been actively maintaining it. Marcin is also a baculum developer. Basically, Bacularis is the better, more optimized tool. This doesn't mean that the source of your problem is certainly baculum, but trying the actively maintained and more optimized tool Bacularis is certainly a good first step. :) Bacularis is available in docker containers if you want to simply load a container and give it a try. It is naturally available for bare metal installations as well. Https://bacularis.app Robert Gerber 402-237-8692 r...@craeon.net On Sat, Apr 6, 2024, 12:56 PM Unkdef unkdef wrote: > i use only 45 backup jobs a day... > > Marcin Haba 6 апреля 2024 г. 20:14:30 написал: > >> Hello Unkdef, >> >> Yes, Baculum 11.0.6 is a bit old and can be a little slow, specially for >> many jobs and MySQL. There exists a couple of techniques to speed it up >> like described here: >> >> >> https://baculum.app/doc/brief/troubleshooting.html#the-baculum-job-tables-are-loading-slowly >> >> or here: >> >> >> https://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg72737.html >> >> If I could suggest something, I would propose to try Bacularis that is >> much faster and contains significant optimizations. >> >> Best regards, >> Marcin Haba (gani) >> >> On Fri, 5 Apr 2024 at 09:07, Unkdef unkdef >> wrote: >> >>> Hello! I use bacula 13.0.4(mysql) and baculum 11.0.6 on Ubuntu 22.04 >>> Problem: When I refresh page in baculum i wait 10-15 sec before i see >>> information of backups and other... how to fix it? >>> ___ >>> Bacula-users mailing list >>> Bacula-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>> >> >> >> -- >> >> "Greater love hath no man than this, that a man lay down his life for his >> friends." Jesus Christ >> >> "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za >> przyjaciół swoich." Jezus Chrystus >> >> > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy Job Configuration Question
Thanks Bill, that's a good plan that I'll implement.
Chris
On Sat, 6 Apr 2024, 22:17 Bill Arlofski via Bacula-users, <
bacula-users@lists.sourceforge.net> wrote:
> On 4/6/24 10:53 AM, Chris Wilkinson wrote:
> > I am attempting to write a copy job to copy uncopied jobs from one SD to
> another. It seems that the client and fileset
> > directives are required or the syntax check will fail. The documentation
> (v9) is not explicit on this point.
> >
> > Since the client is not involved in a copy job, it seems that these
> clauses are redundant. If they really are required, does
> > it matter what value they have, even a non-existent one?
> >
> > This is what I have for the copy job:
> >
> > Job {
> >Name = "catalog-copy"
> >Description = "copy of catalog"
> >Type = "Copy"
> >Level = "Full"
> >Messages = "Standard"
> >Storage = "dns-325-sd" #source storage
> >Pool = "catalog"
> >NextPool = "catalog-copy" #destination storage
> >Client = "catalog-fd" #why is this needed, what value?
> >Fileset = "Catalog" #why is this needed, what value?
> >Schedule = "sched_none"
> >SelectionPattern = "catalog" #copy only job names matching "catalog"
> >SelectionType = "PoolUncopiedJobs"
> > }
> >
> > Many Thanks
> > Chris Wilkinson
>
> Hello Chris,
>
> The parser sees that it is parsing a Job resource, and then requires all
> the settings for a Job resource, and does not
> distinguish a Backup type job from an Admin one, or Copy, or Verify etc.
> This had annoyed me also for some time, but I
> suspect the developers will never want to spend time on making this
> distinction when parsing resources. :)
>
> What I have been doing in my Bacula environments for many years is I
> create some "dummy/fake" resources and use them in
> places where the parser requires them but they are clearly not needed/used.
>
> The nice (OCD?) thing here is that in my Copy, Migration, Admin, Restore,
> etc job logs and summaries, it is clear that no
> Fileset, or Client, or Storage, etc was really used.
>
> The same is true when viewing Job listings in BWeb, Baculaum, Bacula-Web,
> Bacularis, or in my
> https://github.com/waa/baculabackupreport script. ie: It is explicitly
> clear that a Copy/Migration Control job (for example)
> in the list contacted no Client.
>
> In each of my fake resources, I have just the bare minimum required to
> satisfy the parser fro that type of resource. I name
> them all "None" (there is funny bug in my reporting script story about
> this - Python programmers will know straight away :)
> and I use them in special jobs as mentioned above.
>
>
> Fake Client for copy jobs, etc:
> 8<
> Client {
>Name = None
>Address = localhost
>Password = N/A
>@/opt/comm-bacula/include/Clients-Defaults # Some required things for
> all Clients like FDPort, Catalog are in here
> }
> 8<
>
> Fake Fileset for copy jobs, etc:
> 8<
> Fileset {
>Name = None
> Include {
>Options {
>Signature = md5
> }
>}
> }
> 8<
>
> Fake Storage for admin jobs, etc
> 8<
> Autochanger {
>Name = None
>Address = localhost
>Enabled = no
>Device = N/A
>Password = N/A
>Media Type = None
> }
> 8<
>
> Fake Pool for copy jobs, etc:
> 8<
> Pool {
>Name = None
>PoolType = Backup
> }
> 8<
>
> Fake Schedule with no run times. This way I can implicitly see what are my
> "Manually run" jobs:
> 8<
> Schedule {
>Name = Manual
> }
> 8<
>
>
>
> Hope this helps!
> Bill
>
> --
> Bill Arlofski
> w...@protonmail.com
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] baculum + bacula 13.0.4 slow web interface
Hello Unkdef, Yes, Baculum 11.0.6 is a bit old and can be a little slow, specially for many jobs and MySQL. There exists a couple of techniques to speed it up like described here: https://baculum.app/doc/brief/troubleshooting.html#the-baculum-job-tables-are-loading-slowly or here: https://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg72737.html If I could suggest something, I would propose to try Bacularis that is much faster and contains significant optimizations. Best regards, Marcin Haba (gani) On Fri, 5 Apr 2024 at 09:07, Unkdef unkdef wrote: > Hello! I use bacula 13.0.4(mysql) and baculum 11.0.6 on Ubuntu 22.04 > Problem: When I refresh page in baculum i wait 10-15 sec before i see > information of backups and other... how to fix it? > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Copy Job Configuration Question
I am attempting to write a copy job to copy uncopied jobs from one SD to
another. It seems that the client and fileset directives are required or
the syntax check will fail. The documentation (v9) is not explicit on this
point.
Since the client is not involved in a copy job, it seems that these clauses
are redundant. If they really are required, does it matter what value they
have, even a non-existent one?
This is what I have for the copy job:
Job {
Name = "catalog-copy"
Description = "copy of catalog"
Type = "Copy"
Level = "Full"
Messages = "Standard"
Storage = "dns-325-sd" #source storage
Pool = "catalog"
NextPool = "catalog-copy" #destination storage
Client = "catalog-fd" #why is this needed, what value?
Fileset = "Catalog" #why is this needed, what value?
Schedule = "sched_none"
SelectionPattern = "catalog" #copy only job names matching "catalog"
SelectionType = "PoolUncopiedJobs"
}
Many Thanks
Chris Wilkinson
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacularis/Baculum] Restore with web doesn't create job before 1 hour
Hello Timothé, Thanks for providing all details about this restore problem. I did tests on a set of 150K files with old Bacula 9.0.6 and the latest 15.0.2. It looks that something has changed in Bacula MySQL/MariaDB support in a version between because on the version 9 it works well, while on 15 it hangs on the restore commands. Your restore using bconsole can work much better because it does not use Bacula Bvfs interface. I will try to look at it deeper in the next few days. I will let know if I find something. Best regards, Marcin Haba (gani) On Thu, 4 Apr 2024 at 16:14, timothe pennec wrote: > Hi, > > I'm using mariadb for database and it's him seems to be slow : > top result : > > top - 16:08:51 up 2 days, 20:10, 6 users, load average: 0,97, 0,60, 0,29 > Tasks: 307 total, 1 running, 306 sleeping, 0 stopped, 0 zombie > %Cpu(s): 12,5 us, 0,0 sy, 0,0 ni, 87,3 id, 0,0 wa, 0,0 hi, 0,2 si, > 0,0 st > MiB Mem : 7936,5 total,177,7 free,592,8 used, 7166,1 buff/cache > MiB Swap: 4096,0 total, 4095,0 free, 1,0 used. 7026,5 avail Mem > PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+ COMMAND > 29267 mysql 20 0 1346196 142604 23396 S 100,0 1,8 2:53.58 > mariadbd > > I've 8 vCPU but only one is at 100 % (can see it with htop) all other at 0. > > I've enabled debug level 500 and i can see many log right after submit > button "Run" and stuck at : > > bacula-dir: ua_restore.c:2015-0 JobId=4792 JobIds= FileIndex=10 > [All 53000 files ...] > bacula-dir: ua_restore.c:2015-0 JobId=4792 JobIds= FileIndex=53895 > bacula-dir: ua_restore.c:2015-0 JobId=4792 JobIds= FileIndex=53896 > bacula-dir: mysql.c:584-0 db_sql_query finished > bacula-dir: ua_restore.c:1476-0 query=SELECT Path.Path, File.Filename FROM > File JOIN Path USING (PathId) JOIN b210091 AS T ON (File.JobId = T.JobId > AND File.FileIndex = T.FileIndex) WHERE File.Filename LIKE > ':component_info_%' AND File.JobId IN (4792) > bacula-dir: mysql.c:527-0 db_sql_query starts with SELECT Path.Path, > File.Filename FROM File JOIN Path USING (PathId) JOIN b210091 AS T ON > (File.JobId = T.JobId AND File.FileIndex = T.FileIndex) WHERE File.Filename > LIKE ':component_info_%' AND File.JobId IN (4792) > > > Any idea ? > > Thank you > Timothé > > -- > *De :* Marcin Haba > *Envoyé :* mercredi 3 avril 2024 21:53 > *À :* timothe pennec > *Cc :* bacula-users@lists.sourceforge.net < > bacula-users@lists.sourceforge.net> > *Objet :* Re: [Bacula-users] [Bacularis/Baculum] Restore with web doesn't > create job before 1 hour > > Hello Timothé, > > Thanks for your detailed description of the problem. > > For start I would propose to check what is going inside Bacula and the > Catalog when this restore is trying to start. > > Yes, observing 'ps aux' is a good idea, that you did. Also useful can be > 'top' command with watching what process takes the most resources. > > From the Bacula side you can set debug for Director by this bconsole > command: > > setdebug level=500 tags=bvfs,sql trace=1 dir > > in the Bacula Director working directory you will see the > youdirector-dir.trace file where this debug lines will be written. > > If your Catalog database is PostgreSQL you ran run in the time of the > restore hanging this SQL query to see currently pending SQL queries: > > SELECT * FROM pg_stat_activity; > > Please let us know about the results. > > Good luck. > > Best regards, > Marcin Haba (gani) > > On Wed, 3 Apr 2024 at 20:40, timothe pennec wrote: > > Hi everyone, > > I've one issue with Bacularis (and same with Baculum) WEB when i follow > the restore wizzard : > >- I select my client and let checked "Enable restore from copy job >feature" >- I select the Last full Backup of my client >- I select one folder with add button (total of files about 55 000 and >total syse about 4Go) >- I select the same client and "/tm/restore" folder for testing >- I let default setting for Restore job (RestoreFiles and no replace >file and no relocate files) >- I submit with "Run restore" > > > When i submit nothing happen (i got only "loading" favicon and not > redirect or validation message for create job). > At the same time if i open new tab to default web page it seems the server > not responding for a while and after a time the page loaded for the default > tab and error "504 Gateway timeout" for restore tab > If i ssh and launch bconsole and enter "status director" i can see that no > job was created (terminated or running or whatever state). > > Here a "ps aux | grep bconsole" at this time : > www-data 21059 0.0 0.0 2892 964 ?S19:45 0:00 sh -c > sudo /usr/bin/bconsole -c "/etc/bacula/bconsole.conf" -D bacula-dir 2>&1 > < replace="never" restorejob="RestoreFiles" where="/tmp/restore" yes quit > END_OF_DATA > root 21060 0.0 0.0 8488 5228 ?S19:45 0:00 sudo > /usr/bin/bconsole -c /etc/bacula/bconsole.conf -D
Re: [Bacula-users] baculum + bacula 13.0.4 slow web interface
i use only 45 backup jobs a day... Marcin Haba 6 апреля 2024 г. 20:14:30 написал: Hello Unkdef, Yes, Baculum 11.0.6 is a bit old and can be a little slow, specially for many jobs and MySQL. There exists a couple of techniques to speed it up like described here: https://baculum.app/doc/brief/troubleshooting.html#the-baculum-job-tables-are-loading-slowly or here: https://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg72737.html If I could suggest something, I would propose to try Bacularis that is much faster and contains significant optimizations. Best regards, Marcin Haba (gani) On Fri, 5 Apr 2024 at 09:07, Unkdef unkdef wrote: Hello! I use bacula 13.0.4(mysql) and baculum 11.0.6 on Ubuntu 22.04 Problem: When I refresh page in baculum i wait 10-15 sec before i see information of backups and other... how to fix it?___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
