Re: [Bacula-users] Scheduling example
On 3/6/24 01:58, Stefan G. Weichinger wrote: As I try to come up with an initial setup for weekly-daily-monthly I found this: https://www.bacula.org/13.0.x-manuals/en/main/Automatic_Volume_Recycling.html#SECTION003060 Shouldn't there also be Jobs for Weekly and Monthly? No, they aren't needed. Read carefully. The job "NightlySave" uses the schedule "NightlySave", which runs the NightlySave job monthly, weekly and daily at Full, Differential and Incremental levels, each to its own Pool. Which Pool to use for the Catalog Backups? This example doesn't include a catalog backup. But I run my catalog backup to whichever Pool the other jobs for that night use. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-fd hangs during backup
On 2/12/24 18:35, Peter Sjoberg wrote: Hi all I have a strange problem and (on my system) reproducible problem. When I do backup of some directories then bacula-fd just hangs and never complete. The directories in question are not very strange and backup of them works find with older versions of -fd Silly question: Do the problem directories contain named pipes or sockets? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] BAT Installation
On 2/2/24 10:25, Bill Arlofski via Bacula-users wrote: All good information. And I would strongly urge to move away from BAT in favor of Bacularis (for example). Bacularis, unlike BAT allows you to configure every aspect of your Bacula environment, and as Rob mentioned, Marcin Haba, a Bacula Systems employee maintains it as an open-source and free tool, and he is quite active and helpful on this list. :) Bottom line: Abandon BAT. It has not received any love in a very long time, and quite frankly I am sure it will not. It did what it was designed to do, but now that we have some very powerful web gui administration tools, there is no reason not to abandon it.* (Just my two cents) *Having said that, years ago, I had customers that LOVE LOVE LOVED BAT, and would not move away from it even at my urging, so there is one reason, I guess. :) BAT still does everything I want a Bacula console to do. The things that it does NOT do, like configuration, I don't WANT it to; I want to do those myself, by hand, using a proper editor. To be truthful, I detest the "Everything is a web page/application" model. I have not, to be fair, tried either Baculum or Bacularis, but my general experience is that web-based applications tend to be uniformly awful. Especially web-based admin interfaces. For example, I briefly tried to deploy a QNAP NAS that used a web admin interface, and there were basic security functions that you simply *COULD NOT DO* via the admin GUI. It was not merely hidden, it was *impossible*. Webmin, cpanel, ... uniformly ghastly. Your mileage may of course vary. I will note that there are a few known bugs in BAT, notably that some purge operations can produce *multiple* simultaneous pop-up confirmation alerts that can be confusing. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] BAT Installation
On 2/2/24 10:53, Heitor Faria wrote: Frankly, I never understood this community's urge to regularly create "one more Bacula GUI", instead of just fixing and improve the state-of-art. IMHO bacula.org <http://bacula.org> should adopt and sponsor BAT and Bacularis/Baculum GUIs development, get people together, integrate bacula-web features and organize this mess. Rgds. I frankly agree with Heitor here. BAT Just Works, isn't subject to browser compatibility issues, and doesn't suffer any of the broken interface behaviors that come as part and parcel of web applications. It does have a few minor bugs that should be fixed, but even those it has do not stop it from working. This isn't to say that a web interface for Bacula *monitoring* is necessarily a bad idea. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ubuntu 22.04: bacula-fd from official repo cannot talk to src-compiled server on 20.04
On 11/9/23 04:09, Uwe Schuerkamp wrote: director (compiled from source): libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x7f3d8ac51000) libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x7f087cbb5000) client (from bacula repos): libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x7f4d8499b000) One presumes you are aware of how old your Director's OpenSSL is? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula copy from tape to tape
On 11/7/23 17:09, Rob Gerber wrote: Technically I do have a huge NAS that dwarfs the production NAS in size (TrueNAS box). It was intended to store one movie, but the movie IT people massively over-estimated the amount of space we needed, so it never got used. It has occurred to me to use this TrueNAS device as a disk based volume storage solution. Can one make multiple copies of a reference volume? Ie, could we have disk based backups managed by bacula on this larger NAS, with copies made to tape for onsite AND offsite storage? You can absolutely to disk-to-disk-to-tape. It's not conceptually difficult to set up. (I'm doing disk to disk to removable ZFS disk pool, but ... same principle.) Maybe at that point we wouldn't need an onsite tape backup set. Disk based backup could speed up our initial backups substantially since both of these systems can operate at 10gbe speeds. Yeah, I love my disk-to-disk setup. Small restores are almost instant. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: Your advice for my re-design of backup layout regarding pools and volumes (backups of and in homelab)
On 10/24/23 15:37, Justin Case wrote:
can you paste how you select the fulls for copy jobs?
This is my methodology:
Job {
Name = "Archive Copy"
Type = Copy
Enabled = Yes
Pool = Full-Disk
Level = Full
Client = ALL
Fileset = DUMMY
Storage = epsilon3-archive
Selection Type = SQL Query
Selection Pattern = "SELECT DISTINCT J.JobId
FROM Job J
JOIN Pool P ON P.PoolId = J.PoolId
LEFT JOIN Job J2 ON J2.PriorJobId = J.JobId
WHERE P.Name = 'Full-Disk'
AND J2.JobId IS NULL
AND J.Type = 'B'
AND J.JobStatus IN ('T','W')
AND J.JobBytes > 0
AND J.StartTime > now() - interval 7 day"
Messages = Daemon
Priority = 20
Rerun Failed Levels = yes
Allow Duplicate Jobs = no
Cancel Queued Duplicates = yes
Schedule = "Full Backup Archive"
}
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: Your advice for my re-design of backup layout regarding pools and volumes (backups of and in homelab)
On 10/24/23 14:42, Justin Case wrote: I realized that my fulls and incrementals have the same retention time, so I made 1 pool for both. I guess you are using X > Y months for fulls and Y > Z months fordifferentials and Z months for incrementals? That's the basic idea, yes. I do a nightly incremental kept for 3 weeks, weekly differentials kept for 3 months, and monthly Fulls which if memory serves I keep for one year. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: Your advice for my re-design of backup layout regarding pools and volumes (backups of and in homelab)
On 10/24/23 10:59, Justin Case wrote: Disclaimer: My situation is a little different from yours, I'm doing disk-to-disk-to-removable-disk. For tier 1: does it make sense to have different volumes for full jobs and incr jobs? Does it make sense to have these volumes in seperate pools, or full and incr together in a pool? Would you recommend having different pools for different groups of jobs? I keep Full, Differential and Incremental jobs in separate Pools. That way I can purge job batches as they become obsolete on their own schedules and recover the space. For tier 2: here I wil use copy jobs. Since copy jobs really do not differentiate full and incr backups, there will be full and incr copies together in volumes. Would you recommend having different pools for different groups of copy jobs? I use copy jobs, but I copy only FULL backups to removable disk. (The removable disk zpools are then exported, unmounted, and stored offline/airgapped.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extraction of a specific file
On 10/2/23 13:39, Rob Gerber wrote: I'm not sure, but maybe you'll need to query your database directly instead of relying on Bacula to simplify that process. *This is not an educated answer, just my novice guess.* I see bacula bconsole has a 'query' command that I believe passes SQL query language commands on to the relevant DB server. Might want to look into that, possibly in the bacula console operators manual. I've used the PDF copy of the documentation because it's easier to search. I'd swear there used to be a 'blist' tool, but I can't for the life of me find it any more. This query will list files in a given Job: SELECT CONCAT(p.Path, f.Filename) FROM File f JOIN Path p ON f.PathId = p.PathId WHERE f.JobId = ; If you want only, say .odt files, you could use: SELECT CONCAT(p.Path, f.Filename) FROM File f JOIN Path p ON f.PathId = p.PathId WHERE f.JobId = AND f.Filename LIKE '%.odt'; For example: MySQL 127.0.0.1> SELECT CONCAT(p.Path, f.Filename) FROM File f JOIN Path p ON f.PathId = p.PathId WHERE f.JobId = 836 AND f.Filename LIKE '%.odt'; +--+ | CONCAT(p.Path, f.Filename) | +--+ | /home/alaric/Documents/Writing/Bearing Gifts.odt | | /home/alaric/Documents/Writing/United Fleet.odt | +--+ 2 rows in set (0.647 sec) You can get the LStat field as well, but you'd have to decode it yourself. I don't remember offhand how LStat is encoded. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and BBR Protocol for Internet Backups, Packet Losses, Disconnections, etc.
On 10/2/23 09:47, hei...@bacula.com.br wrote: > I've enabled this now on all of my hosts, but I'm honestly not seeing a > clear improvement in network throughput so far. The improvements are more notiable if you introduce packet losses or networ interruptions. Ah, so, all the things I've done my best to eliminate from my network :) Well maybe it'll show up in other usages. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and BBR Protocol for Internet Backups, Packet Losses, Disconnections, etc.
On 9/14/23 18:08, hei...@bacula.com.br wrote: Hello Bacula Users, I would like to share this finding regarding the Linux BBR TCP congestion protocol and Bacula. I've tested in a few production environments, but I would like to have a broather population of results: https://www.bacula.lat/bacula-e-prototoclo-bbr-para-backups-via-internet-perdas-de-pacote-desconecoes-etc/?lang=en I've enabled this now on all of my hosts, but I'm honestly not seeing a clear improvement in network throughput so far. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Slow spooling and hashing speed
On 9/23/23 14:25, Rob Gerber wrote: You know, asking them to add a Bacula FD plugin is a good idea! I'm sure it wouldn't be available for quite a while, but that is genuinely a good idea. If they're building on Gentoo, they ought to be able to do it pretty quickly if you can persuade them to. the Gentoo ebuild is mature and, particularly for client-only, is about as close to plug-and-play as you can get. This is a commercial market NAS, aimed at videographers. The devs are actually pretty damn good, and I'm proud of them for choosing Gentoo. They have a bunch of software sitting on top of Gentoo aiming to provide fast access to stored media files for video editors. Good on them, then. The main data array is formatted XFS, by way of trivia for y'all. The core ethic is "get media go the end users as fast as possible so they can edit and render as fast as possible". Everything else is secondary. XFS is probably the ideal choice for that application. It was specifically designed to optimize streaming media performance. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Slow spooling and hashing speed
On 9/23/23 13:34, Rob Gerber wrote: Heitor, Thank you for the reply. The backup source is a studio network solutions Evo NAS appliance. Unfortunately, I have no access to run bacula FD, nor most of other things suggested on this appliance as access is only via appliance developer designer means. I do not have any shell access to this appliance. Off the top of my head, I believe my only access options to this Evo NAS are smb (I believe 3.x), read only iSCSI, or afs (apple file sharing - obviously not something I want to try). This Evo NAS appliance is running Gentoo Linux. Wait, it runs Gentoo and they locked it down that hard somehow? Gut feeling says you should be able to install almost anything you want onto it, but it may be some work depending on how they locked it down. (This would of course probably void your warranty.) That said, if you can't put a fd on it directly, given those options the next best thing would probably be to mount it as an iSCSI target. You might also consider asking them if they would consider a Bacula-fd package available. But my experience is that to NAS consumer-appliance manufacturers, backup is at best an afterthought. I went through this with a NAS appliance which QNAP proudly claimed ran Linux and ZFS, only to find out that through idiotic choices and Byzantine design decisions they had crippled it so badly it might as well have been running Windows. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] access key
On 9/13/23 18:35, Bob McNabb wrote: Where do I obtain the access key for installing the community edition? You go to the download page, follow the instructions, and you will be emailed a download link. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula purges old jobs and I don't want it to do that
On 8/18/23 17:24, Rob Gerber wrote: verification jobs. My guess is that bacula purged job records for some reason, possibly because they were older than 6 months. I have volume retention set to 1000 years, but maybe I need to add something for job retention? There are three retention periods you need to consider: — Volume retention — Job retention — File retention I suggest reading up on these in the documentation so that you understand what each of them does. You can then adjust your pools, and update all of your volumes, as necessary. Background info: I am backing up some large video files with LTO8 tape. My goal is to back these files up once, and do incremental backups to capture any new or changed files in the fileset. I do not expect the large media files to be changed once placed on the disk. Basically, I am using an "incrementals forever" strategy. You might want to consider virtual-full jobs as a part of that strategy. I am using bacula 13.x. We have a qualstar q24 autochanger. We use LTO 8 media. The fileset in question is around 70TiB. I welcome any assistance the list could provide. I'm sure it has to be something simple. You could fix your retention and then scan the pruned volumes back in using bscan. By the sound of it, that would take a while, but it's doable. What you need to keep in mind is that an "incrementals forever" strategy is also a "Bacula database grows forever" strategy. That will eventually become a problem. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with grant_mysql_privileges
On 8/3/23 04:05, Graham Dicker via Bacula-users wrote:
On Thursday, 3 August 2023 08:35:39 BST Graham Dicker via Bacula-users wrote:
On Wednesday, 2 August 2023 17:21:49 BST Phil Stracchino wrote:
On 8/2/23 11:12, Graham Dicker via Bacula-users wrote:
Hello
I am installing Bacula 13.0.2 on Opensuse 15.5 and get this problem when
I
run grant_mysql_privileges:
ERROR 1064 (42000) at line 3: You have an error in your SQL syntax;
check
the manual that corresponds to your MariaDB server version for the right
syntax to use near '%{db_user}@"%"' at line 1
Database version 10.6.14-MariaDB
I guess it's complaining about the statement
db_user=${db_user:-bacula}
Can anyone help with this please?
What does the line 'echo "Created MySQL database user: ${db_user}"'
report as the value of ${db_user}?
It reports just the single word bacula
Graham
It doesn't actually create the user bacula either although when I run the
script it says it has created it.
Martin is correct that the second GRANT line is incorrect, it should be
${db_user} not %{db_user}
Try doing this yourself manually. It's not complex. You know your
MariaDB superuser password I assume. Log into MariaDB and run the
following commands:
GRANT ALL PRIVILEGES ON bacula@'HOST' IDENTIFIED BY "password goes here";
GRANT PROCESS ON *.* TO bacula@'HOST';
FLUSH PRIVILEGES;
HOST here can be one of several things. The script uses '%', a wildcard
meaning 'any host'. If your DB runs on the same host as your Director
and will not be accessed from anywhere else, it would be more secure to
use 'localhost' there. Or if you have a subnet behind a firewall and
want to allow access from a range of hosts or all of your hosts, you can
use something like bacula@'1.2.3.%'.
Report any errors.
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with grant_mysql_privileges
On 8/2/23 13:43, Martin Simmons wrote: It looks like a bug in the script to me (maybe % instead of $): if $bindir/mysql $* -u root -f < You are absolutely right, Martin, I totally looked past that. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with grant_mysql_privileges
On 8/2/23 11:12, Graham Dicker via Bacula-users wrote:
Hello
I am installing Bacula 13.0.2 on Opensuse 15.5 and get this problem when I run
grant_mysql_privileges:
ERROR 1064 (42000) at line 3: You have an error in your SQL syntax; check the
manual that corresponds to your MariaDB server version for the right syntax to
use near '%{db_user}@"%"' at line 1
Database version 10.6.14-MariaDB
I guess it's complaining about the statement
db_user=${db_user:-bacula}
Can anyone help with this please?
What does the line 'echo "Created MySQL database user: ${db_user}"'
report as the value of ${db_user}?
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/7/23 19:35, Phil Stracchino wrote: Then re-update it again, but BEFORE you rerun the updater, find update_mysql_tables_1021_to_1022 and change every instance of CREATE TABLE to CREATE TABLE IF NOT EXISTS. That will make sure that if something else is happening along the way that is creating those tables before they're expected, then the CREATE TABLE will be skipped, instead of the script failing. The *other* way you could do this is to insert a 'DROP TABLE IF EXISTS ;' before each 'CREATE TABLE (...' statement. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/7/23 19:26, Phil Stracchino wrote: On 7/7/23 18:59, Ken Mandelberg wrote: The suggestion below which is essentially to run "update_mysql_tables" manually instead of within the "apt-get install bacula-mysql" runs into the same issue. Specifically Update of Bacula MySQL tables from 16 to 1017 succeeded. Update of Bacula MySQL tables from 1017 to 1018 succeeded. Update of Bacula MySQL tables 1018 to 1019 succeeded. (From 16) Update of Bacula MySQL tables 1019 to 1020 succeeded. Update of Bacula MySQL tables 1020 to 1021 succeeded. ERROR 1050 (42S01) at line 3: Table 'TagJob' already exists Update of Bacula MySQL tables 1021 to 1022 failed. I also tried deleting TagJob and starting again, and then it just moves on to the next table that already exists. In fact I deleted all the Tag* tables, and that it complains about the Object table. I suspect it won't like any. Any ideas? OK, the short version is none of those TAG tables are SUPPOSED to exist before database version 1022. That PROBABLY means that you didn't fully clean from a prior update attempt. So what you can do is drop your bacula DB, *make sure it's empty*, restore it again from your backup copy, connect to mysql, do a SHOW TABLES FROM bacula; and check that the Tag... and Event tables are not in that list. Then re-update it again, but BEFORE you rerun the updater, find update_mysql_tables_1021_to_1022 and change every instance of CREATE TABLE to CREATE TABLE IF NOT EXISTS. That will make sure that if something else is happening along the way that is creating those tables before they're expected, then the CREATE TABLE will be skipped, instead of the script failing. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/7/23 18:59, Ken Mandelberg wrote: The suggestion below which is essentially to run "update_mysql_tables" manually instead of within the "apt-get install bacula-mysql" runs into the same issue. Specifically Update of Bacula MySQL tables from 16 to 1017 succeeded. Update of Bacula MySQL tables from 1017 to 1018 succeeded. Update of Bacula MySQL tables 1018 to 1019 succeeded. (From 16) Update of Bacula MySQL tables 1019 to 1020 succeeded. Update of Bacula MySQL tables 1020 to 1021 succeeded. ERROR 1050 (42S01) at line 3: Table 'TagJob' already exists Update of Bacula MySQL tables 1021 to 1022 failed. I also tried deleting TagJob and starting again, and then it just moves on to the next table that already exists. In fact I deleted all the Tag* tables, and that it complains about the Object table. I suspect it won't like any. Any ideas? Let me take a look at the current update_mysql_tables script and get back to you. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/4/23 19:48, Ken Mandelberg wrote: Another followup. One thing I learned, I need to remove all the Ubuntu bundled bacula 9 packages before doing the apt-get install bacula-mysql for the Community 13 release, otherwise there are package conflicts. So after removing the old packages, apt-get install bacula-mysql tires to install the mysql bacula database. It gets caught at mysql said: ERROR 1050 (42S01) at line 9: Table 'TagJob' already exists I imagine if I removed that table it would just get caught at the next one. Its not really clear on how to proceed preserving the current database 9 info and letting 13 do the update job. OK, try this. - DUMP the bacula schema using mysqldump or your database dump tool of choice. (Personally I prefer mydumper.) - Then DROP the bacula DB. - Then run apt-get install bacula-mysql. - You MIGHT need to drop the bacula schema *again* at this point. to be safe. - Then RESTORE the bacula schema from your dump. - Then run update_mysql_tables to update your v9 Bacula DB to v13. That should put you where you need to be, and in the worst case, you have a dump of your bacula DB in the event you have to roll back. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/1/23 18:30, Ken Mandelberg wrote: So I know I need to copy /etc/bacula to a safe place. Should I also delete it? No, don't do that, you want to keep your configuration. If you delete those files you'll lose it. I will also stop director , sd, and fd. and move those binaries out of the way. You shouldn't need to do that. Let the package system handle it. I'm running mysql. Should I do anything there to the existing bacula data base? See below. Assuming the answers to those the next stop is apt-get install bacula-mysql I would assume I then copy back the saved /etc/bacula back in place and then start the new binaries. What about updating the mysql tables from 9 to 13? Where does that happen. The update-mysql-tables script will take care of that for you. Run it BEFORE you start the new Director. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] reload command
On 6/5/23 10:46, Bill Arlofski via Bacula-users wrote: On 6/2/23 08:59, Phil Stracchino wrote: Based on personal experience, after reloading I also recommend restarting the Director when opportunity presents itself. (i.e when there are no running jobs). I have found in the past that after multiple reloads, stability deteriorates. It SHOULDN'T, in theory, but nevertheless sometimes it does. Hello Phil, For the record, and for completeness, do you think you can describe some of the things that seem to become unstable? I am asking out of curiosity because I don't believe I have seen this, and who knows, maybe I have and just didn't make the correlation and your experiences might turn on a light bulb. :) Bill, I would if I could, but I haven't run into it in a long time because (a) I have an established practice of restarting the Director when feasible after significant configuration changes (which does not include Filesets, Schedules, new Clients etc), and (b) it's been a long time since I've needed to make any significant configuration changes. But I do recall that when I was developing and tuning my configuration, especially things like tweaking tape storage devices, spooling etc, if I applied too many changes via reloads without a restart in between, the Director would become unresponsive. Unfortunately I can't really recall any more detail than that. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacularis data table error
On 6/2/23 23:43, Marcin Haba wrote: Hello Ken, This problem is caused by the MySQL catalog table inconsistency. I opened an issue in the Bacula GitLab here: https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2684 <https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2684> Best regards, Marcin Haba (gani) Good catch Marcin! -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] reload command
On 6/2/23 09:57, Pedro Oliveira wrote: Hi Yateen Yes it works but only for configurations changes made on Director "Bacula-dir.conf", for the other Daemons we need to restart the daemons. Best Pedro Oliveira Keep in mind that a reload will not affect already-running jobs, and *some* changes may not take effect in already *scheduled* jobs. Based on personal experience, after reloading I also recommend restarting the Director when opportunity presents itself. (i.e when there are no running jobs). I have found in the past that after multiple reloads, stability deteriorates. It SHOULDN'T, in theory, but nevertheless sometimes it does. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Index for Table File is corrupt
On 5/28/23 02:38, Dr. Thorsten Brandau wrote: Well, thank you. I tried everything above, including going from -1 instead (tried different ranges). WHATEVER I DO, I only get ERROR 1034 (HY000): Index for table 'File' is corrupt; try to repair it back. And you'd already tried a REPAIR TABLE and it failed, as I recall. Yeah, at that point there's nothing else to do but drop and restore, or reinitialize the entire database. Right call. Hopefully the tuning we did will help prevent any future problems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Index for Table File is corrupt
On 5/27/23 14:22, Phil Stracchino wrote: So first create a new empty table: CREATE TABLE `File_NEW` ( `FileId` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `FileIndex` int(11) DEFAULT 0, `JobId` int(10) unsigned NOT NULL, `PathId` int(10) unsigned NOT NULL, `Filename` blob NOT NULL, `DeltaSeq` smallint(5) unsigned DEFAULT 0, `MarkId` int(10) unsigned DEFAULT 0, `LStat` tinyblob NOT NULL, `MD5` tinyblob DEFAULT NULL, PRIMARY KEY (`FileId`) ) ENGINE=InnoDB; Incidentally ... this table schema is not ideal, and yours is missing indexes in any case. This is what a Bacula 13.0.3 File table SHOULD look like, out of the box: CREATE TABLE File ( FileId BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, FileIndex INTEGER DEFAULT 0, JobId INTEGER UNSIGNED NOT NULL, PathId INTEGER UNSIGNED NOT NULL, Filename BLOB NOT NULL, DeltaSeq SMALLINT UNSIGNED DEFAULT 0, MarkId INTEGER UNSIGNED DEFAULT 0, LStat TINYBLOB NOT NULL, MD5 TINYBLOB, PRIMARY KEY(FileId), INDEX (JobId), INDEX (JobId, PathId, Filename(255)) ); I have made a few changes to mine which *significantly* improve performance without any impact detected so far (in years of use) on operation. This is what my File table now looks like: CREATE TABLE `File` ( `FileId` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `FileIndex` int(11) DEFAULT 0, `JobId` int(10) unsigned NOT NULL, `PathId` int(10) unsigned NOT NULL, `Filename` varbinary(255) NOT NULL, `DeltaSeq` smallint(5) unsigned DEFAULT 0, `MarkId` int(10) unsigned DEFAULT 0, `LStat` varbinary(80) NOT NULL, `MD5` varbinary(130) NOT NULL, PRIMARY KEY (`FileId`), KEY `JobId_2` (`JobId`,`PathId`,`Filename`), KEY `PathId` (`PathId`) ) ENGINE=InnoDB It's up to you whether you want to try this modification. You can always modify the table to this spec later. (And I can give you the rest of the set of schema changes, if you want.) The change from BLOB columns to VARBINARY greatly increases performance of JOINS, because the MEMORY storage engine does not support BLOB/TEXT data types, and as a result any implicit temporary table created by a JOIN which contains BLOB/TEXT tables will always be forced to disk. It also avoids the need to put a range constraint on Filename in the second index. VARBINARY behaves exactly like BLOB in all important respects except that it IS supported in the MEMORY storage engine. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Index for Table File is corrupt
ree reasons for doing it this way: 1. You can see that it's visibly making progress, not just going out to lunch 2. Copying the rows in small batches reduces the memory required for each copy operation. Copying 70 million rows at once is a huge transaction, and if it fails for some reason in the middle, all that work is lost. 3. If the copy is going to fail for any reason, we want to copy as many rows as possible BEFORE it fails *and know where it failed*. If it fails partway through, say at 3500, you MAY be able to resume by passing it a starting offset of 3510 and see if you can resume after the failure. I'm also not deleting the copied rows from the source table, for two reasons: 1. Bulk deletes in MySQL are slow and it would cost us more time than it would save 2. We want to preserve all of the original data until we're sure we've got it all. If you prefer, you can TRY doing a single one-shot batch copy first: INSERT INTO File_NEW SELECT * FROM File; But this may run for hours with no visible output. You can monitor it from another MySQL shell using SHOW PROCESSLIST to verify that it is still running. Once you can (hopefully) get one or the other method to complete without erroring out: RENAME TABLE File TO File_DAMAGED, File_NEW TO File; And then if I were you, I'd run a dbcheck admin job. Then restart your Director and see how it all looks. I can't guarantee this will work, but there's *clearly* something screwy in your DB at the tablespace level or lower, and I can't tell without poking around in your database myself what it is. It's possible there was a corrupted sector where the index is stored. Yeah, I know, RAID, checksums, not supposed to happen; there is a thing that storage admins lose sleep over called the RAID5 write hole. It's a way that silent data corruption can happen on a RAID5 array. (Speaking of which, if you have the ability to do so, if I were you I'd try to provision a separate RAID10, *NOT* RAID5, volume for your database storage. When it comes to large database operations, I/O throughput is next to godliness.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Index for Table File is corrupt
On 5/26/23 11:39, Dr. Thorsten Brandau wrote: When I use mysqlcheck I get Warning : InnoDB: The B-tree of index PRIMARY is corrupted. mysqlcheck -r bacula returns "storage engine does not support repair" So, the B-Tree is still corrup on PRIMARY. OK. If that is the only problem, and there is plenty of free disk space (AT LEAST the size of the File.ibd file), this SHOULD be straightforward to repair. Try this: ALTER TABLE File DROP PRIMARY KEY, ADD PRIMARY KEY (FileId); That will recreate the corrupted index. Expect it to take a while. If NOTHING ELSE is corrupted, you should be good. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy job clarification requested
On 5/18/23 18:37, Chris Wilkinson wrote: I'm not sure I'm getting the motivation for using a copy job in preference to a duplicate job to a second SD. This would also create a second backup. The only reason I can think of is that a duplicate job might be different if the files changed in between. That shouldn't be an issue. I read that a copy job cannot be used for a restore Not true. Just select that job to restore from. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] compression with lto?
On 5/17/23 12:52, Marco Gaiarin wrote: I've LTO tape that work in very dusty environment by at least 6 years. Istead of dust, i've found that setting properly spooling and buffers prevent the spinup/spindown effect, that effectively can be very stressful... Yes, I went to great lengths to try to keep mine streaming and avoid shoe-shining, but with only moderate success. I have had many fewer problems, as well as much better performance, since I abandoned tape and went to disk-to-disk-to-removable-disk (with both of the destination disk stages being RAID). Full backup cycles that used to take 18 hours and two or three media changes, with about a 10% failure rate due to media errors, now take 3 or 4 hours with no media changes and nearly 100% success. However, we are getting further and further off the subject of compression. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] compression with lto?
On 5/17/23 11:48, Dr. Thorsten Brandau wrote: I decided to use the softwarecompression instead, as this actually compresses the files. At last it means 1.4:1 for me, which is a lot better than uncompressed backup (LTO9 tapes are very expensive...). I have to admit I found the cost of the media for the last LTO generation I used (LTO4) to be secondary to the cost of replacing drive after drive after drive as they failed, because in my experience LTO drives fail very quickly outside of a clean-room environment or a sealed tape library. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Storage protocols advice sought
On 5/10/23 11:19, Chris Wilkinson wrote: This is not really a Bacula question but I'm hoping someone has come across this when using NFS to mount a NAS storage device. As I understand it, the UID:GID on the remote storage and the local SD daemon must match or permission denied results. My NAS has NFS v3 and this can't be changed. I can create a new user on the NAS but I have no control over what UID:GID it chooses and will not be the same as the SD daemon user:group. I believe NFS v3 doesn't have the capability to map UIDs; that facility is v4 only. Has anyone found a workaround for this? What is the NAS? I'm guessing if you don't have the ability to control UIDs/GIDs on the NAS you probably can't install a SD on it directly either. NFS-mounting your Bacula backup storage is usually a bad idea unless you have no alternative. Honestly, in my experience off-the-shelf consumer/prosumer NAS appliances are utterly horrible from a management perspective — you WILL do things THEIR way, or not at all. I tried a consumer NAS a few uears ago — I'm blanking on the brand right now — and had to return it because not only was it useless for what I wanted, as well as having some pretty ghastly security issues. (Like making every file on every share world-read/write via NFS.) If you can't control the UID/GID assigned by the NAS, can you change the UID/GID of your storage daemon to match what the NAS assigns? That would at least solve the permission problems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New bacula install
On 4/6/23 17:30, Bill Arlofski via Bacula-users wrote: On 4/6/23 15:27, Andrew Meyer via Bacula-users wrote: I think these were provided from epel. Is there a rpm/yum repo provided by bacula?? Hello Andrew, https://www.bacula.org/bacula-binary-package-download/ Fill out the form and you will be emailed your one, unique download area of Bacula Community packages. Be sure to drop your partly-created bacula database before you reinstall. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New bacula install
On 4/6/23 09:09, Andrew Meyer via Bacula-users wrote: [@bacula01 ~]$ rpm -qa|grep -i bacula bacula-storage-9.0.6-6.el8.x86_64 bacula-libs-9.0.6-6.el8.x86_64 bacula-director-9.0.6-6.el8.x86_64 bacula-client-9.0.6-6.el8.x86_64 bacula-common-9.0.6-6.el8.x86_64 bacula-libs-sql-9.0.6-6.el8.x86_64 bacula-console-9.0.6-6.el8.x86_64 [@bacula01 ~]$ I suggest you get current release packages (13.0.2) from bacula.org and try that. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New bacula install
On 4/5/23 11:10, Andrew Meyer via Bacula-users wrote: I'm trying to set up bacula on AlmaLinux 8.7 w/ MariaDB 10.11 and getting the following error: $ sudo /usr/libexec/bacula/make_mysql_tables -u bacula -p Enter password: ERROR 1005 (HY000) at line 112: Can't create table `bacula`.`Job` (errno: 150 "Foreign key constraint is incorrectly formed") ERROR 1005 (HY000) at line 286: Can't create table `bacula`.`Pool` (errno: 150 "Foreign key constraint is incorrectly formed") Creation of Bacula MySQL tables succeeded. The error was much worse when I tried to run this as mysql root user. Can someone point me in the right direction to resolve this? Andrew, which version of Bacula? If you are seeing foreign key constraint failures, I suspect the general answer is "an old one", because no table in current Bacula releases contains foreign key constraints. They were removed a LONG time ago. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] troubles with automatic labeling after ubuntu upgrade
On 3/31/23 10:16, Peter N. Steinmetz wrote: Hi Matlink and Bill, Thanks for those suggestions and explanation of why Bacula won’t automatically label a new volume because the pool is full. I agree that the puzzle now is why it is not recycling one of the purged volumes in that pool? Are the oldest volumes *actually fully purged*? Do you have Recycle Oldest Volume enabled? (Actually, I don't recycle disk volumes. I delete them on purge and create new ones as needed. I find it simplifies volume tracking and makes better use of disk space.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Truncate Volumes Before Retention Time is Up
On 3/30/23 06:40, Chris Wilkinson wrote: I have an old job that is no longer needed that is consuming some disk space that I'd like to recover. This storage for this job is on S3 cloud and the job has full/diff/incr pools associated. The full backup pool/volumes of this job have a retention time of 365 days so would not normally get pruned out for another year almost. The incr/diff pool/volumes have a retention time of 7days/31days respectively. I'd like to truncate these now, delete the job resources and rm the actual disk volumes, deleting all traces of the defunct job. What would I need to do to truncate these out now? I haven't been able to find bconsole commands that would do this. I tried the truncate command but the volumes status remains unchanged at full or used or append. Thanks for any suggestions. Fundamentally: purge jobs volume=... delete volume=... then rm the file. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fileset with varibale date (130223)
On 2/13/23 10:28, Jose Alberto wrote: Hi. It is possible to backup a folder with the date variable in example format: 130223 (day, month, year) I have a folder that is created every Saturday and I need only that one to be backed up /oracle/rman/180223 (next Saturday) It's possible? A Fileset can be the output of a script. So you could create a script thatfinds and returns just that one location. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 9.6 client managing Windows services
On 1/30/23 11:43, Adam Weremczuk wrote: Hi all, Is it possible to tell a client to stop a list of services in a specific order before starting a backup job? Then start them up after backup run is complete. Bacula server runs on Debian 11 and the client on Windows Server 2019 Standard. You could use client-side RunBefore Job and RunAfterJob scripts. (i.e, "RunsOnClient = yes" in the RunBeforeJob/RunAfterJob directive) Obviously this would need careful testing, and you face the issue that if a job fails for any reason, then those services may not get restarted. If they are mission-critical, this would likely be a problem and you would need to implement a failsafe mechanism for restarting them after the backup if not running (a cron job perhaps). -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy backups to more than one storage
On 1/17/23 11:45, Bill Arlofski via Bacula-users wrote: On 1/17/23 06:05, Ivan Villalba via Bacula-users wrote: > How can I run two differnet copy job that copies the same jobid with the PoolUncopiedJobs ? You can't. The PoolUncopiedJobs does exactly what its name suggests: It copies jobs in a pool that have not been copied to some other pool. If you want to copy the same backup jobs to more than one other pool, you will need to use `Selection type = SQLQuery` and then use an appropriate SQL query for the `SelectionPattern` to generate a list of JobIds to run the second set of copies. Which, I note, is what I do ANYWAY, just in case something goes wrong and I have to *rerun* the copy job. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Storage configuration for just two hard drives in rotation
On 12/27/22 16:24, Bill Arlofski via Bacula-users wrote: The description above is our current working scenario using Retrospect just scaled way down for testing purposes. For reference, we typically backup around 100ish clients and have three sets of drives that are in rotation: onsite online set, an onsite offline set, and an offsite sets. These sets are rotated each week. In my experience, trying to migrate a backup *PLAN* from one backup *SYSTEM* to another rarely works well. It is usually better to define your backup *goals*, and then determine how best to accomplish those goals within the context of the new backup system. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup restore missing a directory
On 9/9/22 13:55, William Muriithi wrote: Hello, I have attempted a restore and I can get all files and directories with exception of one folder. This is on Windows server 2016 (bacula client - 9.0.8 (28May18) Microsoft Standard Edition (build 9200)) and it is running as local system. The storage and director are on 9.0.6 on Centos 8 Question, what would cause an issue where bacula selectively ignore one directory? First question: Have you verified whether the directory actually exists *in the backup*? Second question: What does the backup fileset look like and what is the pathname of the directory? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: Verify differences - how to investigate?
On 9/8/22 20:25, Justin Case wrote: Finally 8 of 15 verify jobs returned with differences. I did not have this before when I was only using level Data and not Accurate. How can I investigate teh root cause of this? What seems counter intuitive to me is that all verify jobs are marked as OK, also those that returned with differences. I guess this means the verify job ran completely and has no relation to potential differences being found. That is correct. 'OK' means that the Verify job ran successfully without incident. It does NOT mean that it did not find any discrepancies. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: is Level a selection criterion for Migration/Copy jobs?
On 9/8/22 13:38, Martin Simmons wrote:
If you can express it in SQL, then you can use that to select a job, e.g. I
use this for testing:
Job {
Name = "CopyLastBackupJob"
Type = Copy
Selection Type = SQLQuery
Selection Pattern = "select jobid from job where type='B' and jobbytes > 1 order
by jobtdate desc limit 1"
...
}
I use the following:
Selection Type = SQL Query
Selection Pattern = "SELECT DISTINCT J.JobId, J.StartTime
FROM Job J JOIN Pool P ON P.PoolId = J.PoolId
WHERE P.Name = 'Full-Disk'
AND J.Type = 'B'
AND J.JobStatus IN ('T','W')
AND J.jobBytes > 0
AND J.StartTime > now() - interval 48 hour"
(I don't actually need the J.StartTime on the first line there, it was
there for testing the query, but it does no harm.)
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Silly question from newbie
On 8/25/22 17:30, Pollard, Jim wrote: Maybe another option would be do a migrate job to external media for enough of the volumes that contain the incrementals. That way If things go completely south I could still restore. I’ll have to take a look at the schedules. I’m feeling like the first thing bacula is going to do is determine that incrementals are missing and initiate fulls. Is that sounding right? Bacula will not initiate a FULL backup because incrementals are missing or have been moved elsewhere. It will self-initiate a FULL backup if it cannot find a valid prior FULL backup job *in the Catalog*. (Side note: You might look into VIRTUAL FULL jobs here and see if that is of any use to you.) My backup strategy varies retention times for different job levels such that there is always overlap. The higher the level, the longer the retention. I run nightly Incrementals which are kept for a month, weekly Differentials that are kept for two months, and monthly Fulls that are kept for four months. I can point-in-time full restore to a specific day anywhere in the past month from no more than eight Jobs (one Full, one Differential, and no more than six Incrementals). Your incremental backups should be small, and should have the shortest retention times. If considering migrating/copying some jobs to external storage, I would suggest migrating the higher-level jobs that are larger and have longer retention, rather than the incrementals. This will make better use of your online storage while not impacting your ability to perform immediate small, near-present-time restores. I copy only my FULL backups to air-gapped external media; Differentials and Incrementals are kept online until pruned. It literally takes me longer to *set up* a small restore of a few recent files than it does to actually perform the restore. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] running Bacula client on Synology
On 8/2/22 15:37, Bill Arlofski via Bacula-users wrote: Thanks Phil. I needed the laughs and the quotes to steal today. :) My work here is done. :) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] running Bacula client on Synology
On 8/2/22 14:42, Andrea Venturoli wrote: I've just tried cross-building 13.0.0 for QNAP/ARM: it just took a few minutes. (I explicitly left OpenSSL out, since I don't use it, so I don't know if it would work). Unfortunately I cannot test this at least until September. I'll happy provide the binaries (of course as-is) if you need them. Huh. Now I'm curious about how you set up the QNAP cross-compiler and development environment. I couldn't find any useful information on that and QNAP support flatly refused to provide any. It's only academic curiosity at this point though, as I returned the QNAP NAS because it manifestly could not do the things I wanted to do with it. (Starting with manage it in any sensible way beyond pushing buttons in its point-and-drool web management interface, which I quickly found to be full of You Can't Do That.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] running Bacula client on Synology
On 8/2/22 07:16, Adam Weremczuk wrote: Hi all, Synology RS1221+ 8-Bay NAS specifically. Are there any officially supported fairly recent clients and installation guides available? If not Synology what other budget NAS brands are supported best? I have not tried Bacula on Synology, but I can tell you one thing for free: Don't even THINK about trying to run a current Bacula on QNAP. There are obsolete Bacula packages for SOME QNAP models in QNAP's proprietary package format, but absolutely no way to install a current Bacula version. I tried it once and couldn't even figure out how to get a viable build environment set up. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Change expiration date of a single volume
On 7/15/22 12:13, Wolfgang Denk wrote: Is this what you want to see? *show pool=ARCH Pool: name=ARCH PoolType=Backup use_cat=1 use_once=0 cat_files=1 max_vols=0 auto_prune=1 VolRetention=12 years 3 days VolUse=0 secs recycle=0 This would be your problem ^ right here. You have TOLD Bacula not to recycle volumes, so it's not recycling them, just as you told it. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Change expiration date of a single volume
On 7/15/22 11:26, Wolfgang Denk wrote: This was my expectation, too, but apparently this is not the case. As long as the tape has not expired, it will not be reused by Bacula. When I change the status from "purged" to "append", Bacula will happyly use the tape - and append to the end of the existing data. What, if any, 'Recycle ...' directives do you have in your Pool definitions? Please post your Pool definitions. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula on Nas
On 7/5/22 11:55, rkasmoll...@gmail.com wrote: Dear Sir or Madam, I am doing my thesis on open source projects and one of them I have chosen is Bacula. My question regarding Bacula would be if it is possible to configure Bacula in NAS even with the community edition model. You need to be a little more specific than this. Start with, "What NAS"? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Compatibility between Dir/SD 9.4 and FD 5.X ...
On 6/22/22 16:51, Marco Gaiarin wrote: Mandi! Phil Stracchino In chel di` si favelave... The first and most obvious question here is: Is there some sound technical reason why you have not updated the file daemons on those servers to something at least moderately recent? Never hear of 'legacy systems'?! ;-) These are a bunch of old servers, that have to be decommissioned but for now have to be backed up, and clearly first choice is not to do so much work, apart 'apt-get install bacula-fd'. OK, so what you're saying is that these are servers that are old enough that you cannot get a post-7.x bacula client package that will work on them? That certainly does limit your options. Is it *feasible* to install enough of a local development environment on these legacy servers to locally compile a 7.x client for them? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Compatibility between Dir/SD 9.4 and FD 5.X ...
On 6/22/22 12:39, Marco Gaiarin wrote: I'm setting up a Bacula installation for a set of servers, using debian buster package (9.4.2-2+deb10u1) for director and storage daemons. All works as expected until we use file daemon versions 7+, but some very old servers that have bacula 5.0 or 5.2 does not work. The first and most obvious question here is: Is there some sound technical reason why you have not updated the file daemons on those servers to something at least moderately recent? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: one job for a group of "identical" machines using the same fileset?
On 6/20/22 06:33, Justin Case wrote: Interesting, ist there an overview of available Community plugins, or where would I best start to look around? On 20. Jun 2022, at 10:37, Radosław Korzeniewski mailto:rados...@korzeniewski.net>> wrote: Hello, niedz., 19 cze 2022 o 23:29 Justin Case <mailto:jus7inc...@gmail.com>> napisał(a): Good to know, I guess for commercial use. For my private homelab commercial plugins are out of reach. Or is there such a plugin for free? FYI, what I do for this application is I have a JobDef for all of the machines that use that fileset, then a Job for each one that basically just specifies that set of defaults, the job name, and the client hostname. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy Job of a bunch of backups?
On 5/27/22 05:27, Pierre Bernhardt wrote: Hello, I create a full backup each 1st Sunday, diff backups of rest of Sundays, incremental backups each day. I would create a copy after the first day of a month backup has been made. The data should be copied from the last backup however it has been made. So for the moment I create a restore before a given date/time and create a full backup of these restore to a special tape. Is there a "virtual copy" possible which which I can configure to create one copy job based on backup before a specified time which made the same without having restore them before? If I understand correctly what you are asking, you could, hypothetically speaking, do a Virtual Full backup (i.e. construct a synthetic full backup job within the database from already-existing backups), and then run a Copy job to copy that Virtual Full job to a new tape. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] query.sql
On 5/25/22 13:23, SO Giuseppe - COMAE via Bacula-users wrote: SELECT count(*) AS Jobs,sum(JobFiles) AS Files,sum(JobBytes) AS Bytes,VolumeName FROM (SELECT DISTINCT Jobmedia.JobId, JobFiles, JobBytes, VolumeName FROM Job,JobMedia,Media WHERE JobMedia.JobId=Job.JobId AND JobMedia.MediaId=Media.MediaId) AS f GROUP BY VolumeName ORDER BY VolumeName; MySQL 127.0.0.1> SELECT count(*) AS Jobs,sum(JobFiles) AS Files,sum(JobBytes) AS Bytes,VolumeName -> FROM (SELECT DISTINCT Jobmedia.JobId, JobFiles, JobBytes, VolumeName -> FROM Job,JobMedia,Media -> WHERE JobMedia.JobId=Job.JobId -> AND JobMedia.MediaId=Media.MediaId) AS f -> GROUP BY VolumeName -> ORDER BY VolumeName; ERROR 1054 (42S22): Unknown column 'Jobmedia.JobId' in 'field list' Straightforward typo in line 3. You have Jobmedia instead of JobMedia. With the typo fixed, yes, this yields much more reasonable and correct-looking results. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about schedule
On 5/24/22 14:12, egoitz--- via Bacula-users wrote:
Good morning,
One little question about Schedules. Imagine I have this schedule :
Schedule{
Name = "WeeklyCycle"
Run = Full 1st sun at 23:05
Run = Differential 2nd-5th sun at 23:05
Run = Incremental mon-sat at 23:05
}
Imagine I set up a new backup... and I set up in the third week of the
month. Imagine I start on a Tuesday. For my experience, I assume that
Bacula would do :
- As no backups have been done previosly, first of all at the first time
it runs at 23:05, perform a full backup.
- The next day it runs, even there is not a differential backup, as
supposedly the second time the job runs is Wednesday it will do an
incremental backup.
- When Sunday arrives, it would do the first differential. At this
moment we will have a Full backup, plus N incrementals. Today we to a
differential and will have a differential too.
- The following times it launches incrementals in the week and on Sunday
new differential, until the first Sunday of month in which a new full
will be created.
Am I wrong?. I think not...
That all looks correct, yes.
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about scratch pool
On 5/23/22 12:05, egoitz--- via Bacula-users wrote: Hi mates, My purpose is the following one. As I run backup of different customers : - For better isolating them for managing permissions properly - For separating quota usage - For using more little databases - And because it's done this way some time ago for several reasons (some of them I can't really know now) we are using different catalogues for our customers. OK, so you are using one Director, but multiple Catalogs, one per customer? As Heitor suggested, you could create a separate named ScratchPool in each Catalog, but there is no way to *safely* share the same Scratch pool across multiple catalogs, and I cannot begin to guess at what would happen if the Director finds the same Pools in different Catalogs with different volumes in them. I would urge you in this case to try to make all resource names unique to customers and NOT try to share anything between them. Trying to share volumes in a common scratch pool would break the isolation you are trying to create, and sounds like a cross-customer data breach mishap waiting to happen. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about scratch pool
On 5/23/22 10:39, egoitz--- via Bacula-users wrote: Good morning, We are planning to use the Scratch pool for automatically having avaible tapes in the different pools of my different catalogues. For this purpose, we planned to create let's say 100 volumes in scratch and later, that volumes to to be requested by the different pooles AND of different catalogues, needing new media for continuating backup jobs. Is this possible or the scratch pool can only share volumes with pools within the same catalogue?. If you have multiple Catalogs, you would need to devise a way to synchronize the state of the Scratch pool across all of them. This is not functionality that is supported in Bacula. If you have multiple Bacula directors running from different Catalogs, they need to be separate, not sort-of-separate-but-with-overlaps. Trying to share resources between Directors running from different Catalogs will get you into trouble, because one director will have no way of knowing what another is using or has used. This could lead to cases where, for example, one Director performs a critical full backup on a certain set of tapes, and then the next day another Director that doesn't know it has done so recycles one of those tapes and uses it for something else, and the first doesn't know the backup is no longer valid. What are you trying to accomplish with this multiple-catalogs scheme? When the answer doesn't make sense, sometimes it means you're asking the wrong question. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Sorry
On 4/25/22 09:52, Alan Polinsky wrote: Folks: Unintentionally I included a email to the group, when I only meant it to go to my sons. Sorry. It's OK. Back door alarm support is planned for a future release. :D -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] 9.4.4 dir and 9.4.2 client interoperability
On 3/25/22 09:24, Shaligram Bhagat, Yateen (Nokia - IN/Bangalore) wrote: Hi All We have bacula-dir v 9.4.4 hosted on Centos 6.x and bacula-fd v 9.4.2 hosted on ubuntu 20.04 Can they interoperate ? This should be no problem. The Director and all Storage daemons should be the same version, but if some clients are a few minor versions behind (even major versions, so long as it's not too many), that will not cause any issues. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula‑users] Link Error 11.0.5 on sles
On 3/10/22 20:06, Phil Stracchino wrote: On 3/10/22 18:37, Mark Casey wrote: Yep, I use bat. The options i used to compile are the same as what I used for the 9.6.4 version qt4 libraries and dev are installed I have downloaded the 11.0.6 version to see if that fixed the issue I had no problems building bat in 11.0.5. 11.0.6 has not hit Gentoo yet. I'm sure it won't be long. Er, I'm sorry, I completely garbled that and only half read what I was replying to. Please ignore. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula‑users] Link Error 11.0.5 on sles
On 3/10/22 18:37, Mark Casey wrote: Yep, I use bat. The options i used to compile are the same as what I used for the 9.6.4 version qt4 libraries and dev are installed I have downloaded the 11.0.6 version to see if that fixed the issue I had no problems building bat in 11.0.5. 11.0.6 has not hit Gentoo yet. I'm sure it won't be long. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] mariadb 10.5.15
Oh, one other thing ... how old was your OLD mysql database? Red Hat had the incredibly stupid idea of shipping MySQL configured by default with 'old_passwords=1', and many other distributions just blindly shrugged and said, "Huh, seems legit to me." If your password hashes in the mysql.user table are less than 41 characters in length, you probably have old pre-MySQL-4.1 insecure passwords which have been deprecated since ... 2004 I think? All such passwords should be regenerated, and preferably changed, after making certain that old_passwords is OFF. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] mariadb 10.5.15
On 3/6/22 14:39, Alan Polinsky wrote: If I use mysql -U bacula, I cannot connect to the Bacula database. It seems to be a permission issue, but things were changed with version 10.4 of MariaDB. Can someone suggest a solution? OK, first: - Can you log into your database as superuser? - Can you verify that the 'bacula' user has a password set? - Can you verify that it is set to what you *think* it is? (Or you could just make sure by CHANGING it to what you think it should be.) - Is the host part for the 'bacula' user correct? - Does the 'bacula' user have the correct permissions? (SHOW GRANTS FOR bacula@...) - Having done all of those, can you log in using 'mysql -u bacula -p' ? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Job copy question
On 2/25/22 17:59, Josip Deanovic wrote:
Several years ago I posted to this list an SQL query that copies only
the jobs that weren't already copied, while starting from the last
successful Full job. The post could still be found in the mailinglist
archives.
I used it when I had to start using Copy jobs but didn't want to
copy the whole pool as it was quite large.
This is what I use:
Job {
Name = "Archive Copy"
Type = Copy
Enabled = Yes
Pool = Full-Disk
Level = Full
Client = ALL
Fileset = DUMMY
Storage = epsilon3-archive
Selection Type = SQL Query
Selection Pattern = "SELECT DISTINCT J.JobId, J.StartTime
FROM Job J JOIN Pool P ON P.PoolId = J.PoolId
WHERE P.Name = 'Full-Disk'
AND J.Type = 'B'
AND J.JobStatus IN ('T','W')
AND J.jobBytes > 0
AND J.StartTime > now() - interval 48 hour"
Messages = Daemon
Priority = 20
Rerun Failed Levels = yes
Allow Duplicate Jobs = no
Cancel Queued Duplicates = yes
Schedule = "Full Backup Archive"
}
This selects every completed Full backup that started in the previous 48
hours for copying. My Full backups run once a month.
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
Just to wrap up, the definitive answer to Bacula on QNAP: There is a third-party Bacula 9.0.5 package for QNAP NAS which can be obtained from https://qnapclub.eu/en. No idea if or when it will be updated to 11.0.5. Other than that, fuhgeddaboudit. And you probably don't want to run anything with QNAP's Hero OS anyway. It has major security problems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/3/22 21:05, Gary R. Schmidt wrote: That can sometimes be fixed by setting the mount to be soft rather than hard, or vice-versa, but the Linux implementation of NFS leaves a lot to be desired, at both ends. (Speaking as a person who was a tech lead at SGI's "File Serving Systems" and wrote many, many tests for NFS and really developed a distaste for the subtle gotchas in NFS. If I ever hear someone defending the, "Oh, it can't fall back from v4 to v3 because they're different protocols", again, I will strangle the .) This is why for many years now my NAS has run Solaris. It is THE reference implementation of NFS. However, Solaris is dead. Oracle hustled it out behind the barn and put a bullet in its head. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/3/22 07:56, Heitor Faria wrote: These providers do that because they think it would be very hard or even impossible to give support to a NAS with an OS that the user can freely modify. Therefore, you could try TrueNAS <https://www.truenas.com/>, which is based on FreeBSD, or build your own (e.g.) <https://www.servethehome.com/buyers-guides/top-hardware-components-for-truenas-core-servers/>. There are a lot of recipes also on YouTube. You can usually also erase proprietary NAS' OS and install TrueNAS. But returning to the main thread, I still think the best solution for Phil's situation would be to use a NAS built-in virtualization capabilities. I hope he has success anyways. This entire discussion is starting to become moot because I am increasingly discovering that the QNAS unit is so terribly, horribly, appallingly buggy and crippled and broken that I'm beginning to doubt how I can possibly in any good conscience use it. It feels as though I would just be storing up years of daily headaches, and I'm looking for alternatives that I have full control over. I did consider completely wiping the thing and installing either Gentoo Linux or Solaris x86. But I hesitate to do that on a brand new unit still under warranty. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/2/22 15:03, Phil Stracchino wrote: I'm going directly to the heart right now. I'm cross-compiling an entire statically-linked gcc/binutils toolchain for it on another machine. THEN I can locally compile Bacula (and probably a few additional library dependencies) and have no further problems. (I note that the ENTIRE REASON this problem arises in the first place is because QNAP sees fit to ship the device with a fully ... well, MOSTLY functional Linux OS *with no system compiler installed*.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/2/22 13:52, Heitor Faria wrote: Hello Phil, Extra workload on the NAS isn't an issue. It has 32GB RAM and eight Xeon D-1622 cores. But then, even you you manage to build the SD you will probably be stuck with the same Bacula version forever. I guess you haven't disclosed this specific NAS model, but you might consider using a VM in the QuTS Hero "Virtual Machine Sation" solution. Ref.: <https://www.qnap.com/en/product/ts-h886> Then the OS is not a problem anymore. That is EXACTLY what I'm using. :) I'm going directly to the heart right now. I'm cross-compiling an entire statically-linked gcc/binutils toolchain for it on another machine. THEN I can locally compile Bacula (and probably a few additional library dependencies) and have no further problems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/2/22 06:59, Josip Deanovic wrote: The problem is with the openssl support. File daemon would probably statically compile without openssl support. Last time I did that was with the 7.x version of bacula. I just tried disabling openssl support but that did not solve the problem. I'm building a static gcc/binutils toolchain to install on the NAS and I will build sd and fd with that. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/2/22 10:08, Heitor Faria wrote: Hello Phil, (a) trying to figure out how to get a Bacula sd and fd installed on it The happiest days of my life were after I got dedicated equipment for home network modem, routing, and wireless mesh accessing points. That said, I never understood this very common user fetish of installing Bacula components in the NAS. OK: you avoid one more network node but the hardware is not designed to handle the extra workload. Because when the backup scheme is disk to disk to removable disk, the transfer rate going directly across the backplane is much higher than I can get across a gigabit network. Also, I don't want to burn the power to run another node that functionally does nothing but run a storage daemon. Extra workload on the NAS isn't an issue. It has 32GB RAM and eight Xeon D-1622 cores. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/1/22 17:50, Phil Stracchino wrote: Without the static directives the entire build completes successfully. ...But didn't work because of the wrong glibc version. QNAP says look, golly gosh gee whiz wow, kernel 5.10! But is still using glibc 2.21, 13 releases behind. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/1/22 13:15, Phil Stracchino wrote: This QNAP unit has Intel CPUs and a proprietary-Linux-distribution internal OS. I suppose compiling a statically linked binary package for it on one of my other machines would be one way to solve the problem. Thanks for the thought. The following configure invocation on a Gentoo system also with Intel processors: ./configure --prefix=/opt/bacula --with-dump-email=r...@caerllewys.net --with-job-email=r...@caerllewys.net --with-smtp-host=smtp.caerllewys.net --with-subsys-dir=/opt/bacula/var --with-working-dir=/opt/bacula/var --enable-build-stored --disable-build-dird --enable-smartalloc --disable-batch-insert --with-mysql --enable-static-fd --enable-static-sd --enable-static-tools --disable-libtool successfully built lib and findlib, then failed in filed: /usr/bin/g++ -static -L../lib -L../findlib -o static-bacula-fd filed.o authenticate.o backup.o crypto.o win_efs.o estimate.o fdcollect.o fd_plugins.o accurate.o bacgpfs.o filed_conf.o runres_conf.o heartbeat.o hello.o job.o fd_snapshot.o restore.o status.o verify.o verify_vol.o fdcallsdir.o suspend.o org_filed_dedup.o bacl.o bacl_linux.o bxattr.o bxattr_linux.o \ -lacl -lz -lbacfind -lbaccfg -lbac -lm -lpthread -ldl -ldl \ -lssl -lcrypto -lcap -llzo2 /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lacl /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lz /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: ../lib/libbac.a(plugins.o): in function `load_plugins(void*, void*, char const*, char const*, bool (*)(Plugin*))': /netstore/src/bacula-11.0.5/src/lib/plugins.c:150: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lssl /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lcrypto /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lcap /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -llzo2 collect2: error: ld returned 1 exit status make[1]: *** [Makefile:240: static-bacula-fd] Error 1 Do we actually still support static builds? Without the static directives the entire build completes successfully. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/1/22 13:09, Andrea Venturoli wrote: On 1/1/22 18:13, Phil Stracchino wrote: (a) trying to figure out how to get a Bacula sd and fd installed on it I cross-compiled them myself. Actually, only the SD (9.6.3 for arm): I think I shared it in the past, but if you need it just ask. This QNAP unit has Intel CPUs and a proprietary-Linux-distribution internal OS. I suppose compiling a statically linked binary package for it on one of my other machines would be one way to solve the problem. Thanks for the thought. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/1/22 11:32, Andrea Venturoli wrote: On 12/31/21 22:22, Phil Stracchino wrote: Does anyone have any experience with, or useful tips for, using Bacula to back up a QNAP NAS with the QuTS "Hero" OS? I'm backing up *to* a couple of QNAPs (running bacula SD daemon, and not with ZFS). I didn't choose them, but I'm quite happy about them. IIC you are trying to backup up *from* a QNAP, right? Perhaps you could be more specific: what problems are you encountering? (a) trying to figure out how to get a Bacula sd and fd installed on it (b) dear gods, the POINTLESS OBFUSCATION of shared volumes ... it seems like they fucked around with it because they could. Yes, it's ZFS internal filesystems, but they clearly have no idea what they're doing with ZFS. I absolutely cannot imagine what they were thinking. (a) is somewhat complicated by the fact that I need the sd compiled with attribute batching disabled, because batching is broken as currently implemented and as a result is not compatible with Galera clusters. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula for QNAP?
I replaced my failed NAS with a QNAP NAS that is advertised as having a Linux OS and ZFS filesystem. Both of these are technically true, but ... well, here is not the place to rant about what a mess they made of it. Does anyone have any experience with, or useful tips for, using Bacula to back up a QNAP NAS with the QuTS "Hero" OS? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-fd version
On 12/28/21 18:22, Richard Couture wrote: Does the bacula-fd client version have to be the same or newer than the server version of bacula or does the server version of bacula have to be the same or newer than the bacula-fd client version? my server version is 9.6.7 and the bacula-fd client version on one of the machines is much older How much older is "much older"? Bacula directors and storage daemons should always be the same version. Clients (file daemons) can be older *within reason*, but you may run into problems if the client is *too* old. Is there a reason why that one client cannot be updated? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] make_catalog_backup stopped working
On 12/28/21 11:33, Graham Dicker wrote: Looking good. A full backup completed and make_catalog_backup performed flawlessly, which was the original problem. All done manually. Tomorrow morning it'll be back on automatic and I have every expectation that when I finish breakfast there will be emails waiting for me reporting successful backups. I can't tell you how grateful I am for all the help from Graham Sparks, Phil Stracchino and Josip Deanovic. Thank you very much guys, ever in your debt. Glad to hear everything is working as it should be now. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] make_catalog_backup stopped working
On 12/27/21 15:07, Dmitri Maziuk wrote: On 12/27/2021 2:02 PM, Josip Deanovic wrote: On 2021-12-27 19:19, Phil Stracchino wrote: ... The blazes? If mysql.user is a VIEW, something is VERY BADLY WRONG. Are you still using Akonadi's MySQL instance? Yes, that doesn't look normal. I hope it's not something Suse specific. Google sez it's one of the many improvements in mariadb 10.4. Does bacula support mariadb at all? Not *officially*. But I'm using it with MariaDB Galera Cluster (10.5.10-r2) and I know it to work. And to my complete surprise, yes, mysql.user is now a VIEW in MariaDB. I have not yet formed an opinion as to whether I think this is a good idea. I consider quite a few of the ways in which mariaDB is diverging from MySQL to be at best questionable. Starting with the Aria storage engine that is supposed to somehow be all things to all people. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] make_catalog_backup stopped working
On 12/27/21 13:38, Graham Dicker wrote: On Monday, 27 December 2021 18:19:23 GMT Phil Stracchino wrote: > The blazes? If mysql.user is a VIEW, something is VERY BADLY WRONG. > Are you still using Akonadi's MySQL instance? Not as far as I can tell. I changed /etc/my.cnf to say: port = 3306 And you are certain that the mysqld listening on that port is NOT Akonadi's? If Akonadi's mysqld is listening on port 3306, and you install a second mysqld, it will be unable to bind port 3306. You cannot have two different processes listening on the same port. Try this command as root: lsof -i | grep 3306 and see which mysql process is listening on that port. Then look at the process tree (ps axf) and see what that process belongs to. Remember that if you install a second mysqld, it will need to have its own separate mysql configuration file(s), and listen on its own separate socket and its own separate port, and you will have to configure bacula to use that socket and/or port. Personally I'm not a fan of PostgreSQL. Speaking as a computer scientist there are things about its internals that make me want to tear my hair out. But in this case you may actually have an easier time dumping your catalog schema from MySQL and importing it into PostgreSQL than in keeping two distinct MySQL instances properly separate when (and I do not intend any offense by this) it is clear that you don't really understand how to manage MySQL. The other option might be to set up a Bacula catalog DB on a different host that is not running Akonadi, migrate your catalog DB to that instance, and then configure the Director to talk to that host. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] make_catalog_backup stopped working
On 12/27/21 12:10, Josip Deanovic wrote: On 2021-12-27 14:24, Graham Dicker wrote: I can't get those commands to work MariaDB [mysql]> CREATE USER 'bacula'@'localhost' IDENTIFIED BY 'xxx'; ERROR 1396 (HY000): Operation CREATE USER failed for 'bacula'@'localhost' MariaDB [mysql]> UPDATE mysql.user SET Host='localhost' WHERE User='bacula'; ERROR 1356 (HY000): View 'mysql.user' references invalid table(s) or column(s) The blazes? If mysql.user is a VIEW, something is VERY BADLY WRONG. Are you still using Akonadi's MySQL instance? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] make_catalog_backup stopped working
On 12/26/21 12:59, Josip Deanovic wrote: When using MySQL roles and grant statement to change privileges, the command "flush privileges" is not necessary but it doesn't hurt. When modifying mysql/user and similar tables then it is necessary to issue "flush privileges" in order to re-read and apply changes made to the tables. Never hurts to be sure. You will never cause a problem by calling FLUSH PRIVILEGES when you didn't need to. Unfortunately, many years ago KDE PIM application suite moved to akonadi which uses and manages its own mysql instance. That made it awfully slow and unstable, hard to debug and understand and even harder to trace the problem and fix but the wheel of "progress" doesn't care much about functionality. I totally hear you. Prime case in point: systemd. Also, you mentioned "embedded instance" but I am not sure if this can be called embedded instance because the there is an actual mysql daemon running. It just uses dedicated config file and datadir. Embedded as in "the application sets up its own dedicated mysql instance that it assumes nothing else will use". -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] make_catalog_backup stopped working
On 12/26/21 07:47, Graham Dicker wrote: I see that my attempts to paste messages copied from a console window into an email are not working as I expect. Trying again...The message above should have read: 26-Dec 09:39 bacula-dir JobId 0: Fatal error: Could not open Catalog "MyCatalog", database "bacula". 26-Dec 09:39 bacula-dir JobId 0: Fatal error: mysql.c:293 Unable to connect to MySQL server. Database=bacula User=bacula MySQL connect failed either server not running or your authorization is incorrect. 26-Dec 09:39 bacula-dir ERROR TERMINATION Please correct configuration file: /opt/bacula/etc/bacula-dir.conf From all of the above it appears that your MySQL credentials are incorrectly configured. You say you can log into the mysql instance as root. Do this: mysql> select user, host from mysql.user; See if there is a bacula user or another such user that you created for your catalog database. You should see something like: +-+-+ | User| Host| +-+-+ | alaric | 10.24.32.% | | bacula | 10.24.32.% | | books | 10.24.32.% | ... If so: mysql> GRANT ALL PRIVILEGES ON bacula.* TO 'bacula'@'localhost' (or whatever DB user you want bacula to connect as) IDENTIFIED BY 'bacula password goes here'; mysql> FLUSH PRIVILEGES; If there is not an existing user for bacula, run the same command, but make up a new username. (For sanity's sake I suggest using 'bacula', then you'll never be in any doubt what it is.) Then go back into your Bacula director config, set the mysql credentials to what you just set above, restart the Director, and try again. KMail/akonadi is a separate issue. I'm not a fan of services that set up their own embedded mysql instances. Multiple mysql (or most other database) instances running on the same host will ALWAYS, *ALWAYS*, perform worse, for all services, than a single properly tuned instance utilizing the same resources. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy job restores, again (solved)
On 12/25/21 12:54, Phil Stracchino wrote: Aha! There WAS a permissions problem! *GROUP* root had only RO access. I can now restore. And at this moment I have Bacula compiling under Solaris 11.4. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy job restores, again (solved)
Aha! There WAS a permissions problem! *GROUP* root had only RO access. I can now restore. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Copy job restores, again
For the first time I'm trying to run a restore directly from a copy job, after a complete failure of my NAS. I've got a second storage daemon temporarily installed on my workstation, I have the external disk chassis that holds my rotating archive copy sets attached to a temporary server running a clean Solaris 11.4 install, and I have the most recent full backup set copy set mounted. And I'm trying to do a test restore of 30 or so small files. Status dir says: JobId Type Level Files Bytes Name Status == 34562 Rest Rest 0 0 Restore is waiting for a mount request Status storage says: Reading: Full Restore job Restore JobId=34562 Volume="ARCHIVE-20211206-14:00" pool="Scratch" device="ArchiveCopy" (/arcpool) newbsr=0 Files=0 Bytes=0 AveBytes/sec=0 LastBytes/sec=0 FDReadSeqNo=7 in_msg=7 out_msg=7 fd=6 Director connected at: 24-Dec-21 23:59 Jobs waiting to reserve a drive: Terminated Jobs: JobId LevelFiles Bytes Status FinishedName === 34558 Rest 0 0 Cancel 24-Dec-21 23:17 Restore 34559 Rest 0 0 Cancel 24-Dec-21 23:45 Restore Device status: Device File: "ArchiveCopy" (/arcpool) is not open. Device is BLOCKED waiting for mount of volume "ARCHIVE-20211206-14:00", Pool:Scratch Media type: File Available Space=711.4 GB And if I try to mount: *mount Automatically selected Catalog: Catalog Using Catalog "Catalog" Automatically selected Storage: babylon5-archive 3906 File device ""ArchiveCopy" (/arcpool)" is always mounted. There is one complication. The machine running the storage daemon does not have enough local storage to copy the ARCHIVE-20211206-14:00 volume over. And I've had no luck so far trying to get a working development environment onto the server that has the external chassis attached. I can't attach it to the machine running the storage daemon because OpenZFS and Solaris ZFS zpools are incompatible. Thank you OH SO VERY MUCH ORACLE. So, the zfs filesystem containing the archive file is remotely mounted via NFS. Is that what's causing the problem here? Does anyone know a solution? I don't suppose anyone has a set of Bacula 11.0.5 packages for Solaris 11.4 amd64 compiled...? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] make_catalog_backup stopped working
On 12/23/21 11:19, Graham Dicker wrote: Thank you for your reply Graham. I don't think Bacula itself has any problems, it is only the script mysqldump that has a problem. The script is just supposed to create a dump of the database and (if it is successful) then Bacula goes on to first back it up and then delete it. I'm gonna be very blunt here. mysqldump is a piece of obsolete garbage and nobody should be using it any more. It CANNOT, period, CORRECTLY back up both My ISAM and InnoDB tables in the same dump. CANNOT. (Nobody should be using the MyISAM storage engine any more either, but that's another issue.) Take a look at mydumper instead. It is better than mysqldump in almost every imaginable way. It is parallelized, storage engine aware, and immensely easier to do partial restores with (i.e a single schema or a handful of tables). It is what mysqldump SHOULD have evolved into fifteen years ago. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] "Airgapping" USB drives used for backup?
On 12/22/21 09:03, Neil Balchin wrote: I’ve been down that road, yes bacula can certainly be configured to handle that at least on a ’nix system. Before your get too far though, you should do your own research on the reliability of spinning disk drives (internal or external). They pale in comparison to Tape. I would include s.m.a.r.t. disk checks as a regular part of my schedule and don’t expect to do this in an archiving scenario, hard drives sitting on a shelf unpowered for in excess of a year have frightening failure rates. LTO tapes are meant to let for 30 yrs Yeah, but the LTO drives are dead in eighteen months unless you have a filtered environment for them. (That is my personal experience at least.) I abandoned tape backup because I got tired of replacing the drives. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Verify jobs and "Warning: sql_get.c:186 More than one Filename!"
On 12/21/21 04:25, Mark Dixon wrote: On Fri, 17 Dec 2021, Phil Stracchino wrote: ... I've been running my Bacula jobs in parallel for literally decades now and have never seen it result in a duplicate Path or Filename warning. And they ARE only warnings. I would not worry about it as long as it is only a warning. You could manually clean up the duplicates, but it is probably not worth your while. When did you last run a dbcheck? The dbcheck tool should clean up most orphaned and duplicate records. Hi Phil, Great, I'd rummaged around the create/update table scripts in etc, but hadn't spotted dbcheck - thanks for that. Alternatively, I might just comment-out the warning in the source. You really should be running dbcheck on a regular basis. Weekly is a good idea. It's also a good idea to stop the Director while dbcheck runs. That is easy to automate as long as it is not constantly busy. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Verify jobs and "Warning: sql_get.c:186 More than one Filename!"
On 12/17/21 05:16, Mark Dixon wrote: Hi all, I find the verify job feature of bacula really useful to see what has changed since the last verifyinit job, but the job output is becoming difficult to read as it's clustered with lots and lots of messages of the form: 17-Dec 18:28 foo-dir JobId 67663: Warning: sql_get.c:186 More than one Filename!: 2 for file: AnySimpleTypeDatatypeValidator.hpp And: 17-Dec 18:28 foo-dir JobId 67663: Warning: sql_get.c:236 More than one Path!: 2 for path: /usr/include/xercesc/validators/common/ Looking at the database, the messages are absolutely right: select * from Filename where Name = "AnySimpleTypeDatatypeValidator.hpp"; +++ | FilenameId | Name | +++ |6421754 | AnySimpleTypeDatatypeValidator.hpp | |6421755 | AnySimpleTypeDatatypeValidator.hpp | +++ 2 rows in set (0.00 sec) select * from Path where Path = "/usr/include/xercesc/validators/common/"; +-+-+ | PathId | Path| +-+-+ | 2518839 | /usr/include/xercesc/validators/common/ | | 2518840 | /usr/include/xercesc/validators/common/ | +-+-+ 2 rows in set (0.00 sec) I've been using the verify job feature for years (thanks!), but have only come across this relatively recently - presumably it's a race condition since switching to running jobs in parallel. Firstly: 1) Should I be worried about these messages? 2) If not, is there a good way to get rid of them?! I'm using bacula 9.6.5 + mariadb on centos7. In current Bacula versions: * The Filename table is gone; filenames are now in the File table * Duplicate Filenames are fine, you can have multiple Files with the same name and different paths * The Path table now has a unique index over Path I've been running my Bacula jobs in parallel for literally decades now and have never seen it result in a duplicate Path or Filename warning. And they ARE only warnings. I would not worry about it as long as it is only a warning. You could manually clean up the duplicates, but it is probably not worth your while. When did you last run a dbcheck? The dbcheck tool should clean up most orphaned and duplicate records. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New Bacula Server with multiple disks
On 12/16/21 12:41, dmitri maziuk wrote: What kind of drives? In what kind of chassis? If I were setting this up from scratch, I'd go with a ZFS raidz-1 on 3 drives and a (large-ish) SSD in the 4th slot for ZFS write cache and job spool. Unless there's 5th disk: system, in which case I'd use an SSD there as well and put ZFS write cache on that instead. That is ALMOST exactly what I am doing. I have four rotating sets of 1TB drives set up as a ZFS pool in an external hot-plug SATA chassis, as the destination pool for my archive copy jobs. Media change is as simple as this: # zpool export arcpool [physically swap drives] # zpool import arcpool # rm /arcpool/* All four sets of drives have the same ZFS pool name, so it all Just Works. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
On 11/30/21 14:54, Josip Deanovic wrote: On Tuesday 2021-11-30 12:07:37 Phil Stracchino wrote: The question then ceases to be "I have twenty possible volumes from which your restore could be performed, which five of the twenty volumes do you want to use?", and instead becomes, "I have four possible SOURCES to perform this restore from, please pick *one restore source*." The restore command of the bconsole tool already has the "pool" command line argument which could be used for volume selection. In my case with Bacula 9.6.7 it didn't work with copy jobs. It would be nice to make it work with newer Bacula versions (e.g. 11.x). Eric mentioned that MediaType might potentially be used in the selection algorithm. I am not sure if this is necessary if we get a working "pool=" argument. Indeed, and Pool is probably a better discriminator in this case than MediaType. In my case, the two relevant pools for Full backups are Full-Disk and Full-Archive, and both are Media Type = File. In any random hypothetical environment with copy jobs, I would expect it to be far more likely that Copy jobs will be in a different Pool than that they will have a different media type. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Restoring from a copy job
On 11/30/21 11:01, Bill Arlofski via Bacula-users wrote: On 11/30/21 01:38, Eric Bollengier wrote: Note, that it's also possible to have multiple copies of each job, on different kind of devices, and some jobs involved may not have copies. Now, if we can determine a selection algorithm (via MediaType?), we can probably implement it. Bconsole is not the best tool for advanced selection screens, however Baculum GUI has an excellent supports the copy job selection and can also display the different versions of a file, and let you restore the one you want for example. Yes, this is true, and something people (like me for example) often forget - A backup job may be copied many times, and to every different storage type - So making a guess which copy the user wants to restore from would be impossible. :) So, the workflow as I described in my follow-up replay last night currently is the right one and it makes more sense considering the multiple copies scenario. However, in a case where there are two or more complete sets of jobs on different pools or devices, both containing all of the jobs necessary to perform the desired restore, it SEEMS that it should be conceptually simple to list the jobs and sources and allow the user/administrator to select a restore SOURCE, then automatically select the volumes and jobs required to perform the restore from that source. This seems much simpler than trying to choose from all possible sets of jobs/volumes. The question then ceases to be "I have twenty possible volumes from which your restore could be performed, which five of the twenty volumes do you want to use?", and instead becomes, "I have four possible SOURCES to perform this restore from, please pick *one restore source*." -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup Slow NFS (HPE StoreOnce 3620 VTL and NFS)
On 11/26/21 14:53, Jose Alberto wrote: RMAN exports are hosted on the NFS server. And Bacula mount NFS peer-to-peer to tape backup. If at all possible, it would be preferable to back up the NFS-shared volumes using a file daemon running directly on the NFS host. However, if you can't do that, then the best you can do is tune your NFS mounts for throughput as you have. Increasing both rsize and wsize well beyond the default is standard practice to improve NFS performance. Also consider mounting with -o noatime, nodiratime (if supported). File and directory atime updates don't really get you anything useful, and disabling them reduces overhead. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup Slow NFS (HPE StoreOnce 3620 VTL and NFS)
On 11/26/21 08:31, Jose Alberto wrote: Hi. Adjust mount.nfs -o rsize=32768 10.29.5.3:/nas/ /NFS/ add -o rsize=32768 Now Backup rate 150MB/s Is this recommended, what am I doing? rsize is a value for reading This would be a recommended and normal NFS performance tuning step, yes. Though backing up over NFS is still not recommended if you have a choice. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
