Re: [Bacula-users] Performance testing simultaneous spooling and despooling
On 9/16/24 18:58, B. Smith wrote: Good evening, I have a ZFS pool as a dedicated Bacula spool. The pool contains six 4TB drives, configured as three mirrors of two striped disks. OK, just one observation: That's generally considered the wrong way to do it. The normally preferred arrangement is to first build mirrored pairs, then stripe the mirrors. -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] versions table has two rows
On 9/16/24 20:34, Dan Langille wrote: On Mon, Sep 16, 2024, at 5:46 PM, Phil Stracchino wrote: On 9/16/24 15:17, Dan Langille wrote: delete from version order by version limit 1; bacula=*# delete from version order by version limit 1; ERROR: syntax error at or near "order" LINE 1: delete from version order by version limit 1; Doh, my bad. Forgot you can't use ORDER BY in a DELETE FROM. (Which means you have to use shenanigans when you want to do that.) -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] versions table has two rows
On 9/16/24 15:17, Dan Langille wrote: Hello, I discovered this situation today when reviewing upgrade instructions for Bacula 9 -> Bacula 15. FYI, Bacula 9 and 13 are both deprecated in FreeBSD and will be removed from the ports tree at the end of the month. The problem: duplicate entries: bacula=# select * from version ; versionid --- 1026 1026 (2 rows) bacula=# I reckon I can just delete one of them. How to accomplish that is left as an exercise for the reader. delete from version order by version limit 1; -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] full and diff backups in 2 different jobs
On 9/3/24 06:24, Mehrdad R. wrote: Hi Mostly to see if it is possible, i am fully aware that it can be done in one job, was just wondering if there was a way to have them in separate jobs, maybe save them on separate disks eventualy The jobs that i have set up fail in that respect as i mentioned and the diff job does not see the full job Yes, this is what would be expected if you try to do this because they are different jobs. I understand what you're trying to do, what I don't understand is why you're trying to do it. This is not the way the tool works, and it would greatly complicate matters. Let's take a step back. What are you trying to ACCOMPLISH by trying to make the Full and Differential separate Jobs, instead of different levels of the same Job? What are you hoping to gain? -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Director migration question
On 9/2/24 02:42, Gary R. Schmidt wrote: On 02/09/2024 05:02, Phil Stracchino wrote: On 9/1/24 14:48, Phil Stracchino wrote: Actually I already see that one issue is I don't want both directors trying to run the same jobs. In this situation, if I disable a job in one Director, does that GLOBALLY disable the job, or only in that Director? If so, I could — for now — disable all jobs in the "read-only" Director ... ...And answering my own question again, yes, this looks fine Apart from the problem when one director tries to do something - like a restore - while the other is running a backup. Data corruption is the name of the game... Since nobody else but me will be starting or stopping jobs, I don't see that being an issue. It looks like nothing went amiss up to the point of verifying that everything is correctly configured. It looks as though there may be some issues around two Directors talking to the same SD at the same time, even if one of them is only "watching". -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Director migration question
On 9/2/24 10:57, Jose Alberto wrote: n my case I have a Director and an SD in the same location. and 3 additional SDs in remote locations are your tape. What I worry about is supporting the Catalog in all locations, so that when the Director fails for X reason, I can build it in the most convenient location. My Catalog is on a three-node redundant cluster. It does mean I have to turn off attribute spooling, but the truth is the way attribute spooling is implemented (at least in Bacula's mysql driver) is very badly misguided in the first place. -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Director migration question
On 9/1/24 14:48, Phil Stracchino wrote: Actually I already see that one issue is I don't want both directors trying to run the same jobs. In this situation, if I disable a job in one Director, does that GLOBALLY disable the job, or only in that Director? If so, I could — for now — disable all jobs in the "read-only" Director ... ...And answering my own question again, yes, this looks fine -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Director migration question
On 9/1/24 13:57, Phil Stracchino wrote: It's about to become prudent to move my Director to a different host. Is there any TECHNICAL reason why I cannot or should not have two Directors on the network, both using the same catalog and talking to the same clients, with the new one in a "read-only" role running no jobs until I'm ready to cut over? Actually I already see that one issue is I don't want both directors trying to run the same jobs. In this situation, if I disable a job in one Director, does that GLOBALLY disable the job, or only in that Director? If so, I could — for now — disable all jobs in the "read-only" Director ... -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Director migration question
It's about to become prudent to move my Director to a different host. Is there any TECHNICAL reason why I cannot or should not have two Directors on the network, both using the same catalog and talking to the same clients, with the new one in a "read-only" role running no jobs until I'm ready to cut over? -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] postgres load at restore
On 8/31/24 01:56, Stefan G. Weichinger wrote:
As mentioned I added that Runscript and it seems to work.
Now I got 2 emails per job this night.
For each job like:
Bacula: Backup OK of samba-fd Full
there is also one with
Bacula: Console Unknown term code of samba-fd
that's "new" ;-)
Yes, I'm getting the same odd message.
For reference one of my 2 JobDefs:
JobDefs {
Name = "DefaultJob"
...
Runscript {
Runs When = After
Runs On Client = no
Console = ".bvfs_update jobid=%i"
}
That part looks just like mine, but I'm doing it in a separate admin job.
--
Phil Stracchino
Fenian House Publishing
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] postgres load at restore
On 8/29/24 20:28, Marcin Haba wrote: On Fri, 30 Aug 2024 at 01:22, Phil Stracchino <mailto:ph...@caerllewys.net>> wrote: > Yes, I see your idea. This full rebuild will cost creating all cache > from scratch. For many jobs and paths it will take lots of resources. Well, I just tested my revised dbcheck script with inline bvfs cache clear and rebuild, and it worked fine, runs for about two hours. My bvfs update job to run after all backups needs a little work though. The job definition looks like this: Hello Phil, Great. Two hours isn't that bad. But when run manually from BAT, BAT won't let me specify a level, runs .bvfs_update with an empty level, then errors out because "" is not a valid level. It looks like a bug. It LOOKS as though when run by the Director from the schedule, it completed successfully according to the Jobs Run log. It generates TWO notification mail messages, though, the first of which has subject "-RunScript- ( ) Unknown term code" and contains only the following: 30-Aug 05:30 minbar-dir JobId 0: Using Catalog "Catalog" The second looks like this: Subject: BVFS_Update (Full) OK Date: Fri, 30 Aug 2024 05:30:29 -0400 X-Original-To: r...@babylon5.caerllewys.net 30-Aug 05:30 minbar-dir JobId 1359: console command: run BeforeJob ".bvfs_update" 30-Aug 05:30 minbar-dir JobId 1359: Start Admin JobId 1359, Job=BVFS_Update.2024-08-30_05.30.00_11 30-Aug 05:30 minbar-dir JobId 1359: Bacula 15.0.2 (21Mar24): 30-Aug-2024 05:30:29 JobId: 1359 Job:BVFS_Update.2024-08-30_05.30.00_11 Scheduled time: 30-Aug-2024 05:30:00 Start time: 30-Aug-2024 05:30:29 End time: 30-Aug-2024 05:30:29 Termination:Admin OK 30-Aug 05:30 minbar-dir JobId 1359: Begin pruning Jobs older than 12 months . 30-Aug 05:30 minbar-dir JobId 1359: No Jobs found to prune. 30-Aug 05:30 minbar-dir JobId 1359: Begin pruning Files. 30-Aug 05:30 minbar-dir JobId 1359: No Files found to prune. 30-Aug 05:30 minbar-dir JobId 1359: End auto prune. Did Bacula effectively "hallucinate" a nonexistent Job with id 0 to run the script "before"? Or is there still a bug triggering when running bvfs_update as a RunScript? This does not happen with my other similarly-structured admin jobs, such as the one that expires volumes past th. I tried running a restore of my workstation this morning, and it still took more than four minutes to build the directory tree. Does this seem as though the bvfs cache is being built correctly and used? Is there a way I can test and verify that it is working correctly? -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] postgres load at restore
On 8/29/24 14:32, Marcin Haba wrote:
On Thu, 29 Aug 2024 at 20:14, Phil Stracchino <mailto:ph...@caerllewys.net>> wrote:
NIGHTLY
Run all scheduled client backup jobs
Incremental rebuild bvfs cache
Run catalog backup
WEEKLY
Clear bvfs cache
Run dbcheck
Full rebuild bvfs cache
Yes, I see your idea. This full rebuild will cost creating all cache
from scratch. For many jobs and paths it will take lots of resources.
Well, I just tested my revised dbcheck script with inline bvfs cache
clear and rebuild, and it worked fine, runs for about two hours. My
bvfs update job to run after all backups needs a little work though.
The job definition looks like this:
Job {
Name = "BVFS Update"
Type = Admin
Enabled = Yes
Pool = Scratch
Storage = epsilon3-file
FileSet = Dummy
Client = epsilon3
Level = Full
Runscript {
Runs When = Before
Runs On Client = no
Console = ".bvfs_update"
}
Messages = Daemon
Priority = 18
Rerun Failed Levels = no
Allow Duplicate Jobs = no
Cancel Queued Duplicates = yes
Schedule = "BVFS Update"
}
Its schedule looks like this:
# BVFS maintenance schedule
Schedule {
Name = "BVFS Update"
Run = Full mon-sun at 05:30
}
But when run manually from BAT, BAT won't let me specify a level, runs
.bvfs_update with an empty level, then errors out because "" is not a
valid level.
--
Phil Stracchino
Fenian House Publishing
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] postgres load at restore
On 8/29/24 13:39, Marcin Haba wrote: On Thu, 29 Aug 2024 at 18:42, Phil Stracchino <mailto:ph...@caerllewys.net>> wrote: The bvfs cache DOES greatly improve restore performance, but the one showstopper drawback I found (beyond the runscript bug) was that if you pre-populate the bvfs cache, dbcheck will refuse to clear orphaned Path entries unless you clear the bvfs cache first. Marcin, do you have any insights on this issue? Hello Phil, I think that it's due to the nature of the Bvfs cache. In the Bvfs 'pathhierarchy' table is stored the directory tree structure in form pathid -> parent pathid (ppathid). If some paths are orphaned, it means that theoretically they could to be removed from Bvfs cache as well. But it is hard to remove any path from the 'pathhierarchy' without rebuilding the cache because otherwise the cache will be incomplete. For example, we have the following path hierarchy relation written in the 'pathhierarchy' table (I use pathids): 1 => 2 => 3 => 4 => 5 If the path with pathid 3 is orphaned and removed, then in the Bvfs cache we have the path relation: 1 => 2 => ??? => 4 => 5 In this meaning the dbcheck behavior and block removing orphaned paths for me is understandable to avoid rebuilding everything. Oh yeah, the consistency problem is fully understandable, it just seems an oversight that dbcheck seems unable to fully handle the presence of a populated bvfs cache. It seems like a part of dbcheck should be checking and updating the bvfs cache if present. BTW: When the cache is manually cleared by '.bvfs_clear_cache yes' it will be completely (re)built anyway. Gotcha. If I do a .bvfs_update WITHOUT specifying a jobid, will it do a full rebuild? I'm thinking what I'd LIKE to do is the following: NIGHTLY Run all scheduled client backup jobs Incremental rebuild bvfs cache Run catalog backup WEEKLY Clear bvfs cache Run dbcheck Full rebuild bvfs cache -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] postgres load at restore
On 8/29/24 14:32, Marcin Haba wrote: On Thu, 29 Aug 2024 at 20:14, Phil Stracchino <mailto:ph...@caerllewys.net>> wrote: If I do a .bvfs_update WITHOUT specifying a jobid, will it do a full rebuild? I'm thinking what I'd LIKE to do is the following: The .bvfs_update without specifying a jobid does update for all jobs :-) I mean all jobs for which the cache has not been created yet. It is still the incremental process. OK, great. NIGHTLY Run all scheduled client backup jobs Incremental rebuild bvfs cache Run catalog backup WEEKLY Clear bvfs cache Run dbcheck Full rebuild bvfs cache Yes, I see your idea. This full rebuild will cost creating all cache from scratch. For many jobs and paths it will take lots of resources. Yeah, but I can probably live with it as a once-a-week maintenance. I'm already shutting down my Director while dbcheck runs anyway, because I've found that to be more reliable. -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] postgres load at restore
On 8/29/24 12:25, Marcin Haba wrote:
1) This is an incremental process. It means that the cache is not
re-created for jobs and paths that already exist there. Added are only
new ones. Because of that for newly configured backup jobs this cache
can take longer than for others.
2) This process can be triggered asynchronously to the restore process.
There is possible to update this cache not in a moment when the files
from restore are urgently needed, but in a moment when the backup job
finishes work. This way when you go to the restore wizard, the cache is
ready. It can be done in the Job Runscript directives like these:
Runscript {
Runs When = After
Runs On Client = no
Console = ".bvfs_update jobid=%i"
}
In older Director versions there was a bug in running the .bvfs_update
command in the Runscript Console directive:
https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2419
<https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2419>
but I think in newer Bacula versions it should be fixed already. For
older Director versions it can be done by using script and putting the
.bvfs_update command inside it, like this:
Runscript {
Runs When = After
Runs On Client = no
Command = "/path/to/script/script.sh %i"
}
The bvfs cache DOES greatly improve restore performance, but the one
showstopper drawback I found (beyond the runscript bug) was that if you
pre-populate the bvfs cache, dbcheck will refuse to clear orphaned Path
entries unless you clear the bvfs cache first.
Marcin, do you have any insights on this issue?
--
Phil Stracchino
Fenian House Publishing
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula rescue cdrom source code
On 8/1/24 13:35, Bill Arlofski via Bacula-users wrote: On 8/1/24 7:31 AM, William Rice wrote: > Hello I'm trying to locate the bacula-rescue cdrom source code so I can make a Bare Metal Recovery cdrom for bacula-9.0.8 Any help would be greatly appreciated! For what it's worth: What I do for bare metal recovery is I restore the most recent backup for the client onto my NAS, boot the client from a system restore disk, prep the system, mount the NAS, then rsync the restored image over. That basically Just Works, and if something goes wrong part-way I don't have to restart the restore, it's still there on the NAS. -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Does anybody have a working Linux ISO that will allow me to restore from an offsite Bacula backup?
On 6/25/24 09:39, MylesDearBusiness via Bacula-users wrote: After trying and failing to use more low-level manual means, I`m now pivoting and trying to adapt an existing Live Ubuntu ISO and install appropriate Bacula packages and storage backend linkages using CUBIC. I plan to test the ISO on local VirtualBox and ultimately bring it up in a virtual CDROM on my cloud server`s ASMB9-iKV. I then want the ISO to come up with all Bacula services primed and ready for a full system restore. I`m not sure what the history was that led up to the removal of the Live ISO from the community Bacula builds, but it`s taking a LOT of my time to figure out how to do restores on an unstable system. Is there a reason a basic universal Bacula recovery ISO isn't being built with each community release ? For what it's worth, this is my bare-metal restore procedure in my systems: — Use another system to do a full restore of the target to an image on NAS — Boot the target using SystemRescue — Perform necessary system preparation — Mount the NAS — Copy the restored image from NAS to the target This has always worked for me, and if I get something wrong the first time around, the restore image is still there, I can just fix the problem and re-copy without having to do another restore. -- Phil Stracchino Fenian House Publishing ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Scheduling example
On 3/6/24 01:58, Stefan G. Weichinger wrote: As I try to come up with an initial setup for weekly-daily-monthly I found this: https://www.bacula.org/13.0.x-manuals/en/main/Automatic_Volume_Recycling.html#SECTION003060 Shouldn't there also be Jobs for Weekly and Monthly? No, they aren't needed. Read carefully. The job "NightlySave" uses the schedule "NightlySave", which runs the NightlySave job monthly, weekly and daily at Full, Differential and Incremental levels, each to its own Pool. Which Pool to use for the Catalog Backups? This example doesn't include a catalog backup. But I run my catalog backup to whichever Pool the other jobs for that night use. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-fd hangs during backup
On 2/12/24 18:35, Peter Sjoberg wrote: Hi all I have a strange problem and (on my system) reproducible problem. When I do backup of some directories then bacula-fd just hangs and never complete. The directories in question are not very strange and backup of them works find with older versions of -fd Silly question: Do the problem directories contain named pipes or sockets? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] BAT Installation
On 2/2/24 10:25, Bill Arlofski via Bacula-users wrote: All good information. And I would strongly urge to move away from BAT in favor of Bacularis (for example). Bacularis, unlike BAT allows you to configure every aspect of your Bacula environment, and as Rob mentioned, Marcin Haba, a Bacula Systems employee maintains it as an open-source and free tool, and he is quite active and helpful on this list. :) Bottom line: Abandon BAT. It has not received any love in a very long time, and quite frankly I am sure it will not. It did what it was designed to do, but now that we have some very powerful web gui administration tools, there is no reason not to abandon it.* (Just my two cents) *Having said that, years ago, I had customers that LOVE LOVE LOVED BAT, and would not move away from it even at my urging, so there is one reason, I guess. :) BAT still does everything I want a Bacula console to do. The things that it does NOT do, like configuration, I don't WANT it to; I want to do those myself, by hand, using a proper editor. To be truthful, I detest the "Everything is a web page/application" model. I have not, to be fair, tried either Baculum or Bacularis, but my general experience is that web-based applications tend to be uniformly awful. Especially web-based admin interfaces. For example, I briefly tried to deploy a QNAP NAS that used a web admin interface, and there were basic security functions that you simply *COULD NOT DO* via the admin GUI. It was not merely hidden, it was *impossible*. Webmin, cpanel, ... uniformly ghastly. Your mileage may of course vary. I will note that there are a few known bugs in BAT, notably that some purge operations can produce *multiple* simultaneous pop-up confirmation alerts that can be confusing. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] BAT Installation
On 2/2/24 10:53, Heitor Faria wrote: Frankly, I never understood this community's urge to regularly create "one more Bacula GUI", instead of just fixing and improve the state-of-art. IMHO bacula.org <http://bacula.org> should adopt and sponsor BAT and Bacularis/Baculum GUIs development, get people together, integrate bacula-web features and organize this mess. Rgds. I frankly agree with Heitor here. BAT Just Works, isn't subject to browser compatibility issues, and doesn't suffer any of the broken interface behaviors that come as part and parcel of web applications. It does have a few minor bugs that should be fixed, but even those it has do not stop it from working. This isn't to say that a web interface for Bacula *monitoring* is necessarily a bad idea. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Ubuntu 22.04: bacula-fd from official repo cannot talk to src-compiled server on 20.04
On 11/9/23 04:09, Uwe Schuerkamp wrote: director (compiled from source): libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x7f3d8ac51000) libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x7f087cbb5000) client (from bacula repos): libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x7f4d8499b000) One presumes you are aware of how old your Director's OpenSSL is? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula copy from tape to tape
On 11/7/23 17:09, Rob Gerber wrote: Technically I do have a huge NAS that dwarfs the production NAS in size (TrueNAS box). It was intended to store one movie, but the movie IT people massively over-estimated the amount of space we needed, so it never got used. It has occurred to me to use this TrueNAS device as a disk based volume storage solution. Can one make multiple copies of a reference volume? Ie, could we have disk based backups managed by bacula on this larger NAS, with copies made to tape for onsite AND offsite storage? You can absolutely to disk-to-disk-to-tape. It's not conceptually difficult to set up. (I'm doing disk to disk to removable ZFS disk pool, but ... same principle.) Maybe at that point we wouldn't need an onsite tape backup set. Disk based backup could speed up our initial backups substantially since both of these systems can operate at 10gbe speeds. Yeah, I love my disk-to-disk setup. Small restores are almost instant. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: Your advice for my re-design of backup layout regarding pools and volumes (backups of and in homelab)
On 10/24/23 15:37, Justin Case wrote:
can you paste how you select the fulls for copy jobs?
This is my methodology:
Job {
Name = "Archive Copy"
Type = Copy
Enabled = Yes
Pool = Full-Disk
Level = Full
Client = ALL
Fileset = DUMMY
Storage = epsilon3-archive
Selection Type = SQL Query
Selection Pattern = "SELECT DISTINCT J.JobId
FROM Job J
JOIN Pool P ON P.PoolId = J.PoolId
LEFT JOIN Job J2 ON J2.PriorJobId = J.JobId
WHERE P.Name = 'Full-Disk'
AND J2.JobId IS NULL
AND J.Type = 'B'
AND J.JobStatus IN ('T','W')
AND J.JobBytes > 0
AND J.StartTime > now() - interval 7 day"
Messages = Daemon
Priority = 20
Rerun Failed Levels = yes
Allow Duplicate Jobs = no
Cancel Queued Duplicates = yes
Schedule = "Full Backup Archive"
}
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: Your advice for my re-design of backup layout regarding pools and volumes (backups of and in homelab)
On 10/24/23 14:42, Justin Case wrote: I realized that my fulls and incrementals have the same retention time, so I made 1 pool for both. I guess you are using X > Y months for fulls and Y > Z months fordifferentials and Z months for incrementals? That's the basic idea, yes. I do a nightly incremental kept for 3 weeks, weekly differentials kept for 3 months, and monthly Fulls which if memory serves I keep for one year. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: Your advice for my re-design of backup layout regarding pools and volumes (backups of and in homelab)
On 10/24/23 10:59, Justin Case wrote: Disclaimer: My situation is a little different from yours, I'm doing disk-to-disk-to-removable-disk. For tier 1: does it make sense to have different volumes for full jobs and incr jobs? Does it make sense to have these volumes in seperate pools, or full and incr together in a pool? Would you recommend having different pools for different groups of jobs? I keep Full, Differential and Incremental jobs in separate Pools. That way I can purge job batches as they become obsolete on their own schedules and recover the space. For tier 2: here I wil use copy jobs. Since copy jobs really do not differentiate full and incr backups, there will be full and incr copies together in volumes. Would you recommend having different pools for different groups of copy jobs? I use copy jobs, but I copy only FULL backups to removable disk. (The removable disk zpools are then exported, unmounted, and stored offline/airgapped.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Extraction of a specific file
On 10/2/23 13:39, Rob Gerber wrote: I'm not sure, but maybe you'll need to query your database directly instead of relying on Bacula to simplify that process. *This is not an educated answer, just my novice guess.* I see bacula bconsole has a 'query' command that I believe passes SQL query language commands on to the relevant DB server. Might want to look into that, possibly in the bacula console operators manual. I've used the PDF copy of the documentation because it's easier to search. I'd swear there used to be a 'blist' tool, but I can't for the life of me find it any more. This query will list files in a given Job: SELECT CONCAT(p.Path, f.Filename) FROM File f JOIN Path p ON f.PathId = p.PathId WHERE f.JobId = ; If you want only, say .odt files, you could use: SELECT CONCAT(p.Path, f.Filename) FROM File f JOIN Path p ON f.PathId = p.PathId WHERE f.JobId = AND f.Filename LIKE '%.odt'; For example: MySQL 127.0.0.1> SELECT CONCAT(p.Path, f.Filename) FROM File f JOIN Path p ON f.PathId = p.PathId WHERE f.JobId = 836 AND f.Filename LIKE '%.odt'; +--+ | CONCAT(p.Path, f.Filename) | +--+ | /home/alaric/Documents/Writing/Bearing Gifts.odt | | /home/alaric/Documents/Writing/United Fleet.odt | +--+ 2 rows in set (0.647 sec) You can get the LStat field as well, but you'd have to decode it yourself. I don't remember offhand how LStat is encoded. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and BBR Protocol for Internet Backups, Packet Losses, Disconnections, etc.
On 10/2/23 09:47, hei...@bacula.com.br wrote: > I've enabled this now on all of my hosts, but I'm honestly not seeing a > clear improvement in network throughput so far. The improvements are more notiable if you introduce packet losses or networ interruptions. Ah, so, all the things I've done my best to eliminate from my network :) Well maybe it'll show up in other usages. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula and BBR Protocol for Internet Backups, Packet Losses, Disconnections, etc.
On 9/14/23 18:08, hei...@bacula.com.br wrote: Hello Bacula Users, I would like to share this finding regarding the Linux BBR TCP congestion protocol and Bacula. I've tested in a few production environments, but I would like to have a broather population of results: https://www.bacula.lat/bacula-e-prototoclo-bbr-para-backups-via-internet-perdas-de-pacote-desconecoes-etc/?lang=en I've enabled this now on all of my hosts, but I'm honestly not seeing a clear improvement in network throughput so far. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Slow spooling and hashing speed
On 9/23/23 14:25, Rob Gerber wrote: You know, asking them to add a Bacula FD plugin is a good idea! I'm sure it wouldn't be available for quite a while, but that is genuinely a good idea. If they're building on Gentoo, they ought to be able to do it pretty quickly if you can persuade them to. the Gentoo ebuild is mature and, particularly for client-only, is about as close to plug-and-play as you can get. This is a commercial market NAS, aimed at videographers. The devs are actually pretty damn good, and I'm proud of them for choosing Gentoo. They have a bunch of software sitting on top of Gentoo aiming to provide fast access to stored media files for video editors. Good on them, then. The main data array is formatted XFS, by way of trivia for y'all. The core ethic is "get media go the end users as fast as possible so they can edit and render as fast as possible". Everything else is secondary. XFS is probably the ideal choice for that application. It was specifically designed to optimize streaming media performance. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Slow spooling and hashing speed
On 9/23/23 13:34, Rob Gerber wrote: Heitor, Thank you for the reply. The backup source is a studio network solutions Evo NAS appliance. Unfortunately, I have no access to run bacula FD, nor most of other things suggested on this appliance as access is only via appliance developer designer means. I do not have any shell access to this appliance. Off the top of my head, I believe my only access options to this Evo NAS are smb (I believe 3.x), read only iSCSI, or afs (apple file sharing - obviously not something I want to try). This Evo NAS appliance is running Gentoo Linux. Wait, it runs Gentoo and they locked it down that hard somehow? Gut feeling says you should be able to install almost anything you want onto it, but it may be some work depending on how they locked it down. (This would of course probably void your warranty.) That said, if you can't put a fd on it directly, given those options the next best thing would probably be to mount it as an iSCSI target. You might also consider asking them if they would consider a Bacula-fd package available. But my experience is that to NAS consumer-appliance manufacturers, backup is at best an afterthought. I went through this with a NAS appliance which QNAP proudly claimed ran Linux and ZFS, only to find out that through idiotic choices and Byzantine design decisions they had crippled it so badly it might as well have been running Windows. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] access key
On 9/13/23 18:35, Bob McNabb wrote: Where do I obtain the access key for installing the community edition? You go to the download page, follow the instructions, and you will be emailed a download link. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula purges old jobs and I don't want it to do that
On 8/18/23 17:24, Rob Gerber wrote: verification jobs. My guess is that bacula purged job records for some reason, possibly because they were older than 6 months. I have volume retention set to 1000 years, but maybe I need to add something for job retention? There are three retention periods you need to consider: — Volume retention — Job retention — File retention I suggest reading up on these in the documentation so that you understand what each of them does. You can then adjust your pools, and update all of your volumes, as necessary. Background info: I am backing up some large video files with LTO8 tape. My goal is to back these files up once, and do incremental backups to capture any new or changed files in the fileset. I do not expect the large media files to be changed once placed on the disk. Basically, I am using an "incrementals forever" strategy. You might want to consider virtual-full jobs as a part of that strategy. I am using bacula 13.x. We have a qualstar q24 autochanger. We use LTO 8 media. The fileset in question is around 70TiB. I welcome any assistance the list could provide. I'm sure it has to be something simple. You could fix your retention and then scan the pruned volumes back in using bscan. By the sound of it, that would take a while, but it's doable. What you need to keep in mind is that an "incrementals forever" strategy is also a "Bacula database grows forever" strategy. That will eventually become a problem. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with grant_mysql_privileges
On 8/3/23 04:05, Graham Dicker via Bacula-users wrote:
On Thursday, 3 August 2023 08:35:39 BST Graham Dicker via Bacula-users wrote:
On Wednesday, 2 August 2023 17:21:49 BST Phil Stracchino wrote:
On 8/2/23 11:12, Graham Dicker via Bacula-users wrote:
Hello
I am installing Bacula 13.0.2 on Opensuse 15.5 and get this problem when
I
run grant_mysql_privileges:
ERROR 1064 (42000) at line 3: You have an error in your SQL syntax;
check
the manual that corresponds to your MariaDB server version for the right
syntax to use near '%{db_user}@"%"' at line 1
Database version 10.6.14-MariaDB
I guess it's complaining about the statement
db_user=${db_user:-bacula}
Can anyone help with this please?
What does the line 'echo "Created MySQL database user: ${db_user}"'
report as the value of ${db_user}?
It reports just the single word bacula
Graham
It doesn't actually create the user bacula either although when I run the
script it says it has created it.
Martin is correct that the second GRANT line is incorrect, it should be
${db_user} not %{db_user}
Try doing this yourself manually. It's not complex. You know your
MariaDB superuser password I assume. Log into MariaDB and run the
following commands:
GRANT ALL PRIVILEGES ON bacula@'HOST' IDENTIFIED BY "password goes here";
GRANT PROCESS ON *.* TO bacula@'HOST';
FLUSH PRIVILEGES;
HOST here can be one of several things. The script uses '%', a wildcard
meaning 'any host'. If your DB runs on the same host as your Director
and will not be accessed from anywhere else, it would be more secure to
use 'localhost' there. Or if you have a subnet behind a firewall and
want to allow access from a range of hosts or all of your hosts, you can
use something like bacula@'1.2.3.%'.
Report any errors.
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with grant_mysql_privileges
On 8/2/23 13:43, Martin Simmons wrote: It looks like a bug in the script to me (maybe % instead of $): if $bindir/mysql $* -u root -f < You are absolutely right, Martin, I totally looked past that. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Problem with grant_mysql_privileges
On 8/2/23 11:12, Graham Dicker via Bacula-users wrote:
Hello
I am installing Bacula 13.0.2 on Opensuse 15.5 and get this problem when I run
grant_mysql_privileges:
ERROR 1064 (42000) at line 3: You have an error in your SQL syntax; check the
manual that corresponds to your MariaDB server version for the right syntax to
use near '%{db_user}@"%"' at line 1
Database version 10.6.14-MariaDB
I guess it's complaining about the statement
db_user=${db_user:-bacula}
Can anyone help with this please?
What does the line 'echo "Created MySQL database user: ${db_user}"'
report as the value of ${db_user}?
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/7/23 19:35, Phil Stracchino wrote: Then re-update it again, but BEFORE you rerun the updater, find update_mysql_tables_1021_to_1022 and change every instance of CREATE TABLE to CREATE TABLE IF NOT EXISTS. That will make sure that if something else is happening along the way that is creating those tables before they're expected, then the CREATE TABLE will be skipped, instead of the script failing. The *other* way you could do this is to insert a 'DROP TABLE IF EXISTS ;' before each 'CREATE TABLE (...' statement. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/7/23 19:26, Phil Stracchino wrote: On 7/7/23 18:59, Ken Mandelberg wrote: The suggestion below which is essentially to run "update_mysql_tables" manually instead of within the "apt-get install bacula-mysql" runs into the same issue. Specifically Update of Bacula MySQL tables from 16 to 1017 succeeded. Update of Bacula MySQL tables from 1017 to 1018 succeeded. Update of Bacula MySQL tables 1018 to 1019 succeeded. (From 16) Update of Bacula MySQL tables 1019 to 1020 succeeded. Update of Bacula MySQL tables 1020 to 1021 succeeded. ERROR 1050 (42S01) at line 3: Table 'TagJob' already exists Update of Bacula MySQL tables 1021 to 1022 failed. I also tried deleting TagJob and starting again, and then it just moves on to the next table that already exists. In fact I deleted all the Tag* tables, and that it complains about the Object table. I suspect it won't like any. Any ideas? OK, the short version is none of those TAG tables are SUPPOSED to exist before database version 1022. That PROBABLY means that you didn't fully clean from a prior update attempt. So what you can do is drop your bacula DB, *make sure it's empty*, restore it again from your backup copy, connect to mysql, do a SHOW TABLES FROM bacula; and check that the Tag... and Event tables are not in that list. Then re-update it again, but BEFORE you rerun the updater, find update_mysql_tables_1021_to_1022 and change every instance of CREATE TABLE to CREATE TABLE IF NOT EXISTS. That will make sure that if something else is happening along the way that is creating those tables before they're expected, then the CREATE TABLE will be skipped, instead of the script failing. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/7/23 18:59, Ken Mandelberg wrote: The suggestion below which is essentially to run "update_mysql_tables" manually instead of within the "apt-get install bacula-mysql" runs into the same issue. Specifically Update of Bacula MySQL tables from 16 to 1017 succeeded. Update of Bacula MySQL tables from 1017 to 1018 succeeded. Update of Bacula MySQL tables 1018 to 1019 succeeded. (From 16) Update of Bacula MySQL tables 1019 to 1020 succeeded. Update of Bacula MySQL tables 1020 to 1021 succeeded. ERROR 1050 (42S01) at line 3: Table 'TagJob' already exists Update of Bacula MySQL tables 1021 to 1022 failed. I also tried deleting TagJob and starting again, and then it just moves on to the next table that already exists. In fact I deleted all the Tag* tables, and that it complains about the Object table. I suspect it won't like any. Any ideas? Let me take a look at the current update_mysql_tables script and get back to you. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/4/23 19:48, Ken Mandelberg wrote: Another followup. One thing I learned, I need to remove all the Ubuntu bundled bacula 9 packages before doing the apt-get install bacula-mysql for the Community 13 release, otherwise there are package conflicts. So after removing the old packages, apt-get install bacula-mysql tires to install the mysql bacula database. It gets caught at mysql said: ERROR 1050 (42S01) at line 9: Table 'TagJob' already exists I imagine if I removed that table it would just get caught at the next one. Its not really clear on how to proceed preserving the current database 9 info and letting 13 do the update job. OK, try this. - DUMP the bacula schema using mysqldump or your database dump tool of choice. (Personally I prefer mydumper.) - Then DROP the bacula DB. - Then run apt-get install bacula-mysql. - You MIGHT need to drop the bacula schema *again* at this point. to be safe. - Then RESTORE the bacula schema from your dump. - Then run update_mysql_tables to update your v9 Bacula DB to v13. That should put you where you need to be, and in the worst case, you have a dump of your bacula DB in the event you have to roll back. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Community Installation UPGRADE
On 7/1/23 18:30, Ken Mandelberg wrote: So I know I need to copy /etc/bacula to a safe place. Should I also delete it? No, don't do that, you want to keep your configuration. If you delete those files you'll lose it. I will also stop director , sd, and fd. and move those binaries out of the way. You shouldn't need to do that. Let the package system handle it. I'm running mysql. Should I do anything there to the existing bacula data base? See below. Assuming the answers to those the next stop is apt-get install bacula-mysql I would assume I then copy back the saved /etc/bacula back in place and then start the new binaries. What about updating the mysql tables from 9 to 13? Where does that happen. The update-mysql-tables script will take care of that for you. Run it BEFORE you start the new Director. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] reload command
On 6/5/23 10:46, Bill Arlofski via Bacula-users wrote: On 6/2/23 08:59, Phil Stracchino wrote: Based on personal experience, after reloading I also recommend restarting the Director when opportunity presents itself. (i.e when there are no running jobs). I have found in the past that after multiple reloads, stability deteriorates. It SHOULDN'T, in theory, but nevertheless sometimes it does. Hello Phil, For the record, and for completeness, do you think you can describe some of the things that seem to become unstable? I am asking out of curiosity because I don't believe I have seen this, and who knows, maybe I have and just didn't make the correlation and your experiences might turn on a light bulb. :) Bill, I would if I could, but I haven't run into it in a long time because (a) I have an established practice of restarting the Director when feasible after significant configuration changes (which does not include Filesets, Schedules, new Clients etc), and (b) it's been a long time since I've needed to make any significant configuration changes. But I do recall that when I was developing and tuning my configuration, especially things like tweaking tape storage devices, spooling etc, if I applied too many changes via reloads without a restart in between, the Director would become unresponsive. Unfortunately I can't really recall any more detail than that. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacularis data table error
On 6/2/23 23:43, Marcin Haba wrote: Hello Ken, This problem is caused by the MySQL catalog table inconsistency. I opened an issue in the Bacula GitLab here: https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2684 <https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2684> Best regards, Marcin Haba (gani) Good catch Marcin! -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] reload command
On 6/2/23 09:57, Pedro Oliveira wrote: Hi Yateen Yes it works but only for configurations changes made on Director "Bacula-dir.conf", for the other Daemons we need to restart the daemons. Best Pedro Oliveira Keep in mind that a reload will not affect already-running jobs, and *some* changes may not take effect in already *scheduled* jobs. Based on personal experience, after reloading I also recommend restarting the Director when opportunity presents itself. (i.e when there are no running jobs). I have found in the past that after multiple reloads, stability deteriorates. It SHOULDN'T, in theory, but nevertheless sometimes it does. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Index for Table File is corrupt
On 5/28/23 02:38, Dr. Thorsten Brandau wrote: Well, thank you. I tried everything above, including going from -1 instead (tried different ranges). WHATEVER I DO, I only get ERROR 1034 (HY000): Index for table 'File' is corrupt; try to repair it back. And you'd already tried a REPAIR TABLE and it failed, as I recall. Yeah, at that point there's nothing else to do but drop and restore, or reinitialize the entire database. Right call. Hopefully the tuning we did will help prevent any future problems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Index for Table File is corrupt
On 5/27/23 14:22, Phil Stracchino wrote: So first create a new empty table: CREATE TABLE `File_NEW` ( `FileId` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `FileIndex` int(11) DEFAULT 0, `JobId` int(10) unsigned NOT NULL, `PathId` int(10) unsigned NOT NULL, `Filename` blob NOT NULL, `DeltaSeq` smallint(5) unsigned DEFAULT 0, `MarkId` int(10) unsigned DEFAULT 0, `LStat` tinyblob NOT NULL, `MD5` tinyblob DEFAULT NULL, PRIMARY KEY (`FileId`) ) ENGINE=InnoDB; Incidentally ... this table schema is not ideal, and yours is missing indexes in any case. This is what a Bacula 13.0.3 File table SHOULD look like, out of the box: CREATE TABLE File ( FileId BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, FileIndex INTEGER DEFAULT 0, JobId INTEGER UNSIGNED NOT NULL, PathId INTEGER UNSIGNED NOT NULL, Filename BLOB NOT NULL, DeltaSeq SMALLINT UNSIGNED DEFAULT 0, MarkId INTEGER UNSIGNED DEFAULT 0, LStat TINYBLOB NOT NULL, MD5 TINYBLOB, PRIMARY KEY(FileId), INDEX (JobId), INDEX (JobId, PathId, Filename(255)) ); I have made a few changes to mine which *significantly* improve performance without any impact detected so far (in years of use) on operation. This is what my File table now looks like: CREATE TABLE `File` ( `FileId` bigint(20) unsigned NOT NULL AUTO_INCREMENT, `FileIndex` int(11) DEFAULT 0, `JobId` int(10) unsigned NOT NULL, `PathId` int(10) unsigned NOT NULL, `Filename` varbinary(255) NOT NULL, `DeltaSeq` smallint(5) unsigned DEFAULT 0, `MarkId` int(10) unsigned DEFAULT 0, `LStat` varbinary(80) NOT NULL, `MD5` varbinary(130) NOT NULL, PRIMARY KEY (`FileId`), KEY `JobId_2` (`JobId`,`PathId`,`Filename`), KEY `PathId` (`PathId`) ) ENGINE=InnoDB It's up to you whether you want to try this modification. You can always modify the table to this spec later. (And I can give you the rest of the set of schema changes, if you want.) The change from BLOB columns to VARBINARY greatly increases performance of JOINS, because the MEMORY storage engine does not support BLOB/TEXT data types, and as a result any implicit temporary table created by a JOIN which contains BLOB/TEXT tables will always be forced to disk. It also avoids the need to put a range constraint on Filename in the second index. VARBINARY behaves exactly like BLOB in all important respects except that it IS supported in the MEMORY storage engine. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Index for Table File is corrupt
ase configuration, I'm doing this on a 3-node synchronous cluster.) There's three reasons for doing it this way: 1. You can see that it's visibly making progress, not just going out to lunch 2. Copying the rows in small batches reduces the memory required for each copy operation. Copying 70 million rows at once is a huge transaction, and if it fails for some reason in the middle, all that work is lost. 3. If the copy is going to fail for any reason, we want to copy as many rows as possible BEFORE it fails *and know where it failed*. If it fails partway through, say at 3500, you MAY be able to resume by passing it a starting offset of 3510 and see if you can resume after the failure. I'm also not deleting the copied rows from the source table, for two reasons: 1. Bulk deletes in MySQL are slow and it would cost us more time than it would save 2. We want to preserve all of the original data until we're sure we've got it all. If you prefer, you can TRY doing a single one-shot batch copy first: INSERT INTO File_NEW SELECT * FROM File; But this may run for hours with no visible output. You can monitor it from another MySQL shell using SHOW PROCESSLIST to verify that it is still running. Once you can (hopefully) get one or the other method to complete without erroring out: RENAME TABLE File TO File_DAMAGED, File_NEW TO File; And then if I were you, I'd run a dbcheck admin job. Then restart your Director and see how it all looks. I can't guarantee this will work, but there's *clearly* something screwy in your DB at the tablespace level or lower, and I can't tell without poking around in your database myself what it is. It's possible there was a corrupted sector where the index is stored. Yeah, I know, RAID, checksums, not supposed to happen; there is a thing that storage admins lose sleep over called the RAID5 write hole. It's a way that silent data corruption can happen on a RAID5 array. (Speaking of which, if you have the ability to do so, if I were you I'd try to provision a separate RAID10, *NOT* RAID5, volume for your database storage. When it comes to large database operations, I/O throughput is next to godliness.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Index for Table File is corrupt
On 5/26/23 11:39, Dr. Thorsten Brandau wrote: When I use mysqlcheck I get Warning : InnoDB: The B-tree of index PRIMARY is corrupted. mysqlcheck -r bacula returns "storage engine does not support repair" So, the B-Tree is still corrup on PRIMARY. OK. If that is the only problem, and there is plenty of free disk space (AT LEAST the size of the File.ibd file), this SHOULD be straightforward to repair. Try this: ALTER TABLE File DROP PRIMARY KEY, ADD PRIMARY KEY (FileId); That will recreate the corrupted index. Expect it to take a while. If NOTHING ELSE is corrupted, you should be good. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy job clarification requested
On 5/18/23 18:37, Chris Wilkinson wrote: I'm not sure I'm getting the motivation for using a copy job in preference to a duplicate job to a second SD. This would also create a second backup. The only reason I can think of is that a duplicate job might be different if the files changed in between. That shouldn't be an issue. I read that a copy job cannot be used for a restore Not true. Just select that job to restore from. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] compression with lto?
On 5/17/23 12:52, Marco Gaiarin wrote: I've LTO tape that work in very dusty environment by at least 6 years. Istead of dust, i've found that setting properly spooling and buffers prevent the spinup/spindown effect, that effectively can be very stressful... Yes, I went to great lengths to try to keep mine streaming and avoid shoe-shining, but with only moderate success. I have had many fewer problems, as well as much better performance, since I abandoned tape and went to disk-to-disk-to-removable-disk (with both of the destination disk stages being RAID). Full backup cycles that used to take 18 hours and two or three media changes, with about a 10% failure rate due to media errors, now take 3 or 4 hours with no media changes and nearly 100% success. However, we are getting further and further off the subject of compression. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] compression with lto?
On 5/17/23 11:48, Dr. Thorsten Brandau wrote: I decided to use the softwarecompression instead, as this actually compresses the files. At last it means 1.4:1 for me, which is a lot better than uncompressed backup (LTO9 tapes are very expensive...). I have to admit I found the cost of the media for the last LTO generation I used (LTO4) to be secondary to the cost of replacing drive after drive after drive as they failed, because in my experience LTO drives fail very quickly outside of a clean-room environment or a sealed tape library. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Storage protocols advice sought
On 5/10/23 11:19, Chris Wilkinson wrote: This is not really a Bacula question but I'm hoping someone has come across this when using NFS to mount a NAS storage device. As I understand it, the UID:GID on the remote storage and the local SD daemon must match or permission denied results. My NAS has NFS v3 and this can't be changed. I can create a new user on the NAS but I have no control over what UID:GID it chooses and will not be the same as the SD daemon user:group. I believe NFS v3 doesn't have the capability to map UIDs; that facility is v4 only. Has anyone found a workaround for this? What is the NAS? I'm guessing if you don't have the ability to control UIDs/GIDs on the NAS you probably can't install a SD on it directly either. NFS-mounting your Bacula backup storage is usually a bad idea unless you have no alternative. Honestly, in my experience off-the-shelf consumer/prosumer NAS appliances are utterly horrible from a management perspective — you WILL do things THEIR way, or not at all. I tried a consumer NAS a few uears ago — I'm blanking on the brand right now — and had to return it because not only was it useless for what I wanted, as well as having some pretty ghastly security issues. (Like making every file on every share world-read/write via NFS.) If you can't control the UID/GID assigned by the NAS, can you change the UID/GID of your storage daemon to match what the NAS assigns? That would at least solve the permission problems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New bacula install
On 4/6/23 17:30, Bill Arlofski via Bacula-users wrote: On 4/6/23 15:27, Andrew Meyer via Bacula-users wrote: I think these were provided from epel. Is there a rpm/yum repo provided by bacula?? Hello Andrew, https://www.bacula.org/bacula-binary-package-download/ Fill out the form and you will be emailed your one, unique download area of Bacula Community packages. Be sure to drop your partly-created bacula database before you reinstall. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New bacula install
On 4/6/23 09:09, Andrew Meyer via Bacula-users wrote: [@bacula01 ~]$ rpm -qa|grep -i bacula bacula-storage-9.0.6-6.el8.x86_64 bacula-libs-9.0.6-6.el8.x86_64 bacula-director-9.0.6-6.el8.x86_64 bacula-client-9.0.6-6.el8.x86_64 bacula-common-9.0.6-6.el8.x86_64 bacula-libs-sql-9.0.6-6.el8.x86_64 bacula-console-9.0.6-6.el8.x86_64 [@bacula01 ~]$ I suggest you get current release packages (13.0.2) from bacula.org and try that. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] New bacula install
On 4/5/23 11:10, Andrew Meyer via Bacula-users wrote: I'm trying to set up bacula on AlmaLinux 8.7 w/ MariaDB 10.11 and getting the following error: $ sudo /usr/libexec/bacula/make_mysql_tables -u bacula -p Enter password: ERROR 1005 (HY000) at line 112: Can't create table `bacula`.`Job` (errno: 150 "Foreign key constraint is incorrectly formed") ERROR 1005 (HY000) at line 286: Can't create table `bacula`.`Pool` (errno: 150 "Foreign key constraint is incorrectly formed") Creation of Bacula MySQL tables succeeded. The error was much worse when I tried to run this as mysql root user. Can someone point me in the right direction to resolve this? Andrew, which version of Bacula? If you are seeing foreign key constraint failures, I suspect the general answer is "an old one", because no table in current Bacula releases contains foreign key constraints. They were removed a LONG time ago. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] troubles with automatic labeling after ubuntu upgrade
On 3/31/23 10:16, Peter N. Steinmetz wrote: Hi Matlink and Bill, Thanks for those suggestions and explanation of why Bacula won’t automatically label a new volume because the pool is full. I agree that the puzzle now is why it is not recycling one of the purged volumes in that pool? Are the oldest volumes *actually fully purged*? Do you have Recycle Oldest Volume enabled? (Actually, I don't recycle disk volumes. I delete them on purge and create new ones as needed. I find it simplifies volume tracking and makes better use of disk space.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Truncate Volumes Before Retention Time is Up
On 3/30/23 06:40, Chris Wilkinson wrote: I have an old job that is no longer needed that is consuming some disk space that I'd like to recover. This storage for this job is on S3 cloud and the job has full/diff/incr pools associated. The full backup pool/volumes of this job have a retention time of 365 days so would not normally get pruned out for another year almost. The incr/diff pool/volumes have a retention time of 7days/31days respectively. I'd like to truncate these now, delete the job resources and rm the actual disk volumes, deleting all traces of the defunct job. What would I need to do to truncate these out now? I haven't been able to find bconsole commands that would do this. I tried the truncate command but the volumes status remains unchanged at full or used or append. Thanks for any suggestions. Fundamentally: purge jobs volume=... delete volume=... then rm the file. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Fileset with varibale date (130223)
On 2/13/23 10:28, Jose Alberto wrote: Hi. It is possible to backup a folder with the date variable in example format: 130223 (day, month, year) I have a folder that is created every Saturday and I need only that one to be backed up /oracle/rman/180223 (next Saturday) It's possible? A Fileset can be the output of a script. So you could create a script thatfinds and returns just that one location. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula 9.6 client managing Windows services
On 1/30/23 11:43, Adam Weremczuk wrote: Hi all, Is it possible to tell a client to stop a list of services in a specific order before starting a backup job? Then start them up after backup run is complete. Bacula server runs on Debian 11 and the client on Windows Server 2019 Standard. You could use client-side RunBefore Job and RunAfterJob scripts. (i.e, "RunsOnClient = yes" in the RunBeforeJob/RunAfterJob directive) Obviously this would need careful testing, and you face the issue that if a job fails for any reason, then those services may not get restarted. If they are mission-critical, this would likely be a problem and you would need to implement a failsafe mechanism for restarting them after the backup if not running (a cron job perhaps). -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy backups to more than one storage
On 1/17/23 11:45, Bill Arlofski via Bacula-users wrote: On 1/17/23 06:05, Ivan Villalba via Bacula-users wrote: > How can I run two differnet copy job that copies the same jobid with the PoolUncopiedJobs ? You can't. The PoolUncopiedJobs does exactly what its name suggests: It copies jobs in a pool that have not been copied to some other pool. If you want to copy the same backup jobs to more than one other pool, you will need to use `Selection type = SQLQuery` and then use an appropriate SQL query for the `SelectionPattern` to generate a list of JobIds to run the second set of copies. Which, I note, is what I do ANYWAY, just in case something goes wrong and I have to *rerun* the copy job. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Storage configuration for just two hard drives in rotation
On 12/27/22 16:24, Bill Arlofski via Bacula-users wrote: The description above is our current working scenario using Retrospect just scaled way down for testing purposes. For reference, we typically backup around 100ish clients and have three sets of drives that are in rotation: onsite online set, an onsite offline set, and an offsite sets. These sets are rotated each week. In my experience, trying to migrate a backup *PLAN* from one backup *SYSTEM* to another rarely works well. It is usually better to define your backup *goals*, and then determine how best to accomplish those goals within the context of the new backup system. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Backup restore missing a directory
On 9/9/22 13:55, William Muriithi wrote: Hello, I have attempted a restore and I can get all files and directories with exception of one folder. This is on Windows server 2016 (bacula client - 9.0.8 (28May18) Microsoft Standard Edition (build 9200)) and it is running as local system. The storage and director are on 9.0.6 on Centos 8 Question, what would cause an issue where bacula selectively ignore one directory? First question: Have you verified whether the directory actually exists *in the backup*? Second question: What does the backup fileset look like and what is the pathname of the directory? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: Verify differences - how to investigate?
On 9/8/22 20:25, Justin Case wrote: Finally 8 of 15 verify jobs returned with differences. I did not have this before when I was only using level Data and not Accurate. How can I investigate teh root cause of this? What seems counter intuitive to me is that all verify jobs are marked as OK, also those that returned with differences. I guess this means the verify job ran completely and has no relation to potential differences being found. That is correct. 'OK' means that the Verify job ran successfully without incident. It does NOT mean that it did not find any discrepancies. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: is Level a selection criterion for Migration/Copy jobs?
On 9/8/22 13:38, Martin Simmons wrote:
If you can express it in SQL, then you can use that to select a job, e.g. I
use this for testing:
Job {
Name = "CopyLastBackupJob"
Type = Copy
Selection Type = SQLQuery
Selection Pattern = "select jobid from job where type='B' and jobbytes > 1 order
by jobtdate desc limit 1"
...
}
I use the following:
Selection Type = SQL Query
Selection Pattern = "SELECT DISTINCT J.JobId, J.StartTime
FROM Job J JOIN Pool P ON P.PoolId = J.PoolId
WHERE P.Name = 'Full-Disk'
AND J.Type = 'B'
AND J.JobStatus IN ('T','W')
AND J.jobBytes > 0
AND J.StartTime > now() - interval 48 hour"
(I don't actually need the J.StartTime on the first line there, it was
there for testing the query, but it does no harm.)
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Silly question from newbie
On 8/25/22 17:30, Pollard, Jim wrote: Maybe another option would be do a migrate job to external media for enough of the volumes that contain the incrementals. That way If things go completely south I could still restore. I’ll have to take a look at the schedules. I’m feeling like the first thing bacula is going to do is determine that incrementals are missing and initiate fulls. Is that sounding right? Bacula will not initiate a FULL backup because incrementals are missing or have been moved elsewhere. It will self-initiate a FULL backup if it cannot find a valid prior FULL backup job *in the Catalog*. (Side note: You might look into VIRTUAL FULL jobs here and see if that is of any use to you.) My backup strategy varies retention times for different job levels such that there is always overlap. The higher the level, the longer the retention. I run nightly Incrementals which are kept for a month, weekly Differentials that are kept for two months, and monthly Fulls that are kept for four months. I can point-in-time full restore to a specific day anywhere in the past month from no more than eight Jobs (one Full, one Differential, and no more than six Incrementals). Your incremental backups should be small, and should have the shortest retention times. If considering migrating/copying some jobs to external storage, I would suggest migrating the higher-level jobs that are larger and have longer retention, rather than the incrementals. This will make better use of your online storage while not impacting your ability to perform immediate small, near-present-time restores. I copy only my FULL backups to air-gapped external media; Differentials and Incrementals are kept online until pruned. It literally takes me longer to *set up* a small restore of a few recent files than it does to actually perform the restore. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] running Bacula client on Synology
On 8/2/22 15:37, Bill Arlofski via Bacula-users wrote: Thanks Phil. I needed the laughs and the quotes to steal today. :) My work here is done. :) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] running Bacula client on Synology
On 8/2/22 14:42, Andrea Venturoli wrote: I've just tried cross-building 13.0.0 for QNAP/ARM: it just took a few minutes. (I explicitly left OpenSSL out, since I don't use it, so I don't know if it would work). Unfortunately I cannot test this at least until September. I'll happy provide the binaries (of course as-is) if you need them. Huh. Now I'm curious about how you set up the QNAP cross-compiler and development environment. I couldn't find any useful information on that and QNAP support flatly refused to provide any. It's only academic curiosity at this point though, as I returned the QNAP NAS because it manifestly could not do the things I wanted to do with it. (Starting with manage it in any sensible way beyond pushing buttons in its point-and-drool web management interface, which I quickly found to be full of You Can't Do That.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] running Bacula client on Synology
On 8/2/22 07:16, Adam Weremczuk wrote: Hi all, Synology RS1221+ 8-Bay NAS specifically. Are there any officially supported fairly recent clients and installation guides available? If not Synology what other budget NAS brands are supported best? I have not tried Bacula on Synology, but I can tell you one thing for free: Don't even THINK about trying to run a current Bacula on QNAP. There are obsolete Bacula packages for SOME QNAP models in QNAP's proprietary package format, but absolutely no way to install a current Bacula version. I tried it once and couldn't even figure out how to get a viable build environment set up. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Change expiration date of a single volume
On 7/15/22 12:13, Wolfgang Denk wrote: Is this what you want to see? *show pool=ARCH Pool: name=ARCH PoolType=Backup use_cat=1 use_once=0 cat_files=1 max_vols=0 auto_prune=1 VolRetention=12 years 3 days VolUse=0 secs recycle=0 This would be your problem ^ right here. You have TOLD Bacula not to recycle volumes, so it's not recycling them, just as you told it. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Change expiration date of a single volume
On 7/15/22 11:26, Wolfgang Denk wrote: This was my expectation, too, but apparently this is not the case. As long as the tape has not expired, it will not be reused by Bacula. When I change the status from "purged" to "append", Bacula will happyly use the tape - and append to the end of the existing data. What, if any, 'Recycle ...' directives do you have in your Pool definitions? Please post your Pool definitions. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula on Nas
On 7/5/22 11:55, rkasmoll...@gmail.com wrote: Dear Sir or Madam, I am doing my thesis on open source projects and one of them I have chosen is Bacula. My question regarding Bacula would be if it is possible to configure Bacula in NAS even with the community edition model. You need to be a little more specific than this. Start with, "What NAS"? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Compatibility between Dir/SD 9.4 and FD 5.X ...
On 6/22/22 16:51, Marco Gaiarin wrote: Mandi! Phil Stracchino In chel di` si favelave... The first and most obvious question here is: Is there some sound technical reason why you have not updated the file daemons on those servers to something at least moderately recent? Never hear of 'legacy systems'?! ;-) These are a bunch of old servers, that have to be decommissioned but for now have to be backed up, and clearly first choice is not to do so much work, apart 'apt-get install bacula-fd'. OK, so what you're saying is that these are servers that are old enough that you cannot get a post-7.x bacula client package that will work on them? That certainly does limit your options. Is it *feasible* to install enough of a local development environment on these legacy servers to locally compile a 7.x client for them? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Compatibility between Dir/SD 9.4 and FD 5.X ...
On 6/22/22 12:39, Marco Gaiarin wrote: I'm setting up a Bacula installation for a set of servers, using debian buster package (9.4.2-2+deb10u1) for director and storage daemons. All works as expected until we use file daemon versions 7+, but some very old servers that have bacula 5.0 or 5.2 does not work. The first and most obvious question here is: Is there some sound technical reason why you have not updated the file daemons on those servers to something at least moderately recent? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Q: one job for a group of "identical" machines using the same fileset?
On 6/20/22 06:33, Justin Case wrote: Interesting, ist there an overview of available Community plugins, or where would I best start to look around? On 20. Jun 2022, at 10:37, Radosław Korzeniewski mailto:rados...@korzeniewski.net>> wrote: Hello, niedz., 19 cze 2022 o 23:29 Justin Case <mailto:jus7inc...@gmail.com>> napisał(a): Good to know, I guess for commercial use. For my private homelab commercial plugins are out of reach. Or is there such a plugin for free? FYI, what I do for this application is I have a JobDef for all of the machines that use that fileset, then a Job for each one that basically just specifies that set of defaults, the job name, and the client hostname. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Copy Job of a bunch of backups?
On 5/27/22 05:27, Pierre Bernhardt wrote: Hello, I create a full backup each 1st Sunday, diff backups of rest of Sundays, incremental backups each day. I would create a copy after the first day of a month backup has been made. The data should be copied from the last backup however it has been made. So for the moment I create a restore before a given date/time and create a full backup of these restore to a special tape. Is there a "virtual copy" possible which which I can configure to create one copy job based on backup before a specified time which made the same without having restore them before? If I understand correctly what you are asking, you could, hypothetically speaking, do a Virtual Full backup (i.e. construct a synthetic full backup job within the database from already-existing backups), and then run a Copy job to copy that Virtual Full job to a new tape. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] query.sql
On 5/25/22 13:23, SO Giuseppe - COMAE via Bacula-users wrote: SELECT count(*) AS Jobs,sum(JobFiles) AS Files,sum(JobBytes) AS Bytes,VolumeName FROM (SELECT DISTINCT Jobmedia.JobId, JobFiles, JobBytes, VolumeName FROM Job,JobMedia,Media WHERE JobMedia.JobId=Job.JobId AND JobMedia.MediaId=Media.MediaId) AS f GROUP BY VolumeName ORDER BY VolumeName; MySQL 127.0.0.1> SELECT count(*) AS Jobs,sum(JobFiles) AS Files,sum(JobBytes) AS Bytes,VolumeName -> FROM (SELECT DISTINCT Jobmedia.JobId, JobFiles, JobBytes, VolumeName -> FROM Job,JobMedia,Media -> WHERE JobMedia.JobId=Job.JobId -> AND JobMedia.MediaId=Media.MediaId) AS f -> GROUP BY VolumeName -> ORDER BY VolumeName; ERROR 1054 (42S22): Unknown column 'Jobmedia.JobId' in 'field list' Straightforward typo in line 3. You have Jobmedia instead of JobMedia. With the typo fixed, yes, this yields much more reasonable and correct-looking results. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about schedule
On 5/24/22 14:12, egoitz--- via Bacula-users wrote:
Good morning,
One little question about Schedules. Imagine I have this schedule :
Schedule{
Name = "WeeklyCycle"
Run = Full 1st sun at 23:05
Run = Differential 2nd-5th sun at 23:05
Run = Incremental mon-sat at 23:05
}
Imagine I set up a new backup... and I set up in the third week of the
month. Imagine I start on a Tuesday. For my experience, I assume that
Bacula would do :
- As no backups have been done previosly, first of all at the first time
it runs at 23:05, perform a full backup.
- The next day it runs, even there is not a differential backup, as
supposedly the second time the job runs is Wednesday it will do an
incremental backup.
- When Sunday arrives, it would do the first differential. At this
moment we will have a Full backup, plus N incrementals. Today we to a
differential and will have a differential too.
- The following times it launches incrementals in the week and on Sunday
new differential, until the first Sunday of month in which a new full
will be created.
Am I wrong?. I think not...
That all looks correct, yes.
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about scratch pool
On 5/23/22 12:05, egoitz--- via Bacula-users wrote: Hi mates, My purpose is the following one. As I run backup of different customers : - For better isolating them for managing permissions properly - For separating quota usage - For using more little databases - And because it's done this way some time ago for several reasons (some of them I can't really know now) we are using different catalogues for our customers. OK, so you are using one Director, but multiple Catalogs, one per customer? As Heitor suggested, you could create a separate named ScratchPool in each Catalog, but there is no way to *safely* share the same Scratch pool across multiple catalogs, and I cannot begin to guess at what would happen if the Director finds the same Pools in different Catalogs with different volumes in them. I would urge you in this case to try to make all resource names unique to customers and NOT try to share anything between them. Trying to share volumes in a common scratch pool would break the isolation you are trying to create, and sounds like a cross-customer data breach mishap waiting to happen. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Question about scratch pool
On 5/23/22 10:39, egoitz--- via Bacula-users wrote: Good morning, We are planning to use the Scratch pool for automatically having avaible tapes in the different pools of my different catalogues. For this purpose, we planned to create let's say 100 volumes in scratch and later, that volumes to to be requested by the different pooles AND of different catalogues, needing new media for continuating backup jobs. Is this possible or the scratch pool can only share volumes with pools within the same catalogue?. If you have multiple Catalogs, you would need to devise a way to synchronize the state of the Scratch pool across all of them. This is not functionality that is supported in Bacula. If you have multiple Bacula directors running from different Catalogs, they need to be separate, not sort-of-separate-but-with-overlaps. Trying to share resources between Directors running from different Catalogs will get you into trouble, because one director will have no way of knowing what another is using or has used. This could lead to cases where, for example, one Director performs a critical full backup on a certain set of tapes, and then the next day another Director that doesn't know it has done so recycles one of those tapes and uses it for something else, and the first doesn't know the backup is no longer valid. What are you trying to accomplish with this multiple-catalogs scheme? When the answer doesn't make sense, sometimes it means you're asking the wrong question. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Sorry
On 4/25/22 09:52, Alan Polinsky wrote: Folks: Unintentionally I included a email to the group, when I only meant it to go to my sons. Sorry. It's OK. Back door alarm support is planned for a future release. :D -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] 9.4.4 dir and 9.4.2 client interoperability
On 3/25/22 09:24, Shaligram Bhagat, Yateen (Nokia - IN/Bangalore) wrote: Hi All We have bacula-dir v 9.4.4 hosted on Centos 6.x and bacula-fd v 9.4.2 hosted on ubuntu 20.04 Can they interoperate ? This should be no problem. The Director and all Storage daemons should be the same version, but if some clients are a few minor versions behind (even major versions, so long as it's not too many), that will not cause any issues. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula‑users] Link Error 11.0.5 on sles
On 3/10/22 20:06, Phil Stracchino wrote: On 3/10/22 18:37, Mark Casey wrote: Yep, I use bat. The options i used to compile are the same as what I used for the 9.6.4 version qt4 libraries and dev are installed I have downloaded the 11.0.6 version to see if that fixed the issue I had no problems building bat in 11.0.5. 11.0.6 has not hit Gentoo yet. I'm sure it won't be long. Er, I'm sorry, I completely garbled that and only half read what I was replying to. Please ignore. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] [Bacula‑users] Link Error 11.0.5 on sles
On 3/10/22 18:37, Mark Casey wrote: Yep, I use bat. The options i used to compile are the same as what I used for the 9.6.4 version qt4 libraries and dev are installed I have downloaded the 11.0.6 version to see if that fixed the issue I had no problems building bat in 11.0.5. 11.0.6 has not hit Gentoo yet. I'm sure it won't be long. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] mariadb 10.5.15
Oh, one other thing ... how old was your OLD mysql database? Red Hat had the incredibly stupid idea of shipping MySQL configured by default with 'old_passwords=1', and many other distributions just blindly shrugged and said, "Huh, seems legit to me." If your password hashes in the mysql.user table are less than 41 characters in length, you probably have old pre-MySQL-4.1 insecure passwords which have been deprecated since ... 2004 I think? All such passwords should be regenerated, and preferably changed, after making certain that old_passwords is OFF. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] mariadb 10.5.15
On 3/6/22 14:39, Alan Polinsky wrote: If I use mysql -U bacula, I cannot connect to the Bacula database. It seems to be a permission issue, but things were changed with version 10.4 of MariaDB. Can someone suggest a solution? OK, first: - Can you log into your database as superuser? - Can you verify that the 'bacula' user has a password set? - Can you verify that it is set to what you *think* it is? (Or you could just make sure by CHANGING it to what you think it should be.) - Is the host part for the 'bacula' user correct? - Does the 'bacula' user have the correct permissions? (SHOW GRANTS FOR bacula@...) - Having done all of those, can you log in using 'mysql -u bacula -p' ? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Job copy question
On 2/25/22 17:59, Josip Deanovic wrote:
Several years ago I posted to this list an SQL query that copies only
the jobs that weren't already copied, while starting from the last
successful Full job. The post could still be found in the mailinglist
archives.
I used it when I had to start using Copy jobs but didn't want to
copy the whole pool as it was quite large.
This is what I use:
Job {
Name = "Archive Copy"
Type = Copy
Enabled = Yes
Pool = Full-Disk
Level = Full
Client = ALL
Fileset = DUMMY
Storage = epsilon3-archive
Selection Type = SQL Query
Selection Pattern = "SELECT DISTINCT J.JobId, J.StartTime
FROM Job J JOIN Pool P ON P.PoolId = J.PoolId
WHERE P.Name = 'Full-Disk'
AND J.Type = 'B'
AND J.JobStatus IN ('T','W')
AND J.jobBytes > 0
AND J.StartTime > now() - interval 48 hour"
Messages = Daemon
Priority = 20
Rerun Failed Levels = yes
Allow Duplicate Jobs = no
Cancel Queued Duplicates = yes
Schedule = "Full Backup Archive"
}
This selects every completed Full backup that started in the previous 48
hours for copying. My Full backups run once a month.
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
Just to wrap up, the definitive answer to Bacula on QNAP: There is a third-party Bacula 9.0.5 package for QNAP NAS which can be obtained from https://qnapclub.eu/en. No idea if or when it will be updated to 11.0.5. Other than that, fuhgeddaboudit. And you probably don't want to run anything with QNAP's Hero OS anyway. It has major security problems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/3/22 21:05, Gary R. Schmidt wrote: That can sometimes be fixed by setting the mount to be soft rather than hard, or vice-versa, but the Linux implementation of NFS leaves a lot to be desired, at both ends. (Speaking as a person who was a tech lead at SGI's "File Serving Systems" and wrote many, many tests for NFS and really developed a distaste for the subtle gotchas in NFS. If I ever hear someone defending the, "Oh, it can't fall back from v4 to v3 because they're different protocols", again, I will strangle the .) This is why for many years now my NAS has run Solaris. It is THE reference implementation of NFS. However, Solaris is dead. Oracle hustled it out behind the barn and put a bullet in its head. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/3/22 07:56, Heitor Faria wrote: These providers do that because they think it would be very hard or even impossible to give support to a NAS with an OS that the user can freely modify. Therefore, you could try TrueNAS <https://www.truenas.com/>, which is based on FreeBSD, or build your own (e.g.) <https://www.servethehome.com/buyers-guides/top-hardware-components-for-truenas-core-servers/>. There are a lot of recipes also on YouTube. You can usually also erase proprietary NAS' OS and install TrueNAS. But returning to the main thread, I still think the best solution for Phil's situation would be to use a NAS built-in virtualization capabilities. I hope he has success anyways. This entire discussion is starting to become moot because I am increasingly discovering that the QNAS unit is so terribly, horribly, appallingly buggy and crippled and broken that I'm beginning to doubt how I can possibly in any good conscience use it. It feels as though I would just be storing up years of daily headaches, and I'm looking for alternatives that I have full control over. I did consider completely wiping the thing and installing either Gentoo Linux or Solaris x86. But I hesitate to do that on a brand new unit still under warranty. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/2/22 15:03, Phil Stracchino wrote: I'm going directly to the heart right now. I'm cross-compiling an entire statically-linked gcc/binutils toolchain for it on another machine. THEN I can locally compile Bacula (and probably a few additional library dependencies) and have no further problems. (I note that the ENTIRE REASON this problem arises in the first place is because QNAP sees fit to ship the device with a fully ... well, MOSTLY functional Linux OS *with no system compiler installed*.) -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/2/22 13:52, Heitor Faria wrote: Hello Phil, Extra workload on the NAS isn't an issue. It has 32GB RAM and eight Xeon D-1622 cores. But then, even you you manage to build the SD you will probably be stuck with the same Bacula version forever. I guess you haven't disclosed this specific NAS model, but you might consider using a VM in the QuTS Hero "Virtual Machine Sation" solution. Ref.: <https://www.qnap.com/en/product/ts-h886> Then the OS is not a problem anymore. That is EXACTLY what I'm using. :) I'm going directly to the heart right now. I'm cross-compiling an entire statically-linked gcc/binutils toolchain for it on another machine. THEN I can locally compile Bacula (and probably a few additional library dependencies) and have no further problems. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/2/22 06:59, Josip Deanovic wrote: The problem is with the openssl support. File daemon would probably statically compile without openssl support. Last time I did that was with the 7.x version of bacula. I just tried disabling openssl support but that did not solve the problem. I'm building a static gcc/binutils toolchain to install on the NAS and I will build sd and fd with that. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/2/22 10:08, Heitor Faria wrote: Hello Phil, (a) trying to figure out how to get a Bacula sd and fd installed on it The happiest days of my life were after I got dedicated equipment for home network modem, routing, and wireless mesh accessing points. That said, I never understood this very common user fetish of installing Bacula components in the NAS. OK: you avoid one more network node but the hardware is not designed to handle the extra workload. Because when the backup scheme is disk to disk to removable disk, the transfer rate going directly across the backplane is much higher than I can get across a gigabit network. Also, I don't want to burn the power to run another node that functionally does nothing but run a storage daemon. Extra workload on the NAS isn't an issue. It has 32GB RAM and eight Xeon D-1622 cores. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/1/22 17:50, Phil Stracchino wrote: Without the static directives the entire build completes successfully. ...But didn't work because of the wrong glibc version. QNAP says look, golly gosh gee whiz wow, kernel 5.10! But is still using glibc 2.21, 13 releases behind. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/1/22 13:15, Phil Stracchino wrote: This QNAP unit has Intel CPUs and a proprietary-Linux-distribution internal OS. I suppose compiling a statically linked binary package for it on one of my other machines would be one way to solve the problem. Thanks for the thought. The following configure invocation on a Gentoo system also with Intel processors: ./configure --prefix=/opt/bacula --with-dump-email=r...@caerllewys.net --with-job-email=r...@caerllewys.net --with-smtp-host=smtp.caerllewys.net --with-subsys-dir=/opt/bacula/var --with-working-dir=/opt/bacula/var --enable-build-stored --disable-build-dird --enable-smartalloc --disable-batch-insert --with-mysql --enable-static-fd --enable-static-sd --enable-static-tools --disable-libtool successfully built lib and findlib, then failed in filed: /usr/bin/g++ -static -L../lib -L../findlib -o static-bacula-fd filed.o authenticate.o backup.o crypto.o win_efs.o estimate.o fdcollect.o fd_plugins.o accurate.o bacgpfs.o filed_conf.o runres_conf.o heartbeat.o hello.o job.o fd_snapshot.o restore.o status.o verify.o verify_vol.o fdcallsdir.o suspend.o org_filed_dedup.o bacl.o bacl_linux.o bxattr.o bxattr_linux.o \ -lacl -lz -lbacfind -lbaccfg -lbac -lm -lpthread -ldl -ldl \ -lssl -lcrypto -lcap -llzo2 /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lacl /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lz /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: ../lib/libbac.a(plugins.o): in function `load_plugins(void*, void*, char const*, char const*, bool (*)(Plugin*))': /netstore/src/bacula-11.0.5/src/lib/plugins.c:150: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lssl /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lcrypto /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lcap /usr/lib/gcc/x86_64-pc-linux-gnu/11.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -llzo2 collect2: error: ld returned 1 exit status make[1]: *** [Makefile:240: static-bacula-fd] Error 1 Do we actually still support static builds? Without the static directives the entire build completes successfully. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/1/22 13:09, Andrea Venturoli wrote: On 1/1/22 18:13, Phil Stracchino wrote: (a) trying to figure out how to get a Bacula sd and fd installed on it I cross-compiled them myself. Actually, only the SD (9.6.3 for arm): I think I shared it in the past, but if you need it just ask. This QNAP unit has Intel CPUs and a proprietary-Linux-distribution internal OS. I suppose compiling a statically linked binary package for it on one of my other machines would be one way to solve the problem. Thanks for the thought. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula for QNAP?
On 1/1/22 11:32, Andrea Venturoli wrote: On 12/31/21 22:22, Phil Stracchino wrote: Does anyone have any experience with, or useful tips for, using Bacula to back up a QNAP NAS with the QuTS "Hero" OS? I'm backing up *to* a couple of QNAPs (running bacula SD daemon, and not with ZFS). I didn't choose them, but I'm quite happy about them. IIC you are trying to backup up *from* a QNAP, right? Perhaps you could be more specific: what problems are you encountering? (a) trying to figure out how to get a Bacula sd and fd installed on it (b) dear gods, the POINTLESS OBFUSCATION of shared volumes ... it seems like they fucked around with it because they could. Yes, it's ZFS internal filesystems, but they clearly have no idea what they're doing with ZFS. I absolutely cannot imagine what they were thinking. (a) is somewhat complicated by the fact that I need the sd compiled with attribute batching disabled, because batching is broken as currently implemented and as a result is not compatible with Galera clusters. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bacula for QNAP?
I replaced my failed NAS with a QNAP NAS that is advertised as having a Linux OS and ZFS filesystem. Both of these are technically true, but ... well, here is not the place to rant about what a mess they made of it. Does anyone have any experience with, or useful tips for, using Bacula to back up a QNAP NAS with the QuTS "Hero" OS? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] bacula-fd version
On 12/28/21 18:22, Richard Couture wrote: Does the bacula-fd client version have to be the same or newer than the server version of bacula or does the server version of bacula have to be the same or newer than the bacula-fd client version? my server version is 9.6.7 and the bacula-fd client version on one of the machines is much older How much older is "much older"? Bacula directors and storage daemons should always be the same version. Clients (file daemons) can be older *within reason*, but you may run into problems if the client is *too* old. Is there a reason why that one client cannot be updated? -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] make_catalog_backup stopped working
On 12/28/21 11:33, Graham Dicker wrote: Looking good. A full backup completed and make_catalog_backup performed flawlessly, which was the original problem. All done manually. Tomorrow morning it'll be back on automatic and I have every expectation that when I finish breakfast there will be emails waiting for me reporting successful backups. I can't tell you how grateful I am for all the help from Graham Sparks, Phil Stracchino and Josip Deanovic. Thank you very much guys, ever in your debt. Glad to hear everything is working as it should be now. -- Phil Stracchino Babylon Communications ph...@caerllewys.net p...@co.ordinate.org Landline: +1.603.293.8485 Mobile: +1.603.998.6958 ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
