Re: [Bacula-users] HP 1/8 G2 Autoloader

[Rob Gerber]

Stefan, are your bacula catalog backups being made to a disk volume, as is
default, or to a tape volume? If being made to a disk volume you could
restore a catalog backup. If your catalog backups were being made to that
same machine whose backups were purged, and you lost the database entries
for the catalog backups, you can still run bscan against the catalog
backups to do a restore.

By default, a database restore will purge all other entries in the
database. So if you're going to make a restore of the bacula catalog you'll
need to do it soon vs later OR maybe try to do some hybrid method where you
restore entries partially later, but I have no knowledge of how to do that
and I don't know how dangerous or foolish such a thing may be.

I assume right now you aren't doing many or any backups because your tape
drive is down (except for the backups made to disk that you're trying to
purge/prune because you accidentally backed up data that's too big for your
disk storage). However, once you start running more backups you
increasingly accumulate more catalog data you don't want to lose.

If catalog entries for volumes other than the disk volumes on your storage
were lost (backups made to tape via that client), your system might attempt
to automatically reuse tape volumes once your tape system is back online,
thereby overwriting backups you'd want to keep. Maybe watch out for that.

The data on your tape volumes is obviously still available, and worst case
you could do a bscan operation against those volumes once tape drive is
operational again. Would just have to be sure any impacted tapes aren't
automatically overwritten. A bscan operation would definitely take a while,
depending on how many tapes are impacted (if any).

A catalog restore is almost certainly what you want to do, presuming there
isn't a bunch of new catalog entries made after the accidental purge event.

To manually remove the disk volumes, and associated job and file entries,
do a delete operation. Easily done in Bacularis or baculum, (I haven't done
it in bconsole though I'm sure it's doable there too). I would have a
catalog backup restored BEFORE I performed the delete operation, and I
would be very careful to select the correct volumes to delete.

Running to a meeting, will check back later.

Robert Gerber
402-237-8692
r...@craeon.net

On Wed, May 15, 2024, 2:17 AM Stefan G. Weichinger  wrote:

> Am 08.05.24 um 13:13 schrieb Stefan G. Weichinger:
> >
> > I assume the drive has a problem.
>
> we used a new cleaning tape and a new LTO tape, btape still fails
>
> I tend to say this drive is defective
>
> Today the IT and the CEOs decide how to proceed.
>
> For now I created a temporary file-based storage on disk.
>
> As if it isn't dangerous enough right now I made a mistake:
>
> at first I had the huge veeam-files dumped to disk, I didn't want that.
> So I tried to delete jobs or volumes in bacularis, but that didn't
> remove the files on disk.
>
> googled and found some hint to "purge" ... now I purged all jobs from my
> single client even the good ones :-(
>
> Now I have volumes on disk with no jobs in the DB. I re-run the valid
> jobs right now, this creates new "virtual tapes" now, but I have to
> (manually?) remove all those orphaned volumes to free space.
>
>
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Bacula community repo SHA1 pubkey

[Rob Gerber]

Hello,

The bacula community repo currently signs their packages with a SHA1 key.
SHA1 is deprecated in EL9 onwards, and poses a security risk that only
increases over time.

Do the community package maintainers have any plans to update the package
signing process to use a SHA256 or greater SHA cipher? This would be a good
move for a project which positions itself in the enterprise software space.

I appreciate that this change would entail change and difficulty, and that
there might be some downsides for users of older bacula distributions, or
for those who have previously installed bacula using an older key. I do not
know if it is possible to sign a package with both the old SHA1 key and a
newer SHA256+ key (I suspect not, but this isn't my field of expertise).

Given that bacula 15.x is in beta, this might be a good time to sign the
next 15.x release with a new SHA256+ key, so at least packages 15.x onwards
are signed with a more secure cipher standard.

Here is a brief writeup on the subject. I hope it is useful.
https://www.redhat.com/en/blog/rhel-security-sha-1-package-signatures-distrusted-rhel-9

Regards,
Robert Gerber
402-237-8692
r...@craeon.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] bacula 13.0.4 community repo missing bacula-cloud-storage package?

[Rob Gerber]

Hello,

I have been running bacula 13.0.3 on rocky linux 9.3, and am considering
upgrading to bacula 13.0.4. However, for EL9 I see that the bacula
community repo for bacula 13.0.4 does not appear to include the
bacula-cloud-storage package, though bacula 13.0.3 does include that
package. The release notes and changelog are silent about this omission.
The package appears to be present in the 13.0.4 repos for EL7 and EL8.

Why was the bacula-cloud package omitted from the 13.0.4 EL9 release, and
what could we do to work around that omission? Do you think it'd be
reasonable to install the bacula-cloud package from 13.0.3 to a system
running 13.0.4, or reasonable to upgrade a 13.0.3 system with the
bacula-cloud-storage package installed to 13.0.4?

Would it be feasible for the project to add this package to the 13.0.4 EL9
repo?

I think it would be unfortunate if users were forced to choose between
13.0.3 or 13.0.4 on the basis of the cloud plugin's availability, since
13.0.4 contains the fix to issue #2704 regarding old FD compatibility.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] HP 1/8 G2 Autoloader

[Rob Gerber]

Wait a sec - look at this again:

# mtx -f /dev/sg4 status
   Storage Changer /dev/sg4:1 Drives, 8 Slots ( 0 Import/Export )
Data Transfer Element 0:Full (Storage Element 4 Loaded):VolumeTag =
CMR921L6
   Storage Element 1:Full :VolumeTag=CMR904L6
   Storage Element 2:Full :VolumeTag=CMR933L6
   Storage Element 3:Full :VolumeTag=CMR902L6
   Storage Element 4:Empty
   Storage Element 5:Full :VolumeTag=ARC954L6
   Storage Element 6:Full :VolumeTag=ARC946L6
   Storage Element 7:Full :VolumeTag=ARC943L6
   Storage Element 8:Full :VolumeTag=ARC939L6

It says data transfer element full with the tape from slot 4. Is that still
the current status? Maybe there is a tape stuck in the robot's tape changer!

I suggest doing
mtx -f /dev/sg4 status
See what it says now. Maybe it was in the middle of moving the tape, but if
it still says that, then the tape might be stuck.

Also,

mtx: Request Sense: Error Code=70 (Current)
mtx: Request Sense: Sense Key=Illegal Request
...
mtx: Request Sense: Additional Sense Code = 53
mtx: Request Sense: Additional Sense Qualifier = 03


Chatgpt 4 says that this error code with qualifying sense codes means the
tape library believes it's physically impossible to remove the media.
Usually caused by mechanical issues, misalignment, jammed media, or
physically damaged tape or drive. Maybe a sticker came off the tape and
it's jammed?

Chatgpt 4:
"MTX Load/Unload Failure: The mtx command failure with the sense key
"Illegal Request" and the additional sense code of 53 03 indicates a "Media
load or eject failed". This usually happens because the tape drive believes
it is physically impossible to load or unload the tape, potentially due to
mechanical issues, a misalignment, or a physically damaged tape or drive."

I am unable to find a cross reference to validate chatgpt's claim about the
meaning of the error code. On my phone right now, so limited access to
systems I administer.



Robert Gerber
402-237-8692
r...@craeon.net

On Tue, May 7, 2024, 10:57 AM Rob Gerber  wrote:

> Checking simple stuff:
>
> What output for 'lsscsi'?
>
> What output for 'ps aux| grep -i bacula'?
>
> Try removing tape, physically sliding tape protection slider all the way
> closed (write protected), then all the way open (not write protected). I
> think should maybe be a click each direction?
>
> Take different known blank tape with definitely no useful information on
> it, try same mt commands.
>
> Remember that after rewind / writing eof you must relabel the tape. Since
> existing tape has same name in bacula catalog (assuming you name with
> barcodes), will need to delete old volume with same name to allow
> relabeling as that name. BE CERTAIN YOU DELETE CORRECT VOLUME.
> Alternatively, can rename as oldname-1. I think it is better to delete old
> volume since rewind / writing eof definitely makes recovery of data on that
> tape impossible without specialized firmware on tape drive.
>
> Good idea to remove all tapes with important information from the tape
> changer prior to doing any destructive operations like mt rewind/ write eof.
>
> Remember, after manually moving tapes around in changer, should update
> bacula's knowledge of the tape changer inventory by doing update slots. Be
> sure to update slots 1-24 or how ever many you have (0 does not mean "all
> slots").
>
> I have typically done Volume deletes, labeling, update slots, etc, in
> Bacularis / baculum.
>
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
> On Tue, May 7, 2024, 10:42 AM Stefan G. Weichinger  wrote:
>
>> Am 07.05.24 um 17:04 schrieb Stefan G. Weichinger:
>>
>> > Could it be related to something like blocksizes?
>> >
>> > These tapes were used with Amanda before.
>> >
>> > trying another tape now
>>
>> I really get lost here. Wanted to re-label a tape to start over:
>>
>> # systemctl stop  bacula-sd
>> # mt -f /dev/st0 rewind
>> # mt -f /dev/st0 weof
>> /dev/st0: Input/output error
>> mt: The tape is write-protected.
>>
>> The physical "slider" is set to non-protected.
>>
>>
>>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] HP 1/8 G2 Autoloader

[Rob Gerber]

Checking simple stuff:

What output for 'lsscsi'?

What output for 'ps aux| grep -i bacula'?

Try removing tape, physically sliding tape protection slider all the way
closed (write protected), then all the way open (not write protected). I
think should maybe be a click each direction?

Take different known blank tape with definitely no useful information on
it, try same mt commands.

Remember that after rewind / writing eof you must relabel the tape. Since
existing tape has same name in bacula catalog (assuming you name with
barcodes), will need to delete old volume with same name to allow
relabeling as that name. BE CERTAIN YOU DELETE CORRECT VOLUME.
Alternatively, can rename as oldname-1. I think it is better to delete old
volume since rewind / writing eof definitely makes recovery of data on that
tape impossible without specialized firmware on tape drive.

Good idea to remove all tapes with important information from the tape
changer prior to doing any destructive operations like mt rewind/ write eof.

Remember, after manually moving tapes around in changer, should update
bacula's knowledge of the tape changer inventory by doing update slots. Be
sure to update slots 1-24 or how ever many you have (0 does not mean "all
slots").

I have typically done Volume deletes, labeling, update slots, etc, in
Bacularis / baculum.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, May 7, 2024, 10:42 AM Stefan G. Weichinger  wrote:

> Am 07.05.24 um 17:04 schrieb Stefan G. Weichinger:
>
> > Could it be related to something like blocksizes?
> >
> > These tapes were used with Amanda before.
> >
> > trying another tape now
>
> I really get lost here. Wanted to re-label a tape to start over:
>
> # systemctl stop  bacula-sd
> # mt -f /dev/st0 rewind
> # mt -f /dev/st0 weof
> /dev/st0: Input/output error
> mt: The tape is write-protected.
>
> The physical "slider" is set to non-protected.
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Install Bacula-Community from repository?

[Rob Gerber]

Personally, I allowed the use of sha1 to sign packages. This was the only
way to use the bacula community packages from the repository. The packages
are signed with SHA1. Can't determine authenticity without SHA1. Once the
Bacula community project issues a SHA512 key or something similar and signs
their packages with that, then SHA1 will be unnecessary.

Here is my runbook from my notes on installing bacula on rocky linux 9.
secrets removed and instances of my username have been replaced by
'YOURUSERNAME'.

Please note that I run bacula 13.x. Bacula 15.x is in beta and I personally
decided not to deploy it in production. The text below proceeds from that
perspective.

Bacula rocky 9 install and configuration process


# most of the following has to be done as root. I'm assuming you've done
"sudo su" prior to start.
# RL 9 doesn't allow sha1 signing by default. gotta enable it for bacula.
update-crypto-policies --set DEFAULT:SHA1

# import bacula project key
cd /tmp
wget
https://www.bacula.org/downloads/Bacula-4096-Distribution-Verification-key.asc
rpm --import Bacula-4096-Distribution-Verification-key.asc
rm Bacula-4096-Distribution-Verification-key.asc -y

# Add the following entries to your /etc/yum.repos.d/Bacula.repo file:
nano /etc/yum.repos.d/Bacula.repo
# note this URL is customized from the install guide to be for RHEL 9!
[Bacula-Community]
name=CentOS - Bacula - Community
baseurl=
https://www.bacula.org/packages/YOURLICENSEKEY/rpms/13.0.3/el9/x86_64/
enabled=1
protect=0
gpgcheck=1


# correct syntax to find all bacula packages on every repo is dnf list|grep
-i bacula
# we want to disable all the bacula packages in the RHEL appstream repo.
they can break bacula installs from the bacula community repo.
# lets try adding some exclude lines to /etc/yum.conf. this is symlinked
with /etc/dnf/dnf.conf so isn't necessary to edit both
nano /etc/yum.conf
exclude=bacula-common.x86_64 bacula-console.x86_64 bacula-director.x86_64
bacula-libs-sql.x86_64 bacula-logwatch.noarch bacula-storage.x86_64
# with the above string in yum.conf, yum list|grep -i bacula only shows
13.x bacula repo packages and doesn't show any appstream repo bacula
packages, which were version 11.x
# same applies to dnf.
#WARNING: BACULA 15.X APPEARS TO FEATURE A BACULA-CONSOLE PACKAGE, WHICH
MIGHT BE BLACKLISTED BY THE ABOVE PROCESS DURING AN INSTALLATION/UPGRADE OF
15.X

# install postgresql and bacula
yum install postgresql-server -y
service postgresql initdb
#output: Hint: the preferred way to do this is now
"/usr/bin/postgresql-setup --initdb --unit postgresql"
yum install chkconfig -y
chkconfig postgresql on
yum install bacula-postgresql -y
systemctl start postgresql.service
su - postgres
/opt/bacula/scripts/create_postgresql_database
/opt/bacula/scripts/make_postgresql_tables
/opt/bacula/scripts/grant_postgresql_privileges
exit

# give bacula user a shell so I can su into that user
chsh -s /bin/bash bacula

# add bacula user to tape group
usermod -a -G tape bacula

# start bacula
/opt/bacula/scripts/bacula start

# give my user rwx access to bacula dir. used so I can filezilla into the
server and edit stuff from windows
setfacl -R -m YOURUSERNAME:rwx /opt/bacula/

# make symlinks to all bacula programs in /usr/sbin so they can be ran
without a full path
cp /opt/bacula/bin/* /usr/sbin -s



Robert Gerber
402-237-8692
r...@craeon.net

On Fri, May 3, 2024, 3:40 PM James Israel via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> Thanks for the suggestion, Davide.
>
>
>
> However, I had tried that script before (used the one for CentOS, as that
> OS is pretty close to RHEL), and I get the following errors. (I tried it
> again just now, same result):
>
>
>
> First, SHA1 checksums don’t work on this RHEL 9 server, as they don’t with
> many modern OSes, as they’ve been deemed insecure. So, I get:
>
>
>
> warning: Signature not supported. Hash algorithm SHA1 not available.
>
> error: /tmp/Bacula-4096-Distribution-Verification-key.asc: key 1 import
> failed.
>
>
>
> As a work around, I downloaded the .asc file to my local Windows machine,
> which can still do SHA1, and used the resulting hash in the URL in the
> script, commenting out the hash creation parts.
>
>
>
> After doing that and running the script again, I get:
>
>
>
> Errors during downloading metadata for repository 'Bacula-Community':
>
>   - Status code: 404 for
> https://www.bacula.org/packages/bf417a80d9108b58a8a3fc8b78110f9f5b181ae1/rpms/11.0.5/el7/repodata/repomd.xml
> (IP: 94.103.98.87)
>
> Error: Failed to download metadata for repo 'Bacula-Community': Cannot
> download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were
> tried
>
>
>
> As you can see, the url looks correct, but the needed xml file doesn’t
> seem to be hosted anywhere anymore, as no mirrors had it.
>
>
>
> Is there another way to get that file?
>
>
>
> Thanks.
>
>
>
> James
>
>
>
>
>
> *From: *Davide F. via Bacula-users 
> *Sent: *Friday, May 3, 2024 4:11 AM
> *To: 

Re: [Bacula-users] advice about tape drives

[Rob Gerber]

Alan,

>From the Wikipedia article on LTO:


   - Up to and including LTO-7, an Ultrium drive *can read* data from a
   cartridge in its own generation and the two prior generations. LTO-8 drives
   can read LTO-7 and LTO-8 tape, but not LTO-6 tape.[29]
   
   [30]
   

   [31]
   

   - An Ultrium drive *can write* data to a cartridge in its own generation
   and to a cartridge from the one prior generation *in the prior
   generation's format*.
   - Some LTO-8 drives may write previously unused LTO-7 tapes with an
   increased, uncompressed capacity of 9 TB (*Type M (M8)*).[32]
    Only
   new, unused LTO-7 cartridges may be initialized as LTO-7 Type M. Once a
   cartridge is initialized as Type M it may not be changed back to a 6 TB
   LTO-7 cartridge. LTO-7 Type M cartridges are only initialized to Type M in
   an LTO-8 drive. LTO-7 drives are not capable of reading LTO-7 Type M
   cartridges.[33]
   
   - An Ultrium drive *cannot make any use* of a cartridge from a more
   recent generation. For example, an LTO-2 cartridge can never be used by an
   LTO-1 drive; and even though it can be used in an LTO-3 drive, it performs
   as if it were in an LTO-2 drive.

So based on that, I'd guess that LTO 4 is the highest version of LTO
technology you could make use of with your requirement to be able to read
LTO 2 media.

I do have an alternative suggestion however: what if you purchased a drive
newer than LTO 4, and either used your LTO 3 drive to read any LTO 2 media
as needed, OR used migrate jobs to migrate the backups off those LTO 2
tapes to the newer media type?

Just fyi, a migrate or copy job type requires two devices be simultaneously
available, a read device and a write device. So you'd have to have the
ability to connect two LTO drives at once to your system, or maybe would
have to migrate tape data to a file volume, them migrate the file volumes
to the new LTO standard. I don't know how complex it would be to migrate
LTO volumes to disk volumes, then migrate those disk volumes back to the
newer LTO tapes. I think it should be feasible in theory.

I don't have practical experience with migrate / copy jobs, or LTO variants
below 8. Just fyi. Others here may be able to provide more information
about migrate / copy jobs, or about your choice of hardware.

However
Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Apr 22, 2024, 10:00 AM Alan Polinsky 
wrote:

> I have used Bacula for many years, since version 5. In the past, I have
> mentioned my two Nas's along with various Windows and Linux machines get
> backed up on a nightly basis to tape. Currently that tape drive is an
> LTO3 based drive. Some of the older backups are on LTO2 tapes. My tape
> drive is starting to show its age, and within a period of time it will
> have to be replaced. (Since I am a retired programmer on a fixed income,
> cost, as always becomes an issue.) I need to understand the backward
> capabilities of more recent drives. How high could I go with LTO based
> machines while still maintaining the ability to read (and hopefully
> write) those old LTO2 tapes?
>
>
> Thank you everyone for your help.
>
>
> Alan
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Backup from windows

[Rob Gerber]

Mehrdad,

I don't know if I can help with all of your questions but I have some
information for you.

I back up samba shares for a machine that I cannot get shell access onto.
The only way for me to access the files on this Nas is via smb. For this
case, on the bacula server, I have mounted the relevant shares read only. I
have a run-before script defined for my backup job, which script runs
basically
mountpoint -q /mnt/sharename || mount /mnt/sharename

This ensures that my smb targets are mounted. If mountpoint exits >0
(error, nothing mounted there), it tries to mount the shares. If the mount
command fails, it exists > 0 (fails), and the script then exits > 0. The
job then ends with an error code and doesn't proceed further. This is the
desired behavior since I don't want bacula to back up an empty mount folder
and happily report success.

My mountpoint script lists each of the relevant shares that are included in
my fileset.

By default, bacula backs up to file volumes, which you could think of as
virtual tapes. The default configuration includes setup to write the
catalog backup to said file volumes. I don't use file volumes (yet), so I
can't help much there. However, I would suggest you examine the default
configuration and adapt it to your needs. Probably add new pool resources,
etc for your file volumes, so the catalog backup infrastructure isn't
changed or mixed in with your regular backups.

Let's say a volume (whether tape or file volume) has aged out of whatever
file, job, and volume retention periods are defined. However, the tape
isn't in an autochanger, the file volume is read only or has been copied
elsewhere, volume recycling is disabled, etc. Basically, the catalog
entries for the volume in question have been removed from the database, so
bacula has no knowledge of what could be on that volume, and bacula has
been prevented from reusing the volume. In this case, you can use the
bacula bscan tool to read the volume and see what data is stored on it.

You may notice I provided many caveats to the above hypothetical situation.
This is because while I could fairly easily withhold a tape from bacula for
archive purposes merely by refusing to insert it into the tape library,
withholding a file volume is more complicated. In any case, operating
within defined retention periods is better, so perhaps what you could do is
have an archive pool and job definition and use copy or migrate jobs to
move or copy an existing job into that pool. I think the existing retention
period might apply there, so maybe that would be better than manually
copying a file volume and then bscanning it later. In any case, the idea is
to work within bacula to define an increased retention period for a certain
job instead of seeking to work outside bacula. I haven't done this, so I
recommend seeking the advice of others to confirm how best to do this.

With bscan it is not necessary for you to have any information about the
backup.

Bacula does generate BSR (Boot Strap Record) files when backing up or when
restoring (can also configure a restore, then at last step tell bacula just
to save a BSR instead of doing the restore). These BSR files are small and
human readable. They say which volumes contain the relevant data, and where
in the volumes the data is stored. If you have your BSR files backed up,
this will make recovering from a volume much easier and faster. One
suggestion I have heard is that maybe you could configure a run-after
script to email the BSR file generated by a job to yourself after the
bsckup job runs.

I have discussed tape and file volumes. One exception to this behavior of
backing up files into volumes is if bacula is used to back up to an S3
object storage target. In that case I am led to understand that bacula
backs up files directly to the object storage and does not use volumes.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Apr 16, 2024, 10:38 AM Mehrdad Ravanbod <
mehrdad.ravan...@ampfield.se> wrote:

> Hi guys
>
> I am new to bacula and trying to figure it out and testing atm
>
> I have it installed on a RHEL9 server with clients on 2 windows computer
> ( one server, one win7 client) and trying to set up backups both via VSS
> and indiviual file/folders to disk(mainly a NAS and shares)
>
> Only problem is i am having trouble finding any guides or resources
> handling this, almost everything i have found s far is backup of Linux
> machines and concentrated on Tapes/autochangers etc
> I would appreciate any pointers towards relevant material or if anyone
> has any exple configuration files(dir, fd, sd conf files etc)
>
> Also, is there anyone who has experience of long term back up with
> bacula?? How easy is it to archive backups/data for several years?? How
> easy is it to access such archives?? Do you need teh records to be in
> the database for such archives?? Can they be accessed even if something
> happens to database??
>
> Regards /Mehrdad
>
>
>
>
> 

Re: [Bacula-users] Backup from windows

[Rob Gerber]

Oh!

One thing I forgot to mention: if you do not define the file, job, or
volume retention periods, this does not mean that everything will be
retained forever. Instead, bacula uses the default retention periods (I
think they are 365 days, not sure).

Any volume or job retention period specified in a pool resource overrides
the volume or job retention periods specified elsewhere.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Apr 16, 2024, 11:35 AM Rob Gerber  wrote:

> Mehrdad,
>
> I don't know if I can help with all of your questions but I have some
> information for you.
>
> I back up samba shares for a machine that I cannot get shell access onto.
> The only way for me to access the files on this Nas is via smb. For this
> case, on the bacula server, I have mounted the relevant shares read only. I
> have a run-before script defined for my backup job, which script runs
> basically
> mountpoint -q /mnt/sharename || mount /mnt/sharename
>
> This ensures that my smb targets are mounted. If mountpoint exits >0
> (error, nothing mounted there), it tries to mount the shares. If the mount
> command fails, it exists > 0 (fails), and the script then exits > 0. The
> job then ends with an error code and doesn't proceed further. This is the
> desired behavior since I don't want bacula to back up an empty mount folder
> and happily report success.
>
> My mountpoint script lists each of the relevant shares that are included
> in my fileset.
>
> By default, bacula backs up to file volumes, which you could think of as
> virtual tapes. The default configuration includes setup to write the
> catalog backup to said file volumes. I don't use file volumes (yet), so I
> can't help much there. However, I would suggest you examine the default
> configuration and adapt it to your needs. Probably add new pool resources,
> etc for your file volumes, so the catalog backup infrastructure isn't
> changed or mixed in with your regular backups.
>
> Let's say a volume (whether tape or file volume) has aged out of whatever
> file, job, and volume retention periods are defined. However, the tape
> isn't in an autochanger, the file volume is read only or has been copied
> elsewhere, volume recycling is disabled, etc. Basically, the catalog
> entries for the volume in question have been removed from the database, so
> bacula has no knowledge of what could be on that volume, and bacula has
> been prevented from reusing the volume. In this case, you can use the
> bacula bscan tool to read the volume and see what data is stored on it.
>
> You may notice I provided many caveats to the above hypothetical
> situation. This is because while I could fairly easily withhold a tape from
> bacula for archive purposes merely by refusing to insert it into the tape
> library, withholding a file volume is more complicated. In any case,
> operating within defined retention periods is better, so perhaps what you
> could do is have an archive pool and job definition and use copy or migrate
> jobs to move or copy an existing job into that pool. I think the existing
> retention period might apply there, so maybe that would be better than
> manually copying a file volume and then bscanning it later. In any case,
> the idea is to work within bacula to define an increased retention period
> for a certain job instead of seeking to work outside bacula. I haven't done
> this, so I recommend seeking the advice of others to confirm how best to do
> this.
>
> With bscan it is not necessary for you to have any information about the
> backup.
>
> Bacula does generate BSR (Boot Strap Record) files when backing up or when
> restoring (can also configure a restore, then at last step tell bacula just
> to save a BSR instead of doing the restore). These BSR files are small and
> human readable. They say which volumes contain the relevant data, and where
> in the volumes the data is stored. If you have your BSR files backed up,
> this will make recovering from a volume much easier and faster. One
> suggestion I have heard is that maybe you could configure a run-after
> script to email the BSR file generated by a job to yourself after the
> bsckup job runs.
>
> I have discussed tape and file volumes. One exception to this behavior of
> backing up files into volumes is if bacula is used to back up to an S3
> object storage target. In that case I am led to understand that bacula
> backs up files directly to the object storage and does not use volumes.
>
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
> On Tue, Apr 16, 2024, 10:38 AM Mehrdad Ravanbod <
> mehrdad.ravan...@ampfield.se> wrote:
>
>> Hi guys
>>
>> I am new to bacula and trying to figure it out and testing atm
>>
>> I have it installe

Re: [Bacula-users] Wildfile

[Rob Gerber]

I don't think it is correct, no.

I am not an expert, but last time I tried something like this I believe the
suggestion I was given was to use a runbefore script to run a find command
or similar to locate and build a file list, then give that list to bacula
as input for the job.

Perhaps for your runbefore use something like the following line.
mountpoint -q /mnt/backup/ && find '/mnt/backup/vmbackup/Backup xxx-y/'
-type f -name '*.vbk' > /opt/bacula/working/VM_xxx-y.list ||
mount /mnt/backup/ && find /mnt/backup/vmbackup/Backup xxx-y/ -type f -name
'*.vbk' > /opt/bacula/working/VM_xxx-y.list

If any part of this fails it SHOULD exit nonzero and the job will fail
(which is what you want). I recommend testing this before relying on it.
I've only written it in an email client. I assume using bash here.

I'm not certain how to include files from a list elsewhere, but I know it
can be done. I think you do @/opt/bacula/working/VM_xxx-y.list in your
fileset definition. Check the docs to be sure.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net

On Wed, Apr 10, 2024, 1:39 AM Stefan G. Weichinger  wrote:

>
> Is this Fileset correct?
>
> Fileset {
>Name = "VM_xxx-y"
>Include {
>  Options {
>WildFile = "\"/mnt/backup/vmbackup/Backup xxx-y/*.vbk\""
>  }
>}
> }
>
>
> I don't get files with this ... seems not to match.
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacularis in a container

[Rob Gerber]

I know what your problem is. You need a Bacularis API instance. Bacularis
web connects to a Bacularis API instance, which connects to the bacula
director. Right now you only have a Bacularis web instance.

If your bacula instance is entirely bare metal, not in a container, then
you probably don't want to use containers to install Bacularis, or at least
don't want to use containers for the Bacularis API. I recommend installing
Bacularis api + web on bare metal in that case.

There is not a container featuring a Bacularis web instance with a
Bacularis API instance but NO bacula components. I don't know why this is
the case. The various containers that do exist support a number of
different circumstances where Bacularis is being used to manage a
distributed, modular bacula system. There isn't a single container that
only contains the functionality and packages to support Bacularis
management of an existing bare metal bacula instance.


Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Apr 9, 2024, 4:06 PM Thing  wrote:

> Hi,
>
> What am I missing here to connect my bacularis-web container to the
> bacula-director pls?
>
>
> https://imgur.com/a/No8S74f
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Install.sh error trying to install manually Bacularis

[Rob Gerber]

What happens when you click 'update slots' on the volumes page, and tell it
to update slots 1-24? Idk if this is relevant, but perhaps it is. Should
update bacula's knowledge of what is in the library, as per mtx. The output
once you click that button and tell it to update slots 1-24 should show
that it knew about every tape's position in various slots in the library.
If it says anything about updating a record to reflect a tape being in a
certain slot, then its knowledge was incorrect and has been corrected.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Apr 9, 2024, 1:14 PM Neil Balchin  wrote:

> Ok Great,  that solves that problem
>
> I’m almost there
>
> I’m using a Dell TL2000 Tape library and a HH LTO7 drive
>
> I’ve added these to my bacula-sd ( configs pasted below). Also added the
> Autochanger and Drives to the API device list
>
> From the API gui. It’s clearly running the slots and status commands
> properly. Because it shows the 19 tapes with barcodes ,  however when I try
> to label these tapes from the volumes page it reports Zero slots
>
> Autochanger {
>
>   Name = "TL2000"
>
>   Device = "HH-LTO7"
>
>   ChangerDevice = "/dev/pass1"
>
>   ChangerCommand = "/etc/bacula/scripts/mtx-changer %c %o %S %a %d"
>
> }
>
>
> Device {
>
>   Name = "HH-LTO7"
>
>   MediaType = "LTO-7"
>
>   ArchiveDevice = "/dev/sa0"
>
>   HardwareEndOfMedium = no
>
>   BackwardSpaceRecord = no
>
>   BsfAtEom = yes
>
>   TwoEof = yes
>
>   FastForwardSpaceFile = yes
>
>   RemovableMedia = yes
>
>   RandomAccess = no
>
>   AutomaticMount = yes
>
>   AlwaysOpen = yes
>
>   Autochanger = yes
>
>   ControlDevice = "/dev/pass1"
>
>   AlertCommand = "/usr/local/share/bacula/tapealert %l"
>
> }
>
>
> I can run mt and mtx commands freely from the command line as both the
> bacula user and www user so I think my /dev/ permissions are ok
>
>
>
>
>
>
> On 9 Apr 2024, at 09:39, Marcin Haba  wrote:
>
> Hello Neil,
>
> Great to hear that you were able to set up the Bacularis instance.
>
> This error that you see is because in the php.ini file on FreeBSD by
> default is error reporting set to display all errors. To fix this error you
> need to edit php.ini file:
>
> /usr/local/etc/php.ini
>
> and change error reporting line from this:
>
> error_reporting = E_ALL
>
> into this one:
>
> error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
>
> At the end you need to restart PHP.
>
> Best regards,
> Marcin Haba (gani)
>
> On Tue, 9 Apr 2024 at 15:22, Neil Balchin  wrote:
>
>> Thanks so much,
>>
>>
>> I got a little further along my path
>>
>> The Bacularis web guide is up and running
>>
>> I’ve configured my web server user (www) to have appropriate file
>> permissions for the bacula config files and directories and added the sudo
>> permissions for the necessary binaries
>>
>> I think I’m missing something in my server setup because
>>
>> I get this error when I try to config anything:
>>
>>
>>
>> Error 1000 - Internal error. [Unknown Error] substr(): Passing null to
>> parameter #1 ($string) of type string is deprecated (@line 232 in file
>> /usr/local/www/bacularis/protected/vendor/pradosoft/prado/framework/Data/Common/Pgsql/TPgsqlMetaData.php).
>>
>>
>>
>> On 8 Apr 2024, at 23:35, Marcin Haba  wrote:
>>
>> Hello Neil,
>>
>> I have checked installing Bacularis on FreeBSD and I have one more note.
>>
>> By default in Apache is used PHP as Apache module. If you want to use it
>> this way, it is fine. Nothing else to do. If you will need to use PHP
>> through PHP-FPM, then you need to do a few changes in your httpd.conf to
>> disable the Apache PHP module and enable PHP-FPM.
>>
>> First, you need to disable/comment the MPM Prefork module and libphp:
>>
>> #LoadModule mpm_prefork_module libexec/apache24/mod_mpm_prefork.so
>> #LoadModule php_module libexec/apache24/libphp.so
>>
>> and enable/uncomment these modules:
>>
>> LoadModule mpm_event_module libexec/apache24/mod_mpm_event.so
>> LoadModule proxy_module libexec/apache24/mod_proxy.so
>> LoadModule proxy_fcgi_module libexec/apache24/mod_proxy_fcgi.so
>>
>> Then you need to add these lines to the same httpd.conf:
>>
>> 
>> SetEnvIfNoCase ^Authorization$ "(.+)" HTTP_AUTHORIZATION=$1
>> 
>> SetHandler  "proxy:unix:/tmp/php-fpm.sock|fcgi://localhost/"
>> 
>> 
>>
>> At the end in /usr/local/etc/php-fpm.d/www.conf you need to set
>> permissions for PHP-FPM unix socket:
>>
>> listen.owner = www
>> listen.group = www
>> listen.mode = 0660
>>
>> and restart the services:
>>
>> # service php-fpm restart
>> # service apache24 restart
>>
>> Best regards,
>> Marcin Haba (gani)
>>
>> On Tue, 9 Apr 2024 at 04:54, Marcin Haba  wrote:
>>
>>> Hello Neil,
>>>
>>> Thanks for your feedback with installing Bacularis on FreeBSD.
>>>
>>> It looks that in your environment this script uses the default FreeBSD
>>> shell /bin/sh (or any other) that does not support arrays.
>>>
>>> This is the Bash script so to use it you need to install Bash:
>>>
>>> # pkg install 

Re: [Bacula-users] baculum + bacula 13.0.4 slow web interface

[Rob Gerber]

Unkdef, one important thing to know is that Bacularis is a friendly fork of
baculum. It's very similar, and Marcin has been actively maintaining it.
Marcin is also a baculum developer.

Basically, Bacularis is the better, more optimized tool.

This doesn't mean that the source of your problem is certainly baculum, but
trying the actively maintained and more optimized tool Bacularis is
certainly a good first step. :)

Bacularis is available in docker containers if you want to simply load a
container and give it a try. It is naturally available for bare metal
installations as well.

Https://bacularis.app

Robert Gerber
402-237-8692
r...@craeon.net

On Sat, Apr 6, 2024, 12:56 PM Unkdef unkdef  wrote:

> i use only 45 backup jobs a day...
>
> Marcin Haba  6 апреля 2024 г. 20:14:30 написал:
>
>> Hello Unkdef,
>>
>> Yes, Baculum 11.0.6 is a bit old and can be a little slow, specially for
>> many jobs and MySQL. There exists a couple of techniques to speed it up
>> like described here:
>>
>>
>> https://baculum.app/doc/brief/troubleshooting.html#the-baculum-job-tables-are-loading-slowly
>>
>> or here:
>>
>>
>> https://www.mail-archive.com/bacula-users@lists.sourceforge.net/msg72737.html
>>
>> If I could suggest something, I would propose to try Bacularis that is
>> much faster and contains significant optimizations.
>>
>> Best regards,
>> Marcin Haba (gani)
>>
>> On Fri, 5 Apr 2024 at 09:07, Unkdef unkdef 
>> wrote:
>>
>>> Hello! I use bacula 13.0.4(mysql) and baculum 11.0.6 on Ubuntu 22.04
>>> Problem: When I refresh page in baculum i wait 10-15 sec before i see
>>> information of backups and other... how to fix it?
>>> ___
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>
>>
>>
>> --
>>
>> "Greater love hath no man than this, that a man lay down his life for his 
>> friends." Jesus Christ
>>
>> "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za 
>> przyjaciół swoich." Jezus Chrystus
>>
>>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacularis API host initial setup with postgresql and bacula.

[Rob Gerber]

Marcin,

Thank you for the very quick response! I will experiment with this when I
have some time.

I am guessing because I've never configured a password for the bacula
database, and based on my pg_hba.conf, that bacula is authenticating to the
database server via peer connections, so bacula doesn't need a password to
access the database (in my default configuration). This detail kind of
stumped me at first. Please let me know if you think this is incorrect. My
original fear was that resetting the database password would lock bacula
out of the database.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Apr 2, 2024, 11:40 PM Marcin Haba  wrote:

> Hello Rob,
>
> At the moment I prepared a section about the authentication methods in the
> Bacularis documentation. You can read it here:
>
>
> https://bacularis.app/doc/brief/configuration.html#bacula-catalog-database-access
>
> Best regards,
> Marcin Haba (gani)
>
> On Wed, 3 Apr 2024 at 04:51, Marcin Haba  wrote:
>
>> Hello Rob,
>>
>> Thanks for your question.
>>
>> As the authentication method in PostgreSQL pg_hba.conf you can use
>> scram-sha-256 This method is available starting from PostgreSQL 10.
>>
>> For the Bacularis documentation, yes, it looks to be missing there. I
>> will add to the doc this section about configuring the Catalog Database
>> access in Bacularis. Thanks for pointing it.
>>
>> Best regards,
>> Marcin Haba (gani)
>>
>> On Wed, 3 Apr 2024 at 00:50, Rob Gerber  wrote:
>>
>>> I have previously set up bacula and bacularis on Rocky Linux 9. At the
>>> time I ran into issues giving bacularis access to bacula's postgresql
>>> database. I didn't know the bacula database password. At that time I set
>>> the postgresql pg_hba.conf file to use "trust". However I am concerned that
>>> this isn't a good security best practice.
>>>
>>> I am setting up a second bacula / Bacularis system and I'd like to
>>> revisit this postgres authentication issue so I can follow best practice
>>> better.
>>>
>>> The bacularis documentation doesn't make it clear how I am supposed to
>>> authenticate bacularis to bacula's database.
>>>
>>> I suspect there is a process that is obvious to many others and not to
>>> me.
>>>
>>> Does anyone here have suggestions for how I might authenticate bacularis
>>> to the bacula database?
>>>
>>> Robert Gerber
>>> 402-237-8692
>>> r...@craeon.net
>>> ___
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>
>>
>>
>> --
>>
>> "Greater love hath no man than this, that a man lay down his life for his 
>> friends." Jesus Christ
>>
>> "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za 
>> przyjaciół swoich." Jezus Chrystus
>>
>>
>
> --
>
> "Greater love hath no man than this, that a man lay down his life for his 
> friends." Jesus Christ
>
> "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za 
> przyjaciół swoich." Jezus Chrystus
>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Bacularis API host initial setup with postgresql and bacula.

[Rob Gerber]

I have previously set up bacula and bacularis on Rocky Linux 9. At the time
I ran into issues giving bacularis access to bacula's postgresql database.
I didn't know the bacula database password. At that time I set the
postgresql pg_hba.conf file to use "trust". However I am concerned that
this isn't a good security best practice.

I am setting up a second bacula / Bacularis system and I'd like to revisit
this postgres authentication issue so I can follow best practice better.

The bacularis documentation doesn't make it clear how I am supposed to
authenticate bacularis to bacula's database.

I suspect there is a process that is obvious to many others and not to me.

Does anyone here have suggestions for how I might authenticate bacularis to
the bacula database?

Robert Gerber
402-237-8692
r...@craeon.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula support LTO9?

[Rob Gerber]

Bacula should work with LTO 9 just fine. I would be very surprised if it
had issues with LTO 9. Some users on the mailing list have mentioned that
they use LTO 9.



Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Apr 2, 2024, 4:53 PM Jose Alberto  wrote:

> Hi.
>
> I work with Bacula (11 and 13)  with lto8,  all fine.
>
> with bacula 13 or 15   work  LTO9 ?
>
>
> --
> #
> #   Sistema Operativo: Debian  #
> #Caracas, Venezuela  #
> #
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Loose ends upgrading to 15.0.2?

[Rob Gerber]

KM,

Look into Bacularis. It's a friendly fork of baculum. Containers available,
bare metal packages, the works. Actively maintained by a baculum dev,
Marcin. He's on this list.
https://bacularis.app/

Baculum-web is a good package, from what I have heard. Haven't tried it.
>From what I am led to understand, it is a reporting-focused application and
cannot currently exert control over bacula. Developer Davide is also on
this list.
https://www.bacula-web.org/

No knowledge on other subjects. Be aware that error reporting from older
FDs may be broken for FDs older than 13.x.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Wed, Mar 27, 2024 at 2:40 PM K. M. Peterson <
kmp.lists+bacula-us...@gmail.com> wrote:

> Hi all,
>
> Decided it's time to go to Bacula 15 from Bacula 11.
>
> I'm trying to figure out a few hanging issues...
>
>1. For starters, using the CentOS RPMs, Bacula daemons upgraded and
>any database fixes appear to have worked... thanks!
>2. I think I missed something around whether Baculum is still viable
>and being used or whether the community has converged on Baculum-Web - or
>maybe BWeb is going to be a community project at some point?  Or - one or
>the other alternatives?
>3. One reason why I'm asking: as far as I can tell, there is nothing
>newer at downloads.bacula.org later
>than /downloads/baculum/stable-11/centos8 - while it seems there are
>commits for v13?
>4. Source is available, but I'd like to avoid having to build the *API*
>and *Baculum* components myself - though my target is Podman
>containers and if I do have to build I'd be happy to put the
>Containerfile out for others.  (Not 100% sure I can build this, though, as
>PHP-related stuff isn't my strong point.)
>5. I'm one of those weirdos that like to read through documentation in
>PDF format.  The PDFs for 15.0.2 for the "main" and a couple of the others
>are missing from the bacula.org site (404s ensue when clicking on
>links for these).  The bacula-docs-15.0.2 tarball also doesn't have PDFs of
>these; and again maybe there's a build from LaTex I could do, but...
>
> Best to open bugs on what seem to be omissions or any other ideas?
>
> Many thanks in advance!
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] configure CentOS/RedHat repository - first time usage

[Rob Gerber]

You can go here, to the bacula community website under rpm/ Deb / osx
downloads.
https://www.bacula.org/bacula-binary-package-download/

Heads up that the email sent to you on registering might not send. If so
directly after entering your email address and name, a page will load and
in the middle of that page will be a link saying the click 'here'. That
link contains the path component you want.

I recommend saving a copy of that page because the information isn't
available to return to readily without re-registering. I also recommend
saving a copy of your community license key.

If installing under RHEL 9 / RL 9 / AL 9 etc be aware that the bacula
community distribution (version 13.x, at least) hashes its files with sha1,
and that hash algorithm has been deprecated for security reasons. You'll
need to re-enable it if you get error messages related to this.

If you want a GUI to manage bacula (probably do), you'll probably want
bacularis over baculum. Bacularis is a friendly fork of baculum. Dev Marcin
is on this list. :)

Bacula-web is also available maintained by Davide, also on this list.
bacula-web only provides reporting functionality, not management.

I recommend you follow your current course of action and continue seeking
packages from the official repository, as opposed to outdated packages from
the OS repositories. Be careful not to mix and match packages from the OS
repositories with packages from the official repos. I blacklisted bacula
packages on the OS repos for this reason.

Bacula-dir, bacula-sd must be the same version. Traditionally, bacula-fd
did not have to be same version, could be version <= bacula-dir/bacula-sd.
However, I have seen reports that this backwards compatibility for FD is
partially broken now in bacula 13.x. so an FD for 9.x might still work with
bacula-dir 13.x but maybe error reporting doesn't work right, which is
dangerous. So I recommend you ensure your FD are same version or at least
same major version. At no point can your FD be a greater version than
bacula-dir/bacula-sd. Always <=.

Robert Gerber
402-237-8692
r...@craeon.net

On Wed, Mar 27, 2024, 12:15 PM Eduardo Rothe via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> Hi
>
> I am installing Bacula Community for the first time in a Fedora
> environment. According to the white paper Bacula Community Installation
> Guide, section "6.2 yum Package Manager Configuration", I should
> configure the repository using the, and I quote, "*path component sent in
> the registration email*". Which registration is this ? Where can I
> register ? Searching the bacula.org website, I can't find anything to
> register for.
>
> Best to all,
> Eduardo
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Mistakenly erased label headers from LTO with btape test / need help to salvage content

[Rob Gerber]

Standard data recovery processes are to take a bit for bit image of the
troubled media, them attempt all recovery against a copy of the image. This
process is used in disk recovery for block devices but I think it could
apply in your case also.

At minimum, I would write some data to a scratch tape with bacula (at least
20gb or something somewhat substantial consisting of known files which you
have hashed so you can verify the success of the recovery), repeat the
previous mistaken run of 'btape test' with reasonably quick cancellation
(but not too quick as to be overly optimistic!), then attempt recovery.

How recovery is done in this case is something I'm not super familiar with.
As suggested by Pedro, maybe label + bextract (spelling uncertain, check
bacula bin folder)? When dealing with a tape whose data is not in the
bacula catalog we typically want to run bscan, but I don't know if it will
handle this case well.

The wisest case may be to set this tape aside and do a new full backup. If
a recovery is needed then you can attempt recovery of data from this tape.
If no recovery is ever needed, then no problem.



Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Mar 25, 2024, 10:25 AM Pedro Oliveira  wrote:

> try to label again the tape and then try to use bexcrtac
>
> Create your WiseStamp email signature
> 
>
> [image: __tpx__]
> ‌
>
>
> Dedy Yohann  escreveu em seg., 25/03/2024 às
> 17:15 :
>
>> Dear bacula community,
>> I made a mistake today while making maintenance tasks.
>>
>> I unintentionaly ran the "btape test" command on a LTO cartidge that was
>> already labeled and assigned to our main backup pool.
>>
>> When I realized data was written on the tape after a couple of seconds, I
>> stopped the operation (ctrl-c) in panic...
>> Unfortunately the evil was already done, now when I try to restore data
>> from this volume, the cartidge is correctly loaded in the drive (barcode
>> checked by the autochanger) but I get this error once data is read from the
>> tape :
>>
>> bacula-sd JobId 425: Warning: acquire.c:279 Read acquire: Could not
>> unserialize Volume label: ERR=label.c:987 Expecting Volume Label, got FI=0
>> Stream=0 len=64412
>>
>> Is there a way to manually restore the content of the tape or force the
>> relabelling of the tape without erasing the content of the cartidge?
>>
>> Thanks in advance for your suggestions,
>>
>> Yohann
>>
>> Bacula version : 9.6.7
>> Debian 10 (Buster)
>> Kernel 4.19.0-25-amd64
>>
>>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bvfs_clear_cache

[Rob Gerber]

I wasn't really familiar with BVFS so I decided to give it a Google. My
guess is that it means Bacula Virtual File System but I don't know for sure
if this is correct.

Long story short, looks like BVFS is an API layer for GUI interfaces to
communicate with the bacula director.

Here is some information on BVFS, from a 7.x version bacula manual.

https://www.bacula.org/7.0.x-manuals/en/developers/Implementing_GUI_Interface.html

Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Mar 25, 2024, 7:54 AM Adam Weremczuk 
wrote:

> ...obviously TA!
> On 25/03/2024 12:31, Chris Wilkinson wrote:
>
> I believe this is a bconsole command.
>
> -Chris-
>
> On Mon, 25 Mar 2024, 12:28 Adam Weremczuk, 
> wrote:
>
>> Hi all,
>>
>> Bacula 9.6.7-3 (free version)
>> Baculum-web 9.6.6.3
>> Debian 11
>>
>> Every quarter or so I run "dbcheck -f -c /etc/bacula/bacula-dir.conf"
>> and each time I see this this:
>>
>> "Checking for orphaned File entries. This may take some time!
>> To prune orphaned Path entries, it is necessary to clear the BVFS Cache
>> first with the bconsole ".bvfs_clear_cache yes" command."
>>
>> BVFS is nowhere to be found in my installation.
>>
>> Shall I install it and how? Is it free? Shall I bother at all?
>>
>> Regards,
>> Adam
>>
>>
>>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Tapes suddenly unlabeled

[Rob Gerber]

Karl,

Please understand that while I have some bacula experience, I am not as
much of an expert as some of the other people on this list. As such, my
suggestions are based on my thoughts on the matter and not based on long,
deep experience with bacula, or with issues specifically like yours.

What OS is your bacula SD and bacula director running on?

What version of bacula are you using?

Assuming the command is available on your OS, what is the output of
'lsscsi'?

Please paste the definitions for your tape devices from bacula-sd.conf

What happens if you eject the tape, and then reinsert it?

What happens if you attempt to restore data from this tape, or another tape
that was previously used?

What happens if you stop bacula, change the device type back to st from the
preferred nst, restart bacula, then attempt to restore data?

###
THE FOLLOWING SUGGESTIONS ARE DATA DESTRUCTIVE AND SHOULD BE RAN WITH A
NON-BACULA TAPE THAT DOES NOT HAVE IMPORTANT DATA ON IT
###
What do you see if you take a scratch tape with no important data on it,
insert this tape into the drive, stop bacula, start bacula's btape utility,
and run the 'test' command? *Please note that all data on this tape will be
erased! *Since you do have some new hardware, maybe initial testing like
that done when first setting up bacula could reveal whether the tape drive
/ library are working the way we expect that they should be.

Here are some resources on testing a tape drive, using btape, and
troubleshooting tape system settings.

btape utility manual entry:
https://www.bacula.org/13.0.x-manuals/en/utility/Volume_Utility_Tools.html#SECTION00190

Tape testing chapter from the problem resolution guide:
https://www.bacula.org/13.0.x-manuals/en/problems/Testing_Your_Tape_Drive_Wit.html#blb:TapeTestingChapter


Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Fri, Mar 22, 2024 at 8:56 AM MMag. Dr. Karl Kashofer 
wrote:

> Dear all !
>
> I try to work with a 50 slot Quantum Scalar i3 Library with an IBM LTO7
> drive. I have a set of 10 LTO7 tapes as a
> storage pool in bacula and it has been backing up user directories for a
> while.
>
> Recently we needed to swap the library chassis due to drive issues. I
> unmounted the library, shut down bacula,
> transferred the tapes to the new library (again a Quantum Scalar i3 with
> the drives from the old one), adjusted the scsi
> paths of the storage demon config and fired up bacula. During the change I
> did switch from the "st" to the "nst" devices
> as apparently these are preferred.
>
> Bacula now sees the changer device, sees the pool of tapes, but when it
> tried to do the next backup it horribly died on
> the current append volume with:
>
> 22-Mar 08:42 bacula-sd JobId 1245: Error: [SE0203] The Volume=MU0014L7 on
> device="LTO7-0" (/dev/tape/by-id/scsi-
> C3EB53C004-nst) appears to be unlabeled.
>
> There is already 45GB of data on that tape and i really dont see why it
> suddenly calls it "unlabeled".
>
> Any idea what could be wrong ?
>
> I did find this thread and am unsure if thats related:
> https://bacula-users.narkive.com/JWcvt8eZ/bacula-loses-tape-label
>
> Please help,
> Thanks,
> Karl
>
>
> --
> MMag. Dr. Karl Kashofer 
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Pool Configuration

[Rob Gerber]

It's fairly likely that the oldest volumes have had their job records
pruned because the jobs aged out after they became older than the allowed
job retention period. With the job records pruned, there were no longer any
jobs on the volumes, so the volumes were recycled. However, my guess is
that you haven't needed those volumes yet so so bacula has not written to
them. If you did find a need for the data on those volumes, you could bscan
the volumes and recover the backups you needed despite their having been
pruned.

Bacula's default behavior is to preserve the data on pruned volumes as long
as possible, just in case that data is needed.

Personally I might like to see a longer retention period, 20 days is a
little scary since ransomware could incubate for a period between 14 days
and 4 months. 4 months is a figure I heard on a major news outlet regarding
a high profile breach, so it's probably the more extreme end of that
spectrum, but still, 20 days could be pushing your luck.

On the other hand, if you didn't have enough volume space to hold your
backups with a longer retention period, you could wind up with no available
volumes for your newest data.  You'd have to examine your situation and
decide what is best for you.

I wouldn't rely on expired volumes to be there in the event of a need for
backup restore. A ransomware attack alone incubating long enough could
result in a situation where an influx of newly encrypted data could flush
out previously pruned volumes which happened to contain the last good
unencrypted backups.

As far as trivia goes, there is file, job, and volume retention periods.
The retention periods specified under the pool resource over-ride the
retention periods specified under other resources.

Hope this is helpful.

Robert Gerber
402-237-8692
r...@craeon.net

On Thu, Mar 21, 2024, 8:49 AM Ken Mandelberg  wrote:

> I have my pool set as 15 50GB files, with a 20 day retention, autoprune,
> recycle. Typically the oldest one is about 30 days old, and several of
> the oldest show no jobs on volume.
>
> Are those oldest volumes doing me any good with no jobs, just waiting
> for their time to be recycled?
>
> Should I adjust the parameters to make them more valuable?
>
>
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Using LTO drives with bacula

[Rob Gerber]

Ah yes, the qualstar Q24 library front panel looks identical to the BDT
Flexstor 2U panel.

My Q24 library says it is a FlexStor II in the web GUI.

BDT states they have around 70% market share in the tape library field.

https://www.bdt.de/en/products-solutions/storage-automation/

Heh, Overland Tandberg states they have a 24 tape library they call the
T24. Front panel also identical to the BDT FlexStor 2u library.

I believe BDT might only sell to OEMs, and not directly sell their
libraries.

Overall, when selecting a library you're probably going to buy a BDT
library regardless of whether it says BDT on the front or not. :)

I'd say select a library based on price, and non-encumbrance with silly
"pay us more to use the rest of your slots" licensing.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Mar 19, 2024, 10:24 AM Rob Gerber  wrote:

> When reviewing our options we concluded that a tape changer was a small
> additional cost compared to the cost of a tape drive. It also enabled
> automatic tape handling. We are in the USA and purchased a Qualstar Q24
> library. The library is network connected and has room for 2 LTO drives. It
> can use either FC or SAS, depending on the adapter you use in the library.
>
> The Q24 has a mail slot for tape import/export, but so far I've been
> unable to find a way to use mtx to move tapes to or from the mail slot. The
> only way to open or close the mail slot is the front console of the
> library, from what I've seen. I can move tapes to/from the mail slot using
> the changer web GUI. I am somewhat disappointed by this because I had hoped
> to enable automated tape interchange workflows where a script moves a tape
> from the mail slot, runs a job that deposits data to the tape, then ejects
> the tape, moves it to mail slot, then ejects mail slot. An office staff
> person removes the tape in the morning, places it into its case, and
> inserts the next tape. If the tape isn't ejected, the staff person calls IT
> since that means something went wrong. That workflow isn't possible to
> automate as far as I can see because mtx cannot see or interact with the
> mail slot. Not a huge deal, but something you might want to know. The
> workflow I've described is more commonly used with a single tape drive and
> no tape changer, but in the interest of air gapping as soon and as
> regularly as possible in my opinion it's probably among the best practices
> for routinely removing small single tape backup jobs from the changer.
>
>
> If you want to do virtual full, copy, or migration jobs from LTO to LTO,
> know that you'll need two LTO drives. For these job types there must be an
> input device and an output device. Since you're already using bacula for
> backup to disk volumes, I THINK you should be able to use the disk backup
> as the input device and the LTO drive as an output device. In the case of
> having a disk backup set and an LTO drive, I think you could find copy and
> migration jobs useful since you could perhaps occasionally run off copy
> jobs from your disk backups to tape, then remove your tape media from the
> library.
>
> I have read about ransomware that automatically seeks out and attempts to
> attack tape backup systems, over-writing tape media. You already mentioned
> air gapping, but I must emphasize that actually removing backup media from
> the changer is a very good practice in defense against that sort of threat.
>
> We selected SAS, primarily because we wanted to directly contact the
> library to the tape drive, and because we didn't know anything substantial
> about FC. FC also appeared to be more expensive. When considering SAS
> adapters, we were informed by our backup hardware vendor that some SAS
> adapters don't properly support tape communication. For that reason, an
> ATTO SAS card was recommended. Our cost for that card was around $400 USD.
>
> Our vendor of choice was backupworks.com. I don't have a business
> relationship with them or any incentive for recommending them. They've just
> been really helpful in selecting hardware and their service has been good
> as we ordered additional media. I believe they are usa based, so if you are
> not then they might not be the vendor for you.
>
> I will note that some libraries in this space locked away the usage of
> some slots and perhaps some features behind upgrade licenses. I didn't like
> that idea at all, and didn't select such devices. I have been informed that
> only a couple companies are still manufacturing tape drives, only ibm is
> still researching new tape drive mechanisms. I believe the qualstar
> libraries are actually manufactured in Europe, perhaps in Germany. I have
> been told that the libraries are also being manufactured by one company and
> rebadged elsewhere. Don't know if true.

Re: [Bacula-users] Using LTO drives with bacula

[Rob Gerber]

When reviewing our options we concluded that a tape changer was a small
additional cost compared to the cost of a tape drive. It also enabled
automatic tape handling. We are in the USA and purchased a Qualstar Q24
library. The library is network connected and has room for 2 LTO drives. It
can use either FC or SAS, depending on the adapter you use in the library.

The Q24 has a mail slot for tape import/export, but so far I've been unable
to find a way to use mtx to move tapes to or from the mail slot. The only
way to open or close the mail slot is the front console of the library,
from what I've seen. I can move tapes to/from the mail slot using the
changer web GUI. I am somewhat disappointed by this because I had hoped to
enable automated tape interchange workflows where a script moves a tape
from the mail slot, runs a job that deposits data to the tape, then ejects
the tape, moves it to mail slot, then ejects mail slot. An office staff
person removes the tape in the morning, places it into its case, and
inserts the next tape. If the tape isn't ejected, the staff person calls IT
since that means something went wrong. That workflow isn't possible to
automate as far as I can see because mtx cannot see or interact with the
mail slot. Not a huge deal, but something you might want to know. The
workflow I've described is more commonly used with a single tape drive and
no tape changer, but in the interest of air gapping as soon and as
regularly as possible in my opinion it's probably among the best practices
for routinely removing small single tape backup jobs from the changer.


If you want to do virtual full, copy, or migration jobs from LTO to LTO,
know that you'll need two LTO drives. For these job types there must be an
input device and an output device. Since you're already using bacula for
backup to disk volumes, I THINK you should be able to use the disk backup
as the input device and the LTO drive as an output device. In the case of
having a disk backup set and an LTO drive, I think you could find copy and
migration jobs useful since you could perhaps occasionally run off copy
jobs from your disk backups to tape, then remove your tape media from the
library.

I have read about ransomware that automatically seeks out and attempts to
attack tape backup systems, over-writing tape media. You already mentioned
air gapping, but I must emphasize that actually removing backup media from
the changer is a very good practice in defense against that sort of threat.

We selected SAS, primarily because we wanted to directly contact the
library to the tape drive, and because we didn't know anything substantial
about FC. FC also appeared to be more expensive. When considering SAS
adapters, we were informed by our backup hardware vendor that some SAS
adapters don't properly support tape communication. For that reason, an
ATTO SAS card was recommended. Our cost for that card was around $400 USD.

Our vendor of choice was backupworks.com. I don't have a business
relationship with them or any incentive for recommending them. They've just
been really helpful in selecting hardware and their service has been good
as we ordered additional media. I believe they are usa based, so if you are
not then they might not be the vendor for you.

I will note that some libraries in this space locked away the usage of some
slots and perhaps some features behind upgrade licenses. I didn't like that
idea at all, and didn't select such devices. I have been informed that only
a couple companies are still manufacturing tape drives, only ibm is still
researching new tape drive mechanisms. I believe the qualstar libraries are
actually manufactured in Europe, perhaps in Germany. I have been told that
the libraries are also being manufactured by one company and rebadged
elsewhere. Don't know if true.

Because your goal is to air gap backups, definitely consider two things
regarding which drive you select: cost per TB, and cost per cartridge. As
of a while ago, LTO 8 media was $5 USD / TB (raw). I haven't checked LTO 9
media recently, but I imagine it might be more per TB. Of course, cost per
cartridge might matter a lot if you have otherwise small backups, and if
you want to have many cartridges as part of a routine LTO export / data
versioning / air gap process.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Mar 19, 2024, 9:35 AM Gary R. Schmidt 
wrote:

> On 20/03/2024 00:56, Anders Gustafsson wrote:
> > Hi!
> >
> > We used to use tape a long time ago. Then it was CA Arcserve for Linux,
> ran fine, but CA dropped the product
> > so we switched to Bacula and backing up to disk. The recent surges in
> ransom attacks has made us think of
> > airgapping again and even a resonably small LTO drive would work for our
> immediate needs. So the plan is to
> > get one, plus a SAS card and install in a separate PC and run the
> storage agent on it. At this point I am just
> > fishing for advice and pointers. As I see it, the PC does not even need
> 

Re: [Bacula-users] Permission Issues

[Rob Gerber]

 4.0K Oct 25 10:19 plugins
> drwxr-xr-x  3 root   root   4.0K Dec 28 11:35 scripts
> drwxr-x---  3 bacula bacula 4.0K Mar  6 12:29 working
>
> /opt/bacula/working/:
> total 622M
> drwxr-x---  3 bacula bacula 4.0K Mar  6 12:29 .
> drwxr-xr-x 11 root   root   4.0K Jul 11  2023 ..
> -rw-r--r--  1 root   root190 Jul 11  2023 BackupCatalog.bsr
> -rw-r-  1 bacula bacula 2.2K Mar  6 12:28 bacula-dir.9101.state
> -rw-r--r--  1 root   root414 Jul 11  2023
> bacula-dir.bacula-dir.-1636748120.mail
> -rw-r-  1 root   root   2.2K Mar  6 12:28 bacula-fd.9102.state
> -rw-r-  1 bacula tape  5 Mar  6 12:29 bacula-sd.9103.pid
> -rw-r-  1 bacula tape   2.2K Mar  6 12:28 bacula-sd.9103.state
> -rw---  1 bacula bacula 622M Mar  5 23:19 bacula.sql
> drwx--  2 bacula root   4.0K May  3  2023 bkp
> -rw---  1 bacula bacula   56 Mar  5 23:19 .my.cnf
> -rw---  1 bacula bacula0 Mar  6 12:07 orac-dir.conmsg
> -rw-r--r--  1 bacula root19K Mar  5 23:19 orac-fd.bsr
>
>
>
>
> Rob Gerber wrote on 3/6/24 13:57:
>
> I am concerned that these backups might not be successful (in the sense
> that they are quite probably not actually backing your catalog up). If the
> catalog export via "/opt/bacula/scripts/make_catalog_backup.pl
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmake_catalog_backup.pl%2F=05%7C02%7Ckm%40mathcs.emory.edu%7C5f4f45453a1e48ddc3e208dc3e0f5d81%7Ce004fb9cb0a4424fbcd0322606d5df38%7C0%7C0%7C638453482917134191%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=zYK5okl%2F2Zbm2EHPgxk3OF8dsy%2FpHnc12rpYWZg6bpg%3D=0>
> MyCatalog" is not successful and if bacula cannot access the bacula.sql
> file written to /opt/bacula/working/bacula.sql then I'd be very concerned
> that the backups did not succeed in their purpose.
>
> Obviously you likely hold said concern because you messaged the list. I
> just want to make that clear.
>
> While I don't know everything one would need to know to troubleshoot this
> issue, I want to check something:
>
> When you do an 'ls -lah /opt/bacula/ /opt/bacula/working/' what is your
> output?
>
>
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
> On Wed, Mar 6, 2024, 12:29 PM Ken Mandelberg  wrote:
>
>> I I notice these two permission errors in my logs. I'm on Ubuntu. What do I 
>> need to do to correct them.
>> The backups succeed but I guess they are missing info.
>>
>> 05-Mar 23:19 orac-dir JobId 7661: shell command: run BeforeJob 
>> "/opt/bacula/scripts/make_catalog_backup.pl 
>> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmake_catalog_backup.pl%2F=05%7C02%7Ckm%40mathcs.emory.edu%7C5f4f45453a1e48ddc3e208dc3e0f5d81%7Ce004fb9cb0a4424fbcd0322606d5df38%7C0%7C0%7C638453482917134191%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C=zYK5okl%2F2Zbm2EHPgxk3OF8dsy%2FpHnc12rpYWZg6bpg%3D=0>
>>  MyCatalog"05-Mar 23:19 orac-dir JobId 7661: BeforeJob: mysqldump: Error: 
>> 'Access denied; you need (at least one of) the PROCESS privilege(s) for this 
>> operation' when trying to dump tablespaces
>> 05-Mar 23:19 orac-dir JobId 7661: Start Backup JobId 7661, 
>> Job=BackupCatalog.2024-03-05_23.10.00_05
>> 05-Mar 23:19 orac-dir JobId 7661: Connected to Storage "File" at 
>> localhost:9103 with TLS
>> 05-Mar 23:19 orac-dir JobId 7661: Using Device "FileStorage" to write.
>> 05-Mar 23:19 orac-dir JobId 7661: Connected to Client "orac-fd" at orac:9102 
>> with TLS
>> 05-Mar 23:19 orac-fd JobId 7661: Connected to Storage at localhost:9103 with 
>> TLS
>> 05-Mar 23:19 orac-sd JobId 7661: Volume "Vol0010" previously written, moving 
>> to end of data.
>> 05-Mar 23:19 orac-sd JobId 7661: Ready to append to end of Volume "Vol0010" 
>> size=52,690,472,345
>> 05-Mar 23:19 orac-sd JobId 7661: Elapsed time=00:00:02, Transfer rate=325.8 
>> M Bytes/second
>> 05-Mar 23:19 orac-sd JobId 7661: Sending spooled attrs to the Director. 
>> Despooling 233 bytes ...05-Mar 23:19 orac-dir JobId 7661: Error: Could not 
>> open WriteBootstrap file:/opt/bacula/working/BackupCatalog.bsr: 
>> ERR=Permission denied05-Mar 23:19 orac-dir JobId 7661: Error: Bacula 
>> Enterprise orac-dir 13.0.3 (02May23):
>>   Build OS:   x86_64-pc-linux-gnu-bacula-enterprise ubuntu 22.04
>>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>> <https://nam11.safelinks.protection.out

Re: [Bacula-users] Permission Issues

[Rob Gerber]

I am concerned that these backups might not be successful (in the sense
that they are quite probably not actually backing your catalog up). If the
catalog export via "/opt/bacula/scripts/make_catalog_backup.pl MyCatalog"
is not successful and if bacula cannot access the bacula.sql file written
to /opt/bacula/working/bacula.sql then I'd be very concerned that the
backups did not succeed in their purpose.

Obviously you likely hold said concern because you messaged the list. I
just want to make that clear.

While I don't know everything one would need to know to troubleshoot this
issue, I want to check something:

When you do an 'ls -lah /opt/bacula/ /opt/bacula/working/' what is your
output?


Robert Gerber
402-237-8692
r...@craeon.net

On Wed, Mar 6, 2024, 12:29 PM Ken Mandelberg  wrote:

> I I notice these two permission errors in my logs. I'm on Ubuntu. What do I 
> need to do to correct them.
> The backups succeed but I guess they are missing info.
>
> 05-Mar 23:19 orac-dir JobId 7661: shell command: run BeforeJob 
> "/opt/bacula/scripts/make_catalog_backup.pl MyCatalog"05-Mar 23:19 orac-dir 
> JobId 7661: BeforeJob: mysqldump: Error: 'Access denied; you need (at least 
> one of) the PROCESS privilege(s) for this operation' when trying to dump 
> tablespaces
> 05-Mar 23:19 orac-dir JobId 7661: Start Backup JobId 7661, 
> Job=BackupCatalog.2024-03-05_23.10.00_05
> 05-Mar 23:19 orac-dir JobId 7661: Connected to Storage "File" at 
> localhost:9103 with TLS
> 05-Mar 23:19 orac-dir JobId 7661: Using Device "FileStorage" to write.
> 05-Mar 23:19 orac-dir JobId 7661: Connected to Client "orac-fd" at orac:9102 
> with TLS
> 05-Mar 23:19 orac-fd JobId 7661: Connected to Storage at localhost:9103 with 
> TLS
> 05-Mar 23:19 orac-sd JobId 7661: Volume "Vol0010" previously written, moving 
> to end of data.
> 05-Mar 23:19 orac-sd JobId 7661: Ready to append to end of Volume "Vol0010" 
> size=52,690,472,345
> 05-Mar 23:19 orac-sd JobId 7661: Elapsed time=00:00:02, Transfer rate=325.8 M 
> Bytes/second
> 05-Mar 23:19 orac-sd JobId 7661: Sending spooled attrs to the Director. 
> Despooling 233 bytes ...05-Mar 23:19 orac-dir JobId 7661: Error: Could not 
> open WriteBootstrap file:/opt/bacula/working/BackupCatalog.bsr: 
> ERR=Permission denied05-Mar 23:19 orac-dir JobId 7661: Error: Bacula 
> Enterprise orac-dir 13.0.3 (02May23):
>   Build OS:   x86_64-pc-linux-gnu-bacula-enterprise ubuntu 22.04
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] HP Store Ever MSL 1/8 autocharger LTO-8

[Rob Gerber]

I cannot vouch for your library /drive, but bacula certainly should work
with them. Changer must work with mtx command in Linux (or apparently chio
in bsd). That is how tape changes are done by bacula.

Bacula certainly provides that core functionality you described.

You will want to use the bacula project repo to get your binaries, not the
system provided repos. Just so you know, on rocky Linux 9 I had to enable
deprecated sha1 hash function in the package manager because the bacula
binaries are hashed by sha1, even the recent ones like 13.0.3 (I think I'm
getting that version number right)

Hopefully the bacula developers start hashing their binaries with sha256 /
sha512 so such configuration change is no longer necessary.

Robert Gerber
402-237-8692
r...@craeon.net

On Sat, Mar 2, 2024, 10:52 AM enrico.becche...@gmail.com <
enrico.becche...@gmail.com> wrote:

>
> Dear all, I need to build a backup system and I would like to use Bacula.
> My project involves an HP Store Ever MSL 1/8 library with an LTO-8 fiber
> channel drive. The Bacula server will be an HP Proliant DL380 Gen8 running
> Almalinux 8 or 9. My idea is to have automatic tape management with Bacula
> sending the tape change command when needed, and the library identifying
> the correct cartridge using the barcode. Do you think Bacula can handle all
> of this with the hardware I have specified?
> Thanks a lot
> Best Regards
> Enrico
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Overland-Tandberg TD-LTO9xSA LTO9HH SAS External Tape Drive Kit

[Rob Gerber]

My guess is that this will probably work with Bacula but if spending
substantial money (which I imagine an LTO 9 drive costs substantial money),
I'd check for other more informed opinions.

My general understanding is that bacula works with most modern tape drives.
There aren't many drive manufacturers around anymore so isn't a lot of
hardware variance to track.

I think if drive works with your system probably works fine with Bacula.

For a changer, which I think you will not have with your system, it mainly
needs to work with the mtx command (I think same command is available on
bsd?).

Robert Gerber
402-237-8692
r...@craeon.net

On Thu, Feb 29, 2024, 7:40 AM Andrea Venturoli  wrote:

> Hello.
>
> Anyone using this hardware?
> Does it work well for you?
>
> I'm in need of a tape drive for a FreeBSD machine and I've been offered
> this one.
> Any comments welcome.
>
>   bye & Thanks
> av.
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] (no subject)

[Rob Gerber]

There should be a link saying to click "here" in the middle of the page
that you are redirected to after entering your information.

The email has been broken on and off for a while now. I actually thought it
was fixed.

Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Feb 26, 2024, 2:06 PM Aimé Bamo  wrote:

> I'm unable to receive the sent mail to download Bacula
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] backing up cifs-shares

[Rob Gerber]

In bacularis, when you go to volumes does the tape show as "inchanger=
yes"? If not, you need to do a scan of the changer inventory slots
(possible there in bacularis on volumes page). I forget what the button
says but it's at the top of volumes page. I select our changer and specify
to scan slots 1-24. You'd probably select your changer and type 1-8 for
which slots to scan since iirc you have an 8 bay changer.

Also in bacularis, what does the does the job itself say after running job?
Does job hang and wait forever saying "please load a tape" (i am
paraphrasing the message it will give)? Or does job fail automatically and
end the job running state?



Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Feb 26, 2024, 11:48 AM Stefan G. Weichinger  wrote:

> Am 26.02.24 um 14:25 schrieb Rob Gerber:
> > Mixing topics is ok. Better in this case. All the information is in one
> > email thread.
>
> fine
>
> > I have read that if you use mt to rewind the tape and then write eof to
> > the tape (end of file), it will fool bacula into thinking that the tape
> > is empty. If you want to wipe the bacula catalog in your postgres
> > server, there should be scripts that do this in /opt/bacula/scripts
> > (default location, maybe different on debian).
> >
> > The Mt command is something like
> > mt -f /dev/yourtapedrive rewind
> > mt -f /dev/yourtapedrive -write eof
> > Check the command syntax I only vaguely remember them.
>
> Used that already before, yes, thanks
>
> > If not clearing bacula database prob need to purge volumes from catalog
> > (or maybe delete). Remember, these commands are dangerous and have sharp
> > edges. After using the Mt command may need to relabel tape.
>
> currently a job seems to hang:
>
> 26-Feb 18:33 debian1-sd JobId 34: Warning: mount.c:216 Open of Tape
> device "HP-Ultrium4" (/dev/nst0) Volume "Vol07" failed:
> ERR=tape_dev.c:170 Unable to open device "HP-Ultrium4" (/dev/nst0):
> ERR=Kein Medium gefunden
>
> 26-Feb 18:43 debian1-sd JobId 34: Warning: mount.c:216 Open of Tape
> device "HP-Ultrium4" (/dev/nst0) Volume "Vol07" failed:
> ERR=tape_dev.c:170 Unable to open device "HP-Ultrium4" (/dev/nst0):
> ERR=Kein Medium gefunden
>
> I don't know why. Tests with "btape" are completely fine.
>
> That Volume seems not to exist on tape but somewhere in the DB or so.
>
> I would expect the software to skip that and look for another usable tape.
>
> Is there a check routine possible *before* starting the job?
>
> It dumps ~120GB data into the spooling area, then detects that the tape
> isn't there and hangs ... when I stop the job, the spooled data is deleted.
>
> How can I fix that? thanks!
>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] backing up cifs-shares

[Rob Gerber]

Mixing topics is ok. Better in this case. All the information is in one
email thread.

26-Feb 12:41 debian1-sd JobId 31: Warning: For Volume "08L4":
The number of files mismatch! Volume=4 Catalog=3
Correcting Catalog
This message is fairly normal if you restore your catalog backup and lose
some information on backups that exist. There is catalog information
encoded on disk with each backup.

26-Feb 12:41 debian1-sd JobId 31: Error: Backspace record at EOT failed.
I have not seen this error before, but I am not extremely experienced with
Bacula. I suspect your theory is correct.

Warning: destructive / dangerous process ahead.

I have read that if you use mt to rewind the tape and then write eof to the
tape (end of file), it will fool bacula into thinking that the tape is
empty. If you want to wipe the bacula catalog in your postgres server,
there should be scripts that do this in /opt/bacula/scripts (default
location, maybe different on debian).

The Mt command is something like
mt -f /dev/yourtapedrive rewind
mt -f /dev/yourtapedrive -write eof
Check the command syntax I only vaguely remember them.

If not clearing bacula database prob need to purge volumes from catalog (or
maybe delete). Remember, these commands are dangerous and have sharp edges.
After using the Mt command may need to relabel tape.

Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Feb 26, 2024, 5:49 AM Stefan G. Weichinger  wrote:

> Am 24.02.24 um 20:53 schrieb Stefan G. Weichinger:
>
> > I was already able to get the windows-client to work by
> > upgrading/reinstalling the bacula-server part (basically starting from
> > scratch ... didn't matter much, but was a bit of work).
> >
> > The server was older than the client, that seems to have lead to the
> > mentioned error.
> >
> > I also took the chance to get rid of baculum and install bacularis.
>
> The windows-client part seems to work now, but by "losing" my
> postgres-DB/catalog in the process of upgrading/reinstalling I have
> issues with the tapes/volumes.
>
> Is there a fast way to start from scratch? I re-labelled volumes,
> updated slots, edited tapes to status "Append" etc, but Bacula seems
> confused by the tapes ;-)
>
> Stuff like this right now:
>
>
>
> 26-Feb 12:39 debian1-sd JobId 31: Volume "08L4" previously written,
> moving to end of data.
> 26-Feb 12:41 debian1-sd JobId 31: Warning: For Volume "08L4":
> The number of files mismatch! Volume=4 Catalog=3
> Correcting Catalog
> 26-Feb 12:41 debian1-sd JobId 31: New volume "08L4" mounted on
> device "HP-Ultrium4" (/dev/nst0) at 26-Feb-2024 12:41.
> 26-Feb 12:41 debian1-sd JobId 31: Error: Backspace record at EOT failed.
> ERR=Eingabe-/Ausgabefehler
> 26-Feb 12:41 debian1-sd JobId 31: End of medium on Volume "08L4"
> Bytes=53,687,079,936 Blocks=0 at 26-Feb-2024 12:41.
> 26-Feb 12:41 debian1-sd JobId 31: 3307 Issuing autochanger "unload
> Volume 08L4, Slot 2, Drive 0" command.
> 26-Feb 12:45 debian1-sd JobId 31: 3304 Issuing autochanger "load Volume
> 09L4, Slot 3, Drive 0" command.
> 26-Feb 12:46 debian1-sd JobId 31: 3305 Autochanger "load Volume
> 09L4, Slot 3, Drive 0", status is OK.
> 26-Feb 12:46 debian1-sd JobId 31: Volume "09L4" previously written,
> moving to end of data.
>
> I have to add that I edited MaximumFileSize from 20GB to 2GB after
> reading the info. Maybe the tapes I wrote with the larger value aren't
> "compatible" now anymore?
>
> Sorry for mixing topics and asking FAQs ... still learning many details.
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] backing up cifs-shares

[Rob Gerber]

Hello Stefan!

I have a server that I back up via bacula that I can only access via SMB.
It is a high-end NAS appliance running gentoo linux, with absolutely no
shell access by anyone but the vendor who provides it. The appliance does
its job well, but I have no ability to run an FD on it, nor should I since
it is a highly tuned and customized environment. As such, I back it up via
SMB to LTO-8. This works fairly well.

I will emphasize that best practice as I understand it is to use a local
bacula FD whenever possible. So the windows FD you located may be a good
choice. However, you'll have to experiment with it as I have no experience
with windows FD software.

I have not found any issue backing up multiple shares in one job/fileset
definition. Ultimately I found that all shares in one job was best for my
case. Keep in mind for ongoing backups that if some data is static and
doesn't change much and other data changes very frequently, you might want
to back those different datasets up in different jobs so the large static
body of data isn't frequently written to tape again and again as part of
full jobs, when you could be running rare incrementals vs the large body of
static data and frequent fulls, differentials, and incrementals against
frequently changing data. Might not apply to your case, but I believe that
reasons to split share data among different jobs would primarily be for
other reasons besides bacula limitations. Bacula can back up the shares as
part of a single fileset just fine.

I did find that when backing up "large static dataset that doesn't change
often" separately from "small dataset that changes often" was slightly more
inconvenient when doing general data restores, because I needed to restore
data from both jobs/filesets/pools and the files from the small active
dataset weren't included in the large static dataset. I wound up adding the
shares for the small dataset to the fileset for the large dataset, but
didn't increase the backup frequency for the large dataset backup job. This
wouldn't be suitable for routine backups of the small dataset, but because
it is so small it really cost me nothing to back it up twice, routinely and
often in the job/fileset/pool for the small active dataset, and
occasionally for the large, static, and mostly unchanging dataset. Small
tweak, but convenient.

Good job using a mountpoint script! without that, bacula will probably
happily back up /mnt/s01/sharename even if unmounted, report "nothing to
back up! mission accomplished!" and exit 0. This sort of thing is one point
in favor of running a local windows fd (if they work well - i don't know
any details about them).

regarding fileset elegance, for includes you could simply use  a single
line: /mnt/bacula/s01/
After all, this folder contains all the mountpoints and should be
sufficient. Bacula will be backing up with the perspective that the file
paths all start with /mnt/bacula/s01/foo/bar so specifying individual
shares when every share in /mnt/bacula/s01/ is a backup target isn't really
necessary. This only gains you 2 lines. The excludes are much longer, and
I'm not sure there's a way to make them more elegant. You need to exclude
those items, after all.

You should be aware that even if not defined, File, Job, and Volume catalog
records all have default retention periods. If you back data up and don't
define those periods, a retention period will be enforced for you. If the
job records are pruned for a volume, the volume will be automatically
pruned as well. As such, be aware and define those retention periods as you
deem appropriate.

Practice restoring your bacula catalog now. In my general experience, the
restore process is fairly straightforward. You'll need to restore the file
from the backup job defined with the default install. It'll give you a
bacula.sql file. assuming you're using postgres sql, you'll have to run
something like 'psql bacula < bacula.sql'. My command syntax or even the
command used could be inaccurate. VERIFY EVERYTHING, I'm only typing this
from memory. I do recall that the bacula.sql file appeared to contain
everything needed to drop the existing postgres tables, create new ones,
and import all the relevant data.

Know that the bconsole purge and delete commands are DANGEROUS. They tell
you that, what they don't tell you is that there isn't much in the way of
confirmation before they go ahead and delete all your records for poorly
formatted / misunderstood command entries. The same level of care given to
the design when restoring or backing up files wasn't used when designing
the purge command at minimum. I expected some level of confirmation before
it did its business, but two levels in it happily announced "ok! I deleted
all the records associated with the FD you selected!" I was shocked. I was
also unharmed in the end because I knew how to restore my frequently backed
up bacula database.

The purge command is dangerous not just because of what it does (remove
catalog 

Re: [Bacula-users] BAT Installation

[Rob Gerber]

As far as I know, BAT is available only with enterprise bacula.

I believe your choices with regard to bacula community are baculum,
bacularis, and bacula-web.

Bacularis is a friendly fork of baculum, and is the better maintained
package. Developer is on this list.

Bacula-web - I have not tested it, but I hear it provides good reporting,
but no active control. Developer is on this list.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Feb 2, 2024, 1:49 AM  wrote:

> Hi,
>
> I am working off the attached set of instructions for installing Bacula.
> This was obtained from this :
> https://www.bacula.org/packages/x/debs/13.0.3/dists/jammy/main/binary-amd64/INSTALL
> where x was the code sent to me.
>
>
>
> According to this document I should be able to install Bacula
> Administration Tool (BAT) by entering: apt-get install bacula-bat. This
> doesn’t work.
>
>
>
> This is for a server being used for development. It is running Ubuntu
> 22.04.3 LTS on a Dell Optiplex 990 with MySQL, Apache and PHP. Gnome is
> installed.
>
>
>
> The Bacula director version is: Version 13.0.3 (02 May 2023)
> x86_64-pc-linux-gnu-bacula-eneterprise unbuntu 22.04
>
>
>
> although I thought I was running the free community version. The catalog
> database server is MySQL.
>
>
>
> Ubuntu and Bacula are up to date.
>
>
>
> The entry for Bacula in the apt source list is: deb
> https://www.bacula.org/packages/X/debs/13.0.3 jammy main. This is
> taken from the document.
>
>
>
> I had installed Bacularis, but it has been removed. The reason I removed
> it is that I didn’t want to give Apache/PHP access to the backup directory.
>
>
>
> Is BAT still available with the community version and, if so, how do I get
> it installed? I can provide more configuration details if necessary.
>
>
>
> I’d appreciate any help you can provide.
>
>
>
> Regards
>
>
>
> Howard Viccars
>
>
>
> Computer Manager
> Family History ACT
> email: computer...@familyhistoryact.org.au
> url: familyhistoryact.org.au
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] My backup files aren't being reused and my backups are stuck. Help ?

[Rob Gerber]

 don't want to be forced
> to go in manually and remove old backup files and jobs as I wish this to be
> done automatically, as long as my backup retention policy similar to the
> example I gave is followed.
>
> Thanks,
>
> 
> On 2024-01-28 9:57 p.m., Rob Gerber wrote:
>
> I don't know of a way to make bacula backups always fit within a set
> amount of space. This could be dangerous. Depends on how far back you want
> your backup data to go. If such a thing was possible using bacula, you
> wouldn't want bacula to delete the last full backup or you wouldn't have
> any valid backups for a given host. Even assuming bacula could do this and
> was smart enough to not delete the latest full backup, you could encounter
> a situation where you previously had a certain amount of data retention
> (say, 2 months), but as backup sizes increased that time slowly went down
> until you only had 1 week of retention. Could be very dangerous, esp if
> ransomware struck some of your clients. In the event of ransomware, you
> will absolutely want backups going back further than 2 weeks. Also, a usage
> based purge of backup data could be initiated by the ransomware encrypting
> many files on your customer systems, resulting in a rapidly ballooning
> backup size as your server backed up the encrypted files. In a worst case
> scenario, a system configured the way you described could be prompted to
> throw out its last good backups of a system by the influx of data from a
> new ransomware infection.
>
> Now on to discussion of your current issue:
> If I recall correctly, you first started messaging this list within 2023.
> I suspect that no volume is older than 365 days, and as such, no volume
> will be reused, end of story. The volume, file, and job retention options
> are all time based. If you've hit 100GB of utilization before 365 days was
> up, you will need to change one of several things:
>
> Increase available storage space.
> Decrease storage utilization. For instance, maybe do full backups less
> frequently, relying on differential and incremental backups for a longer
> period of time.
> Decrease your volume/job/file retention.
>
> With the exception of increasing your storage space, as far as I know your
> situation requires that you manually purge older volumes. I strongly advise
> caution when doing this. The purge command is DANGEROUS. Running it without
> parameters will delete all your database records for whichever file daemon
> you select. Take catalog backups and restore those backups to a safe place
> before doing anything related to purging or deleting existing volumes.
> Understand the database restore process BEFORE you need it. If possible set
> up a test environment to test any proposed changes of this scale.
>
> Manually changing the volume retention period on a per volume basis might
> make the volumes valid targets for pruning. Not sure. What I am pretty sure
> about is that changing the default volume retention period in your config
> won't retroactively change that value for your volumes that already exist.
>
> Double check everything I've said here. My knowledge is limited, and I may
> be incorrect on some points. Pretty sure on the first paragraph diagnosing
> your issue and your list of options, though.
>
> You can confirm the validity of my diagnosis by checking the dates
> associated with your volumes.
>
> As a matter of opinion, your 1GB volumes seem small, though that's not
> necessarily something I can back up with anything other than opinion. I do
> imagine that smaller 1GB volumes would be easier to export from cloud
> hosted storage (which I think you are using, iirc). Don't feel pressured to
> change this unless it bothers you or you have other knowledge supporting my
> opinion. Changing maximum volume sizes could impact access to and use of
> existing volumes. Be careful.
>
> Regards,
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
>
> On Sat, Jan 27, 2024 at 8:33 AM MylesDearBusiness via Bacula-users <
> bacula-users@lists.sourceforge.net> wrote:
>
>> Hello,
>>
>> I'm having some problems with backup file configuration.
>>
>> I want the backup file area to consume 100GB and no more.
>>
>> I decided on file sizes of 1GB
>>
>> Pool {
>>
>> ...
>>
>>   Volume Retention = 365 days # one year
>>   Maximum Volume Bytes = 1G   # Limit Volume size to something
>> reasonable
>>   Maximum Volumes = 100   # Limit number of Volumes in Pool
>>
>>   Recycle = yes   # Bacula can automatically recycle
>> Volumes
>>   AutoPrune = yes # Prune expired volumes
>>
>> ...
&

Re: [Bacula-users] Bacula purges old jobs and I don't want it to do that

[Rob Gerber]

Radosław, thank you for your reply! Useful information.

Regarding virtual full backups, because I am backing up via my single SAS
LTO8 drive, and because making a copy job requires an input device and an
output device, the information you shared leads me to expect that I am
unable to do virtual full backups with my Bacula community edition unless I
either upgrade to bacula enterprise edition or purchase another LTO8 drive.
My tape changer does have room for such a drive, but the expense would have
to be justified to my client.

I agree with you about manually purging volumes being dangerous and really
not advisable at all. The problem I'm facing is that I don't see a better
way to meet my client's needs otherwise.

Basically, I have two classes of data. The first (and easiest to handle)
consists of project files and minor media files that the video editors use
in their day to day work. These files are created, changed, and deleted all
the time, and the total dataset is around 2TB. So, very manageable. I have
a nightly backup of these files running on an automated schedule. I think I
will discuss with the client and set a retention period around one year.

The second dataset is the root of my issues here. This dataset consists of
the video files captured by film crews on location. These files are filmed
in 8k, and are very large. Dataset size is around 130TB, and is only
expected to grow. The raw video files should be expected to NEVER change.
Any edits are not applied to the original raws. Effectively, if bit rot or
any edit has changed these files, then this is to be regarded as
corruption. There are smaller downscaled versions of these files (called
"proxies") that do change periodically. The proxies aren't as valuable or
important as the original raw files, as they can always be regenerated from
the raws.

My goal regarding the media share is to back it up manually only, as media
is added. Media will only be added when film crews have gone onsite and
filmed new media, or when an editor has generated or regenerated proxies
for that media. Because system performance is so important to the video
editors, they don't want any backup action to impact their access speeds
when working on the raws. Additionally, none of us want a partially
completed file to be saved to LTO. As such, backups to this media share
will be manually ran, as and when media is added or changes made. Because
the proxies are stored right next to the raws, in the same share and folder
tree, I don't have a good idea so far for detecting comparatively small
changes and backing those up automatically. This leaves me with manual
backup for this large dataset.

My largest concern is that I don't want to lose the original backup of
these essential raw files. I presume that the original backup was correct,
and any subsequent re-backup of that original could include damage from bit
rot or an unwanted change. My incremental backups will capture any wanted
changes, but I don't want the original backup to be eliminated as I reuse
tape. So, I plan to re-use tape very infrequently. I have been thinking
about this problem and I think I might be able to confirm that data on the
media share is valid and not corrupt by a combination of querying the
bacula catalog for any file in the media share which has been backed up
twice (modification date/size changed, so this was a valid filesystem
operation), and then using a catalog to disk verify operation to hash files
presently on disk and determine if any of them have changed due to bit rot.
The only problem is that some raw files may have been deleted by video
editors for valid reasons, but tracking such would be very difficult for
purposes of validating if a deleted file was really unwanted or is missing.
Reusing my tapes would then eliminate backups of the missing file.

My client's stated goal for data retention of this media is 7 years.

I think I could set the retention period for volumes to be around 7 years.
I don't know if this period is doable between full backups. I imagine I'm
going to have to adjust my expectations as I encounter more situations and
learn. Meeting my customer's goals with bacula might involve buying a lot
of tape, or mixing analysis of data on disk with analysis of catalog data
to verify if discarding the old full backup would be safe.

For reference, a full backup of this dataset takes 2-3 weeks at the average
tape write rate of about 170MB/s. For further reference, acquisition of the
raw files is expensive, requiring that professionals be paid to go to
location and capture footage. This places a value on the data, not counting
the cost if we disappoint my client's customer by losing the data for the
shoots they arranged.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Sat, Jan 27, 2024 at 7:45 AM Radosław Korzeniewski <
rados...@korzeniewski.net> wrote:

> Hello,
>
> wt., 7 lis 2023 o 21:57 Rob Gerber  napisał(a):
>
>> To update 

Re: [Bacula-users] My backup files aren't being reused and my backups are stuck. Help ?

[Rob Gerber]

I don't know of a way to make bacula backups always fit within a set amount
of space. This could be dangerous. Depends on how far back you want your
backup data to go. If such a thing was possible using bacula, you wouldn't
want bacula to delete the last full backup or you wouldn't have any valid
backups for a given host. Even assuming bacula could do this and was smart
enough to not delete the latest full backup, you could encounter a
situation where you previously had a certain amount of data retention (say,
2 months), but as backup sizes increased that time slowly went down until
you only had 1 week of retention. Could be very dangerous, esp if
ransomware struck some of your clients. In the event of ransomware, you
will absolutely want backups going back further than 2 weeks. Also, a usage
based purge of backup data could be initiated by the ransomware encrypting
many files on your customer systems, resulting in a rapidly ballooning
backup size as your server backed up the encrypted files. In a worst case
scenario, a system configured the way you described could be prompted to
throw out its last good backups of a system by the influx of data from a
new ransomware infection.

Now on to discussion of your current issue:
If I recall correctly, you first started messaging this list within 2023. I
suspect that no volume is older than 365 days, and as such, no volume will
be reused, end of story. The volume, file, and job retention options are
all time based. If you've hit 100GB of utilization before 365 days was up,
you will need to change one of several things:

Increase available storage space.
Decrease storage utilization. For instance, maybe do full backups less
frequently, relying on differential and incremental backups for a longer
period of time.
Decrease your volume/job/file retention.

With the exception of increasing your storage space, as far as I know your
situation requires that you manually purge older volumes. I strongly advise
caution when doing this. The purge command is DANGEROUS. Running it without
parameters will delete all your database records for whichever file daemon
you select. Take catalog backups and restore those backups to a safe place
before doing anything related to purging or deleting existing volumes.
Understand the database restore process BEFORE you need it. If possible set
up a test environment to test any proposed changes of this scale.

Manually changing the volume retention period on a per volume basis might
make the volumes valid targets for pruning. Not sure. What I am pretty sure
about is that changing the default volume retention period in your config
won't retroactively change that value for your volumes that already exist.

Double check everything I've said here. My knowledge is limited, and I may
be incorrect on some points. Pretty sure on the first paragraph diagnosing
your issue and your list of options, though.

You can confirm the validity of my diagnosis by checking the dates
associated with your volumes.

As a matter of opinion, your 1GB volumes seem small, though that's not
necessarily something I can back up with anything other than opinion. I do
imagine that smaller 1GB volumes would be easier to export from cloud
hosted storage (which I think you are using, iirc). Don't feel pressured to
change this unless it bothers you or you have other knowledge supporting my
opinion. Changing maximum volume sizes could impact access to and use of
existing volumes. Be careful.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Sat, Jan 27, 2024 at 8:33 AM MylesDearBusiness via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> Hello,
>
> I'm having some problems with backup file configuration.
>
> I want the backup file area to consume 100GB and no more.
>
> I decided on file sizes of 1GB
>
> Pool {
>
> ...
>
>   Volume Retention = 365 days # one year
>   Maximum Volume Bytes = 1G   # Limit Volume size to something
> reasonable
>   Maximum Volumes = 100   # Limit number of Volumes in Pool
>
>   Recycle = yes   # Bacula can automatically recycle
> Volumes
>   AutoPrune = yes # Prune expired volumes
>
> ...
>
> }
>
> I enabled the following schedule:
>
> #
> # When to do the backups, full backup on first sunday of the month,
> #  differential (i.e. incremental since full) every other sunday,
> #  and incremental backups other days
> Schedule {
>   Name = "WeeklyCycle"
>   Run = Full 1st sun at 23:05
>   Run = Differential 2nd-5th sun at 23:05
>   Run = Incremental mon-sat at 23:05
> }
>
> ... and what I saw is 100 files written and filled, and then all backups
> stopping in their tracks and no further backups produced.
>
> Every new job errors out.
>
> None of the older backup files were reused.
>
> Backups only resumed when I manually deleted the oldest volumes from
> bconsole, and stopped again when the maximum number of files had been
> reached
>
> I want automatic space management 

Re: [Bacula-users] Using Bacula in a cloud environment with a jumphost

[Rob Gerber]

Could you establish a site-to-site VPN link from your director's lan to the
remote lan that is currently only accessible from the jump host?

If you're concerned about the remote site having access to the central lan
with director on it, you could vlan tag all packets from remote lan VPN and
pass tagged traffic to director server, forbidding other clients.

If need be, maybe modify the idea so that the central director's server has
a site-to-site VPN link to the remote lan. Maybe more difficult to do if
the director doesn't have a public IP (so maybe the remote VPN server will
have difficulty reaching the director to complete the tunnel?) Also, a
network infrastructure link will be maintained on something that isn't a
piece of core network equipment (director server), hiding the configuration
from network admins.

MAYBE, you could give director access to remote lan via standard VPN (one
way, client initiated, road warrior, whichever term means "not site to site
VPN"). You could run into issues with the VPN connection disconnecting.
Maybe solve those issues by having a runbeforejob script that verifies the
tunnel is up, and if it isn't restarts the VPN connection prior to the
backup starting. However, if there's any instance where the clients would
need to reach out to the director, and if the client initiated VPN proves
to be unstable, you could have an issue. I have no reason to believe that
client initiated VPN is unstable, but I guess it's possible. Also you would
probably need to initiate this connection entirely using command line
tools, which I haven't done but imagine is possible using openvpn or
similar.

I'm sure there might be bacula features that cover these eventualities, but
I'm not a big enough bacula expert to know about them.



Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Dec 15, 2023, 3:59 AM Martin Reissner  wrote:

> Hello and sorry for the generic subject. My issue is as follows:
>
> I have a centralized director which should be used to backup several
> setups with multiple clients/fds in a cloud environment. In those setups
> there is only one gateway/jumphost with a public ip, the actual
> clients/fds only have an address in an internal subnet and are reachable
> from outside via ssh-proxyjump from the gw/jumphost or via a loadbalancer.
>
> So far the only solutions I have come up with are portforwardings on the
> gw eg. port 19102 gets forwarded to client1 port 9102, 29102 to client2
> 9102 and so on. This works but is kind of tedious with many clients.
>
> I read something about client initiated backups using the tray monitor.
> I will look into that but scheduling backups on the clients/fds takes
> away one of the main advantages of bacula, which is the centralized
> scheduling.
>
> Are there any further options that I might not have found or thought of?
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restoring a file from an unknown backup

[Rob Gerber]

If you have access to a baculum or bacularis web console you can
graphically browse through files on a job by job basis. Start a restore,
click through. A little slow, in my experience, but if you are totally lost
trying to navigate bconsole and find what you need to that way, maybe this
will help. I suspect bconsole will be much more useful for searching,
though. baculum/bacularis don't have good database search functionality as
far as I can tell.

Bacularis is a friendly fork of Baculum, so they're very close to each
other in features and functions.

baculum is typically hosted on http://yourbaculadirectorserverip:9096
bacularis is typically hosted on http://yourbaculadirectorserverip:909*7*

To determine if bacularis or baculum are running on your bacula server, in
your system shell do:
ps aux|grep bacularis
ps aux|grep baculum

In my case, I am running bacularis, and the ps aux command told me that I
have various http servers running bacularis. I also got a line telling me
that the grep operation was searching for bacularis, which of course is not
an actual bacularis process.

I am not running baculum, so I only got the output for the grep process
filtering ps aux output for "baculum".

I think the bconsole search commands, if you can make them work for you,
will give you more information more quickly. But they will give you what
you ask for, and nothing more. If you ask for the wrong things, or ask in
the wrong way, you won't get the information you need. That is my limited
experience on the subject.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Wed, Dec 13, 2023 at 11:14 AM Rob Gerber  wrote:

> In my experience with Bacula 13.0.3 in bconsole you can do
> restore
>
> option 2 "search for a filename" give it a filename and it will tell you
> every job that filename has ever been backed up in, with complete path
> option 11 "enter a list of directories to restore for found jobs" also
> looks juicy.
>
> You will have to select an FD (File Daemon) as part of restores and some
> of the commands listed under the "restore" command. The FD is the part of
> bacula that (typically) runs on the client machine and sends the files
> selected for backup to the Bacula system for backup. Keep in mind that in
> some odd cases (like mine!) the FD could run on the Bacula server and make
> backups of remote filesystems that are mounted on the bacula server (if you
> cannot run the FD on the client machine for instance).
>
> Remember that providing a period . to bconsole cancels the current command
> and exits back to the bconsole shell.
> . [press enter]
>
> See my bconsole output below.
>
> Connecting to Director nsf-rocky:9101
> 1000 OK: 10002 nsf-rocky-dir Version: 13.0.3 (02 May 2023)
> Enter a period to cancel a command.
> *restore
> Automatically selected Catalog: MyCatalog
> Using Catalog "MyCatalog"
>
> First you select one or more JobIds that contain files
> to be restored. You will be presented several methods
> of specifying the JobIds. Then you will be allowed to
> select which files from those JobIds are to be restored.
>
> To select the JobIds, you have the following choices:
>  1: List last 20 Jobs run
>  2: List Jobs where a given File is saved
>  3: Enter list of comma separated JobIds to select
>  4: Enter SQL list command
>  5: Select the most recent backup for a client
>  6: Select backup for a client before a specified time
>  7: Enter a list of files to restore
>  8: Enter a list of files to restore before a specified time
>  9: Find the JobIds of the most recent backup for a client
> 10: Find the JobIds for a backup for a client before a specified time
> 11: Enter a list of directories to restore for found JobIds
> 12: Select full restore to a specified Job date
> 13: Select object to restore
> 14: Cancel
>
>
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
> On Wed, Dec 13, 2023, 10:48 AM Borut Rozman 
> wrote:
>
>> Hi,
>> I inherited a bacula backup solution, and now I got a request to
>> restore a file or set of files from backups. I only know a folder name,
>> and nothing else. Is there a way inside bconsole to search for a
>> specific string.
>>
>> Query/option 20 does not give me any results.
>>
>> Using bacula 11 with pgsql 14, any help appreciated.
>> B.
>>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restoring a file from an unknown backup

[Rob Gerber]

In my experience with Bacula 13.0.3 in bconsole you can do
restore

option 2 "search for a filename" give it a filename and it will tell you
every job that filename has ever been backed up in, with complete path
option 11 "enter a list of directories to restore for found jobs" also
looks juicy.

You will have to select an FD (File Daemon) as part of restores and some of
the commands listed under the "restore" command. The FD is the part of
bacula that (typically) runs on the client machine and sends the files
selected for backup to the Bacula system for backup. Keep in mind that in
some odd cases (like mine!) the FD could run on the Bacula server and make
backups of remote filesystems that are mounted on the bacula server (if you
cannot run the FD on the client machine for instance).

Remember that providing a period . to bconsole cancels the current command
and exits back to the bconsole shell.
. [press enter]

See my bconsole output below.

Connecting to Director nsf-rocky:9101
1000 OK: 10002 nsf-rocky-dir Version: 13.0.3 (02 May 2023)
Enter a period to cancel a command.
*restore
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"

First you select one or more JobIds that contain files
to be restored. You will be presented several methods
of specifying the JobIds. Then you will be allowed to
select which files from those JobIds are to be restored.

To select the JobIds, you have the following choices:
 1: List last 20 Jobs run
 2: List Jobs where a given File is saved
 3: Enter list of comma separated JobIds to select
 4: Enter SQL list command
 5: Select the most recent backup for a client
 6: Select backup for a client before a specified time
 7: Enter a list of files to restore
 8: Enter a list of files to restore before a specified time
 9: Find the JobIds of the most recent backup for a client
10: Find the JobIds for a backup for a client before a specified time
11: Enter a list of directories to restore for found JobIds
12: Select full restore to a specified Job date
13: Select object to restore
14: Cancel


Robert Gerber
402-237-8692
r...@craeon.net

On Wed, Dec 13, 2023, 10:48 AM Borut Rozman 
wrote:

> Hi,
> I inherited a bacula backup solution, and now I got a request to
> restore a file or set of files from backups. I only know a folder name,
> and nothing else. Is there a way inside bconsole to search for a
> specific string.
>
> Query/option 20 does not give me any results.
>
> Using bacula 11 with pgsql 14, any help appreciated.
> B.
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Is there a containerized version of the Bacula GUI available ?

[Rob Gerber]

Regarding Bacularis, I see I made an error in my previous email (and
previous testing of the bacularis docker containers!).

Bacularis is available in several flavors of docker containers,
including one with all bacula components, and one which only contains the
web interface. I clearly must have tested with the wrong container in the
past.
More info on bacularis docker containers here:
https://bacularis.app/doc/brief/installation.html#install-using-docker

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Tue, Dec 5, 2023 at 4:40 PM Rob Gerber  wrote:

> Certainly!
>
> You have several options for bacula web gui.
> There is Bacula-web, which I haven't used so cannot speak to it. I
> believe Bacula-web is primarily a reporting tool, and not a control tool. I
> haven't examined it closely, so I will have to do so later. Looks
> interesting. The developer is on this mailing list!
> https://www.bacula-web.org/
>
> There is Baculum, which is provided and maintained by the bacula project.
> Find it in the bacula project repo or your OS repo if that's what you used
> to install bacula (I hope you used the bacula project repo).
>
> Finally, there is Bacularis, a friendly fork of Baculum. Bacularis makes
> some UI changes, and pushes some bugfixes up to Baculum as well. I think
> the goal of the Bacularis dev is to make baculum more approachable.
> I use bacularis, though I have used baculum as well in the past. They are
> superficially similar in some ways.
> Bacularis is available in a docker container, though the docker container
> version of the app comes with its own director, sd, and fd. You will need
> to modify the container to point to your own bacula installation.
> I installed bacularis natively (not via container) in rockylinux 9. It
> wasn't terribly difficult, as things go. Slightly more involved than merely
> issuing a package manager install command and flying along (this is also
> true of baculum). Generally after installation of the program, you must
> ensure it has appropriate sudoers access to some bacula files, and tell it
> where the bacula executables are located.
> The bacularis developer is also on this list!
> https://bacularis.app/
>
> I personally recommend bacularis, and might advise against using the
> bacularis docker container unless you're comfortable modifying its
> bacularis instance to point to your bacula install. I might be misguided on
> the necessity or difficulty of this, but when I tried the bacularis docker
> container it came with its own bacula install, and I didn't want the
> complication or potential for error. I'm also not very familiar with docker
> containers, and wanted to go with a native install to avoid any surprises.
>
> When installing bacularis OR baculum, I personally used lighttpd, though
> apache is also supported.
>
> Regards,
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
>
> On Tue, Dec 5, 2023 at 4:20 PM MylesDearBusiness via Bacula-users <
> bacula-users@lists.sourceforge.net> wrote:
>
>> Hello,
>>
>> I'm not a PHP expert and am looking for a lower barrier to entry.
>>
>> Is there a Docker container available holding the entire environment
>> required by the Bacula GUI that I can hook up to a client server?
>>
>> II've already set up director/sd/fd daemons on my cloud server and am
>> looking for an easier way to manage the system.
>>
>>
>> Thanks,
>>
>> 
>>
>>
>>
>>
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Is there a containerized version of the Bacula GUI available ?

[Rob Gerber]

Certainly!

You have several options for bacula web gui.
There is Bacula-web, which I haven't used so cannot speak to it. I
believe Bacula-web is primarily a reporting tool, and not a control tool. I
haven't examined it closely, so I will have to do so later. Looks
interesting. The developer is on this mailing list!
https://www.bacula-web.org/

There is Baculum, which is provided and maintained by the bacula project.
Find it in the bacula project repo or your OS repo if that's what you used
to install bacula (I hope you used the bacula project repo).

Finally, there is Bacularis, a friendly fork of Baculum. Bacularis makes
some UI changes, and pushes some bugfixes up to Baculum as well. I think
the goal of the Bacularis dev is to make baculum more approachable.
I use bacularis, though I have used baculum as well in the past. They are
superficially similar in some ways.
Bacularis is available in a docker container, though the docker container
version of the app comes with its own director, sd, and fd. You will need
to modify the container to point to your own bacula installation.
I installed bacularis natively (not via container) in rockylinux 9. It
wasn't terribly difficult, as things go. Slightly more involved than merely
issuing a package manager install command and flying along (this is also
true of baculum). Generally after installation of the program, you must
ensure it has appropriate sudoers access to some bacula files, and tell it
where the bacula executables are located.
The bacularis developer is also on this list!
https://bacularis.app/

I personally recommend bacularis, and might advise against using the
bacularis docker container unless you're comfortable modifying its
bacularis instance to point to your bacula install. I might be misguided on
the necessity or difficulty of this, but when I tried the bacularis docker
container it came with its own bacula install, and I didn't want the
complication or potential for error. I'm also not very familiar with docker
containers, and wanted to go with a native install to avoid any surprises.

When installing bacularis OR baculum, I personally used lighttpd, though
apache is also supported.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Tue, Dec 5, 2023 at 4:20 PM MylesDearBusiness via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> Hello,
>
> I'm not a PHP expert and am looking for a lower barrier to entry.
>
> Is there a Docker container available holding the entire environment
> required by the Bacula GUI that I can hook up to a client server?
>
> II've already set up director/sd/fd daemons on my cloud server and am
> looking for an easier way to manage the system.
>
>
> Thanks,
>
> 
>
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Please help me to unblock my backup run

[Rob Gerber]

(resending, deleted quoted text from previous messages so my message will
pass the 40kb limit)

Myles,

1. Basically, I suspect rclone filled its cache and bacula stopped the
backup at that time. My guess is that if you were to run a backup of less
than 1GiB right now in bacula, it would succeed. Guaranteed not to fill the
rclone buffer. Maybe adjust rclone cache size or otherwise fiddle with that
portion of it? Definitely exclude the rclone cache (if written to disk)
from the bacula backup!

2. I think there may be a mistake in how your bacula-sd.conf defines the
bacula FileChgr1-Dev1 and FileChgr1-Dev2 devices.

Your bacula virtual devices (FileChgr1-Dev1 and FileChgr1-Dev2) point to
actual files on the system. Mine in the default config (working, backs up
bacula database routinely) points to the folder that holds volume files. In
my configuration the volume filenames are never defined in the FileChgr*
resources.

Did you create these files yourself?
root@c1:~# ls -l /mnt/MylesDearDropBox/Backup/bacula/archive/
total 20971544
-rw-r--r-- 1 root root 21474860756 Dec 4 03:27 MylesMpwrware1
-rw-r--r-- 1 root root 0 Dec 4 03:04 MylesMpwrware1.l
-rw-r--r-- 1 root root 0 Dec 4 01:00 MylesMpwrware2

I would expect bacula to create volume files itself, and that it doesn't
need to be told what the volume names will be (except for "Vol-" defined in
configuration elsewhere, as you've noted).

in your bacula-sd.conf, for both devices (FileChgr1-Dev1 and
FileChgr1-Dev2) I recommend adjusting your ArchiveDevice=  line to read the
same:
ArchiveDevice=  /mnt/MylesDearDropBox/Backup/bacula/archive # no trailing
foreward slash! "/"

FileChgr1-Dev1 and FileChgr1-Dev2 should both point to the same folder!
There are two such devices defined in part so that one could read
information from a volume and another could write to another volume, such
as in a copy or migration job. There may be other reasons, but in any case
the FileChgr1-Dev* devices should certainly point to the same folder.

In your current configuration, I think the Vol-0001 and Vol-0002 are labels
written to the volumes themselves internally, for bacula's reference. If
these were tapes, bscan or some other bacula tool could read the tapes
(volumes) and find the labels. The label name internally to bacula doesn't
necessarily match the actual filename or label written on the tape.
However, it usually DOES match in a default configuration, like mine. I
think once you make this change to the ArchiveDevice paths and let bacula
create volumes, the bacula labels will match the filenames of the volumes.

In the case of Vol-0002 it might be that bacula intended to write to the
volume but couldn't create it because FileChgr1-Dev1 and FileChgr1-Dev2
both point to actual files and NOT to a folder. Not sure.

For reference, here's what I see on my bacula system. See how I have files
in /opt/bacula/archive/
[root@NSF-rocky]# find /  -path /mnt -prune  -o -type f -print | grep
"Vol-0"
/opt/bacula/archive/Vol-0001
/opt/bacula/archive/Vol-0040

[root@NSF-rocky EVO-Media]# ls -lah /opt/bacula/archive/
total 15G
drwxrwxr-x+  3 bacula disk   4.0K Oct 20 12:25 .
drwxrwxr-x+ 11 root   root   4.0K Sep 13 14:17 ..
-rw-r-.  1 bacula disk   1.5G Sep  6 12:25 Vol-0001
-rw-r-.  1 bacula disk13G Dec  4 05:45 Vol-0040

in bconsole:
list volumes pool=File
Pool: File
+-++---+-++--+--+-+--+---+---+-+--+-++
| mediaid | volumename | volstatus | enabled | volbytes   | volfiles |
volretention | recycle | slot | inchanger | mediatype | voltype | volparts
| lastwritten | expiresin  |
+-++---+-++--+--+-+--+---+---+-+--+-++
|   1 | Vol-0001   | Read-Only |   1 |  1,248,082,548 |0 |
  31,536,000 |   1 |0 | 0 | File1 |   1 |0
| 2023-06-27 23:10:02 | 17,729,032 |
|  23 | Vol-0022   | Purged|   1 |  0 |0 |
  31,536,000 |   1 |0 | 0 | File1 |   1 |0
| |  0 |
|  24 | Vol-0024   | Read-Only |   1 |  0 |0 |
  31,536,000 |   1 |0 | 0 | File1 |   1 |0
| |  0 |
|  25 | Vol-0025   | Read-Only |   1 |  0 |0 |
  31,536,000 |   1 |0 | 0 | File1 |   1 |0
| |  0 |
|  26 | Vol-0026   | Read-Only |   1 |  0 |0 |
  31,536,000 |   1 |0 | 0 | File1 |   1 |0
| |  0 |
|  27 | Vol-0027   | Error |   1 |  0 |0 |
  31,536,000 |   1 |0 | 0 | File1 |   1 |0

Re: [Bacula-users] Please help me to unblock my backup run

[Rob Gerber]

Maybe Dropbox or rclone or some combination of the two are limiting you to
1GiB file sizes?

In fact, for your rclone process I see it has a 1GB cache size limit. "
--vfs-cache-max-size 1G" I bet in the case of the dd command you did, we
filled the write cache and then dd exited. If the cache was larger or the
input command was rate limited, we might not have that issue. Maybe if
bacula backups took longer in some cases you wouldn't run into this
problem. Bacula does have a bandwidth rate limit feature, but I'd work on
the cache size or a more graceful failure mode first (like "is cache full?
Make bacula wait a while" - admittedly something I don't know to be
possible).


By default, Bacula's file based backup writes to file volumes. Think of it
as being like writing the backed up field and directories to tar or zip
files - the files and directories that are backed up by bacula are stored
in single large archive files, using Bacula's own file format.
Conceptually, Bacula isn't using the tar format, but the bacula file
writers (by default) are using something like a tar file. I think it's done
this way because way back when Kern started developing bacula, the original
destination for the backups was a tape drive. Later hard drives became
cheaper and Kern realized that bacula could also write to "file volumes"
that were stored on a hard drive. This means some customers who couldn't
afford a tape drive but could afford a larger hard drive could use bacula.
Bill recently mentioned that bacula can write backed up files and
directories to some cloud storage solutions directly, so I think the file
volume method isn't used in every case by bacula, but without special
configuration on your part bacula is probably using these file volumes as
described above.

Please do the following, probably as root:
sudo find / |grep -i Vol-0

Also please do
find /mnt/MylesDearDropBox/Backup/bacula/archive/

and please do
df -h

Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Dec 4, 2023, 1:58 PM MylesDearBusiness  wrote:

> Hi, Rob,
>
> Thanks for the response.
>
> 1.
> I'm only using 25% of my 2TB Dropbox account, so I don't expect storage
> to be full.
>
> This particular cloud server is tiny, just a single CPU, 50GB storage,
> 2GB RAM.
>
> The biggest file I managed to write successfully to my rclone/Dropbox
> mount is 1GB:
>
> When I tried to write a bigger file, I got an "out of memory" error, in
> hindsight I suppose this was to be expected.
> I'm trying to keep costs down by renting only a very small cloud machine
> until such time I need the capacity increase.
>
> root@c1:~# dd if=/dev/urandom
> of=/mnt/MylesDearDropBox/Backup/someuniquefilename.img bs=1G count=1
> 1+0 records in
> 1+0 records out
> 1073741824 bytes (1.1 GB, 1.0 GiB) copied, 9.12953 s, 118 MB/s
> root@c1:~# ls -l /mnt/MylesDearDropBox/Backup/someuniquefilename.img
> -rw-r--r-- 1 root root 1073741824 Dec  4 19:31
> /mnt/MylesDearDropBox/Backup/someuniquefilename.img
> root@c1:~#
>
>
> so I'll tune down my bacula director config for max file size of 1G.
>
> 2. I'm still confused by what exactly "Vol-xxx" is supposed to be, I see
> there are config settings for setting this name, but I only create the
> device files MylesMpwrware and point to them in in bacula-sd
> configuration as "Archive Device".  Should I also be creating the
> "Vol-xxx" files as well?   I did see the first of my "Archive Device"
> files filling up:
>
> root@c1:~# ls -l /mnt/MylesDearDropBox/Backup/bacula/archive/
> total 20971544
> -rw-r--r-- 1 root root 21474860756 Dec  4 03:27 MylesMpwrware1
> -rw-r--r-- 1 root root   0 Dec  4 03:04 MylesMpwrware1.l
> -rw-r--r-- 1 root root   0 Dec  4 01:00 MylesMpwrware2
> root@c1:~#
>
>
> I'm sure with a little more banging my head against the wall things will
> start to make sense.
>
> Thanks,
>
> 
>
> On 2023-12-04 2:26 p.m., Rob Gerber wrote:
> > dd if=/dev/urandom
> > of=/mnt/yourdropboxmountpoint/someuniquefilename.img bs=50G count=1
>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Please help me to unblock my backup run

[Rob Gerber]

Myles,

Some thoughts (apologies if I missed something obvious in your GitHub post):

1. I recommend testing your setup to verify that a 50gb file can be stored
the way you think it can. Maybe storage is full. Maybe it is rate limiting
you. Maybe there is a maximum file size set somewhere. To test for this and
maybe some other possibilities, as whichever appropriate user, try writing
a (mostly true random) 50gb file to the Dropbox storage. In my mind
randomness is important here because it will evade any possible
compression, or deduplication.
dd if=/dev/urandom of=/mnt/yourdropboxmountpoint/someuniquefilename.img
bs=50G count=1

WARNING: DD WILL SILENTLY OVERWRITE ANY TARGET SPECIFIED UNDER "of="! BE
CERTAIN THAT DD'S OUTPUT TARGET IS UNIQUE/DOESN'T EXIST/ISN'T IMPORTANT!

2. Is Vol-001 really stored where you think it is? Please post an 'ls -lah'
of the storage location where Vol-001 is stored.

If the volume isn't being stored where you told Bacula to store it, has
Bacula been restarted since you last told it where to store the volume?
Maybe Bacula hasn't loaded the new configuration.

Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Dec 4, 2023, 10:43 AM MylesDearBusiness via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> Hello again,
>
> I'm using a cloud server with rclone / Dropbox back end (which is working).
>
> I'm having trouble with a stuck Bacula run.  I have ample storage space
> but Bacula appears to be having trouble creating additional volumes.  I
> have one volume created, which was sized to a maximum of 50G, but
> appears to have bottomed out at around 30G.
>
> I want to be able to back up my entire server without any blockages, and
> to save multiple daily/weekly/monthly backups.
>
> As I've been receiving "message too long" errors from the mailing list
> server, I have placed most of the details in the following link (sorry
> for the inconvenience) :
>
> https://gist.github.com/mdear/1f15e51584d17d070cb13290a48419d7
>
> Can you help me get unstuck ?  Any concepts I'm missing?  Any
> extra/missing configuration ?
>
>
> Thanks,
>
> 
> >
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Error closing volume

[Rob Gerber]

How much free space left on the hard disk that stores the volumes?

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Dec 1, 2023, 1:33 PM Senor Pascual  wrote:

> Hello everyone,
>
> [SI0202] End of Volume "Backup-test-0050" at 1201665184 on device "test"
> (/mnt/test). Write of 64512 bytes got 2912.
> Error: Error closing volume "Backup-test-0050" device "test" (/mnt/test).
> ERR=No space left on device.
>
> I am having these errors in the volumes lately. I don't relate it to
> anything. I have not modified anything in terms of versions of services,
> storage configuration, volumes, pools, etc... The volumes are being stored
> on a hard disk.
> Previously this error did not happen and they were created or purged
> adding information in them without any problem.
>
> I started to jump these errors for no reason. Can anyone tell me about
> this particular error? I have not found anything in the email list about
> this error.
>
> Versions:
> Bacula 13.0.3 director, storage and client.
> PostgreSQL 16 Catalog
>
> I hope someone can help me. Thanks. Best regards,
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"

[Rob Gerber]

Myles,

Just a brief note: Bacula uses "virtual tape" volumes, which are files that
bacula mounts internally and writes backup data to. If a given volume is
unreachable/unusable/full/etc bacula will attempt to create another similar
volume in the designated location.

Again, the volume will be a file, which will contain the files Bacula has
backed up. Bacula volumes are written to using Bacula's own format. In the
event of a disaster (loss of your bacula catalog database), you can use the
bacula bscan tool to scan the volume(s) (or tapes if you use actual
magnetic media tape), which will rebuild a catalog database.

Are you sure bacula has write permissions for the destination directory,
and are you sure bacula is using the user you think it is?

I haven't heard of a bacula director and storage daemon using different
user credentials before. Doesn't mean it won't supported, but I haven't
heard of that.

What happens when you temporarily set the target directory to be accessible
by any user? (chmod -R 777 /mnt/my_backups)

Does the find command + grep locate any of the volume names given by
bacula? (find /|grep -i volume name here)


Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Nov 28, 2023, 2:29 PM MylesDearBusiness via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> Thanks for the reply, Bill.
>
> First of all, Bacula seems to be looking for a different volume name
> every time, Vol-0014 doesn't match any folder I created (and the storage
> space, minus a few boilerplate files and directories, was empty
> initially as I would expect a newly configured cloud storage service to
> be).  I'm not quite understanding where Vol-0014 and other similarly
> named volumes are coming from.  All I know is that I have a curlftpfs
> based userspace mounting daemon running that presents remote storage
> using a familiar file-system-based integration and that I'm trying to
> point Bacula to back up into it.
>
> I originally created the local directory /mnt/my_backup as a directory
> owned by backupuser:backupuser and then under user backupuser I mounted
> my curlftpfs remote storage space under that directory.
>
> Only backupuser has the rights to read from and write to this directory.
>
> As shown in my gist copied again below, I'm only running the Director
> under the bacula username.
>
> I'm running the SD and FD processes under the backupuser username
> because that's the username I gave sole permission to access the storage
> mount.
>
> I also added more detail to the gist link to try to address some of your
> questions :
>
> https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b
>
> More help is needed and help already given is much appreciated,
>
> Thanks,
>
> 
>
>
> On 2023-11-27 7:18 p.m., Bill Arlofski via Bacula-users wrote:
> > On 11/27/23 15:26, MylesDearBusiness via Bacula-users wrote:
> >> Hello, Bacula experts.
> >>
> >> My cloud provider offers only curlftpfs based storage, which I have
> enabled and secured.  I created a username “backupuser”
> >> and a system service that launches the service, effectively mounting
> the remote storage to /mnt/my_backup and running the
> >> userspace curlftpfs command as the user “backupuser”.  This user has
> demonstrable read/write privileges in the file system as
> >> expected.  So far so good.
> >>
> >> I installed bacula on my Ubuntu 22.04.3 LTS server and got bacula-dir,
> bacula-fd, and bacula-sd all running.
> >>
> >> In summary, when I try to run my backup job I get an error:
> >>
> >> Device is BLOCKED waiting for mount of volume
> >>
> >> ChatGPT4 doesn’t know nearly as much as this august body, I’m hoping a
> kindly member may be able to give me a hand up.
> >>
> >>
> >> More details:
> https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b
> >>
> >>Thanks,
> >>
> >> 
> >
> > What does this show?:
> >
> > # ls -la /mnt/my_backup/backup/bacula/archive
> >
> >
> > Is there a Bacula File volume named `Vol-0014` in there?
> > Is it rw for the `bacula` user which the SD (normally) runs as?
> >
> > If it is not there, but you know where it is, them you must move it
> there, and set the ownership to `bacula:disk` and the
> > permissions to allow the bacula user to, well... Read and write to it. :)
> >
> > It is just a guess, but is it possible that this Bacula file volume
> lives under a directory which you mounted
> > `/mnt/khapbackup/backup/bacula/archive` on top of, so it exists, but is
> not visible currently?  This is just a guess, but it
> > is also a common mistake. :)
> >
> > If you truly do not know where this Bacula file volume is, then you need
> to delete it from the catalog so the director no
> > longer thinks it is accessible:
> >
> > * delete yes volume=Vol-0014
> >
> > Might want to run the bconsole `query` command and select option 14 to
> be sure there are no jobs on it that you might need.
> >
> > Once deleted from the catalog, the Director should select a new volume
> (or create a new one if `LabelFormat = 

Re: [Bacula-users] Q: how to locate files that generate verify differences?

[Rob Gerber]

What level of verify job are you running (data or the other one, can't
recall name), and are you using accurate mode?

I got verify failures in the past when I was verifying old jobs with the
verify option that isn't data, or when I verified with level=data and
accurate=yes. The message I got was similar to yours - verify failure and
no files listed as bad.

The verify job type I can't recall is only for verifying the most recent
job. It checks file consistency and catalog information on the volume to be
sure it is the most recent. Obviously a backup that isn't the most recent
in the chain will have outdated catalog information, so that level of
verify will fail.

The gotcha here is that a data level verify ran with accurate=yes is
equivalent to the other type of verify - it will then check volume catalog
entries in addition to file integrity.

For any job which you cannot be certain is the most recent job, I recommend
level=data and accurate=no. Must specify jobid and run for each job.

Alternative is to run the other type of verify after each backup job, or at
least while that job is the newest and most recent job.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Nov 10, 2023, 7:41 AM Justin Case  wrote:

> Hi there,
>
> I am doing a verify for each backup job. The verify happens no immediately
> after the backup, but on the same day for sure.
>
> For some hosts I do have verify differences, and for FDs version 9.x I get
> reports on which files that, so I am able to exclude them from the fileset.
>
> However, FDs version 11.x and 13.x do not report the files that are
> generating the differences (and an according bug report is open for 8
> months already:
> https://gitlab.bacula.org/bacula-community-edition/bacula-community/-/issues/2670
> ).
>
> I would like to exclude the files that are generating differences on
> machines runnid FD 11.x and 13.x, too, but I have no good idea how to
> determine which files that are. (Using FD 9.x is not an option for these
> machines, as either not available, not supported, or the machines have
> "special needs” regarding connection directions, i.e. due to firewalls the
> default connection initiation directions do not work - an this can be
> remediated using FD 11.x and higher)
>
> Does someone have an idea how I could efficiently determine, which files
> are generating the verify differences?
>
> ATB
>  JC
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula copy from tape to tape

[Rob Gerber]

Well, now I know that whole thing is impossible. Simpler, that way.

Thank you for letting me know!

Technically I do have a huge NAS that dwarfs the production NAS in size
(TrueNAS box). It was intended to store one movie, but the movie IT people
massively over-estimated the amount of space we needed, so it never got
used. It has occurred to me to use this TrueNAS device as a disk based
volume storage solution.

Can one make multiple copies of a reference volume? Ie, could we have disk
based backups managed by bacula on this larger NAS, with copies made to
tape for onsite AND offsite storage?

Maybe at that point we wouldn't need an onsite tape backup set. Disk based
backup could speed up our initial backups substantially since both of these
systems can operate at 10gbe speeds.

I have it set up with LTO as the primary backup solution because
LTO infrastructure was the only backup plan until management suddenly
demanded I build a massive NAS for housing this movie (then failed to use
said NAS).

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Tue, Nov 7, 2023 at 3:28 PM Bill Arlofski via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> On 11/7/23 13:04, Rob Gerber wrote:
>  >
> > How difficult will it be to run copy jobs with only 1 LTO8 tape drive?
>
> Hello Rob,
>
> Sorry to be the bringer of bad news, but "difficult" is not the correct
> word.
>
> The word you are looking for is "impossible"
>
> Bacula needs one read device and one write device for copy or migration
> jobs.
>
>
> Best regards,
> Bill
>
> --
> Bill Arlofski
> w...@protonmail.com
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula purges old jobs and I don't want it to do that

[Rob Gerber]

To update this thread, I ultimately was able to avoid a bscan of all my
backed up media by restoring a backup of my catalog database. I have set
file, job, and volume retention to 1000 years.

Does a virtual full job strategy eliminate information about changed files?
ie, if the full backup captured fileA in one state, and if a later
incremental backup captured fileA in a different state, would the virtual
full consolidation process eliminate reference to the first backup of
fileA? Lets assume that once a tape is full it, nor its associated files or
jobs will never be recycled, at least not for a 7 year period or so.

Incrementals forever could scale very badly in a larger enterprise, but my
objective is to protect a single set of files on a single system. My
largest concern is tapes going missing in an incremental chain, and for
that reason I'm probably going to need to do differential backups
periodically.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Fri, Aug 18, 2023 at 6:25 PM Phil Stracchino 
wrote:

> On 8/18/23 17:24, Rob Gerber wrote:
> > verification jobs. My guess is that bacula purged job records for some
> > reason, possibly because they were older than 6 months. I have volume
> > retention set to 1000 years, but maybe I need to add something for job
> > retention?
>
> There are three retention periods you need to consider:
> — Volume retention
> — Job retention
> — File retention
> I suggest reading up on these in the documentation so that you
> understand what each of them does.  You can then adjust your pools, and
> update all of your volumes, as necessary.
>
>
> > Background info: I am backing up some large video files with LTO8 tape.
> > My goal is to back these files up once, and do incremental backups to
> > capture any new or changed files in the fileset. I do not expect
> > the large media files to be changed once placed on the disk. Basically,
> > I am using an "incrementals forever" strategy.
>
>
> You might want to consider virtual-full jobs as a part of that strategy.
>
> > I am using bacula 13.x. We have a qualstar q24 autochanger. We use LTO 8
> > media. The fileset in question is around 70TiB. I welcome any assistance
> > the list could provide. I'm sure it has to be something simple.
>
>
> You could fix your retention and then scan the pruned volumes back in
> using bscan.  By the sound of it, that would take a while, but it's doable.
>
> What you need to keep in mind is that an "incrementals forever" strategy
> is also a "Bacula database grows forever" strategy.  That will
> eventually become a problem.
>
>
> --
>Phil Stracchino
>Babylon Communications
>ph...@caerllewys.net
>p...@co.ordinate.org
>Landline: +1.603.293.8485
>Mobile:   +1.603.998.6958
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Bacula copy from tape to tape

[Rob Gerber]

I need to back up a 76TiB dataset to LTO8 tape. This data is primarily
large video files that won't change very often. I need two backups, one set
of tapes for onsite storage and one set for offsite storage.

My plan has been to back up the server data using two different job
definitions, with two different pools, sharing a single file definition.
However, the initial backup process takes a long time. This is interfering
with our use of the NAS appliance because we need to clear some items from
the NAS to free up available space, but if we remove them then the second
backup won't back up the removed files a second time. Overall, this "run
the backup twice using different tapes" idea isn't great because it could
lead to backup synchronization issues where the two backups aren't
identical snapshots of a moment in time, and couldn't necessarily be relied
upon to have captured the same data. IE, we might not have two backup
copies of all data, exactly like in this scenario where we want to delete
some data from the NAS for longterm storage on tape.

I have read about bacula's copy job process, whereby an existing backup is
copied to a new set of media, and the job media won't be restored directly
unless the original job media is unavailable.

With my current plan, I am confident that it will be easy to tell the
difference between the offsite and onsite jobs and their associated media.
I mostly distinguish between them by the associated pool.

If I run copy jobs against my first backup, will it be similarly easy to
tell which media is intended for offsite storage and which is intended for
onsite storage?

How difficult will it be to run copy jobs with only 1 LTO8 tape drive?I
know I'd have to provide suitable spool storage to unspool a given tape. My
current spool is 75GiB, so it's woefully undersized for the task.

Can copy jobs copy from 1 tape to the other? I've seen references to copy
jobs as a primary means to move a copy from a disk volume to a tape volume.
I assume this is possible from tape to tape as well, but want to verify.

If I rename a bacula job, pool, or fileset in the bacula director config,
will it cause any issues with data already backed up to tape when I go to
restore? My current backup is named "sharename offsite" and since I can't
easily restore from the copy unless the original job is unavailable or I
specify the copy job directly by jobid, I wouldn't want to take the
original backup offsite and leave the copy onsite. The original full backup
took 2 weeks+ to complete and re-running that backup would stink.

If I were to do a bscan of those tapes later in some disaster recovery
scenario, would I have any issues because the job changed names partway
through the full >> incremental lifecycle?

Would the copied tapes be interchangable with their original counterpart? I
assume not, though it'd be great if so. I imagine a situation where 1 of
several tapes are unavailable from the onsite backup set, and we substitute
the appropriate tape from the offsite backup set. It does strike me as
concerning the possibility that 1 tape is missing from the onsite set and a
different tape missing from the offsite set. In that case, even though the
tapes are different and the data should be identical, would we be unable to
do a full restore of the data from either backup set?

background information:
I am using bacula to back up a nas appliance holding 196TiB of data, mostly
large video files that should not change (though they might be moved to
different folders by video editors).

I am backing this data up to LTO 8 tape using a Qualstar Q24 tape changer.
I have 1 LTO 8 tape drive.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] fd lose connection

[Rob Gerber]

Lionel,

I found this post from 10 years ago. That user had a similar problem. Kern
suggested add "Heartbeat Interval = 300" to the appropriate resources on
both sides (dir and FD in your case). I see you mentioned setting
heartbeat, did you set it on both sides?

Source: old post to this list
https://bacula-users.narkive.com/i6Y7gQ1J/lib-bsock-c-397-write-error-sending-1-bytes-to-storage-daemon


Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Nov 3, 2023, 1:51 PM Lionel PLASSE  wrote:

> Hello ,
>
> I am losing connection between sd and windows fd  and  I don't understand
> why.
> The  machine lose every time the connection, sometimes after an hour  and
> often after 2 hours of job.
> And the transfer rate is so slow around 3 to 5 Mb/s (instead of up to 50 in
> other configurations)
>
> I tried many Network Maximum Rate values (32768 and higher and smaller) but
> every time I have more byte written to client (65566 > 65536) and
> connection
> fall down.
>
> The server is in win2019 version  with bacula-fd  win32 v13 and the Bacula
> Dir is in v13.0.3 on  a Debian 12 (with libssl1.1 and 6.1.0-13-amd kernel)
>
> I before was using bacula v9.6 and all wad good through my Lan network
> when
> backing up this same windows server.
>
> I am now using a Debian server  in a different place, through a site to
> site
> permanent ipsec VPN ( general upload wan bandwith  80Mb/s)
> The jobs are good for the other machines (win2012-r2 & Debian 10) which
> reach a rate of 10 up to 30 Mb/s depending the time and size of files
> backed
> up.
>
> But for this one (and another one in win2008-r2) :
> serv-dc-01-fd JobId 122: Fatal error: filed/backup.c:1008 Network send
> error
> to SD. ERR=Input/output error
> serv-dc-01-fd JobId 122: Error: lib/bsock.c:397 Wrote 65566 bytes to
> client:192.168.0.4:9102, but only 65536 accepted
>
> I check full duplex, opened firewall port, tcp_keep_alive & heartbeat,
> changed net buffer size,installed last driver .
>
> Any suggestion ?
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] fd lose connection

[Rob Gerber]

Oh, reading the error message again, did you set "heartbeat interval = 300"
between Dir and SD? Your error message is from the SD.

I don't know if FD will communicate directly with SD or if data from Fd
goes through dir to SD.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Nov 3, 2023, 2:15 PM Rob Gerber  wrote:

> Lionel,
>
> I found this post from 10 years ago. That user had a similar problem. Kern
> suggested add "Heartbeat Interval = 300" to the appropriate resources on
> both sides (dir and FD in your case). I see you mentioned setting
> heartbeat, did you set it on both sides?
>
> Source: old post to this list
>
> https://bacula-users.narkive.com/i6Y7gQ1J/lib-bsock-c-397-write-error-sending-1-bytes-to-storage-daemon
>
>
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
> On Fri, Nov 3, 2023, 1:51 PM Lionel PLASSE  wrote:
>
>> Hello ,
>>
>> I am losing connection between sd and windows fd  and  I don't understand
>> why.
>> The  machine lose every time the connection, sometimes after an hour  and
>> often after 2 hours of job.
>> And the transfer rate is so slow around 3 to 5 Mb/s (instead of up to 50
>> in
>> other configurations)
>>
>> I tried many Network Maximum Rate values (32768 and higher and smaller)
>> but
>> every time I have more byte written to client (65566 > 65536) and
>> connection
>> fall down.
>>
>> The server is in win2019 version  with bacula-fd  win32 v13 and the Bacula
>> Dir is in v13.0.3 on  a Debian 12 (with libssl1.1 and 6.1.0-13-amd kernel)
>>
>> I before was using bacula v9.6 and all wad good through my Lan network
>> when
>> backing up this same windows server.
>>
>> I am now using a Debian server  in a different place, through a site to
>> site
>> permanent ipsec VPN ( general upload wan bandwith  80Mb/s)
>> The jobs are good for the other machines (win2012-r2 & Debian 10) which
>> reach a rate of 10 up to 30 Mb/s depending the time and size of files
>> backed
>> up.
>>
>> But for this one (and another one in win2008-r2) :
>> serv-dc-01-fd JobId 122: Fatal error: filed/backup.c:1008 Network send
>> error
>> to SD. ERR=Input/output error
>> serv-dc-01-fd JobId 122: Error: lib/bsock.c:397 Wrote 65566 bytes to
>> client:192.168.0.4:9102, but only 65536 accepted
>>
>> I check full duplex, opened firewall port, tcp_keep_alive & heartbeat,
>> changed net buffer size,installed last driver .
>>
>> Any suggestion ?
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Windows filesets setup...

[Rob Gerber]

On C drive, what things do you want to back up?

It looks like you say "include all of C drive" and then also say "back up
folder matching C:/Program Files*/Hocoma", then say "exclude anything on
C".

I think if this is the case maybe you are using conflicting instructions.

If you only want to back up folder matching C:/Program Files*/Hocoma, maybe
only tell it to back up that folder?

Robert Gerber
402-237-8692
r...@craeon.net

On Thu, Oct 26, 2023, 1:41 PM Marco Gaiarin  wrote:

>
> Still there's some things in Bacula that i really dont't understand,
> evidently. ;-)
>
>
> I need to backup a windows box; i need to backup entirely the D: drive and
> partially the C: drive. Following documentation and examples, i've wrote:
>
>  FileSet {
>Name = ArmeoStd
>Description = "Backup dati Hocoma/Armeo"
>Enable VSS = yes
>
>Include {
>  File = "C:/"
>
>  Options {
>Signature = MD5
>Ignore Case = yes
>
>WildDir = "C:/Program Files*/Hocoma"
>  }
>
>  Options {
>Exclude = yes
>
>WildDir = "C:/*"
>  }
>}
>
>Include {
>  File = "D:/"
>
>  Options {
>Signature = MD5
>Ignore Case = yes
>  }
>}
>  }
>
>
> Backup on D: works as expected, on C: i got:
>
>  cwd is: /
>  $ dir
>  drwxrwx--T   1 root root   12288  2023-10-23 08:45:20  C:/
>  drwxrwx--T   1 root root4096  2023-10-23 08:45:17  D:/
>  cd C:/
>  cwd is: C:/
>  dir:/
>  -rwxrwxrwx   1 root root1024  2023-10-18 13:09:20  C:/.rnd
>  -rwxrwx--T   1 root root  3126231040  2023-10-26 11:41:44
> C:/hiberfil.sys
>  -rwxrwx--T   1 root root  4168310784  2023-10-26 11:41:44
> C:/pagefile.sys
>
> What i'm doing wrong?!
>
>
> Thanks...
>
> --
>   Sai come fanno i Serbi ad abbattere i caccia americani?
>   Disegnando una funivia sul terreno...
>
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restoring Incremental Promotes to Full

[Rob Gerber]

As far as I know you should only need one restore job.

Have you tried specifying which level to actually use when you did the
restore? Like in bconsole I think you'd add "level=incremental" to the end
of the restore command. In baculum/bacularis you would click the drop-down
box where it said "full" and change it to incremental.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Oct 27, 2023, 3:47 PM Chris Wilkinson 
wrote:

> Till now I have only ever done restores of full backups using the restore
> wizard of Baculum. When I did a restore of an Incremental backup, I found
> that it reverted to restoring the last full backup. I get exactly the same
> behaviour using bconsole restore.
>
> This is the restore job resource.
>
> Job {
>   Name = "Restore"
>   Description = "Restore template"
>   Type = "Restore"
>   Level = "Full"
>   Messages = "Standard"
>   Storage = "dns-325-sd"
>   Pool = "usb16tb-full"
>   Client = "usb16tb-fd"
>   Fileset = "usb16tb"
>   Schedule = "sched_none"
> }
>
> I had thought that most of these directives are not actually used as is
> but would be overridden by the wizard, i.e. the values here are required
> but unimportant.
>
> I also thought that only one 'dummy' restore job is needed that would be
> populated by the appropriate level, pool etc. from the job/level being
> restored but that isn't what actually happens. It looks like the restore is
> taking its' level from the restore job above.
>
> I have separate pools for each job and level.
>
> Do I need to define individual restore jobs for every job/level?
>
> Best
> Chris Wilkinson
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Configuring Baculum - Error Provided directory path is not writable by web server

[Rob Gerber]

Some of this advice might not follow best practice (basically item 1). You
should assess risk accordingly. I am not a bacula or Apache expert. (Really
not an Apache expert since I'm using lighthttpd). I might get some command
syntax wrong or misremember a process name since I'm typing this from my
phone. Double check my work before implementing it.

0. Are you using php-fpm? Try to restart that process (service php-fpm
status) (service php-fpm restart). Also restart the Apache2 service
(service httpd restart). With lighthttpd and bacularis I have to restart
both php-fpm and the bacularis-lighttpd process.

1. Consider testing with permissions set to 777. See below link. Looks like
this isn't unique to you.
https://github.com/tyler-dane/bacula/blob/master/troubleshooting.md#problem-api-configuration-gives-error-that-your-directory-path-for-new-config-files-is-not-writable-by-apache

2. When you did the chown and chgrp on the relevant folder, did you do it
recursively? Maybe there is a file in there that is not writable.

2.1 su yourself into www-data (su www-data) and try to create a file in the
working directory (touch /etc/baculum/Config-api-cache/filename.txt). Can
you create the file? Can you list directory contents using ls to prove that
the file was created and that it is owned by user www-data? Might need to
set a shell type for user www-data first before you can actually log into
it. Could also just use su -c or something to take action as that user, but
a user session as that user is the surest way to be sure that some program
running as that user had the rights to do things in that folder.

3. Restart the server. Maybe there is a process we aren't familiar with
that needs to be restarted. Sorry about your uptime, but if it's possible
see if a system restart fixes the issue.

4. Completely uninstall baculum and all associated packages. Use the apt
purge command to completely remove any associated configuration files as
well as uninstalling the package. This should in theory let you reinstall
fresh. Try again. Did something just go wrong during initial installation?

5. Install baculum using lighttpd instead of apache2. I'm personally using
lighttpd, and don't have experience with Apache. Be advised that you'll
maybe need to restart the php-fpm service as well as the baculum-lighttpd
service.

6. Install bacularis using lighttpd. See the bacularis website for
instructions.


Robert Gerber
402-237-8692
r...@craeon.net

On Sun, Oct 15, 2023, 5:33 PM Vaughan Wickham  wrote:

> Hello,
>
>
>
> I now have the following:
>
>
>
> ls -ld /etc/baculum/Config-api-apache/
>
> drwxrwx--- 2 www-data www-data 4096 Oct 15 14:32
> /etc/baculum/Config-api-apache
>
>
>
> Which I understand is what is needed for apache to have write access to
> the above file.
>
>
>
> Still seeing this error in Baculum setup:
>
>
>
> Baculum working directory for Bacula config:
>
> /etc/baculum/Config-api-cache
>
>
>
> Error Provided directory path is not writable by web server.
>
>
>
> Regards,
>
> Vaughan
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Configuring Baculum - Error Provided directory path is not writable by web server

[Rob Gerber]

Thoughts, with most important probably being middle and end thought:

1. Can set permissions using ACL with getfacl and setfacl. Google and see
man pages. However, this is probably not the best solution since...
2. You might need to add an exemption to a sudoers allow list file. Odds
are there should be a baculum file in the relevant directory (I think
/etc/sudoers?) with paths pointing to various places baculum is supposed to
be able to access.
2.1 Are you installing bacula from the Debian repositories? This method
uses non-standard (for bacula conventions) file locations sometimes. Also,
code might not be the latest. The baculum API setup should prompt you to
add entries to the sudoers file, but keep in mind that if the files
requested are actually somewhere else you'll have to give the correct
locations. Also, the file must end with a newline.
3. If installing from the Debian repositories, consider switching to the
baacula community project repos instead. To do that, go the link below
("download > Deb, RPM, OSX packages" on the bacula community site), enter
your name and email into the contact info section at the bottom of the
page, and click submit. The main caveat is that it may inform you that a
relevant download link will be emailed to you. This is not correct. It is
present on the page that loads directly after you submit your information,
in a link in the middle of the page, labeled is "here". Not super easy to
find, needs improvement imo. Heads up, the instructions to add the custom
bacula repos hosted by the project currently advise user to use apt-key
which is deprecated. Idk what the preferred solution is in this case, but
you've been warned there.

I figure items 2 / 2.1 are probably what you need. Also, consider looking
into bacularis! It's a more actively maintained friendly fork for baculum.
The developer is on this list, and I've seen him commit patches very
rapidly for issues people report. I'm super grateful for his hard work and
I think his efforts to make bacula more accessible are very important.

Robert Gerber
402-237-8692
r...@craeon.net

On Sat, Oct 14, 2023, 11:18 PM Vaughan Wickham  wrote:

> Hello
>
>
>
> I’m in the middle of first time run of Baculum on Debian 11 with Bacula
> Director >= 11.0
>
>
>
> I’m up to step 4, the Bacula configuration interface.
>
>
>
> When I perform the test, after updating all the paths, I am seeing an
> error on access to the Baculum working directory for Bacula config.
>
>
>
> Error: Provided directory path is not writable by web server.
>
>
>
> The web server that I’m using is apache2.
>
>
>
> I’m guessing that I need to configure apache2 to have write access to:
>
>
>
> /etc/baculum/Config-api-cache
>
>
>
> But I’ve not setup apache previously and having looked on Google, it’s not
> clear to me what the steps are to grant apache2 write access to:
>
>
>
> /etc/baculum/Config-api-cache
>
>
>
> I’m hoping that someone on the list, can provide some direction.
>
>
>
> Thanks
>
>
>
> Regards,
>
> Vaughan
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Configuring Baculum - Error Provided directory path is not writable by web server

[Rob Gerber]

Dangit, I didn't add the link I said I would. Sorry. Bacula project repo
access link:
https://www.bacula.org/bacula-binary-package-download/

Baculum setup info, including more detail on the sudoers.d file I
mentioned.
https://baculum.app/doc/brief/configuration.html

Consider trying lighthttpd instead of apache2 if you can't get it figured
out. Instructions for that also in there.

Robert Gerber
402-237-8692
r...@craeon.net

On Sun, Oct 15, 2023, 12:30 AM Rob Gerber  wrote:

> Thoughts, with most important probably being middle and end thought:
>
> 1. Can set permissions using ACL with getfacl and setfacl. Google and see
> man pages. However, this is probably not the best solution since...
> 2. You might need to add an exemption to a sudoers allow list file. Odds
> are there should be a baculum file in the relevant directory (I think
> /etc/sudoers?) with paths pointing to various places baculum is supposed to
> be able to access.
> 2.1 Are you installing bacula from the Debian repositories? This method
> uses non-standard (for bacula conventions) file locations sometimes. Also,
> code might not be the latest. The baculum API setup should prompt you to
> add entries to the sudoers file, but keep in mind that if the files
> requested are actually somewhere else you'll have to give the correct
> locations. Also, the file must end with a newline.
> 3. If installing from the Debian repositories, consider switching to the
> baacula community project repos instead. To do that, go the link below
> ("download > Deb, RPM, OSX packages" on the bacula community site), enter
> your name and email into the contact info section at the bottom of the
> page, and click submit. The main caveat is that it may inform you that a
> relevant download link will be emailed to you. This is not correct. It is
> present on the page that loads directly after you submit your information,
> in a link in the middle of the page, labeled is "here". Not super easy to
> find, needs improvement imo. Heads up, the instructions to add the custom
> bacula repos hosted by the project currently advise user to use apt-key
> which is deprecated. Idk what the preferred solution is in this case, but
> you've been warned there.
>
> I figure items 2 / 2.1 are probably what you need. Also, consider looking
> into bacularis! It's a more actively maintained friendly fork for baculum.
> The developer is on this list, and I've seen him commit patches very
> rapidly for issues people report. I'm super grateful for his hard work and
> I think his efforts to make bacula more accessible are very important.
>
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
> On Sat, Oct 14, 2023, 11:18 PM Vaughan Wickham  wrote:
>
>> Hello
>>
>>
>>
>> I’m in the middle of first time run of Baculum on Debian 11 with Bacula
>> Director >= 11.0
>>
>>
>>
>> I’m up to step 4, the Bacula configuration interface.
>>
>>
>>
>> When I perform the test, after updating all the paths, I am seeing an
>> error on access to the Baculum working directory for Bacula config.
>>
>>
>>
>> Error: Provided directory path is not writable by web server.
>>
>>
>>
>> The web server that I’m using is apache2.
>>
>>
>>
>> I’m guessing that I need to configure apache2 to have write access to:
>>
>>
>>
>> /etc/baculum/Config-api-cache
>>
>>
>>
>> But I’ve not setup apache previously and having looked on Google, it’s
>> not clear to me what the steps are to grant apache2 write access to:
>>
>>
>>
>> /etc/baculum/Config-api-cache
>>
>>
>>
>> I’m hoping that someone on the list, can provide some direction.
>>
>>
>>
>> Thanks
>>
>>
>>
>> Regards,
>>
>> Vaughan
>> ___
>> Bacula-users mailing list
>> Bacula-users@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Extraction of a specific file

[Rob Gerber]

I'm not sure, but maybe you'll need to query your database directly instead
of relying on Bacula to simplify that process. *This is not an educated
answer, just my novice guess.*

I see bacula bconsole has a 'query' command that I believe passes SQL query
language commands on to the relevant DB server. Might want to look into
that, possibly in the bacula console operators manual. I've used the PDF
copy of the documentation because it's easier to search.

Slightly off-topic:
I think when it comes to interacting with your DB server of choice, bacula
is deliberately vague because they expect you to know how to admin the DB
server yourself. I think they're also vague sometimes because the user
could be using any of several different DB servers (Sqlite, MySQL/mariadb,
postgresql). (Postgresql is the currently recommended DB server of choice.)
I know when it came time to restore a database backup there weren't any
Bacula-centric instructions. Why? Because the DB restore wasn't done
through Bacula. It was done using postgresql psql console. It was
definitely weird though, I found instructions everywhere in Bacula docs for
how to back up the database, and nothing about how to restore it. In the
end, the restore was easy.

Robert Gerber
402-237-8692
r...@craeon.net

On Sat, Sep 30, 2023, 3:27 PM Dr. Thorsten Brandau <
thorsten.bran...@brace.de> wrote:

> Hi
>
> I was trying to get a list of files (via bconsole) of the backups I do on
> tape (most recent for example). The only way I found is to use "restore"
> and then select the last backup. I was able to see some directories (e.g.
> /mnt/data) but not the files that were backup up.
>
> Is there a way to get a list of all files in the backup, preferably with
> date, size etc.?
>
> I am looking for a function like
>
> list files=*officedocument*.odt
>
> or so. It should be wildcard aware.
>
> The onlinedocumentation is not in a way that I would understand how it is
> possible, so maybe I am just too stupid (and there are not a lot of
> examples, especially for restore etc.).
>
> Cheers
>
> T
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Difference between purge volume and purge jobs volume

[Rob Gerber]

I suppose that would be a very thorough way to ensure no records remain.
Remember that I am not 100% certain about what those commands do, and that
others here may have more wisdom.

I have read in the docs somewhere about a way to get Bacula to reuse a tape
that had been previously used before Bacula would usually do such a thing.
By default I believe Bacula's behavior is to overwrite previously written
data on a volume as an absolutely last resort (when that data has been
previously expunged). Somewhere in the docs someone reported learning that
they could rewind the tape, then write a record to the beginning of the
tape that would signal to bacula that the tape was unused. Maybe writing an
EOF marker? Check the docs. You would do this thing I describe to get
bacula to use the whole tape sooner than it would usually do so.

Now, I don't know for sure that you're using tapes. Could be file volumes.

Robert Gerber
402-237-8692
r...@craeon.net

On Wed, Sep 27, 2023, 3:54 AM Felix Brack  wrote:

> Hello Robert,
>
> Many thanks for the fast response!
>
> On 26.09.23 17:36, Rob Gerber wrote:
> > My guess is that purge volume=xyz will purge any records regarding that
> > volume. However, purge jobs volume=xyz will purge any jobs related to
> > that volume, but not the volume itself.
> >
> > I have learned from this list that there are retention periods for
> > records pertaining to files, jobs, and volumes. This implies that bacula
> > considers database entries regarding files, jobs, and volumes to be
> > separate from each other. As a result, I am not sure that purging a
> > volume record would remove all associated file and job records from the
> > database as well.
> >
> > I would say with some confidence that purge jobs volume=xyz will purge
> > job records related to volume xyz, but might not purge file records
> > related to that volume, and almost certainly won't purge volume records
> > related to that volume.
> >
> So, to make manually sure nothing prevents volume xyz to remain in
> 'used' state or turn back to 'used' state after one single job, it would
> be best to enter _all_ of the following commands:
>
> purge volume=xyz
> purge jobs volume=xyz
> purge files volume=xyz
>
> > Keep in mind that purging one record might result in Bacula removing
> > other records automatically. Like maybe if a volume has no jobs or files
> > associated, the volume might be pruned and recycled automatically. I
> > don't know if Bacula does this, but it's plausible.
> >
> > Keep in mind that purge is dangerous and ignores configured retention
> > periods. Prune is safer.
> >
> Thanks for the tip. I use purge intentionally to bypass the configured
> retention policy.
>
> regards Felix
>
> > Those who are more knowledgeable should be able to contribute more.
> >
> > Robert Gerber
> > 402-237-8692
> > r...@craeon.net <mailto:r...@craeon.net>
> >
> > On Tue, Sep 26, 2023, 6:58 AM Felix Brack  > <mailto:f...@ltec.ch>> wrote:
> >
> > Hi,
> >
> > What is the difference between:
> >
> > purge volume="xyz"
> >
> > and
> >
> > purge jobs volume="xyz"
> >
> > Or are they equivalent with respect to the result?
> >
> > regards Felix
> >
> >
> > ___
> > Bacula-users mailing list
> > Bacula-users@lists.sourceforge.net
> > <mailto:Bacula-users@lists.sourceforge.net>
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
> > <https://lists.sourceforge.net/lists/listinfo/bacula-users>
> >
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Difference between purge volume and purge jobs volume

[Rob Gerber]

My guess is that purge volume=xyz will purge any records regarding that
volume. However, purge jobs volume=xyz will purge any jobs related to that
volume, but not the volume itself.

I have learned from this list that there are retention periods for records
pertaining to files, jobs, and volumes. This implies that bacula considers
database entries regarding files, jobs, and volumes to be separate from
each other. As a result, I am not sure that purging a volume record would
remove all associated file and job records from the database as well.

I would say with some confidence that purge jobs volume=xyz will purge job
records related to volume xyz, but might not purge file records related to
that volume, and almost certainly won't purge volume records related to
that volume.

Keep in mind that purging one record might result in Bacula removing other
records automatically. Like maybe if a volume has no jobs or files
associated, the volume might be pruned and recycled automatically. I don't
know if Bacula does this, but it's plausible.

Keep in mind that purge is dangerous and ignores configured retention
periods. Prune is safer.

Those who are more knowledgeable should be able to contribute more.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Sep 26, 2023, 6:58 AM Felix Brack  wrote:

> Hi,
>
> What is the difference between:
>
> purge volume="xyz"
>
> and
>
> purge jobs volume="xyz"
>
> Or are they equivalent with respect to the result?
>
> regards Felix
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Slow spooling and hashing speed

[Rob Gerber]

You know, asking them to add a Bacula FD plugin is a good idea! I'm sure it
wouldn't be available for quite a while, but that is genuinely a good idea.

This is a commercial market NAS, aimed at videographers. The devs are
actually pretty damn good, and I'm proud of them for choosing Gentoo. They
have a bunch of software sitting on top of Gentoo aiming to provide fast
access to stored media files for video editors. It does its job well, but
backup is obviously our responsibility. They do integrate well with Storage
DNA, but given that some of our goals included archiving media (that would
not remain on the NAS), and given the boss's reaction to the storage DNA
price point, we went with Bacula instead. I'm more comfortable using an
open source highly configurable software solution. I'm not shit talking the
storage DNA folks at all - they seemed pretty good and provided some useful
information on tape hardware vendors (backupworks.com were very helpful as
we built our hardware stack).


Overall, I understand why they locked it down. Their target market uses
apple computers for virtually everything, and a modified appliance may well
break when they apply their updates. I would be pretty reluctant to modify
the appliance even if given access, unless I had no other choice. It's
designed and configured to do one thing and I don't know what they've done
with their changes. Breaking the really expensive appliance isn't really on
my to do list ;)

(I wasn't involved in selecting the Evo appliance, and have been tasked
with building infrastructure around it).

The main data array is formatted XFS, by way of trivia for y'all. The core
ethic is "get media go the end users as fast as possible so they can edit
and render as fast as possible". Everything else is secondary.



Robert Gerber
402-237-8692
r...@craeon.net

On Sat, Sep 23, 2023, 2:05 PM Phil Stracchino  wrote:

> On 9/23/23 13:34, Rob Gerber wrote:
> > Heitor,
> >
> > Thank you for the reply.
> >
> > The backup source is a studio network solutions Evo NAS appliance.
> > Unfortunately, I have no access to run bacula FD, nor most of other
> > things suggested on this appliance as access is only via appliance
> > developer designer means. I do not have any shell access to this
> > appliance. Off the top of my head, I believe my only access options to
> > this Evo NAS are smb (I believe 3.x), read only iSCSI, or afs (apple
> > file sharing - obviously not something I want to try). This Evo NAS
> > appliance is running Gentoo Linux.
>
>
> Wait, it runs Gentoo and they locked it down that hard somehow?
>
> Gut feeling says you should be able to install almost anything you want
> onto it, but it may be some work depending on how they locked it down.
> (This would of course probably void your warranty.)
>
> That said, if you can't put a fd on it directly, given those options the
> next best thing would probably be to mount it as an iSCSI target.
>
> You might also consider asking them if they would consider a Bacula-fd
> package available.  But my experience is that to NAS consumer-appliance
> manufacturers, backup is at best an afterthought.  I went through this
> with a NAS appliance which QNAP proudly claimed ran Linux and ZFS, only
> to find out that through idiotic choices and Byzantine design decisions
> they had crippled it so badly it might as well have been running Windows.
>
>
>
> --
>Phil Stracchino
>Babylon Communications
>ph...@caerllewys.net
>p...@co.ordinate.org
>Landline: +1.603.293.8485
>Mobile:   +1.603.998.6958
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Slow spooling and hashing speed

[Rob Gerber]

Heitor,

Thank you for the reply.

The backup source is a studio network solutions Evo NAS appliance.
Unfortunately, I have no access to run bacula FD, nor most of other things
suggested on this appliance as access is only via appliance developer
designer means. I do not have any shell access to this appliance. Off the
top of my head, I believe my only access options to this Evo NAS are smb (I
believe 3.x), read only iSCSI, or afs (apple file sharing - obviously not
something I want to try). This Evo NAS appliance is running Gentoo Linux.

I am able to transfer data using the Linux cp command at link speeds. I am
open to trying anything to see if I can improve these backup times since we
have a lot of data to backup (total dataset about 194 TiB).

Robert Gerber
402-237-8692
r...@craeon.net

On Sat, Sep 23, 2023, 1:24 PM  wrote:

> Hello,
>
> On 9/14/23 15:35, Rob Gerber wrote:
>>
>> Bacula is transferring data at a fraction of the available link speed. I
>> am backing up an SMB share hosted on a fast NAS appliance. The share is
>> mounted on the bacula server in /mnt/NAS/sharename. I have dedicated 10gbe
>> copper interfaces on the NAS and the bacula server.
>>
>>
> Most likely your bottleneck is the SMB data source. Some documentations
> say the protocol has a 30 MBs limitation. I would't use for backup
> apllications.
> Bacula Clients at the source machine, NFS, CIFS and NDMP are better
> options.
> I would also use the Linux TCP BBR protocol.
>
> Rgds.
>
>
> *MSc Heitor Faria (Miami/USA)* Bacula LATAM CIO
> mobile1: + 1 909 655-8971
> mobile2: + 55 61 98268-4220
>
> [image: logo] 
> bacula.lat | bacula.com.br <http://www.bacula.com.br>
> <https://api.whatsapp.com/send?phone=5561982684220>
>
>
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Slow spooling and hashing speed

[Rob Gerber]

Thank you for your reply, josh. Sorry for delay in reply - have been on
vacation. See replies below.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Sep 15, 2023, 9:52 AM Josh Fisher via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

>
> On 9/14/23 15:35, Rob Gerber wrote:
>
> Bacula is transferring data at a fraction of the available link speed. I
> am backing up an SMB share hosted on a fast NAS appliance. The share is
> mounted on the bacula server in /mnt/NAS/sharename. I have dedicated 10gbe
> copper interfaces on the NAS and the bacula server.
>
> When backing up the NAS, cifsiostat shows around 250MB/s during the
> spooling phase (and 0 kb/s during the despool phase). When using cp to copy
> files from the NAS to the Bacula server, I can easily saturate my 10gbe
> link (avg throughput around 1GB/s, or a little lower).
>
>
> So that tells you that there's nothing wrong with the underlying  SMB file
> system. The Bacula client just reads the files like any other directory
> it's backing up.
>
>
>
> I think the problem lies in Bacula because I can copy data much faster
> using cp instead of bacula. Obviously bacula is doing a lot more than cp,
> so there will be differences. However I would hope for transfer speeds
> closer to the available link speed.
>
> top shows that a couple cores are maxed out during the spooling process.
> Maybe hashing speed is the limitation here? If so, could multicore hashing
> support speed this up? I have two e5-2676 v3 processors in this server. I
> am using SHA512 right now, but I saw similar speeds from bacula when using
> MD5.
>
>
> The hashing speed doesn't account for a 4x slower transfer, and likely not
> for saturating 2 cores. Do you have compression enabled for the job? Or
> encryption? You definitely do not want compression, since the tape drive
> will handle compression itself. Also, the client and sd are the same
> machine in this case, but make sure it is not configured to use TLS
> connections.
>

I have not affirmatively enabled compression. Is there any sort of default
compression mode that I could disable?

I'll see what I can find regarding TLS connections. I haven't set up such a
thing, but maybe it works that way by default?

>
>
> Average write speed to LTO-8 media winds up being about 120-150MB/s once
> the times to spool and despool are considered.
>
> My spool is on a 76GB ramdisk (spool size is 75G in bacula dir conf), so I
> don't think spool disk access speed is a factor.
>
>
> Might be overkill. A NVMe SSD is plenty fast enough for both the 10G
> network and for despooling to the LTO8 drive. If the catalog DB is also on
> this server, then you might be better off with the spool on SSD and far
> more RAM dedicated to postgresql. If the DB is on another server, then the
> attributes are being despooled to the DB over the 1G network.
>

I was using a sata SSD for my spool, but switched to a ramdisk for
troubleshooting purposes.

I have about 50gb ram available (not counting ramdisk). Top shows about
1.6gb ram in use, though I don't know how accurate that is.

My postgres DB server is on the same machine. I haven't specifically
configured postgres to use a certain amount of ram. I don't know what
performance enhancements I might be able to use with postgres.

I'm tempted to try setting up a direct to disk backup and test with local
files, or files stored on my TrueNAS server (which has its own dedicated
optical 10gbe link, which I can also a saturate when using cp).

Is there a way I could troubleshoot and try to pinpoint the location of my
bottleneck?

I will investigate the questions you raised in your email and report back.
Some delay is to be expected because a 124tb backup is running and is
currently at 98tb complete, averaging about 125MiB/s according to bacularis.

>
> I have not tested to see if bacula could back up faster if it wasn't
> accessing a share via SMB. I don't think SMB should be an issue here but I
> have to consider every possibility. The SMB share I'm backing up is mounted
> on /mnt/NAS/sharename. Bacula is backing that mount folder up.
>
> Currently, my only access to the NAS appliance is via SMB. The appliance
> does support iscsi in read only mode but i'm not sure if there would be any
> performance improvements.
>
> I don't think the traffic could be going out through the wrong interface.
> The NAS is directly attached to my bacula server using a short cat6 cable.
> The NAS and my server each have 10gbe copper interfaces. The relevant
> interfaces have ip addresses statically assigned. These addresses are
> unique to the LAN configuration (local lan is 10.1.1.0/24, 10gbe
> interfaces assigned to 192.168.6.25 and 192.168.6.100). My bacula
> server's only other co

Re: [Bacula-users] Remote client connection refused

[Rob Gerber]

I do know that best practice is to always specify fully qualified domain
names in the conf files (hostname.domain). What do you have specified in
your bacula-fd.conf file? Additionally, are you specifying localhost
anywhere in your bacula-dir.conf file? If so, should correct it to the
relevant fqdn.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Sep 19, 2023, 5:31 AM Chris Wilkinson 
wrote:

> I have set up a new remote client running FD only on Debian 12. The client
> is on the same LAN as the DIR/SD.
>
> I get a connection refused error from the job.
>
> The FD is running but is listening on 127.0.1.1 port 9102 which is
> probably why the connection is refused.
>
> Client bacula-fd -t returns no errors.
>
> I suppose the FD needs to listen on any address but I can't see how to do
> that.
>
> Is there a bacula-fd.conf setting for this?
>
> Thanks
> Chris Wilkinson
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bscan via Bash Script

[Rob Gerber]

One of the things that the "update slots" command does is tell bacula which
tapes are in which slots in the autochanger.

The error output you are seeing indicates that it doesn't know which slot
to load.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Sep 15, 2023, 9:50 AM Rob Gerber  wrote:

> I understand that you are using bscan. Maybe I am missing something, but
> my understanding is that once you add tapes to an autochanger you need to
> do "update slots" in bconsole. This syncs the autochanger inventory with
> the bacula director.
>
> I have not imported tapes using bscan, so I am sorry if I'm missing
> something important about that process.
>
> Robert Gerber
> 402-237-8692
> r...@craeon.net
>
> On Fri, Sep 15, 2023, 9:23 AM SulEsh  wrote:
>
>> I am using bscan and not directly doing it via bconsole.
>> Thus update slots in actual is not required here.
>>
>> On Fri, Sep 15, 2023 at 3:40 PM Rob Gerber  wrote:
>>
>>> After you add tapes to changer you must tell bacula director what tapes
>>> are in changer. To do this, open bconsole and do
>>>
>>> update slots
>>>
>>>
>>>
>>> Robert Gerber
>>> 402-237-8692
>>> r...@craeon.net
>>>
>>> On Fri, Sep 15, 2023, 7:13 AM SulEsh  wrote:
>>>
>>>> Hi
>>>>
>>>> I want to execute bscan via bash script as I have quite some amount of
>>>> tapes to synchronise with Bacula DB. If I run bscan manually than it is
>>>> working fine but if I try the bscan via Bash Script than it is always stuck
>>>> at a point and unless or until I press Enter, it is not going forward.
>>>> I tried several procedure to automate the Enter in Bash Script but it
>>>> did not work rather getting the following error message(I have trimmed few
>>>> info from original message).
>>>>
>>>> *No Slot Defined in catalog for volume on drive*
>>>> *Cartridge Change or "update slots" may be required*
>>>> *Mount Volume on device and press return when ready.*
>>>>
>>>> Now if i introduce the Automatic Carriage Return in bash script, I am
>>>> starting getting Input/Output Error Message related to device.
>>>> If anyone has ever tried bscan via bash script, please let me know as
>>>> it will be a great help if I automate this procedure.
>>>>
>>>> Regards
>>>>
>>>> Sul
>>>> ___
>>>> Bacula-users mailing list
>>>> Bacula-users@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>>
>>>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bscan via Bash Script

[Rob Gerber]

I understand that you are using bscan. Maybe I am missing something, but my
understanding is that once you add tapes to an autochanger you need to do
"update slots" in bconsole. This syncs the autochanger inventory with the
bacula director.

I have not imported tapes using bscan, so I am sorry if I'm missing
something important about that process.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Sep 15, 2023, 9:23 AM SulEsh  wrote:

> I am using bscan and not directly doing it via bconsole.
> Thus update slots in actual is not required here.
>
> On Fri, Sep 15, 2023 at 3:40 PM Rob Gerber  wrote:
>
>> After you add tapes to changer you must tell bacula director what tapes
>> are in changer. To do this, open bconsole and do
>>
>> update slots
>>
>>
>>
>> Robert Gerber
>> 402-237-8692
>> r...@craeon.net
>>
>> On Fri, Sep 15, 2023, 7:13 AM SulEsh  wrote:
>>
>>> Hi
>>>
>>> I want to execute bscan via bash script as I have quite some amount of
>>> tapes to synchronise with Bacula DB. If I run bscan manually than it is
>>> working fine but if I try the bscan via Bash Script than it is always stuck
>>> at a point and unless or until I press Enter, it is not going forward.
>>> I tried several procedure to automate the Enter in Bash Script but it
>>> did not work rather getting the following error message(I have trimmed few
>>> info from original message).
>>>
>>> *No Slot Defined in catalog for volume on drive*
>>> *Cartridge Change or "update slots" may be required*
>>> *Mount Volume on device and press return when ready.*
>>>
>>> Now if i introduce the Automatic Carriage Return in bash script, I am
>>> starting getting Input/Output Error Message related to device.
>>> If anyone has ever tried bscan via bash script, please let me know as it
>>> will be a great help if I automate this procedure.
>>>
>>> Regards
>>>
>>> Sul
>>> ___
>>> Bacula-users mailing list
>>> Bacula-users@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>
>>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] bscan via Bash Script

[Rob Gerber]

After you add tapes to changer you must tell bacula director what tapes are
in changer. To do this, open bconsole and do

update slots



Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Sep 15, 2023, 7:13 AM SulEsh  wrote:

> Hi
>
> I want to execute bscan via bash script as I have quite some amount of
> tapes to synchronise with Bacula DB. If I run bscan manually than it is
> working fine but if I try the bscan via Bash Script than it is always stuck
> at a point and unless or until I press Enter, it is not going forward.
> I tried several procedure to automate the Enter in Bash Script but it did
> not work rather getting the following error message(I have trimmed few info
> from original message).
>
> *No Slot Defined in catalog for volume on drive*
> *Cartridge Change or "update slots" may be required*
> *Mount Volume on device and press return when ready.*
>
> Now if i introduce the Automatic Carriage Return in bash script, I am
> starting getting Input/Output Error Message related to device.
> If anyone has ever tried bscan via bash script, please let me know as it
> will be a great help if I automate this procedure.
>
> Regards
>
> Sul
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Slow spooling and hashing speed

[Rob Gerber]

Bacula is transferring data at a fraction of the available link speed. I am
backing up an SMB share hosted on a fast NAS appliance. The share is
mounted on the bacula server in /mnt/NAS/sharename. I have dedicated 10gbe
copper interfaces on the NAS and the bacula server.

When backing up the NAS, cifsiostat shows around 250MB/s during the
spooling phase (and 0 kb/s during the despool phase). When using cp to copy
files from the NAS to the Bacula server, I can easily saturate my 10gbe
link (avg throughput around 1GB/s, or a little lower).

I think the problem lies in Bacula because I can copy data much faster
using cp instead of bacula. Obviously bacula is doing a lot more than cp,
so there will be differences. However I would hope for transfer speeds
closer to the available link speed.

top shows that a couple cores are maxed out during the spooling process.
Maybe hashing speed is the limitation here? If so, could multicore hashing
support speed this up? I have two e5-2676 v3 processors in this server. I
am using SHA512 right now, but I saw similar speeds from bacula when using
MD5.

Average write speed to LTO-8 media winds up being about 120-150MB/s once
the times to spool and despool are considered.

My spool is on a 76GB ramdisk (spool size is 75G in bacula dir conf), so I
don't think spool disk access speed is a factor.

I have not tested to see if bacula could back up faster if it wasn't
accessing a share via SMB. I don't think SMB should be an issue here but I
have to consider every possibility. The SMB share I'm backing up is mounted
on /mnt/NAS/sharename. Bacula is backing that mount folder up.

Currently, my only access to the NAS appliance is via SMB. The appliance
does support iscsi in read only mode but i'm not sure if there would be any
performance improvements.

I don't think the traffic could be going out through the wrong interface.
The NAS is directly attached to my bacula server using a short cat6 cable.
The NAS and my server each have 10gbe copper interfaces. The relevant
interfaces have ip addresses statically assigned. These addresses are
unique to the LAN configuration (local lan is 10.1.1.0/24, 10gbe interfaces
assigned to 192.168.6.25 and 192.168.6.100). My bacula server's only other
connection is to the gigabit LAN switch.

Is there any information that I could provide to help the list help me, or
does anyone have any thoughts for me?

Regards,
Robert Gerber
402-237-8692
r...@craeon.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] access key

[Rob Gerber]

Another user ran into this recently - the link may not be emailed. At this
time I don't think the key is emailed for Bacula Community. I don't know
how it's been done historically for Bacula Community, but it does look like
Bacula Enterprise has the access key emailed.

Go to the link below. It looks like a "contact us" form is at the bottom of
the page, but that's actually the registration form. Give name and a valid
email.  When you enter your name and email, a page will load with a line
specifying that your access key is "here". the key is in the url for that
link.

https://www.bacula.org/bacula-binary-package-download/


Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Wed, Sep 13, 2023 at 5:54 PM Phil Stracchino 
wrote:

> On 9/13/23 18:35, Bob McNabb wrote:
> > Where do I obtain the access key for installing the community edition?
>
> You go to the download page, follow the instructions, and you will be
> emailed a download link.
>
>
> --
>Phil Stracchino
>Babylon Communications
>ph...@caerllewys.net
>p...@co.ordinate.org
>Landline: +1.603.293.8485
>Mobile:   +1.603.998.6958
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula recycling volumes it should not recycle

[Rob Gerber]

Don't know if this helps, but file and job retention entries you put in the
pool will override any such entries on the client resource. So afaik the
last word on whether a job will be retained is the job retention entry on a
pool resource. Maybe define job and file retention on the pool and see what
happens, as part of troubleshooting.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Sep 8, 2023, 3:16 AM Andrea Venturoli  wrote:

> On 9/7/23 21:33, Ana Emília M. Arruda wrote:
> > Hello Andrea,
>
> Hello.
>
>
>
> > You have 1 job per volume and automatic pruning and recycling enabled.
>
> Correct.
>
>
>
> > If the only job in the volume is deleted from the Catalog, the recycling
> > algorithm will detect it and the volume will be marked as Purged and
> > reused.
>
> Again, I would expect so if the job was deleted, but why would it?
>
>
>
> > Is it happening with all volumes or only a few cases?
>
> Not sure about "all", but certainly for a lot of volumes.
>
>
>
> > Are you manually deleting jobs from the Catalog?
>
> No.
>
> The only unusual thing I'm doing is:
> > echo "cloud prune AllFromPool Pool=Full" | /usr/local/sbin/bconsole
> > echo "cloud prune AllFromPool Pool=Diff" | /usr/local/sbin/bconsole
> > echo "cloud prune AllFromPool Pool=Inc" | /usr/local/sbin/bconsole
> > echo "cloud prune AllFromPool Pool=Cat" | /usr/local/sbin/bconsole
>
> This was done after each catalog backup jobs (now I disabled it to see
> if it was the culprit).
> AFAICT, this should prune the "cache" not recycle the volume in the cloud.
>
>
>   Bye & Thanks
> av.
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Debian FD v11 and above package

[Rob Gerber]

I don't know for sure because I have only deployed a client with SD / Dir
and FD all on the same machine, but I would imagine that an FD client only
machine wouldn't need a database. I'm pretty confident that if one was
needed, it would be installed as a dependency of the bacula client package.

My expectation is that the FD would communicate files to be backed up to
the Bacula director and that records of files would be kept on the
director. Your mileage may vary, though, so confirm on your end.

You should be aware that Bacula packages in the OS repo may conflict with
the bacula repo. This was true for Ubuntu, though I don't know if it's true
for Debian. When I used it last, the Ubuntu repo had packages for Bacula
9.x, which is pretty far out of date.

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Sep 1, 2023, 3:17 PM Chris Wilkinson  wrote:

> I hadn't spotted that there is a link on this page though it does say an
> email will be sent. That doesn't seem to be the case. I followed that
> through to the directory of debs and found the bullseye/11 clients. As you
> say, that should be OK for Debian 12.
>
> Am I correct in thinking that it isn't necessary to install a database in
> a client-only configuration?
>
> -Chris-
>
> On Fri, 1 Sept 2023, 19:42 Rob Gerber,  wrote:
>
>> Oh no, I gave you wrong information. I'm sorry.
>>
>> I don't think they email you. After you submit your name and email
>> address, the followup page that loads has the repo link in the middle of
>> the page It says " You may now access the bacula binaries located here
>> (please bookmark this link)." where "here" is your own custom repo link.
>> The link will be in format https://www.bacula.org/packages/abc123
>>
>> I clicked through on the link it gave me, and found that they have a
>> package named "bacula-client" that appears to be what you're looking for.
>> look in
>> https://www.bacula.org/packages/YOURSTRINGHERE/debs/BACULAVERSIONOFYOURCHOICE/dists/ONEOFTHEAVAILABLEDEBIANORUBUNTUVERSIONS/
>>
>> So sorry for the incorrect information.
>>
>> FYI release notes for bacula 13.x say:
>>
>> As always, both the Director and Storage daemon(s) must be upgraded at
>> the same time. Any File daemon running on the same machine as a Director
>> or Storage daemon must be of the same version.
>>
>> Older File Daemons should be compatible with the 13.0 Director and Storage
>>
>> I highly recommend browsing through the repo in your web browser of
>> choice to read the release notes and see what packages are available, what
>> debian OS variants are supported for each bacula version number, etc.
>>
>> It looks like bacula 13.x the newest debian version they support is
>> 11/bullseye. It should be fine to use that repo for your Debian 12
>> system, but be sure to verify yourself just to be sure.
>>
>> Regards,
>> Robert Gerber
>> 402-237-8692
>> r...@craeon.net
>>
>>
>> On Fri, Sep 1, 2023 at 11:02 AM Chris Wilkinson 
>> wrote:
>>
>>> I waited a while but never got the registration email.  Tried again and
>>> waited some more so gave up. The only way seems to be to download from
>>> sourceforge and compile from source. There is a compile flag for client
>>> only --enable_client_only, see the manual at;
>>>
>>> https://www.bacula.org/11.0.x-manuals/en/main/Installing_Bacula.html
>>>
>>> -Chris Wilkinson
>>>
>>> On Wed, 30 Aug 2023, 17:24 Chris Wilkinson, 
>>> wrote:
>>>
>>>> Thank-you. I registered and am awaiting the registration email.
>>>>
>>>> -Chris-
>>>>
>>>> On Wed, 30 Aug 2023, 17:15 Rob Gerber,  wrote:
>>>>
>>>>> I don't know about adding the FD only but I do know that bacula
>>>>> maintains a repo for Debian with 13.x binaries in it. To get access to 
>>>>> this
>>>>> repo, you need to go to download on bacula.org, register, get the
>>>>> code they include in an URL in your registration email, and configure the
>>>>> repo in Debian. I guess you could just download the binary you want via
>>>>> http if needed.
>>>>>
>>>>> Register here:
>>>>> https://www.bacula.org/bacula-binary-package-download/
>>>>>
>>>>>
>>>>> Installation guide, has instructions for adding the repo. Outdated in
>>>>> respect to Debian, but shows how to format the link to get access to the
>>>>> repo.
>>>>> https://www.bacula.org/white

Re: [Bacula-users] Debian FD v11 and above package

[Rob Gerber]

Oh no, I gave you wrong information. I'm sorry.

I don't think they email you. After you submit your name and email address,
the followup page that loads has the repo link in the middle of the page It
says " You may now access the bacula binaries located here (please bookmark
this link)." where "here" is your own custom repo link. The link will be in
format https://www.bacula.org/packages/abc123

I clicked through on the link it gave me, and found that they have a
package named "bacula-client" that appears to be what you're looking for.
look in
https://www.bacula.org/packages/YOURSTRINGHERE/debs/BACULAVERSIONOFYOURCHOICE/dists/ONEOFTHEAVAILABLEDEBIANORUBUNTUVERSIONS/

So sorry for the incorrect information.

FYI release notes for bacula 13.x say:

As always, both the Director and Storage daemon(s) must be upgraded at
the same time. Any File daemon running on the same machine as a Director
or Storage daemon must be of the same version.

Older File Daemons should be compatible with the 13.0 Director and Storage

I highly recommend browsing through the repo in your web browser of choice
to read the release notes and see what packages are available, what debian
OS variants are supported for each bacula version number, etc.

It looks like bacula 13.x the newest debian version they support is
11/bullseye. It should be fine to use that repo for your Debian 12
system, but be sure to verify yourself just to be sure.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Fri, Sep 1, 2023 at 11:02 AM Chris Wilkinson 
wrote:

> I waited a while but never got the registration email.  Tried again and
> waited some more so gave up. The only way seems to be to download from
> sourceforge and compile from source. There is a compile flag for client
> only --enable_client_only, see the manual at;
>
> https://www.bacula.org/11.0.x-manuals/en/main/Installing_Bacula.html
>
> -Chris Wilkinson
>
> On Wed, 30 Aug 2023, 17:24 Chris Wilkinson, 
> wrote:
>
>> Thank-you. I registered and am awaiting the registration email.
>>
>> -Chris-
>>
>> On Wed, 30 Aug 2023, 17:15 Rob Gerber,  wrote:
>>
>>> I don't know about adding the FD only but I do know that bacula
>>> maintains a repo for Debian with 13.x binaries in it. To get access to this
>>> repo, you need to go to download on bacula.org, register, get the code
>>> they include in an URL in your registration email, and configure the repo
>>> in Debian. I guess you could just download the binary you want via http if
>>> needed.
>>>
>>> Register here:
>>> https://www.bacula.org/bacula-binary-package-download/
>>>
>>>
>>> Installation guide, has instructions for adding the repo. Outdated in
>>> respect to Debian, but shows how to format the link to get access to the
>>> repo.
>>> https://www.bacula.org/whitepapers/CommunityInstallationGuide.pdf
>>>
>>> Robert Gerber
>>> 402-237-8692
>>> r...@craeon.net
>>>
>>> On Wed, Aug 30, 2023, 10:44 AM Chris Wilkinson 
>>> wrote:
>>>
>>>> I need to install the fd only on a debian 12 system. The v11+ packages
>>>> on sourceforge are for the complete suite and the debian repository is only
>>>> at v9.x today. I don't see a package on sourceforge for the fd alone. Do I
>>>> need to compile it from source?
>>>>
>>>> If so, can anyone help with a script of the needed compile flags?
>>>>
>>>> Best
>>>> -Chris Wilkinson
>>>> ___
>>>> Bacula-users mailing list
>>>> Bacula-users@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>>>>
>>>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Debian FD v11 and above package

[Rob Gerber]

I don't know about adding the FD only but I do know that bacula maintains a
repo for Debian with 13.x binaries in it. To get access to this repo, you
need to go to download on bacula.org, register, get the code they include
in an URL in your registration email, and configure the repo in Debian. I
guess you could just download the binary you want via http if needed.

Register here:
https://www.bacula.org/bacula-binary-package-download/


Installation guide, has instructions for adding the repo. Outdated in
respect to Debian, but shows how to format the link to get access to the
repo.
https://www.bacula.org/whitepapers/CommunityInstallationGuide.pdf

Robert Gerber
402-237-8692
r...@craeon.net

On Wed, Aug 30, 2023, 10:44 AM Chris Wilkinson 
wrote:

> I need to install the fd only on a debian 12 system. The v11+ packages on
> sourceforge are for the complete suite and the debian repository is only at
> v9.x today. I don't see a package on sourceforge for the fd alone. Do I
> need to compile it from source?
>
> If so, can anyone help with a script of the needed compile flags?
>
> Best
> -Chris Wilkinson
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] Bacula purges old jobs and I don't want it to do that

[Rob Gerber]

Hello,

Bacula purged job records for some backup jobs I did a while ago. I don't
want it to do that, and I want to restore the old records. When I went into
baculum to briefly run a backup job to check for any new files, I was
informed that there was no full backup on record, and that as such a new
full backup would need to be ran. I looked in my job history and all my
original backup jobs are missing, though the original backup jobs are still
referenced in much newer data-level verification jobs. My guess is that
bacula purged job records for some reason, possibly because they were older
than 6 months. I have volume retention set to 1000 years, but maybe I need
to add something for job retention?

Background info: I am backing up some large video files with LTO8 tape. My
goal is to back these files up once, and do incremental backups to capture
any new or changed files in the fileset. I do not expect the large media
files to be changed once placed on the disk. Basically, I am using an
"incrementals forever" strategy.

My goal is to have a two complete backups of all the files in my fileset.
To do this I have two jobs, an offsite and an onsite job. these jobs use
the same fileset resource. The respective jobs draw their media from the
scratch pool, and assign a job-specific pool to any tape written to by that
job. ie, tapes written by the offsite job are in the offsite pool. This
ensures that data will not be intermingled on the tapes. In the future, I'm
going to add another fileset, and I will use the same onsite/offsite
strategy as before. We want to keep tapes for these two filesets separate
for various reasons.

I am using bacula 13.x. We have a qualstar q24 autochanger. We use LTO 8
media. The fileset in question is around 70TiB. I welcome any assistance
the list could provide. I'm sure it has to be something simple.


Regards,
Robert Gerber
402-237-8692
r...@craeon.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula Community Installation UPGRADE

[Rob Gerber]

Was the mysql database for bacula 9 used in production, and/or is there
anything important in there? If not, remove the bacula 9 packages and
dependencies with the apt purge command (look up syntax). This will remove
configuration files and things created by the removed packages.

Might have to reinstall the bacula 9 packages then do a purge, can't recall
from when I encountered this.

If you have production data in the bacula 9 catalog, or need to retain an
unrelated mysql database better double check with the people here on this
list.

Robert Gerber
402-237-8692
r...@craeon.net

On Tue, Jul 4, 2023, 6:49 PM Ken Mandelberg  wrote:

> Another followup. One thing I learned, I need to remove all the Ubuntu
> bundled bacula 9 packages before doing  the apt-get install bacula-mysql
> for the Community 13 release, otherwise there are package conflicts.
>
> So after removing the old packages, apt-get install bacula-mysql tires
> to install the mysql bacula database. It gets caught at
>
> mysql said: ERROR 1050 (42S01) at line 9: Table 'TagJob' already exists
>
> I imagine if I removed that table it would just get caught at the next one.
>
> Its not really clear on how to proceed preserving the current database 9
> info and letting 13 do the update job.
>
>
> Ken Mandelberg wrote on 7/1/23 18:30:
> > This is a followup of my previous posts. I'm doing an upgrade from
> > Bacula 9 to 13 on Ubuntu 23.04. I realize there is no specific support
> > for 23.04 so I'm making believe I'm on Jammy.
> >
> > I've gotten past the apt-key issue and sources are set up, and apt-get
> > update has succeeded. Now my question is about the next step dealing
> > with the fact that this is an upgrade of a working Bacula 9
> > installation. (Don't worry, its really on a test machine before I try
> > it on a production machine).
> >
> > So I know I need to copy /etc/bacula to a safe place. Should I also
> > delete it?
> >
> > I will also stop director , sd, and fd. and move those binaries out of
> > the way.
> >
> > I'm running mysql. Should I do anything there to the existing bacula
> > data base?
> >
> > Assuming the answers to those the next stop is
> >
> > apt-get install bacula-mysql
> >
> > I would assume I then copy back the saved /etc/bacula back in place
> > and then start the new binaries.
> >
> > What about updating the mysql tables from 9 to 13? Where does that
> > happen.
> >
> > Sorry for all the needy questions. This is my first time not using the
> > bundled Ubuntu package support for Bacula  which handles updates
> > transparently. Debian/Ubuntu has no Bacula maintainer now.
> >
> > ---
> >
> > Side issue about the Documentation, what is the "concatenate" note
> > about. It makes no sense, deb would just be an ignored comment.
> >
> > Complete example:
> >
> > #Bacula Community
> > deb [arch=amd64]
> > https://www.bacula.org/packages/abc123defxxxyyy/debs/13.0.1 xenial main
> >
> > Note 1: please concatenate the last 2 lines into 1 line.
> >
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula Community Installation UPGRADE

[Rob Gerber]

Ken,

No worries.

I don't have them in front of me, but I'd bet the release notes for bacula
13.x (whatever your version will be, prob 13.0.3) will have info on any
needed upgrade steps. I suspect it'll handle the database transparently
based on my limited recollection from when I browsed said notes earlier
this week. Find the relevant release notes on the bacula website.

Regarding the concatenate note, I suspect they mean to make sure the text
in brackets is all on one line with the URL.

Robert Gerber
402-237-8692
r...@craeon.net

On Sat, Jul 1, 2023, 5:53 PM Ken Mandelberg  wrote:

> This is a followup of my previous posts. I'm doing an upgrade from
> Bacula 9 to 13 on Ubuntu 23.04. I realize there is no specific support
> for 23.04 so I'm making believe I'm on Jammy.
>
> I've gotten past the apt-key issue and sources are set up, and apt-get
> update has succeeded. Now my question is about the next step dealing
> with the fact that this is an upgrade of a working Bacula 9
> installation. (Don't worry, its really on a test machine before I try it
> on a production machine).
>
> So I know I need to copy /etc/bacula to a safe place. Should I also
> delete it?
>
> I will also stop director , sd, and fd. and move those binaries out of
> the way.
>
> I'm running mysql. Should I do anything there to the existing bacula
> data base?
>
> Assuming the answers to those the next stop is
>
> apt-get install bacula-mysql
>
> I would assume I then copy back the saved /etc/bacula back in place and
> then start the new binaries.
>
> What about updating the mysql tables from 9 to 13? Where does that happen.
>
> Sorry for all the needy questions. This is my first time not using the
> bundled Ubuntu package support for Bacula  which handles updates
> transparently. Debian/Ubuntu has no Bacula maintainer now.
>
> ---
>
> Side issue about the Documentation, what is the "concatenate" note
> about. It makes no sense, deb would just be an ignored comment.
>
> Complete example:
>
> #Bacula Community
> deb [arch=amd64]
> https://www.bacula.org/packages/abc123defxxxyyy/debs/13.0.1 xenial main
>
> Note 1: please concatenate the last 2 lines into 1 line.
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula Community Installation Guide

[Rob Gerber]

When I installed bacula 13.0.1 a year or two ago, they only had support for
20.x focal, not the more modern 22.x "whatever the name was". I selected
the most modern available distro in the repo, which was focal. It worked ok.

I believe you should be able to paste the provided repo link into a browser
and see the available os versions as folders.

I used the apt-key option in 22.x. not the best practice, but it's what I
did. I'll have to fix it later.

There isn't a super simple replacement for apt-key at this time. The
general issue is that using apt-key grants global unmitigated trust to
anyone publishing repos with that key, even if they push their code via
someone else's repo servers. Bad guy could steal signing keys from repo A,
subvert repo B servers or domain, then push malicious code to anyone who
trusted repo B.

The most upvoted answer in below stack exchange link gives some information
on how to import a key using better practices that will limit the scope of
your trust to the one repository the key is associated with.
https://askubuntu.com/questions/1286545/what-commands-exactly-should-replace-the-deprecated-apt-key

The most upvoted Unix stack exchange answer here talks about WHY this
matters.
https://unix.stackexchange.com/questions/463072/why-does-the-apt-key-man-page-advise-against-using-its-add-command/463140#463140


Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Jun 30, 2023, 1:12 PM Ken Mandelberg  wrote:

> Ben
>
> Thanks! I now have the access key. But going forward another obstacle.
> I'm running Ubuntu 23.04 Lunar Lobster. apt-key is no longer available
> in 23.04, so the instructions for importing the GPG key won't work. Is
> there a simple replacement, and maybe more to the point if I get as far
> as "deb" for sources, will the repository have Lunar?
>
> Ken
>
> Ben Tyger wrote on 6/30/23 12:12:
> >
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.bacula.org%2Fbacula-binary-package-download%2F=05%7C01%7Ckm%40mathcs.emory.edu%7Ccf20eaabdd404567f8b408db7984dc88%7Ce004fb9cb0a4424fbcd0322606d5df38%7C0%7C0%7C638237383887165100%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C=b2k4q2Gw622smLbaKpbvlZxcoxkF6eHlaCqk%2BsJgdag%3D=0
> > is the registration page. You'll get the specialized link in your e-mail
> >
> >
> > On 6/30/23 11:27, Ken Mandelberg wrote:
> >> I'm trying to use the instructions for installing the community
> >> version binaries for Bacula 13 on Ubuntu.
> >>
> >>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.bacula.org%2Fwhitepapers%2FCommunityInstallationGuide.pdf=05%7C01%7Ckm%40mathcs.emory.edu%7Ccf20eaabdd404567f8b408db7984dc88%7Ce004fb9cb0a4424fbcd0322606d5df38%7C0%7C0%7C638237383887165100%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C=Q5lKyN1XZJHNI%2BjCJJU66hC6uSC7qSRzqOQsf4661hM%3D=0
> >>
> >>
> >> To get started setting up sources for apt it needs an access key
> >> which it says "This is the trailing path component sent in the
> >> registration email".
> >>
> >> I can't find any place to register for the community version which
> >> should be free.
> >>
> >> Any ideas?
> >>
> >>
> >>
> >> ___
> >> Bacula-users mailing list
> >> Bacula-users@lists.sourceforge.net
> >>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fbacula-users=05%7C01%7Ckm%40mathcs.emory.edu%7Ccf20eaabdd404567f8b408db7984dc88%7Ce004fb9cb0a4424fbcd0322606d5df38%7C0%7C0%7C638237383887165100%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000%7C%7C%7C=6UXvNFZgmFnVZcDJGG4QQB5iLXmrEhCHyO24kcMxr4E%3D=0
> >>
> >>
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula Community Installation Guide

[Rob Gerber]

Don't feel bad. I've gotten this access key for my installations before and
I struggled to find it.

Go to the link below. It looks like a "contact us" form is at the bottom of
the page, but that's actually the registration form. Give name and a valid
email, and your access key will be emailed to you. It'll be the
alphanumeric portion of the url they send you. Don't lose that, as you'll
need it in your configuration now and again.

https://www.bacula.org/bacula-binary-package-download/

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Jun 30, 2023, 10:28 AM Ken Mandelberg  wrote:

> I'm trying to use the instructions for installing the community version
> binaries for Bacula 13 on Ubuntu.
>
> https://www.bacula.org/whitepapers/CommunityInstallationGuide.pdf
>
> To get started setting up sources for apt it needs an access key which
> it says "This is the trailing path component sent in the registration
> email".
>
> I can't find any place to register for the community version which
> should be free.
>
> Any ideas?
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Bacula 13.0.x compatibility with 9.4 FD

[Rob Gerber]

This information is in the release notes. :)

https://www.bacula.org/bacula-release-13-0-3/

"Compatibility:
As always, both the Director and Storage daemon(s) must be upgraded at
the same time. Any File daemon running on the same machine as a Director
or Storage daemon must be of the same version.

Older File Daemons should be compatible with the 13.0 Director and Storage
daemons. There should be no need to upgrade older File Daemons."

Robert Gerber
402-237-8692
r...@craeon.net

On Fri, Jun 30, 2023, 6:28 AM Yateen Shaligram Bhagat (Nokia) <
yateen.shaligram_bha...@nokia.com> wrote:

> Hi all,
>
>
>
> Can someone tell if Bacula 13.0.3 Dir & SD are compatible to Bacula 9.4 FD.
>
>
>
> Could not get this info  from the the documentation.
>
>
>
> Thanks
>
> Yateen Bhagat
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Tape Libraries

[Rob Gerber]

Not sure why you want to support LTO-6 tapes. Maybe you want to be able to
read old backups using the new system?

As others have said, definitely recommend using LTO-8 or LTO-9.

If you need the ability to read older backups on LTO-6 media, maybe you
could install a single LTO-6 drive into a modern tape changer. As far as
sourcing an LTO-6 drive, used or refurbished is probably your only option
here unless you already have a suitable LTO-6 drive. LTO-7 drives should be
able to read LTO-6 media, if you can't find an LTO-6 drive.

I bought our organization's hardware from backupworks.com in the USA. They
were really helpful when it came to selecting hardware. I don't have an
affiliation or any profit motive here, just that's who we went through and
I've found them to be very helpful.

We went with a Qualstar q24 library and a single LTO-8 SAS drive. You may
be interested in the Qualstar Q40, which expandable up to 280 slots or so.
The other Qualstar libraries are not expandable. The Q40 base unit contains
the robot, and the expansion modules just have slots for drives. They save
money by using only one robot to service all the drives, so expansion
modules aren't terribly expensive. Most of the cost of a library will be
the drives, not the library itself.

https://www.backupworks.com/Qualstar-Q40-LTO-8-Tape-Library.aspx

AFAIK most libraries are built by the same company these days, some outfit
in germany. Most drives are built by IBM or HP, with only IBM developing
new tape mechanisms at this point in time.

When considering different library manufacturers, knowing that the
libraries were just rebadged offerings from the same company made it a lot
easier to select a product. I can say that I saw some offerings from HP
that locked away some of the library slots behind a license, which I didn't
care for at all. Sell me my hardware and let me use it all, you *$%#^

FYI if you want to be able to read old backups on LTO-6 media, you'll need
to get software set up to read those backups. Bacula uses its own format.

If you're considering sticking with LTO-6 media because you already have a
bunch of LTO-6 tapes, I advise you to strongly consider upgrading to newer
media standards like LTO 8 or LTO 9. Your cost per TB will be MUCH lower
for the newer formats.

Regardless of whether you buy new media or try to use existing LTO-6 media,
you'll need unique barcode labels for each tape. It's imperative that you
apply the labels carefully and that they're within the indented area
designated for the barcodes. Many tape robot jam issues are related to
improper labeling, from what I read.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Tue, Jun 27, 2023 at 1:33 AM Dr. Thorsten Brandau <
thorsten.bran...@brace.de> wrote:

>
> IT Manager via Bacula-users schrieb am 26.06.23 um 12:32:
>
> I’m looking at replacing 4 different backup systems with a single solution
> that can backup Windows (server and desktop), Linux and Unix. Bacula is
> currently very high up the list but there’s one make or break requirement,
> which is tape backups.
>
> Due to the number of systems we need to backup, we need a decent sized
> tape library (40+ tapes) that can handle 3 or 4 tape drives and LTO 6 tapes.
>
>
>
> Can anyone recommend suitable hardware that will work with Bacula without
> too much swearing and cursing being required?
>
>
>
> We are running a Quantum Neolib 24s with one LTO-9 drive.
>
> You can use this lib and install up to 2 drives (LTO what you like). I
> would definetly recommend to use LTO-9 drives, as the money you spend is
> easily and fast recovered by the number of tapes you need, and you will
> probably use it for a long time. Going with an old tape just does not make
> sense. However, the Neolib can use different tapes so you can also start
> with an "older" internal drive and upgrade later.
>
> As the Neolib has two cardridges, you can install different tapes, too.
>
> Bacula works fine with it, if you hack the changer script. There will be a
> new one probably from Bill in the near future that has more options and
> works out of the box, but for now you just have to comment out a time out
> with the Neolib.
>
> The Neo is availabel with up to 80 drives, so plenty of space here.
>
> Cheers
>
> T
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] What is 'voltype'?

[Rob Gerber]

Thank you, Eric!

Ken, I did a little digging through my bacula documentation today and I
discovered that what I was describing is actually "Media type". Media type
is the field used in the storage daemon conf files to define what sort of
tape or other media is being used by bacula. As I described, the media type
defined in the SD conf file is then referenced in the director conf file.
for information on the media type values, search your main.pdf for "media
type". I would say more, but I don't want to contribute misinformation, and
I don't have access to my bacula install at the moment to make sure I am
correct.

As Eric said, I'd check for reasons why the backup might fail. Was it
previously working? Are there available tapes for the backup to run
against? (voltype 2 indicates you're using tape).

Not sure if it'll be helpful, but maybe attempt to run the backup then do

tail /var/log/bacula

Gotta run, so can't say more now. Let me know what you find.


Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Fri, Mar 3, 2023 at 10:04 AM Eric Bollengier via Bacula-users <
bacula-users@lists.sourceforge.net> wrote:

> Hello,
>
> The VolType is an internal field that corresponds to the device type,
> and the device type list is defined in baconfig.h
>
> enum {
> B_FILE_DEV = 1,
> B_TAPE_DEV = 2,
> B_DVD_DEV  = 3,
> B_FIFO_DEV = 4,
> B_VTAPE_DEV= 5,/* change to B_TAPE_DEV after init
> */
> B_FTP_DEV =  6,
> B_VTL_DEV =  7,/* Virtual tape library device */
> B_ADATA_DEV = 8,   /* Aligned data Data file */
> B_ALIGNED_DEV = 9, /* Aligned data Meta file */
> ...
> };
>
> When it is 0, it probably means that the volume was never used. Normally
> it's
> handled automatically, so maybe you have a MediaType or a Pool issue.
>
> Hope it helps!
>
> Best Regards,
> Eric
>
> On 3/3/23 16:48, kjohn...@eclypse.org wrote:
> > Rob,
> >
> > Thanks for this helpful reply.  It had not occurred to me to look
> through the configuration files.
> >
> > As far as the current manual is concerned, "searches in main.pdf dated
> 14 Sep 2022 for "voltype" and "Volume Type" both came up empty."  Perhaps
> different search terms would have been more helpful.
> >
> > I have not used Baculum or BWeb, so I cannot comment there.
> >
> > It may be that my problem had nothing to do with voltype.  The volume I
> was using had recently been added to the pool, and had never been used in
> that pool.  Purging that volume allowed the backup to proceed when I
> restarted the job.  I did not foresee that an empty volume would need to be
> purged before use.
> >
> > Ken
> >
> >
> >
> >
> > From: r...@craeon.net [mailto:r...@craeon.net]
> > Sent: Wednesday, March 01, 2023 3:22 PM
> > To: kjohn...@eclypse.org
> > Cc: bacula-users
> > Subject: Re: [Bacula-users] What is 'voltype'?
> >
> > Voltype is an arbitrary name given to a certain type of volume. It must
> be consistent across all bacula configuration files. For example in my
> setup I have "LTO8" as a voltype. This references my actual LTO 8 tapes,
> but I could have named them anything, like "George" or "duck".
> >
> > I kind of think that maybe a voltype "0" and "2" is not very informative
> for someone who must maintain the system.
> >
> > I think (but can't remember for sure), voltype must be defined in SD
> conf and referenced in bacula director conf for each job.
> >
> > Web tools like Baculum and BWeb may strip out all comments, making it
> more difficult to examine how bacula is configured.
> >
> > I don't have it in front of me, but try to find the current manual pdf
> on the bacula website. I had to dig around to find the correct manual for
> my version.
> > Robert Gerber
> > 402-237-8692
> > r...@craeon.net
> >
> > On Wed, Mar 1, 2023, 1:12 PM  wrote:
> > bconsole list volume displays a field called 'voltype'.  What is this
> used for, and how can I set it?
> >
> > Searches in main.pdf dated 14 Sep 2022 for "voltype" and "Volume Type"
> both came up empty.
> >
> > I have a backup job that should run on a volume that has a voltype of 0,
> and it will not progress.  I observe that all other volumes in that pool
> have a voltype of 2.
> >
> > Thanks.
> >
> >
> > Ken
> >
> >
> >
> >
> >
> > ___
> > Bacula-users mailing list
> > Bacula-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
> >
> >
> >
> > ___
> > Bacula-users mailing list
> > Bacula-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net

Re: [Bacula-users] What is 'voltype'?

[Rob Gerber]

Voltype is an arbitrary name given to a certain type of volume. It must be
consistent across all bacula configuration files. For example in my setup I
have "LTO8" as a voltype. This references my actual LTO 8 tapes, but I
could have named them anything, like "George" or "duck".

I kind of think that maybe a voltype "0" and "2" is not very informative
for someone who must maintain the system.

I think (but can't remember for sure), voltype must be defined in SD conf
and referenced in bacula director conf for each job.

Web tools like Baculum and BWeb may strip out all comments, making it more
difficult to examine how bacula is configured.

I don't have it in front of me, but try to find the current manual pdf on
the bacula website. I had to dig around to find the correct manual for my
version.

Robert Gerber
402-237-8692
r...@craeon.net

On Wed, Mar 1, 2023, 1:12 PM  wrote:

> bconsole list volume displays a field called 'voltype'.  What is this used
> for, and how can I set it?
>
> Searches in main.pdf dated 14 Sep 2022 for "voltype" and "Volume Type"
> both came up empty.
>
> I have a backup job that should run on a volume that has a voltype of 0,
> and it will not progress.  I observe that all other volumes in that pool
> have a voltype of 2.
>
> Thanks.
>
>
> Ken
>
>
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] LTO-8 device or resource busy

[Rob Gerber]

No no, just turn off bacula SD daemon when running tape tests. Otherwise
it's reasonable for SD to be using the tape drive.

Bacula has a script for checking status, starting, stopping, restarting
bacula daemons. Should be in baculadir/scripts iirc.

Robert Gerber
402-237-8692
r...@craeon.net

On Mon, Feb 27, 2023, 11:16 AM Adam Weremczuk 
wrote:

> Thanks, it seems to be working:
>
> btape -c bacula-sd.conf /dev/nst0
> Tape block granularity is 1024 bytes.
> btape: butil.c:290-0 Using device: "/dev/nst0" for writing.
> btape: btape.c:478-0 open device "Quantum LTO-8 HH" (/dev/nst0): OK
> *
>
> Does it mean I should disable bacula-sd service on startup?
>
> On the legacy system that I'm replacing (Debian 7 / Bacula 5.2.6 /
> LTO-4) it's running and auto starts on boot:
>
> service bacula-sd status
> [ ok ] bacula-sd is running.
>
> runlevel
> N 2
>
> ls -l /etc/rc2.d | grep bacula
> lrwxrwxrwx 1 root root  19 Jul  4  2019 S02bacula-sd -> ../init.d/bacula-sd
> lrwxrwxrwx 1 root root  19 Jul  4  2019 S04bacula-fd -> ../init.d/bacula-fd
> lrwxrwxrwx 1 root root  25 Jul  4  2019 S19bacula-director ->
> ../init.d/bacula-director
>
>
> On 27/02/2023 16:14, Bill Arlofski via Bacula-users wrote:
> > On 2/27/23 09:03, Adam Weremczuk wrote:
> >> Hi all,
> >>
> >> My env: Debian 11 / Bacula 9.6.7 / LTO-8. Fresh installation.
> >>
> >> The server has just been rebooted and I'm unable to complete a tape
> test:
> >>
> >> btape -c bacula-sd.conf /dev/nst0
> >> Tape block granularity is 1024 bytes.
> >> btape: butil.c:290-0 Using device: "/dev/nst0" for writing.
> >> btape: device.c:319-0 dev open failed: tape_dev.c:169 Unable to open
> >> device "Quantum LTO-8 HH" (/dev/nst0): ERR=Device or resource busy
> >> 27-Feb 15:57 btape JobId 0: Fatal error: device.c:319 dev open failed:
> >> tape_dev.c:169 Unable to open device "Quantum LTO-8 HH" (/dev/nst0):
> >> ERR=Device or resource busy
> >> btape: butil.c:198-0 Cannot open "Quantum LTO-8 HH" (/dev/nst0)
> >> 27-Feb 15:57 btape JobId 0: Fatal error: butil.c:198 Cannot open
> >> "Quantum LTO-8 HH" (/dev/nst0)
> >
> > Hello Adam,
> >
> > Please make sure that the bacula-sd process is not running. It is
> > probable that it is locking the drive so that other processes cannot
> > access it.
> >
> >
> >> lsof -w | grep /dev/nst
> >> bacula-sd   997 bacula3r CHR
> >> 9,128   0t0218 /dev/nst0
> >> bacula-sd   997  1024 bacula-sd bacula3r CHR
> >> 9,128   0t0218 /dev/nst0
> >
> > ^  And here is the proof ^ :)
> >
> >
> >> cat /etc/bacula/bacula-sd.conf | grep /dev | grep -v ^#
> >> Archive Device = /dev/nst0
> >> Changer Device = /dev/st0
> >
> > Additionally, you will not set a changer device for a stand-alone tape
> > drive.  The human is the "Changer Device" in these cases. :)
> >
> >
> >> What's the reason for my error and how to fix it?
> >
> > Please see above. :)
> >
> >
> > Best regards,
> > Bill
> >
> >
> >
> > ___
> > Bacula-users mailing list
> > Bacula-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] All fields on dashbord are empty

[Rob Gerber]

What messages do you see on baculum API screen? (http://yourIPaddress:9096)
What does the bacula process manager script found in
/opt/bacula/scripts/bacula say?
/opt/bacula/scripts/bacula status

might need to adjust path above to match your bacula install folder.

I don't know if you used apache or lighttpd, but can you restart the
baculum web and baculum api daemons with systemctl or similar?

Was baculum ever working properly? Is this a new installation or an
existing installation that broke?

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Wed, Oct 26, 2022 at 4:35 AM Осташев Сергей Юрьевич <
ostashev...@mpharma.ru> wrote:

> HI!
>
> I checked the settings for connecting the Catalog API to the database.
>
> The test connection was successful.
>
>
>
>
>
>
>
> *From:* Marcin Haba 
> *Sent:* Wednesday, October 26, 2022 12:04 PM
> *To:* Осташев Сергей Юрьевич 
> *Cc:* bacula-users@lists.sourceforge.net
> *Subject:* Re: [Bacula-users] All fields on dashbord are empty
>
>
>
> Hello,
>
> I would propose to look at Baculum API connection to the Bacula Catalog
> database. It looks that something is wrong here. Normally if something goes
> wrong with the connection, you should see a window with a detailed message
> about what is wrong. If you don't see it then I would also advise to check
> if the Baculum API is connected to the same Catalog database as Bacula is.
>
> In any case it might be good to look at the connection Baculum API <=>
> Catalog Database. On the API settings page in the "Catalog" tab there is
> possible to do a connection test.
>
> Good luck.
>
> Best regards,
> Marcin Haba (gani)
>
>
>
> On Wed, 26 Oct 2022 at 10:05, Осташев Сергей Юрьевич <
> ostashev...@mpharma.ru> wrote:
>
> Hello!
>
> I have a problem with baculum.
>
> All fields on dashbord are empty.
>
> Job history list, Client list, Pool list, Volume list - also no any data.
>
> But Job list, Storage list, FileSet list, Schedule list - have data.
>
> I'm already reinstall baculum-web pacage, but nothing changes.
>
>
>
> In developer tools of chrome there is a error in Console:
>
>
>
> graph.js?ver=14:1281 Uncaught TypeError: Cannot convert undefined or null
> to object
>
> at Function.keys ()
>
> at klass.prepare_series (graph.js?ver=14:1281:26)
>
> at klass.initialize (graph.js?ver=14:1275:22)
>
> at new klass (prado.js:103:25)
>
> at Object.update_pie_jobstatus (misc.js?ver=14:569:14)
>
> at Object.update_all (misc.js?ver=14:482:8)
>
> at Object.success ((index):558:16)
>
> at fire (jquery.js:3268:31)
>
> at Object.fireWith [as resolveWith] (jquery.js:3398:7)
>
> at done (jquery.js:9305:14)
>
>
>
> I think it has something to do with my problem.
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
>
>
> --
>
> "Greater love hath no man than this, that a man lay down his life for his
> friends." Jesus Christ
>
> "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za
> przyjaciół swoich." Jezus Chrystus
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] 404 error Apt repository

[Rob Gerber]

Interesting. I typed the IP that gave the 404 into my browser, and it came
up with an invalid certificate. It's registered with lets encrypt. chrome
says the cert's common name isn't valid. I bet our browser sessions are
getting a different server than the server your bacula instance is running
on. or we're getting a different cert for some reason.

I tried to just go to http instead of https, and my browser redirected to
https. still, I think this is a https error. if you request the packages
via http I bet the problem will go away.

Regards,
Robert Gerber
402-237-8692
r...@craeon.net


On Wed, Oct 26, 2022 at 12:46 PM Neil Balchin  wrote:

> I can’t seem to access the Community binaries to update my now out of date
> ubuntu bacula install.
>
>
> Here’s what I’ve done:
>
> Registered for community access
> Received mail with access-keys
>
> Downloaded and added gpg key files using
>
> wget 
> https://www.bacula.org/downloads/Bacula-4096-Distribution-Verification-key.asc
> apt-key add Bacula-4096-Distribution-Verification-key.asc
>
> My output of apt-key list shows :
>
> /etc/apt/trusted.gpg
> 
> pub   rsa4096 2018-09-03 [SC]
>   5235 F5B6 68D8 1DB6 1704  A82D C0BE 2A5F E9DF 3643
> uid   [ unknown] Bacula 4096 Distribution Verification Key (
> www.bacula.org) 
> sub   rsa4096 2018-09-03 [E]
>
> /etc/apt/trusted.gpg.d/ubuntu-keyring-2012-archive.gpg
> --
> pub   rsa4096 2012-05-11 [SC]
>   790B C727 7767 219C 42C8  6F93 3B4F E6AC C0B2 1F32
> uid   [ unknown] Ubuntu Archive Automatic Signing Key (2012) <
> ftpmas...@ubuntu.com>
>
> /etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg
> --
> pub   rsa4096 2012-05-11 [SC]
>   8439 38DF 228D 22F7 B374  2BC0 D94A A3F0 EFE2 1092
> uid   [ unknown] Ubuntu CD Image Automatic Signing Key (2012) <
> cdim...@ubuntu.com>
>
> /etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg
> --
> pub   rsa4096 2018-09-17 [SC]
>   F6EC B376 2474 EDA9 D21B  7022 8719 20D1 991B C93C
> uid   [ unknown] Ubuntu Archive Automatic Signing Key (2018) <
> ftpmas...@ubuntu.com>
>
>
> Created new file at /etc/apt/sources.list.d/.  Called bacula-community.list
>
> Here’s my deb line.  (I’ve x’d out my access-keys,  yes I’m sure they are
> accurate I can paste this link into a web browser and I see the files)
>
> *#Bacula Community*
> deb* [arch=amd64] **https://www.bacula.org/packages/x/debs/9.6.7
> ** focal** main*
>
>
> When I run. apt-get update
>
> That’s when I get the 404 error
>
>
> root@bacula:/tmp# apt-get update
> Hit:1 http://ca.archive.ubuntu.com/ubuntu focal InRelease
> Hit:2 http://ca.archive.ubuntu.com/ubuntu focal-updates InRelease
>
> Hit:3 http://ca.archive.ubuntu.com/ubuntu focal-backports InRelease
>
> Hit:4 http://ca.archive.ubuntu.com/ubuntu focal-security InRelease
>
> Ign:6 https://www.bacula.org/packages/x/debs/9.6.7 focal
> InRelease
> Hit:5 https://www.bacula.org/downloads/baculum/stable/ubuntu focal
> InRelease
> Err:7 https://www.bacula.org/packages/x/debs/9.6.7 focal
> Release
>   404  Not Found [IP: 94.103.98.87 443]
> Reading package lists... Done
> E: The repository 'https://www.bacula.org/packages/x/debs/9.6.7
> focal Release' does not have a Release file.
> N: Updating from such a repository can't be done securely, and is
> therefore disabled by default.
> N: See apt-secure(8) manpage for repository creation and user
> configuration details.1
>
> If I type the deb url into a web browser,  I get a perfect directory
> listing
>
>
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users