[bareos-users] Always incremental looking in wrong device
Hi
I try to implement an always Incremental Scheme, i follow the documentation
guide as much as possible (the documentation how to configure multiple
Storages points to an documentation how to configure multiple devices to one
storage) but finally i have the following Problem:
03-Apr 10:38 bareos-dir JobId 164: Start Virtual Backup JobId 164,
Job=backup-desaster-fd.2022-04-02_17.53.41_11
03-Apr 10:38 bareos-dir JobId 164: Consolidating JobIds
9,13,17,21,25,29,33,37,41,45,53,57,61,65,69,73,77,81,85,89,93,97,101,105,109,113,117,121,125,129
03-Apr 10:38 bareos-dir JobId 164: Bootstrap records written to
/var/lib/bareos/bareos-dir.restore.3.bsr
03-Apr 10:38 bareos-dir JobId 164: Connected Storage daemon at esmeralda:9103,
encryption: TLS_CHACHA20_POLY1305_SHA256 TLSv1.3
03-Apr 10:38 bareos-dir JobId 164: Using Device "Incremental" to read.
03-Apr 10:38 bareos-dir JobId 164: Using Device "Consolidated" to write.
03-Apr 10:38 DiskStorage JobId 164: Volume "AI-Consolidated-0005" previously
written, moving to end of data.
03-Apr 10:38 DiskStorage JobId 164: Ready to append to end of Volume
"AI-Consolidated-0005" size=26770517881
03-Apr 10:38 DiskStorage JobId 164: Warning: stored/acquire.cc:284 Read open
device "Incremental" (/var/lib/bareos/storage/incremental) Volume
"AI-Consolidated-0001" failed: ERR=stored/dev.cc:715 Could not open:
/var/lib/bareos/storage/incremental/AI-Consolidated-0001, ERR=File or directory
not found
IE: For consolidating bareos looks for Volumes in the wrong Device.
Regards, Jan
Configuration:
bareos-dir.d/storage/esmeralda.conf:
Storage {
Name = IncrementalStorage
Address = esmeralda
Password = "*"
Device = Incremental
Media Type = File
}
Storage {
Name = ConsolidatedStorage
Address = esmeralda
Password = "*"
Device = Consolidated
Media Type = File
}
Storage {
Name = LongtermStorage
Address = esmeralda
Password = "*"
Device = Longterm
Media Type = File
}
bareos-dir.d/jobdefs/DefaultJob.conf
JobDefs {
Name = "AiDefault"
Accurate = yes
Always Incremental = yes
Always Incremental Job Retention = 14
Always Incremental Keep Number = 7
Always Incremental Max Full Age = 21
Pool = AI-Incremental
Messages = Standard
Type = Backup
Full Backup Pool = AI-Consolidated
}
bareos-dir.d/job/backup-desaster-fd.conf
Job {
Name = "backup-desaster-fd"
Client = "desaster-fd"
Schedule = "DailyAI"
FileSet = "LinuxServer"
JobDefs = "AiDefault"
}
bareos-dir.d/pool/AI-*
Pool {
Name = AI-Incremental
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 360 days
Maximum Volume Bytes = 10G
Label Format = "AI-Incremental-"
Storage = IncrementalStorage
Next Pool = AI-Consolidated
}
Pool {
Name = AI-Consolidated
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 360 days
Maximum Volume Bytes = 50G
Label Format = "AI-Consolidated-"
Storage = ConsolidatedStorage
Next Pool = AI-Longterm
}
Pool {
Name = AI-Longterm
Pool Type = Backup
Recycle = yes
AutoPrune = yes
Volume Retention = 10 years
Maximum Volume Bytes = 50G
Label Format = "AI-Longterm-"
Storage = LongtermStorage
}
--
You received this message because you are subscribed to the Google Groups
"bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to bareos-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/bareos-users/3920809.YFMqr6NV4i%40esmeralda.
signature.asc
Description: This is a digitally signed message part.
Re: [bareos-users] Always incremental looking in wrong device
Am Mittwoch, 6. April 2022, 19:11:58 CEST schrieb Andreas Rogge: > Hi Jan, Hi Andreas > > You have two different Disk storages with the same Media Type. > This will not work, as is described in the docs[1]: > i reconfigured the Storages as you advised, that looks way better now, Thank you. Maybe it is worth to mention this in https://docs.bareos.org/ TasksAndConcepts/VolumeManagement.html#section-multiplestoragedevices As there is only an example for multiple devices using the same directory which uses (in that case correct) the same media Type. Regards, Jan > > [1] > https://docs.bareos.org/Configuration/Director.html#config-Dir_Storage_Media > Type -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/2004698.P3d1FUV0Cj%40esmeralda. signature.asc Description: This is a digitally signed message part.
Re: [bareos-users] Always incremental looking in wrong device
Hi Andreas Am Freitag, 8. April 2022, 15:55:36 CEST schrieb Andreas Rogge: > Hi Jan, > > Am 08.04.22 um 13:10 schrieb Jan Hebler: > > Maybe it is worth to mention this in https://docs.bareos.org/ > > TasksAndConcepts/VolumeManagement.html#section-multiplestoragedevices > > As there is only an example for multiple devices using the same directory > > which uses (in that case correct) the same media Type. > > you're right, and I would be really happy if it was not so easy to get > it wrong. > However, you're referring to a subsection of the "Concurrent Disk Jobs" > chapter. The third paragraph in that chapter is completely devoted to > the very problem you were running into. You are right, there is an example on how to implement this, unfortunally one section above and under the line "concurrent disk jobs". I for myself was not aware that this will helps with the different pools... > > If you have any idea on how and where to document this so people don't > run into that problem all the time, we're really open to suggestions. In the Chapter Always incremental there are an example configuration with three Pools which uses the Storage File1 and File2 (btw: why uses the longterm-pool file1 as Storage?) If the documentation under "Using multiple devices" would keep these Names, that would help a lot. Or if ALL the Disk- related examples would use an continuous Scheme. Like this: You start with 3 Pools: incremental, differential and full. here you could explain the configuration of the Pools, the Storage, show how to use different devices in one storage to split the pools in separate directories, including the device type and so on. Next you could explain an conventional Setup with the usual incremental, differential and full backups. Next you could explain an Always incremental Backup, using differental for the consolidated and full for the Longterm Backups. That way, you could keep the Storage-definitions. There are only changes to the Pools required, if i got anything right. Finally you could show how to made concurrent Disk Jobs by adding devices to the Incremental and the other example by adding an second incremental pool. At least for me that would help a lot. But, then again, i'm not an Backup nor Bacula-Expert and maybe i'ts just my miserable english :-) Regards, Jan -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/2179408.50RSV7S5vU%40esmeralda. signature.asc Description: This is a digitally signed message part.
Re: [bareos-users] Bareos refuses to backup container subvolumes
Hi Am Samstag, 2. Dezember 2023, 12:20:29 CET schrieb Philipp Storz: > Hello, > > Am 02.12.23 um 07:13 schrieb Jan Hebler: > > Hi > > > > Bareos does it's job very well, except that it not want to backup my > > container subvolumes, which are btrfs-subvolumes as well: > > > > > > 02-Dec 06:22 blaster-fd JobId 153: Disallowed filesystem. Will not > > descend from / into > > /srv/gitlab/.local/share/containers/storage/btrfs/subvolumes/253efabb1b8f > > 1d77d6e481a36fbd429c0a077583740fa548dca112d9516670ee > The subvolume seems to be of a different type than what you have specified > in your fileset. > > Probably it makes sense to set the debug level to 200 and enable tracing in > the filedaemon. > > -> setdebug trace=1 level=200 client= > > That should tell you the exact reason in the trace file. > > see: > https://github.com/bareos/bareos/blob/master/core/src/findlib/find_one.cc#L > 111-L121 > > > > I think this points me in the correct direction. According to https://btrfs.readthedocs.io/en/latest/Subvolumes.html, btrfs subvolumes can, but does not need to be mounted like a "normal" Filesystem, but have their own dev-id anyway. This seems to confuse bareos, as (at least this is my understanding of https://github.com/bareos/bareos/blob/6f35a0436dd9e9428f100f28ea931fb37c2f3199/core/src/lib/mntent_cache.cc#L227-L237) bareos reads /proc/mounts into an cache and search for an entry for the device id afterwards. As the subvolumes is not explicit mounted, this fails. Regards, Jan -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to bareos-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/bareos-users/1918270.taCxCBeP46%40esmeralda. signature.asc Description: This is a digitally signed message part.
[bareos-users] Bareos refuses to backup container subvolumes
Hi
Bareos does it's job very well, except that it not want to backup my container
subvolumes, which are btrfs-subvolumes as well:
02-Dec 06:22 blaster-fd JobId 153: Disallowed filesystem. Will not descend
from / into
/srv/gitlab/.local/share/containers/storage/btrfs/subvolumes/253efabb1b8f1d77d6e481a36fbd429c0a077583740fa548dca112d9516670ee
The stats of the parent directory:
blaster:/etc/bareos # stat
/srv/gitlab/.local/share/containers/storage/btrfs/subvolumes
File: /srv/gitlab/.local/share/containers/storage/btrfs/subvolumes
Size: 1152Blocks: 0 IO Block: 4096 directory
Device: 0,278 Inode: 302 Links: 1
Access: (0700/drwx--) Uid: ( 485/ gitlab) Gid: ( 470/ gitlab)
The stats of the entry:
File:
/srv/gitlab/.local/share/containers/storage/btrfs/subvolumes/253efabb1b8f1d77d6e481a36fbd429c0a077583740fa548dca112d9516670ee
Size: 154 Blocks: 0 IO Block: 4096 directory
Device: 0,280 Inode: 256 Links: 1
Access: (0555/dr-xr-xr-x) Uid: ( 485/ gitlab) Gid: ( 470/ gitlab)
btrfs-info:
blaster:/etc/bareos # btrfs subvolume show
/srv/gitlab/.local/share/containers/storage/btrfs/subvolumes/253efabb1b8f1d77d6e481a36fbd429c0a077583740fa548dca112d9516670ee
.local/share/containers/storage/btrfs/subvolumes/253efabb1b8f1d77d6e481a36fbd429c0a077583740fa548dca112d9516670ee
Name:
253efabb1b8f1d77d6e481a36fbd429c0a077583740fa548dca112d9516670ee
UUID: bf8a44fb-f7d5-5447-9f79-100512e68408
Parent UUID:03f24745-8420-6945-bb31-b0ff20375173
Received UUID: -
Creation time: 2023-10-15 11:25:16 +0200
Subvolume ID: 257
Generation: 33
Gen at creation:32
Parent ID: 5
Top level ID: 5
Flags: -
Send transid: 0
Send time: 2023-10-15 11:25:16 +0200
Receive transid:0
Receive time: -
Snapshot(s):
.local/share/containers/storage/btrfs/subvolumes/86d9eaee6681085d31997c5d96883fbba46d136066a63330c5e749bef070c150
blaster:/etc/bareos #
The Fileset for this is:
FileSet {
Name = "LinuxAll"
Description = "Backup all regular filesystems, determined by filesystem type."
Include {
Options {
Signature = MD5 # calculate md5 checksum per file
One FS = No # change into other filessytems
FS Type = btrfs
FS Type = ext2 # filesystems of given types will be backed up
FS Type = ext3 # others will be ignored
FS Type = ext4
FS Type = reiserfs
FS Type = jfs
FS Type = vfat # UEFI
FS Type = xfs
FS Type = zfs
wilddir = /shared/media/video/2*
}
File = /
}
# Things that usually have to be excluded
# You have to exclude /var/lib/bareos/storage
# on your bareos server
Exclude {
File = /var/lib/bareos
File = /var/lib/bareos/storage
File = /proc
File = /tmp
File = /var/tmp
File = /shared/media/video/other
File = /shared/media/video/receiver
File = /shared/media/video/music
File = /.journal
File = /.fsck
File = /.snapshots
}
}
Regards, Jan
--
You received this message because you are subscribed to the Google Groups
"bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to bareos-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/bareos-users/1951066.PYKUYFuaPT%40esmeralda.
