Ok, now that the memory issues have been resolved, it has uncovered
another issue. It appears that the job that I am attempting to restore
from failed to include the target files, because it did not descend
into the /export directory with the following error message:
CSSPFILE01-fd JobId 51714: /export is a different filesystem. Will not
descend from / into it.
The fileset used for this job specifies "One FS = No", and at some
point in the past this job was descending into /export. / and /export
are separate ext4 partitions. Here are the most relevant configs for
the job and fileset:
FileSet {
Name = agent-fs
Description = "Backup all regular filesystems, including postgres."
Include {
Options {
#compression=GZIP
Signature = MD5
One FS = No # change into other filessytems
FS Type = btrfs
FS Type = ext2
FS Type = ext3
FS Type = ext4
FS Type = jfs
FS Type = xfs
FS Type = zfs
}
Options {
# Wild Dir directives are not allowed in Exclude blocks, so
placing here
Regex Dir = ".*/\.opam/*"
Regex Dir = ".*/\.gradle/*"
Regex Dir = ".*/\.vscode/*"
Wild Dir = "/usr/src"
Wild Dir = "/lib/modules"
Exclude = yes
}
File = /
Exclude Dir Containing = .nobackup
}
Exclude {
File = /var/lib/bareos
File = /var/lib/bareos/storage
File = /proc
File = /tmp
File = /dev
File = /run
File = /sys
File = /var/tmp
File = /.journal
File = /.fscka
File = /var/lib/lxcfs
File = */.local
File = /swapfile
File = /export/archive
File = /snap
}
}
Job {
Name = file01-bu
Client = csspfile01-fd
Type = Backup
FileSet = agent-fs
Schedule = MonthlyCycle
Messages = StandardNew
Pool = IncrPool1
Level = Incremental
Write Bootstrap = "/var/lib/bareos/%c.bsr"
Full Backup Pool = FullPool0
Differential Backup Pool = DiffPool0
Incremental Backup Pool = IncrPool1
Priority = 10
RunScript {
RunsWhen = Before
RunsOnClient = yes
Fail Job On Error = yes
Command = "sudo rear mkrescue"
}
Run Script {
Console = ".bvfs_update"
RunsWhen = After
RunsOnClient = No
}
}
I have been able to force the backup of /export going forward by
adding a "File = /export" directive, but my understanding of how this
should work (based on
https://docs.bareos.org/Configuration/Director.html#config-Dir_Fileset_Include_Options_OneFs)
is that this should not be necessary. Happy to provide any other
configs, traces, etc as needed.
-Toma
On Wednesday, October 30, 2024 at 9:26:25 AM UTC-4 Bruno Friedmann
(bruno-at-bareos) wrote:
Thanks for the feedback and the report.
On Tuesday 29 October 2024 at 16:18:44 UTC+1 Toma Morris wrote:
Thank you, that pointed me in the right direction. That query
succeeded, but took a very long time. I did "explain analyze"
on it, and saw that it was using much more than than the
allotted working memory (using 334Mb, where work_mem in
postgres was set to 4Mb), which caused postgres to use an
external merge rather than quicksort. I set work_mem to 512MB,
which brought the execution time from 224 s to 38 s and
allowed the Bareos restore interface to function.
On Mon, Oct 28, 2024 at 6:59 AM Bruno Friedmann
(bruno-at-bareos) <[email protected]> wrote:
Hi Toma,
It look like the query building the tree didn't finish or
vanish.
This may happen when your PostgreSQL cluster doesn't have
enough free space to store tmp queries/tables.... During a
very limited time, the disk space become full and the
connection is then cleared, PG dropping the running
worker, free space reappear just after.
You may want to try the following query (issue from your
trace) and watch carefully the free space available where
you PG cluster reside and build its temp data
SELECT DISTINCT ON (PathId) 'D', SpecialDir.PathId,
SpecialDir.Path, JobId, LStat, FileId FROM ( SELECT
2711925 AS PathId, '.' AS Path UNION SELECT PPathId AS
PathId, '..' AS Path FROM PathHierarchy WHERE PathId =
2711925 ) AS SpecialDir LEFT JOIN ( SELECT PathId, JobId,
LStat, FileId FROM File WHERE File.Name = '' AND
File.JobId IN
(46255,46283,46436,46623,46653,51205,51516,51538,51560,51582,51604)
ORDER BY PathId ASC, JobId DESC ) AS DirAttribute ON
(SpecialDir.PathId = DirAttribute.PathId) UNION ( SELECT
DISTINCT ON (Path) 'D', PathId, Path, JobId, LStat, FileId
FROM ( SELECT Path1.PathId AS PathId, Path1.Path AS Path,
lower(Path1.Path) AS lpath, listfile1.JobId AS JobId,
listfile1.LStat AS LStat, listfile1.FileId AS FileId FROM
( SELECT listpath1.PathId AS PathId FROM ( SELECT DISTINCT
PathHierarchy1.PathId AS PathId FROM PathHierarchy AS
PathHierarchy1 INNER JOIN Path AS Path2 ON
(PathHierarchy1.PathId = Path2.PathId) INNER JOIN
PathVisibility AS PathVisibility1 ON
(PathHierarchy1.PathId = PathVisibility1.PathId) WHERE
PathHierarchy1.PPathId = 2711925 AND PathVisibility1.JobId
IN
(46255,46283,46436,46623,46653,51205,51516,51538,51560,51582,51604)
) AS listpath1 LEFT JOIN ( SELECT PVD1.PathId AS PathId
FROM ( SELECT PV1.PathId AS PathId, MAX(JobId) AS MaxJobId
FROM PathVisibility AS PV1 WHERE JobId IN
(46255,46283,46436,46623,46653,51205,51516,51538,51560,51582,51604)
GROUP BY PathId ) AS PVD1 INNER JOIN File AS F2 ON
(F2.PathId = PVD1.PathId AND F2.JobId = PVD1.MaxJobId AND
F2.FileIndex = 0 AND F2.Name = '') ) AS listpath2 ON
(listpath1.PathId = listpath2.PathId) WHERE
listpath2.PathId IS NULL ) AS listpath3 INNER JOIN Path
AS Path1 ON (listpath3.PathId = Path1.PathId) LEFT JOIN (
SELECT File1.PathId AS PathId, File1.JobId AS JobId,
File1.LStat AS LStat, File1.FileId AS FileId FROM File AS
File1 WHERE File1.Name = '' AND File1.JobId IN
(46255,46283,46436,46623,46653,51205,51516,51538,51560,51582,51604)
) AS listfile1 ON (listpath3.PathId = listfile1.PathId) )
AS A ORDER BY Path ASC, JobId DESC ) ORDER BY Path
ASC,JobId DESC LIMIT 1000 OFFSET 0
On Friday 25 October 2024 at 10:37:27 UTC+2 Toma Morris wrote:
Here is the debug trace during attempted restore,
resulting in the same error condition. Thank you!
-Toma
On Thursday, October 24, 2024 at 12:40:24 AM UTC-4
Sebastian Sura wrote:
Hi Toma,
could you try doing a debug trace on the director
when you are trying the restore ?
You can enable the debug log by writing
setdebug level=200 trace=1 director
in the bconsole. Once you have reproduced the
issue, you can turn it off with e.g. setdebug
level=10 trace=0 director again.
Having this debug trace would be immensely helpful.
Kind Regards,
Sebastian Sura
Am 23.10.24 um 19:47 schrieb Toma Morris:
Hi,
I am trying to restore files from one of my
backups. In the Restore tab of the webui, I
select the client and then after a delay I get
the following error message:
{"id":"#","xhr":{"readyState":4,"responseText":"Decoding
failed: Syntax errorError: API 2 not available on
director.\n Please upgrade
to verison 15.2.2 or greater and/or compile with
jansson support.","status":200,"statusText":"OK"}
Bareos is running on an Ubuntu 20.04 server with
hostname cscisbu01, and I am trying to access a
backup from a client with hostname csspfile01. I
initially got this error while on
23.0.4~pre169.6644119b5-114. I upgraded bareos on
both the server and client to the most recent
version via apt, which brought me up to
23.0.5~pre146.7e91df1c0-130 with no apparent
change in behavior.
I have successfully restored files within the
last couple of months on this server from the
same client, and have not changed any
configurations since then.
I've attached my apache2 error logs, which
contain some possibly-relevant php errors. I
didn't see anything relevant in
/var/log/bareos/bareos.log or
/var/log/bareos/bareos-audit.log, but can provide
those or anything else if needed.
Thanks,
-Toma
--
You received this message because you are
subscribed to the Google Groups "bareos-users" group.
To unsubscribe from this group and stop receiving
emails from it, send an email to
[email protected].
To view this discussion visit
https://groups.google.com/d/msgid/bareos-users/4ea3f42e-9b0e-4c3f-b28d-a2f382b3d81an%40googlegroups.com
<https://groups.google.com/d/msgid/bareos-users/4ea3f42e-9b0e-4c3f-b28d-a2f382b3d81an%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
Sebastian [email protected]
Bareos GmbH & Co. KG Phone: +49 221 630693-0
https://www.bareos.com
Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646
Komplementär: Bareos Verwaltungs-GmbH
Geschäftsführer: Stephan Dühr, Jörg Steffens, Philipp
Storz
--
You received this message because you are subscribed to a
topic in the Google Groups "bareos-users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/bareos-users/t7mBaL-hNKI/unsubscribe.
To unsubscribe from this group and all its topics, send an
email to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/bareos-users/cb1770ec-d5b0-4744-9d56-46eb8a30ccfdn%40googlegroups.com
<https://groups.google.com/d/msgid/bareos-users/cb1770ec-d5b0-4744-9d56-46eb8a30ccfdn%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google
Groups "bareos-users" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected].
To view this discussion visit
https://groups.google.com/d/msgid/bareos-users/0d04ff3a-043d-4f41-8cee-ae5386e11580n%40googlegroups.com
<https://groups.google.com/d/msgid/bareos-users/0d04ff3a-043d-4f41-8cee-ae5386e11580n%40googlegroups.com?utm_medium=email&utm_source=footer>.
