I'm glad you found a solution.
Can you tell us what issues you found and what the solution was? That might
help someone else in the future.
__Martin
>>>>> On Thu, 19 Dec 2024 15:48:55 +0100, Samuel Zaslavsky said:
>
> Hi Martin,
> I managed to get back on my feet: there were several issues that made
> things difficult to understand, but you helped me a lot.
> Thank you so much !
> And thanks also to everyone who spent time on my problem !
> Best,
> Sam
>
>
> Le ven. 13 déc. 2024 à 19:05, Martin Simmons <mar...@lispworks.com> a
> écrit :
>
> > Using 'o' in the fileset accurate option has caused problems with restore
> > for
> > another user on the list recently. I don't know if it could also break
> > estimate. Have you used the 'o' option in previous incremental backups?
> >
> > Accurate backups assume that the full path to each file is the same as
> > before.
> > Has /path/to/ARCHIVES (the root of the backup) changed since previous
> > backups?
> > Have you renamed/moved any files or directories within the files being
> > backed
> > up?
> >
> > You could try to find more info about some file that is being included
> > unexpectedly by:
> >
> > 1. Add the listing option to the two estimate commands and record the
> > outputs
> > somewhere.
> >
> > 2. Find a file in the accurate=yes listing that doesn't appear in the
> > accurate=no listing.
> >
> > 3. Run this SQL command:
> >
> > SELECT DISTINCT Job.JobId,StartTime AS JobStartTime,VolumeName, File.lstat
> > FROM Job,Path,File,Media,JobMedia,Client
> > WHERE File.JobId=Job.JobId
> > AND File.FileIndex > 0
> > AND Path.Path='%1'
> > AND File.FileName='%2'
> > AND Client.Name='%3'
> > AND Path.PathId=File.PathId
> > AND JobMedia.JobId=Job.JobId
> > AND JobMedia.MediaId=Media.MediaId
> > AND Client.ClientId=Job.ClientId
> > ORDER BY Job.StartTime DESC LIMIT 1;
> >
> > where %1 is the full path to the directory of the file with trailing
> > slash, %2
> > is the filename of the file and %3 is the Bacula client name (lto8-fd).
> >
> > 4. Run the shell commands:
> >
> > stat /full/path/to/filename
> >
> > stat -t /full/path/to/filename
> >
> > __Martin
> >
> >
> > >>>>> On Wed, 11 Dec 2024 11:37:19 +0100, Samuel Zaslavsky said:
> > >
> > > Hello everyone !
> > >
> > > Any ideas for solving this problem?
> > > I'm stuck here...
> > >
> > > Thanks a lot for your help,
> > >
> > > Best,
> > >
> > > Sam
> > >
> > >
> > > Le ven. 6 déc. 2024 à 12:38, Samuel Zaslavsky <s...@w4tch.tv> a écrit :
> > >
> > > > Hello all,
> > > >
> > > > Sorry for this late reply. (According to Murphy's law, our Bacula
> > server
> > > > failed, and it took some time to figure out that the SAS card was
> > dead, and
> > > > get another one ...)
> > > >
> > > > I did some tests, and there's something new :
> > > > In fact, the fileset seems to be OK, and the Incremental "could"
> > resume...
> > > >
> > > > If I do an estimate without accurate, I get around 18K files / 3To ,
> > which
> > > > looks like "OK" for incremental :
> > > >
> > > > *estimate accurate=no level=incremental job=BackupARCHIVES
> > > > Using Catalog "MyCatalog"
> > > > Connecting to Client lto8-fd at localhost:9102
> > > > 2000 OK estimate files=17,949 bytes=3,465,825,854,672
> > > >
> > > > But with accurate=yes , I get 530K files / 23To, which is (almost ? see
> > > > below) a FULL backup :
> > > >
> > > > *estimate accurate=yes level=incremental job=BackupARCHIVES
> > > > Using Catalog "MyCatalog"
> > > > Connecting to Client lto8-fd at localhost:9102
> > > > 2000 OK estimate files=530,779 bytes=23,005,947,679,372
> > > >
> > > > So I have tried to estimate my job with acurate=true and several
> > options
> > > > for accurate in the fileset :
> > > >
> > > > FileSet {
> > > > Name = "ARCHIVES"
> > > > Ignore FileSet Changes=yes
> > > > Include {
> > > > Options {
> > > > signature=MD5
> > > > *#I have tried many accurate options here : mcso5,
> > mso5,
> > > > so5,M,m,o5,sm,s, or nothing (line commented)*
> > > > accurate=s
> > > > mtimeonly=yes
> > > > #Verify = pin5
> > > > }
> > > > File = /path/to/ARCHIVES
> > > > }
> > > >
> > > > Exclude {
> > > > File = "/path/to/ARCHIVES/#recycle"
> > > > }
> > > > }
> > > >
> > > > But all my attempts with Job accurate=yes give me almost the same
> > result -
> > > > but slightly different sometimes ( could be 531,057 files, or 530,719
> > > > or 530,779 files, depending on the accurate option chosen...)
> > > >
> > > > As I understand it, accurate=s in the Fileset definition should tell
> > > > Bacula that a file with "same path, same size" shouldn't be backed up
> > > > again...
> > > > But this isn't the case and my understanding is clearly wrong...
> > > > And as I understand it also, accurate=no would lead, in case of
> > restore,
> > > > to restore even suppressed and moved files, which is not what I want.
> > > >
> > > > So I am stuck : resume incremental backup without accurate option, or
> > > > restart a full backup with accurate option. Or understand better
> > things ...
> > > > :)
> > > > Can anyone help me there ?
> > > >
> > > > Thanks a lot !
> > > >
> > > > Sam
> > > >
> > > >
> > > > Le mar. 12 nov. 2024 à 17:18, Radosław Korzeniewski <
> > > > rados...@korzeniewski.net> a écrit :
> > > >
> > > >> Hello,
> > > >>
> > > >> wt., 12 lis 2024 o 12:00 Samuel Zaslavsky <s...@w4tch.tv> napisał(a):
> > > >>
> > > >>> So, precisely, how does Bacula "decides" a fileset is different from
> > > >>> another one ?
> > > >>>
> > > >>
> > > >> Bacula is checking the fileset changes based on a fileset content.
> > > >>
> > > >>
> > > >>> Where is this information stored in the database ?
> > > >>>
> > > >>
> > > >> It is a fileset table.
> > > >>
> > > >>
> > > >>> How could I trick bacula to believe that all previous jobs have been
> > > >>> made with the new fileset : This would be OK for me !
> > > >>> My idea was that it would be using the fileset.md5 field (or
> > createtime
> > > >>> ?)
> > > >>>
> > > >>
> > > >> From what I know Bacula is checking the md5 column for this purpose.
> > So,
> > > >> if your new fileset md5 will be the same of which is already saved
> > then
> > > >> Bacula won't have any traces it is a different fileset.
> > > >>
> > > >>
> > > >>>
> > > >>> So, a few related questions : When will bacula add another entry in
> > the
> > > >>> fileset database table ?
> > > >>>
> > > >>
> > > >> If your current configuration includes a selected fileset and md5 does
> > > >> not match the one saved in the database then Bacula will create a new
> > entry.
> > > >>
> > > >>
> > > >>> It seems it's not when doing an estimate...
> > > >>> So, should I start ( and abort quickly) a dummy job using the "new"
> > > >>> fileset, to see the "new" fileset in the database ? And then try to
> > trick
> > > >>> Bacula into thinking the old one is the new one ?
> > > >>>
> > > >>
> > > >> Yes, it sounds good to me.
> > > >>
> > > >>
> > > >>> If this is a pertinent approach, how exactly do that ?
> > > >>> For example, where in the database is the information that a "file"
> > > >>> belongs to some files set ?
> > > >>>
> > > >>
> > > >> Database relation.
> > > >> bacula=# \d file
> > > >> Table "public.file"
> > > >> Column | Type | Collation | Nullable | Default
> > > >>
> > > >>
> > > >>
> > -----------+----------+-----------+----------+--------------------------------------
> > > >> fileid | bigint | | not null |
> > > >> nextval('file_fileid_seq'::regclass)
> > > >> jobid | integer | | not null |
> > > >> ...
> > > >> bacula=# \d job
> > > >> Table "public.job"
> > > >> Column | Type | Collation |
> > Nullable |
> > > >> Default
> > > >>
> > > >>
> > -------------------+-----------------------------+-----------+----------+------------------------------------
> > > >> jobid | integer | | not
> > null |
> > > >> nextval('job_jobid_seq'::regclass)
> > > >> filesetid | integer | |
> > |
> > > >> 0
> > > >> ...
> > > >> bacula=# \d fileset
> > > >> Table "public.fileset"
> > > >> Column | Type | Collation | Nullable |
> > > >> Default
> > > >>
> > > >>
> > ------------+-----------------------------+-----------+----------+--------------------------------------------
> > > >> filesetid | integer | | not null |
> > > >> nextval('fileset_filesetid_seq'::regclass)
> > > >>
> > > >> Is it in the file.lstat column ? If yes, how is it "encoded" here ?
> > > >>>
> > > >>
> > > >> This column is a "direct" serialisation of lstat(2), struct stat {};
> > > >> buffer. It is (to some extent) OS dependent.
> > > >>
> > > >>
> > > >>>
> > > >>> Basically, I need to understand how the relationship between fileset
> > > >>> conf <--> fileset in database <--> files in fileset is handled...
> > > >>>
> > > >>>
> > > >> When Bacula detects a fileset change, because the md5 is different
> > then
> > > >> the one saved in the database then it creates a new entry. Then a job
> > will
> > > >> get a new filesetid as a reference for job entry in the database and
> > all
> > > >> files saved will get a jobid reference.
> > > >>
> > > >>
> > > >>> Or maybe I am completely wrong here ?
> > > >>>
> > > >>
> > > >> You are just asking questions. You can't be wrong asking questions.
> > > >>
> > > >>
> > > >>> The filesystem to be backed up is mounted as NFS share on the bacula
> > > >>> host.
> > > >>>
> > > >>
> > > >> Any chance you can access this filesystem directly without mounting
> > it on
> > > >> the remote host? It would be better.
> > > >>
> > > >>
> > > >>> Is it possible that the reboot/remount changed a key parameter for
> > > >>> Bacula ?
> > > >>>
> > > >>
> > > >> Well. Let's assume your first mount point is: /mnt/data. Then you
> > reboot
> > > >> and the new mount point is /mnt/data1. Then it makes a huge
> > difference for
> > > >> Bacula, for sure.
> > > >>
> > > >>
> > > >>> So that Bacula recognizes well the filesystem as the old one,
> > > >>>
> > > >>
> > > >> It doesn't matter. What matters are some metadata for the file, i.e.
> > > >> modify time. You can configure what metadata should be verified in
> > this
> > > >> case. But you can't force Bacula to not backup a file which does not
> > exist
> > > >> at the previous backup jobs.
> > > >>
> > > >>
> > > >>> but doesn't recognize any file anymore ? ( Remember that fileset
> > name,
> > > >>> and all paths in the fileset configuration are the same as before ! )
> > > >>>
> > > >>
> > > >> I'm pretty sure your fileset content is different then the one used
> > > >> before, so Bacula wants to create a full backup.
> > > >>
> > > >> best regards
> > > >> --
> > > >> Radosław Korzeniewski
> > > >> rados...@korzeniewski.net
> > > >>
> > > >
> > >
> >
>
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
