[Bacula-users] Error: bsock.c:609 Decompress error!!!! ERR=-261 (Incremental Fail, but FULL OK with Rescheduled)

[Jose Alberto]

I have the next error:

Backup Incremental:

JobId 2480: Error: bsock.c:615 Read expected 267 got -261 from client:
10.0.0.98:9103

JobId 2480: Fatal error: append.c:320 Network error reading from FD.
ERR=Input/output error
JobId 2480: Elapsed time=00:12:13, Transfer rate=29.10 M Bytes/second
JobId 2480: Error: lib/bsock.c:391 Write error sending 271 bytes to
Storage daemon:10.0.0.190:9103: ERR=Input/output error
srvwin098-fd JobId 2480: Fatal error: filed/backup.c:912 Network send
error to SD. ERR=Input/output error

JobId 2480: Fatal error: filed/backup.c:912 Network send error to SD.
ERR=Input/output error

Error: lib/bsock.c:276 Socket has errors=1 on call to Storage
daemon:10.0.0.190:9103

Elapsed time:   12 mins 33 secs

  Priority:   10
  FD Files Written:   1
  SD Files Written:   1
  FD Bytes Written:   21,764,374,528 (21.76 GB)
  SD Bytes Written:   21,333,672,138 (21.33 GB)
  Rate:   28903.6 KB/s
  Software Compression:   2.0% 1.0:1
  Comm Line Compression:  79.1% 4.8:1

STatus FAIL.


But   Run  FULL

Error: lib/bsock.c:391 Write error sending 271 bytes to Storage
daemon:10.0.0.190:9103: ERR=Input/output error

srvwin-098-fd JobId 2591: Fatal error: filed/backup.c:912 Network
send error to SD. ERR=Input/output error

Error: lib/bsock.c:276 Socket has errors=1 on call to Storage
daemon:10.0.0.190:9103

SD termination status:  Incomplete job

Termination:Backup failed -- Incomplete

srvlnxbacula-dir JobId 2591: Rescheduled Job
srvwin-098-job.2021-12-23_08.08.04_29 at 23-Dec-2021 08:33 to re-run
in 1800 seconds (23-Dec-2021 09:03).
srvlnxbacula-dir JobId 2591: Job srvwin-098-job.2021-12-23_08.08.04_29
waiting 1800 seconds for scheduled start time.


bconsole

Running Jobs:
Console connected at 23-Dec-21 08:55
 JobId  Type Level Files Bytes  Name  Status
==
  2591  Back Full35039.02 G srvwin-098-job   is waiting
for its start time (23-Dec 09:03)


After 30min  (1800 secunds)


srvlnxbacula-dir JobId 2591: Restart Incomplete Backup JobId 2591,
Job=srvwin-098-job.2021-12-23_08.08.04_29
srvlnxbacula-dir JobId 2591: Found 348 files from prior incomplete Job.
srvlnxbacula-dir JobId 2591: Using Device "DRIVE0" to write.
srvlnxbacula-dir JobId 2591: Sending Accurate information to the FD.
 JobId 2591: Generate VSS snapshots. Driver="Win64 VSS"
 JobId 2591: Snapshot mount point: C:\
JobId 2591: Snapshot mount point: D:\


on the 3rd rescheduled the JOB could be completed without problems


Resumen:  Incremental FAIL.

FULL OK  after  variuos rescheduled.


Bacula 11.0.5

Agent: 11.0.5 (03Jun21) Microsoft Windows Server 2012 Standard Edition
(build 9200), 64-bit,Cross-compile,Win64

Debian 10.








-- 
#
#   Sistema Operativo: Debian  #
#Caracas, Venezuela  #
#
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] make_catalog_backup stopped working

[Phil Stracchino]

On 12/23/21 11:19, Graham Dicker wrote:

Thank you for your reply Graham.

I don't think Bacula itself has any problems, it is only the script mysqldump
that has a problem. The script is just supposed to create a dump of the
database and (if it is successful) then Bacula goes on to first back it up and
then delete it.



I'm gonna be very blunt here.  mysqldump is a piece of obsolete garbage 
and nobody should be using it any more.  It CANNOT, period, CORRECTLY 
back up both My ISAM and InnoDB tables in the same dump.  CANNOT. 
(Nobody should be using the MyISAM storage engine any more either, but 
that's another issue.)


Take a look at mydumper instead.  It is better than mysqldump in almost 
every imaginable way.  It is parallelized, storage engine aware, and 
immensely easier to do partial restores with (i.e a single schema or a 
handful of tables).  It is what mysqldump SHOULD have evolved into 
fifteen years ago.



--
  Phil Stracchino
  Babylon Communications
  ph...@caerllewys.net
  p...@co.ordinate.org
  Landline: +1.603.293.8485
  Mobile:   +1.603.998.6958


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore experience report

[Graham Sparks]

Hi.

The backup speed seems reasonable, and the Bacula manual states that restores 
can reasonably be as much as one-third of the speed of the equivalent backup 
(https://www.bacula.org/11.0.x-manuals/en/main/Restore_Command.html#SECTION002890).
  I notice also that you are restoring to an NTFS partition. I'm restoring to 
XFS or ext4, so I'm not sure if the filesystem NTFS is causing the slower 
restore.

I also notice that you have a lot of files for only 32GB-- ~200,000 files.  I'd 
probably call these "small" files.  The smaller the files, the more noticeable 
file creation and attribute application 'slow-down' will be during the restore 
process.

As far as I know, "Comm Line" compression should be enabled by default, and is 
only reported to be "None" if the files are already well-compressed.
Another option that might help, but that should also be enabled by default, is 
the "SpoolAttributes = yes" Job directive.

I'm sorry, but I've run out of suggestions at this point.  Hopefully someone 
with more experience of troubleshooting transfer speeds in Bacula might have 
something more concrete to offer.

I'm hoping to update to a USB3 HDD dock, so I'll be keeping an eye on transfer 
speeds to see what difference this makes.

--
Graham


From: Lionel PLASSE 
Sent: 23 December 2021 15:28
To: Graham Sparks ; bacula-users@lists.sourceforge.net 

Subject: RE: Restore experience report

Thanks

Yes it's on a blue usb port.

I expected too those kind of speed.



I can reach 69614.5 KB/s for a Full backup job for this client (620.6 GB)  and 
3253.3 KB/s  for an other.


And I just see that the difference is only there  is no software compression 
for the first and 53% for the second. I will look after the compression 
settings for the jobs.

And anthor fact is that  my backup job are made 3 by 3. So  Does the concurent 
job affect the performance with perhaps non sequentiel data on the volume ?

I keep  sudying the question  first with compression  settings


--

De : Graham Sparks 
Envoyé : jeudi 23 décembre 2021 16:02
À : bacula-users@lists.sourceforge.net
Objet : Re: Restore experience report

Thanks for the info.  What sort of backup transfer rates are you getting (from 
your clients to the USB-3 backup disk)?

I back up over a 1Gb/s network to a USB2.0 device and get 42MB/s backup. I also 
get 25MB/s restore (restoring from USB2.0 to a directly-connected SSD), so I'd 
expect better for USB-3 without a network bottleneck (perhaps

Forgive me if it's a silly question, but is the USB-3 drive definitely 
connected to a USB-3 (blue, or "SS"-labelled) USB port?

Picking some very (very) arbitrary figures, I'd expect your results to be 
around 100MB/s backup and 50MB/s restore according to the set-up you describe.

--
Graham


From: Lionel PLASSE
Sent: 23 December 2021 13:53
To: mailto:bacula-users@lists.sourceforge.net 

Cc: Graham Sparks 
Subject: RE: Restore experience report

Yes you're right, i explain the situation

The targeted disk for restauration is  a SATA-II directly  connected to the 
mother board with a SATA cable nor USB nor RAID but directly  to the SATA 
controller, it's a small  extension to the front of the computer.

The backup media volumes  are usb disk volume   and  stored  on USB-3  disks .
This  USB disk (XFS formatted)  contains  the volume file and all bsr and 
backup of bacula conf .  It is automatically mounted by linux when plugged on 
the storage dir .
One USB disk file with one media volume file.
All my dayly jobs (9jobs for 9 clients) are run on the volume every day :  
Incremental from Mon to Thu and Differental the Friday and each first Friday it 
is a Full instead of Diff

Bacula-dir sd fd and mariadb are on the same server
I restore on the same machine from the USB volumes to the hotplugged SATA disk.

I  thought that because of  all the restauration process has been made on the 
same server it will be faster ...







--

De : Graham Sparks 
Envoyé : jeudi 23 décembre 2021 14:00
À : mailto:bacula-users@lists.sourceforge.net
Objet : Re: Restore experience report

Hi,

I think a little more info is needed to narrow down whether or not these 
restore transfer figures are slow in this particular case.

How is the hotplug SATA-II restore disk connected (RAID controller, or USB), 
and is it connected to the bacula server directly, or through a client?
Also, where/how is the backup data stored?

--
Graham



From: Lionel PLASSE
Sent: 23 December 2021 09:35
To: mailto:bacula-users@lists.sourceforge.net 

Subject: [Bacula-users] Restore experience report

Hello,

I just post my full restore experience for performance studying purpose.

I use Bacula 9.6.6.3 & - 

Re: [Bacula-users] make_catalog_backup stopped working

[Graham Sparks]

Hi.

I see--it's worth checking then the command that is called by the catalogue 
backup job (the "BeforeJob" directive in the "Job" (or "JobDefs") resource).

On my system the catalogue job calls, in its "BeforeJob" directive, the 
following: "/opt/bacula/etc/make_catalog_backup.pl MyCatalog".

According to the output of Job 65 on your server, "BeforeJob" runs "mysqldump", 
which isn't what I'm expecting (but my config files are from an older version 
of Bacula--so I might be wrong here).

I'd seek out the "BeforeJob =" line of bacula-dir.conf, and check the script it 
executes to make sure it has the credentials it needs.

Thanks.
--
Graham Sparks


From: Graham Dicker
Sent: 23 December 2021 16:19
To: bacula-users@lists.sourceforge.net 
Subject: Re: [Bacula-users] make_catalog_backup stopped working

Thank you for your reply Graham.

I don't think Bacula itself has any problems, it is only the script mysqldump
that has a problem. The script is just supposed to create a dump of the
database and (if it is successful) then Bacula goes on to first back it up and
then delete it.

On Thursday, 23 December 2021 15:14:08 GMT Graham Sparks wrote:
> Hi.
>
> I'm surprised your other backups are working--if you have just one database
> server, your Bacula director should be using the same credentials to
> connect to the database, regardless of the job.
>
> In your "bacula-dir.conf" file, there will be the username and password that
> Bacula uses to connect to the catalogue database:
>
> e.g.
>
> 
>
> Catalog {
>   Name = MyCatalog
>   dbname = "your-bacula-database-name"; dbuser = "your-bacula-user";
> dbpassword = "your-password" }
>
> 
>
>
> Make sure your database name, user and password match the current catalogue
> database settings configured in your bacula-dir.conf.  On your MariaDB
> server, you may need to use:
>
> mysql>  grant all on your-bacula-database-name.* to your-bacula-user
> identified by 'your-password';
>
> to ensure everything is set up correctly again.
>
>
> Thanks.
> --
> Graham Sparks
>
> 
> From: Graham Dicker
> Sent: 23 December 2021 14:51
> To: bacula-users@lists.sourceforge.net 
> Subject: [Bacula-users] make_catalog_backup stopped working
>
>
>
> Further to my previous post I notice that the user table contains the
> following. Is that what it should look like?
>
>
> MariaDB [mysql]> select host,user from user;
>
>
>
>
> 
> From: Graham Dicker
> Sent: 23 December 2021 14:42
> To: bacula-users@lists.sourceforge.net 
> Subject: [Bacula-users] make_catalog_backup stopped working
>
>
> Hi all
>
>
> I'm running Bacula on a desk machine with OpenSuse 15.3 and Bacula 11.0.4.
>
>
> The make_catalog_backup command has stopped working following a complete
> loss and restore of my Bacula database.
>
> I don't know how the database vanished but I restored it from the last
> successful backup thus:
>
>
> ./bextract -b ../working/BackupCatalog.bsr /media/seagate4tb /tmp
>
> mysql
>
> create database bacula;
>
> use bacula;
>
> source bacula.sql;
>
> grant_mysql_privileges
>
>
> But now although my main backup works fine the backup of the catalog gets
> this error:
>
>
> 23-Dec 13:39 vivaldi-dir JobId 65: BeforeJob: mysqldump: Got error: 1044:
> "Access denied for user ''@'localhost' to database 'bacula'" when selecting
> the database
>
>
>
> Any clues? Thanks in advance.






___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] make_catalog_backup stopped working

[Graham Dicker]

Thank you for your reply Graham.

I don't think Bacula itself has any problems, it is only the script mysqldump 
that has a problem. The script is just supposed to create a dump of the 
database and (if it is successful) then Bacula goes on to first back it up and 
then delete it.

On Thursday, 23 December 2021 15:14:08 GMT Graham Sparks wrote:
> Hi.
> 
> I'm surprised your other backups are working--if you have just one database
> server, your Bacula director should be using the same credentials to
> connect to the database, regardless of the job.
> 
> In your "bacula-dir.conf" file, there will be the username and password that
> Bacula uses to connect to the catalogue database:
> 
> e.g.
> 
> 
> 
> Catalog {
>   Name = MyCatalog
>   dbname = "your-bacula-database-name"; dbuser = "your-bacula-user";
> dbpassword = "your-password" }
> 
> 
> 
> 
> Make sure your database name, user and password match the current catalogue
> database settings configured in your bacula-dir.conf.  On your MariaDB
> server, you may need to use:
> 
> mysql>  grant all on your-bacula-database-name.* to your-bacula-user
> identified by 'your-password';
> 
> to ensure everything is set up correctly again.
> 
> 
> Thanks.
> --
> Graham Sparks
> 
> 
> From: Graham Dicker
> Sent: 23 December 2021 14:51
> To: bacula-users@lists.sourceforge.net 
> Subject: [Bacula-users] make_catalog_backup stopped working
> 
> 
> 
> Further to my previous post I notice that the user table contains the
> following. Is that what it should look like?
> 
> 
> MariaDB [mysql]> select host,user from user;
> 
> 
> 
> 
> 
> From: Graham Dicker
> Sent: 23 December 2021 14:42
> To: bacula-users@lists.sourceforge.net 
> Subject: [Bacula-users] make_catalog_backup stopped working
> 
> 
> Hi all
> 
> 
> I'm running Bacula on a desk machine with OpenSuse 15.3 and Bacula 11.0.4.
> 
> 
> The make_catalog_backup command has stopped working following a complete
> loss and restore of my Bacula database.
> 
> I don't know how the database vanished but I restored it from the last
> successful backup thus:
> 
> 
> ./bextract -b ../working/BackupCatalog.bsr /media/seagate4tb /tmp
> 
> mysql
> 
> create database bacula;
> 
> use bacula;
> 
> source bacula.sql;
> 
> grant_mysql_privileges
> 
> 
> But now although my main backup works fine the backup of the catalog gets
> this error:
> 
> 
> 23-Dec 13:39 vivaldi-dir JobId 65: BeforeJob: mysqldump: Got error: 1044:
> "Access denied for user ''@'localhost' to database 'bacula'" when selecting
> the database
> 
> 
> 
> Any clues? Thanks in advance.






___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore experience report

[Lionel PLASSE]

Thanks

Yes it's on a blue usb port. 

I expected too those kind of speed.



I can reach 69614.5 KB/s for a Full backup job for this client (620.6 GB)  and 
3253.3 KB/s  for an other.


And I just see that the difference is only there  is no software compression 
for the first and 53% for the second. I will look after the compression 
settings for the jobs.

And anthor fact is that  my backup job are made 3 by 3. So  Does the concurent 
job affect the performance with perhaps non sequentiel data on the volume ? 

I keep  sudying the question  first with compression  settings 


--

De : Graham Sparks  
Envoyé : jeudi 23 décembre 2021 16:02
À : bacula-users@lists.sourceforge.net
Objet : Re: Restore experience report

Thanks for the info.  What sort of backup transfer rates are you getting (from 
your clients to the USB-3 backup disk)?

I back up over a 1Gb/s network to a USB2.0 device and get 42MB/s backup. I also 
get 25MB/s restore (restoring from USB2.0 to a directly-connected SSD), so I'd 
expect better for USB-3 without a network bottleneck (perhaps 

Forgive me if it's a silly question, but is the USB-3 drive definitely 
connected to a USB-3 (blue, or "SS"-labelled) USB port?

Picking some very (very) arbitrary figures, I'd expect your results to be 
around 100MB/s backup and 50MB/s restore according to the set-up you describe.

-- 
Graham


From: Lionel PLASSE
Sent: 23 December 2021 13:53
To: mailto:bacula-users@lists.sourceforge.net 

Cc: Graham Sparks 
Subject: RE: Restore experience report 
 
Yes you're right, i explain the situation 

The targeted disk for restauration is  a SATA-II directly  connected to the 
mother board with a SATA cable nor USB nor RAID but directly  to the SATA 
controller, it's a small  extension to the front of the computer. 

The backup media volumes  are usb disk volume   and  stored  on USB-3  disks .
This  USB disk (XFS formatted)  contains  the volume file and all bsr and 
backup of bacula conf .  It is automatically mounted by linux when plugged on 
the storage dir .  
One USB disk file with one media volume file.
All my dayly jobs (9jobs for 9 clients) are run on the volume every day :  
Incremental from Mon to Thu and Differental the Friday and each first Friday it 
is a Full instead of Diff  

Bacula-dir sd fd and mariadb are on the same server
I restore on the same machine from the USB volumes to the hotplugged SATA disk. 
 

I  thought that because of  all the restauration process has been made on the 
same server it will be faster ... 







--

De : Graham Sparks  
Envoyé : jeudi 23 décembre 2021 14:00
À : mailto:bacula-users@lists.sourceforge.net
Objet : Re: Restore experience report

Hi,

I think a little more info is needed to narrow down whether or not these 
restore transfer figures are slow in this particular case.

How is the hotplug SATA-II restore disk connected (RAID controller, or USB), 
and is it connected to the bacula server directly, or through a client?
Also, where/how is the backup data stored?

-- 
Graham



From: Lionel PLASSE
Sent: 23 December 2021 09:35
To: mailto:bacula-users@lists.sourceforge.net 

Subject: [Bacula-users] Restore experience report 
 
Hello,

I just post my full restore experience for performance studying purpose.

I use Bacula 9.6.6.3 & - 10.3.30-MariaDB  on  a Debian 11 buster x64 i5-2320 
3Ghz CPU - 32GB ram  

For the 1st restoration (by baculum wizard  interface) FULL on Western Digital 
Blue 500Gb (DOS partition table and  NTFS partition) : 

Where:  /srv
  Replace:    Never
  Start time: 22-déc.-2021 09:19:43
  End time:   22-déc.-2021 12:04:47
  Elapsed time:   2 hours 45 mins 4 secs
  Files Expected: 202,006
  Files Restored: 202,027
  Bytes Restored: 32,650,343,324 (32.65 GB)
  Rate:   3296.7 KB/s
  FD Errors:  0
  FD termination status:  OK
  SD termination status:  OK
  Termination:    Restore OK

~2h25 of restoration cause there was 3 volumes in use for restoration 
(FULL/DIFF) and I didn't noticed the request of changing volume.

The rate is a bit slow I think. (max rate was 5 MB/s)

I'm performing now an other full resto with a greater size of restoration on a 
4TB Westerndigital  Violet (GPT  & NTFS  partition)
The rate is curently 10.62 MB/s

Feed back the result this afternoon (I hope)

What do you think of the performance, is it possible to do better  ? knowing 
that the restored  disk   is on a hotplug SATA II interface. 




___
Bacula-users mailing list
mailto:Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore experience report

[Graham Sparks]

Hi,

I think a little more info is needed to narrow down whether or not these 
restore transfer figures are slow in this particular case.

How is the hotplug SATA-II restore disk connected (RAID controller, or USB), 
and is it connected to the bacula server directly, or through a client?
Also, where/how is the backup data stored?

--
Graham



From: Lionel PLASSE
Sent: 23 December 2021 09:35
To: bacula-users@lists.sourceforge.net 
Subject: [Bacula-users] Restore experience report

Hello,

I just post my full restore experience for performance studying purpose.

I use Bacula 9.6.6.3 & - 10.3.30-MariaDB  on  a Debian 11 buster x64 i5-2320 
3Ghz CPU - 32GB ram

For the 1st restoration (by baculum wizard  interface) FULL on Western Digital 
Blue 500Gb (DOS partition table and  NTFS partition) :

Where:  /srv
  Replace:Never
  Start time: 22-déc.-2021 09:19:43
  End time:   22-déc.-2021 12:04:47
  Elapsed time:   2 hours 45 mins 4 secs
  Files Expected: 202,006
  Files Restored: 202,027
  Bytes Restored: 32,650,343,324 (32.65 GB)
  Rate:   3296.7 KB/s
  FD Errors:  0
  FD termination status:  OK
  SD termination status:  OK
  Termination:Restore OK

~2h25 of restoration cause there was 3 volumes in use for restoration 
(FULL/DIFF) and I didn't noticed the request of changing volume.

The rate is a bit slow I think. (max rate was 5 MB/s)

I'm performing now an other full resto with a greater size of restoration on a 
4TB Westerndigital  Violet (GPT  & NTFS  partition)
The rate is curently 10.62 MB/s

Feed back the result this afternoon (I hope)

What do you think of the performance, is it possible to do better  ? knowing 
that the restored  disk   is on a hotplug SATA II interface.




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] make_catalog_backup stopped working

[Graham Sparks]

Hi.

I'm surprised your other backups are working--if you have just one database 
server, your Bacula director should be using the same credentials to connect to 
the database, regardless of the job.

In your "bacula-dir.conf" file, there will be the username and password that 
Bacula uses to connect to the catalogue database:

e.g.



Catalog {
  Name = MyCatalog
  dbname = "your-bacula-database-name"; dbuser = "your-bacula-user"; dbpassword 
= "your-password"
}




Make sure your database name, user and password match the current catalogue 
database settings configured in your bacula-dir.conf.  On your MariaDB server, 
you may need to use:

mysql>  grant all on your-bacula-database-name.* to your-bacula-user identified 
by 'your-password';

to ensure everything is set up correctly again.


Thanks.
--
Graham Sparks


From: Graham Dicker
Sent: 23 December 2021 14:51
To: bacula-users@lists.sourceforge.net 
Subject: [Bacula-users] make_catalog_backup stopped working



Further to my previous post I notice that the user table contains the 
following. Is that what it should look like?


MariaDB [mysql]> select host,user from user;





From: Graham Dicker
Sent: 23 December 2021 14:42
To: bacula-users@lists.sourceforge.net 
Subject: [Bacula-users] make_catalog_backup stopped working


Hi all


I'm running Bacula on a desk machine with OpenSuse 15.3 and Bacula 11.0.4.


The make_catalog_backup command has stopped working following a complete loss 
and restore of my Bacula database.

I don't know how the database vanished but I restored it from the last 
successful backup thus:


./bextract -b ../working/BackupCatalog.bsr /media/seagate4tb /tmp

mysql

create database bacula;

use bacula;

source bacula.sql;

grant_mysql_privileges


But now although my main backup works fine the backup of the catalog gets this 
error:


23-Dec 13:39 vivaldi-dir JobId 65: BeforeJob: mysqldump: Got error: 1044: 
"Access denied for user ''@'localhost' to database 'bacula'" when selecting the 
database



Any clues? Thanks in advance.

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore experience report

[Graham Sparks]

Thanks for the info.  What sort of backup transfer rates are you getting (from 
your clients to the USB-3 backup disk)?

I back up over a 1Gb/s network to a USB2.0 device and get 42MB/s backup. I also 
get 25MB/s restore (restoring from USB2.0 to a directly-connected SSD), so I'd 
expect better for USB-3 without a network bottleneck (perhaps

Forgive me if it's a silly question, but is the USB-3 drive definitely 
connected to a USB-3 (blue, or "SS"-labelled) USB port?

Picking some very (very) arbitrary figures, I'd expect your results to be 
around 100MB/s backup and 50MB/s restore according to the set-up you describe.

--
Graham


From: Lionel PLASSE
Sent: 23 December 2021 13:53
To: bacula-users@lists.sourceforge.net 
Cc: Graham Sparks 
Subject: RE: Restore experience report

Yes you're right, i explain the situation

The targeted disk for restauration is  a SATA-II directly  connected to the 
mother board with a SATA cable nor USB nor RAID but directly  to the SATA 
controller, it's a small  extension to the front of the computer.

The backup media volumes  are usb disk volume   and  stored  on USB-3  disks .
This  USB disk (XFS formatted)  contains  the volume file and all bsr and 
backup of bacula conf .  It is automatically mounted by linux when plugged on 
the storage dir .
One USB disk file with one media volume file.
All my dayly jobs (9jobs for 9 clients) are run on the volume every day :  
Incremental from Mon to Thu and Differental the Friday and each first Friday it 
is a Full instead of Diff

Bacula-dir sd fd and mariadb are on the same server
I restore on the same machine from the USB volumes to the hotplugged SATA disk.

I  thought that because of  all the restauration process has been made on the 
same server it will be faster ...







--

De : Graham Sparks 
Envoyé : jeudi 23 décembre 2021 14:00
À : bacula-users@lists.sourceforge.net
Objet : Re: Restore experience report

Hi,

I think a little more info is needed to narrow down whether or not these 
restore transfer figures are slow in this particular case.

How is the hotplug SATA-II restore disk connected (RAID controller, or USB), 
and is it connected to the bacula server directly, or through a client?
Also, where/how is the backup data stored?

--
Graham



From: Lionel PLASSE
Sent: 23 December 2021 09:35
To: mailto:bacula-users@lists.sourceforge.net 

Subject: [Bacula-users] Restore experience report

Hello,

I just post my full restore experience for performance studying purpose.

I use Bacula 9.6.6.3 & - 10.3.30-MariaDB  on  a Debian 11 buster x64 i5-2320 
3Ghz CPU - 32GB ram

For the 1st restoration (by baculum wizard  interface) FULL on Western Digital 
Blue 500Gb (DOS partition table and  NTFS partition) :

Where:  /srv
  Replace:Never
  Start time: 22-déc.-2021 09:19:43
  End time:   22-déc.-2021 12:04:47
  Elapsed time:   2 hours 45 mins 4 secs
  Files Expected: 202,006
  Files Restored: 202,027
  Bytes Restored: 32,650,343,324 (32.65 GB)
  Rate:   3296.7 KB/s
  FD Errors:  0
  FD termination status:  OK
  SD termination status:  OK
  Termination:Restore OK

~2h25 of restoration cause there was 3 volumes in use for restoration 
(FULL/DIFF) and I didn't noticed the request of changing volume.

The rate is a bit slow I think. (max rate was 5 MB/s)

I'm performing now an other full resto with a greater size of restoration on a 
4TB Westerndigital  Violet (GPT  & NTFS  partition)
The rate is curently 10.62 MB/s

Feed back the result this afternoon (I hope)

What do you think of the performance, is it possible to do better  ? knowing 
that the restored  disk   is on a hotplug SATA II interface.




___
Bacula-users mailing list
mailto:Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] make_catalog_backup stopped working

[Graham Dicker]

Oops I should have replied to my own message. And the leading pipe symbols 
seem to cause a problem. Sorry about that.

MariaDB [mysql]> select host,user from user;
+---+-+
 Host  | User|
+---+-+
 % | bacula  |
 localhost | |
 localhost | mariadb.sys |
 localhost | mysql   |
 localhost | root|
 vivaldi   | |
+---+-+
6 rows in set (0.001 sec)


On Thursday, 23 December 2021 14:42:12 GMT Graham Dicker wrote:
> Hi all
> 
> I'm running Bacula on a desk machine with OpenSuse 15.3 and Bacula 11.0.4.
> 
> The make_catalog_backup command has stopped working following a complete
> loss and restore of my Bacula database.
> I don't know how the database vanished but I restored it from the last
> successful backup thus:
> 
> ./bextract -b ../working/BackupCatalog.bsr /media/seagate4tb /tmp
> mysql
> create database bacula;
> use bacula;
> source bacula.sql;
> grant_mysql_privileges
> 
> But now although my main backup works fine the backup of the catalog gets
> this error:
> 
> 23-Dec 13:39 vivaldi-dir JobId 65: BeforeJob: mysqldump: Got error: 1044:
> "Access denied for user ''@'localhost' to database 'bacula'" when selecting
> the database
> 
> Any clues? Thanks in advance.






___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] make_catalog_backup stopped working

[Graham Dicker]

Further to my previous post I notice that the user table 
contains the following. Is that what it should look like?

MariaDB [mysql]> select host,user from user; 

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

[Bacula-users] make_catalog_backup stopped working

[Graham Dicker]

Hi all

I'm running Bacula on a desk machine with OpenSuse 15.3 and Bacula 11.0.4.

The make_catalog_backup command has stopped working following a complete loss 
and 
restore of my Bacula database.
I don't know how the database vanished but I restored it from the last 
successful backup 
thus:

./bextract -b ../working/BackupCatalog.bsr /media/seagate4tb /tmp
mysql
create database bacula;
use bacula;
source bacula.sql;
grant_mysql_privileges

But now although my main backup works fine the backup of the catalog gets this 
error:

23-Dec 13:39 vivaldi-dir JobId 65: BeforeJob: mysqldump: Got error: 1044: 
"Access denied 
for user ''@'localhost' to database 'bacula'" when selecting the database

Any clues? Thanks in advance.

___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] "Airgapping" USB drives used for backup?

[Josh Fisher]

On 12/22/21 12:47, Phil Stracchino wrote:

On 12/22/21 09:03, Neil Balchin wrote:
I’ve been down that road,  yes bacula can certainly be configured to 
handle that at least on a ’nix system.  Before your get too far 
though, you should do your own research on the reliability of 
spinning disk drives (internal or external).  They pale in comparison 
to Tape.  I would include s.m.a.r.t. disk checks as a regular part of 
my schedule and don’t expect to do this in an archiving scenario,  
hard drives sitting on a shelf unpowered for in excess of a year have 
frightening failure rates.


LTO tapes are meant to let for 30 yrs



Yeah, but the LTO drives are dead in eighteen months unless you have a 
filtered environment for them.  (That is my personal experience at 
least.)  I abandoned tape backup because I got tired of replacing the 
drives.




My experience as well with tape drives. IMHO, archives must be rotated 
onto newer media periodically anyway. I cannot imagine that it will be 
possible to find a drive that will read a 30 year old LTO tape. Systems 
will likely be able to attach USB drives for many years to come.


I use vchanger to treat USB drives like a magazine full of tapes. 
Management, rotation, etc. is basically the same as with tape. It is a 
good setup for the small businesses that I work with who will never have 
a filtered environment and replacement tape drives, to them, are expensive.


Another factor to consider is that the cost of external SSD drives is 
falling. Currently they are about 4x the cost of hard drives, but I feel 
that they will soon become a viable option for small business. They are 
quite physically durable and are expected to retain data without power 
for around 10 years. However, simply plugging them in once a year 
recharges the cells and should extend that many times over.  In fact, I 
am considering suggesting these to use for archival/off-site storage in 
conjunction with hard drives.




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users] Restore experience report

[Lionel PLASSE]

Yes you're right, i explain the situation 

The targeted disk for restauration is  a SATA-II directly  connected to the 
mother board with a SATA cable nor USB nor RAID but directly  to the SATA 
controller, it's a small  extension to the front of the computer. 

The backup media volumes  are usb disk volume   and  stored  on USB-3  disks .
This  USB disk (XFS formatted)  contains  the volume file and all bsr and 
backup of bacula conf .  It is automatically mounted by linux when plugged on 
the storage dir .  
One USB disk file with one media volume file.
All my dayly jobs (9jobs for 9 clients) are run on the volume every day :  
Incremental from Mon to Thu and Differental the Friday and each first Friday it 
is a Full instead of Diff  

Bacula-dir sd fd and mariadb are on the same server
I restore on the same machine from the USB volumes to the hotplugged SATA disk. 
 

I  thought that because of  all the restauration process has been made on the 
same server it will be faster ... 







--

De : Graham Sparks  
Envoyé : jeudi 23 décembre 2021 14:00
À : bacula-users@lists.sourceforge.net
Objet : Re: Restore experience report

Hi,

I think a little more info is needed to narrow down whether or not these 
restore transfer figures are slow in this particular case.

How is the hotplug SATA-II restore disk connected (RAID controller, or USB), 
and is it connected to the bacula server directly, or through a client?
Also, where/how is the backup data stored?

-- 
Graham



From: Lionel PLASSE
Sent: 23 December 2021 09:35
To: mailto:bacula-users@lists.sourceforge.net 

Subject: [Bacula-users] Restore experience report 
 
Hello,

I just post my full restore experience for performance studying purpose.

I use Bacula 9.6.6.3 & - 10.3.30-MariaDB  on  a Debian 11 buster x64 i5-2320 
3Ghz CPU - 32GB ram  

For the 1st restoration (by baculum wizard  interface) FULL on Western Digital 
Blue 500Gb (DOS partition table and  NTFS partition) : 

Where:  /srv
  Replace:    Never
  Start time: 22-déc.-2021 09:19:43
  End time:   22-déc.-2021 12:04:47
  Elapsed time:   2 hours 45 mins 4 secs
  Files Expected: 202,006
  Files Restored: 202,027
  Bytes Restored: 32,650,343,324 (32.65 GB)
  Rate:   3296.7 KB/s
  FD Errors:  0
  FD termination status:  OK
  SD termination status:  OK
  Termination:    Restore OK

~2h25 of restoration cause there was 3 volumes in use for restoration 
(FULL/DIFF) and I didn't noticed the request of changing volume.

The rate is a bit slow I think. (max rate was 5 MB/s)

I'm performing now an other full resto with a greater size of restoration on a 
4TB Westerndigital  Violet (GPT  & NTFS  partition)
The rate is curently 10.62 MB/s

Feed back the result this afternoon (I hope)

What do you think of the performance, is it possible to do better  ? knowing 
that the restored  disk   is on a hotplug SATA II interface. 




___
Bacula-users mailing list
mailto:Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

Re: [Bacula-users-fr] Incremental backup et interruption faute de LTO

[Lionel PLASSE]

Bonjour,

Je viens de voire le message en m'inscrivant , j'utilise bacula depuis a peine 
2 années sans grands soucis. 

Ce qui me choque dans votre post  c'est que vous avez une stratégie de rotation 
de bande uniquement basée sur de l'incrémentiel? Cela maximise le nombre de LTO 
à l'infini.

Il existe des schémas de rotation de sauvegarde performant et qui minimiserais 
le nombre de LTO.  (Ex INCR day / DIFF Weekly / FULL Montly)
Ainsi toutes les semaines les INCR peuvent être purgés et tous les mois 
l'ensemble INCR et DIFF. 
Il suffit de bloquer les jobs sur 3 mois pour les full pour n'avoir que 3 LTO 
FULL 5 LTO DIFF et 5 LTO INCR pour avoir une persistance de 3 mois des 
sauvegardes (pour faire vite.) 

L'incrémentiel marque dans la base de données les fichiers backés du job. Donc 
la suivante incrémentielle, même s'elle était en attente, ne sauvegardera que 
les nouveaux fichiers depuis le dernier incrémentiel donc rien si elle est 
relancée juste après (sauf erreur de ma part bacula demande d'abord le volume 
et fait ensuite la liste des fichiers à backer) . 

Je pense que vous avez plutôt intérêt à jouer sur les périodes de rétentions 
des job et de configurer un auto purge en fonction d'un schéma de rotation 
minimisant un nombre de LTO.
Faire de l'incrémentiel tous les jours complique le travail de restauration 
puisqu'il faudra passé en revue toutes les LTO pour obtenir un FULL resto

Votre solution est peut-être  déjà trouvée. 
Mais le mailing fr n'est pas très vivant alors si ça peut donner du grain à 
moudre



___
Bacula-users-fr mailing list
Bacula-users-fr@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users-fr

[Bacula-users] Restore experience report

[Lionel PLASSE]

Hello,

I just post my full restore experience for performance studying purpose.

I use Bacula 9.6.6.3 & - 10.3.30-MariaDB  on  a Debian 11 buster x64 i5-2320 
3Ghz CPU - 32GB ram  

For the 1st restoration (by baculum wizard  interface) FULL on Western Digital 
Blue 500Gb (DOS partition table and  NTFS partition) : 

Where:  /srv
  Replace:    Never
  Start time: 22-déc.-2021 09:19:43
  End time:   22-déc.-2021 12:04:47
  Elapsed time:   2 hours 45 mins 4 secs
  Files Expected: 202,006
  Files Restored: 202,027
  Bytes Restored: 32,650,343,324 (32.65 GB)
  Rate:   3296.7 KB/s
  FD Errors:  0
  FD termination status:  OK
  SD termination status:  OK
  Termination:    Restore OK

~2h25 of restoration cause there was 3 volumes in use for restoration 
(FULL/DIFF) and I didn't noticed the request of changing volume.

The rate is a bit slow I think. (max rate was 5 MB/s)

I'm performing now an other full resto with a greater size of restoration on a 
4TB Westerndigital  Violet (GPT  & NTFS  partition)
The rate is curently 10.62 MB/s

Feed back the result this afternoon (I hope)

What do you think of the performance, is it possible to do better  ? knowing 
that the restored  disk   is on a hotplug SATA II interface. 




___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users