For the sake of sharing information I am going to reply to this thread since I
am working on the same type of setup and experiencing similar issues.
I removed the setting ‘prefer mounted volumes = no' and separated my jobs into
3 different pools. All my jobs are running smoothly now, no failures. The
prefer mounted volumes was the issue.
Now I am experiencing interleaving on a single volume/tape drive with multiple
jobs scheduled at the same time, in the same pool, as opposed to bacula using
different volumes and different drives for jobs scheduled at the same time for
the same pool.
Here is my issue:
I have 10 jobs that kick off at the same time every day. Each job takes
anywhere from 10 minutes to 2 hours. The pool behind these jobs has 4 tape
drives and 4 volumes(virtual tapes) assigned to it. The 10 jobs all kick off
at the same time and they all write to the same volume, apparently(assuming)
interleaving the data. See my paste below, Ive bolded start times, end times,
pools and tapes. One can see these jobs all completed near each other and ran
for over an hour. All to the same tape/volume, tape3 and volume AAAACF5F6A .
I assume if I had the "prefer mounted volume=no" for these jobs they would use
all 4 drives in the pool and write to separate volumes as opposed to
interleaving to a single drive and volume. But based on the manual, reported
problems and experienced problems with ‘prefer mounted volumes = no’ I hesitate
to use that.
My question:
Is there another way to get multiple jobs scheduled at the same time to run
concurrently using different drives and volumes, as assigned to the pool,
versus all writing at the same time and interleaving the data to a single drive
and volume even though the pool has 4 drives/volumes assigned to it? Or is
prefer mounted volumes = no the only way to solve this?
any help is greatly appreciated
19-Nov 10:06 backup02-sd JobId 925: Job write elapsed time = 01:01:23, Transfer
rate = 1.489 M Bytes/second
19-Nov 10:06 backup02-dir JobId 925: Bacula backup02-dir 5.2.6 (21Feb12):
Build OS: x86_64-pc-linux-gnu ubuntu 14.04
JobId: 925
Job: labsdb1.2014-11-19_09.05.00_21
Backup Level: Full (upgraded from Differential)
Client: "labsdb1" 2.4.4 (28Dec08)
x86_64-pc-linux-gnu,debian,lenny/sid
FileSet: "Database Backup" 2014-11-07 22:02:50
Pool: "database" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "Tape3" (From Pool resource)
Scheduled time: 19-Nov-2014 09:05:00
Start time: 19-Nov-2014 09:05:02
End time: 19-Nov-2014 10:06:26
Elapsed time: 1 hour 1 min 24 secs
Priority: 10
FD Files Written: 2
SD Files Written: 2
FD Bytes Written: 5,486,802,781 (5.486 GB)
SD Bytes Written: 5,486,802,988 (5.486 GB)
Rate: 1489.4 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s): AAAACF5F6A
Volume Session Id: 155
Volume Session Time: 1415816791
Last Volume Bytes: 1,348,311,508,992 (1.348 TB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
19-Nov 10:06 backup02-dir JobId 925: Begin pruning Jobs older than 1 year .
19-Nov 10:06 backup02-dir JobId 925: No Jobs found to prune.
19-Nov 10:06 backup02-dir JobId 925: Begin pruning Files.
19-Nov 10:06 backup02-dir JobId 925: No Files found to prune.
19-Nov 10:06 backup02-dir JobId 925: End auto prune.
19-Nov 10:41 backup02-sd JobId 919: Job write elapsed time = 01:36:26, Transfer
rate = 2.570 M Bytes/second
19-Nov 10:41 backup02-dir JobId 919: Bacula backup02-dir 5.2.6 (21Feb12):
Build OS: x86_64-pc-linux-gnu ubuntu 14.04
JobId: 919
Job: shopd1.2014-11-19_09.05.00_15
Backup Level: Full (upgraded from Differential)
Client: "shopdb1" 5.0.2 (28Apr10)
x86_64-pc-linux-gnu,debian,squeeze/sid
FileSet: "varlibDatabase Backup" 2014-11-06 20:20:52
Pool: "database" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "Tape3" (From Pool resource)
Scheduled time: 19-Nov-2014 09:05:00
Start time: 19-Nov-2014 09:05:00
End time: 19-Nov-2014 10:41:26
Elapsed time: 1 hour 36 mins 26 secs
Priority: 10
FD Files Written: 3
SD Files Written: 3
FD Bytes Written: 14,875,191,024 (14.87 GB)
SD Bytes Written: 14,875,192,011 (14.87 GB)
Rate: 2570.9 KB/s
Software Compression: None
VSS: no
Encryption: yes
Accurate: no
Volume name(s): AAAACF5F6A
Volume Session Id: 149
Volume Session Time: 1415816791
Last Volume Bytes: 1,375,507,574,784 (1.375 TB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
19-Nov 10:41 backup02-dir JobId 919: Begin pruning Jobs older than 7 years .
19-Nov 10:41 backup02-dir JobId 919: No Jobs found to prune.
19-Nov 10:41 backup02-dir JobId 919: Begin pruning Files.
19-Nov 10:41 backup02-dir JobId 919: No Files found to prune.
19-Nov 10:41 backup02-dir JobId 919: End auto prune.
19-Nov 11:10 backup02-sd JobId 917: Job write elapsed time = 02:05:32, Transfer
rate = 3.894 M Bytes/second
19-Nov 11:10 backup02-dir JobId 917: Bacula backup02-dir 5.2.6 (21Feb12):
Build OS: x86_64-pc-linux-gnu ubuntu 14.04
JobId: 917
Job: pushdb.2014-11-19_09.05.00_13
Backup Level: Full (upgraded from Differential)
Client: "pushdb1" 5.0.2 (28Apr10)
x86_64-pc-linux-gnu,debian,6.0.6
FileSet: "varlibDatabase Backup" 2014-11-06 20:20:52
Pool: "database" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "Tape3" (From Pool resource)
Scheduled time: 19-Nov-2014 09:05:00
Start time: 19-Nov-2014 09:05:00
End time: 19-Nov-2014 11:10:33
Elapsed time: 2 hours 5 mins 33 secs
Priority: 10
FD Files Written: 2
SD Files Written: 2
FD Bytes Written: 29,334,385,684 (29.33 GB)
SD Bytes Written: 29,334,385,894 (29.33 GB)
Rate: 3894.1 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s): AAAACF5F6A
Volume Session Id: 147
Volume Session Time: 1415816791
Last Volume Bytes: 1,397,835,371,520 (1.397 TB)
Non-fatal FD errors: 1
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
19-Nov 11:10 backup02-dir JobId 917: Begin pruning Jobs older than 1 year .
19-Nov 11:10 backup02-dir JobId 917: No Jobs found to prune.
19-Nov 11:10 backup02-dir JobId 917: Begin pruning Files.
19-Nov 11:10 backup02-dir JobId 917: No Files found to prune.
19-Nov 11:10 backup02-dir JobId 917: End auto prune.
19-Nov 11:26 backup02-sd JobId 915: Job write elapsed time = 02:21:25, Transfer
rate = 3.655 M Bytes/second
19-Nov 11:26 backup02-dir JobId 915: Bacula backup02-dir 5.2.6 (21Feb12):
Build OS: x86_64-pc-linux-gnu ubuntu 14.04
JobId: 915
Job: ecommdb1.2014-11-19_09.05.00_11
Backup Level: Differential, since=2014-11-08 09:05:03
Client: "ecommdb" 2.4.4 (28Dec08)
x86_64-pc-linux-gnu,debian,lenny/sid
FileSet: "Database Backup" 2014-11-07 22:02:50
Pool: "database" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Storage: "Tape3" (From Pool resource)
Scheduled time: 19-Nov-2014 09:05:00
Start time: 19-Nov-2014 09:05:00
End time: 19-Nov-2014 11:26:27
Elapsed time: 2 hours 21 mins 27 secs
Priority: 10
FD Files Written: 1
SD Files Written: 1
FD Bytes Written: 31,016,183,168 (31.01 GB)
SD Bytes Written: 31,016,183,295 (31.01 GB)
Rate: 3654.6 KB/s
Software Compression: None
VSS: no
Encryption: no
Accurate: no
Volume name(s): AAAACF5F6A
Volume Session Id: 145
Volume Session Time: 1415816791
Last Volume Bytes: 1,407,107,681,280 (1.407 TB)
Non-fatal FD errors: 0
SD Errors: 0
FD termination status: OK
SD termination status: OK
Termination: Backup OK
From: "hei...@bacula.com.br<mailto:hei...@bacula.com.br>"
<hei...@bacula.com.br<mailto:hei...@bacula.com.br>>
Date: Wednesday, November 19, 2014 at 9:03 AM
To: Josh Fisher <jfis...@pvct.com<mailto:jfis...@pvct.com>>
Cc:
"bacula-users@lists.sourceforge.net<mailto:bacula-users@lists.sourceforge.net>"
<bacula-users@lists.sourceforge.net<mailto:bacula-users@lists.sourceforge.net>>
Subject: Re: [Bacula-users] bacula can't make use of multi drives
I think that the only problem with setting "Prefer Mounted Volumes = no" is
that it will result in a lot of moving of a volume back and forth between
drives. While it might not be a bug, since it works as advertised, it CAN be a
problem. For tape, it will really slow things down because it takes a
substantial time to move a tape out of one drive and into another. For disk
virtual autochangers, though, the moving between "drives" is simply closing a
disk file and then re-opening it, meaning that the delay is very negligible.
Mr. Fisher: It's not just that. Eventually "Prefer Mounted Volumes = no" messes
up the autochanger during a random backup and you have to restart storage
daemon and sometimes powercycle the tape library.
If there is some other issue, like a race condition that I am not aware of,
then that would indeed be a bug in my book. But I have used
PreferMountedVolumes=no for disk virtual autochanger without (so far)
encountering any such problems, just the swapping between drives. And for disk
virtual autochangers, I don't see that as a problem in and of itself.
I don't know if it is already fixed or if this error happens for all
autochangers. But last Bacula System Community Storage whitepapers goes in the
same way. Having 2 pools is still safer and more reliable way to do multiple
volumes writing.
Regards,
=========================================================================
Heitor Medrado de Faria - LPIC-III | ITIL-F
Faltam poucos dias - Treinamento Telepresencial Bacula:
http://www.bacula.com.br/?p=2174
61 2021-8260<tel:%2B55%2061%202021-8260> | 8268-4220<tel:%2B55%2061%208268-4220>
Site: www.bacula.com.br<http://www.bacula.com.br/> | Facebook:
heitor.faria<http://www.facebook.com/heitor.faria> | Gtalk:
heitorfa...@gmail.com<mailto:heitorfa...@gmail.com>
==========================================================================
________________________________
De: "Josh Fisher" <jfis...@pvct.com<mailto:jfis...@pvct.com>>
Para:
bacula-users@lists.sourceforge.net<mailto:bacula-users@lists.sourceforge.net>
Enviadas: Quarta-feira, 19 de novembro de 2014 14:52:42
Assunto: Re: [Bacula-users] bacula can't make use of multi drives
On 11/19/2014 11:10 AM, hei...@bacula.com.br<mailto:hei...@bacula.com.br> wrote:
Is it a bug that bacula cannot write to multiple drives for the jobs in the
same pool? Will there be a fix. Back in March 2014 Wolfgang Denk wrote "it can"
but how?
http://adsm.org/lists/html/Bacula-users/2014-03/msg00212.html
It's not a bug, but just the way it's designed.
"Prefer Mounted Volumes=no" it's a directive that could allow to write multiple
volumes within the same pool at the same time, however this is a little buggy
and not recommended at this time.
It's not a bug, but it is a little buggy??
I think that the only problem with setting "Prefer Mounted Volumes = no" is
that it will result in a lot of moving of a volume back and forth between
drives. While it might not be a bug, since it works as advertised, it CAN be a
problem. For tape, it will really slow things down because it takes a
substantial time to move a tape out of one drive and into another. For disk
virtual autochangers, though, the moving between "drives" is simply closing a
disk file and then re-opening it, meaning that the delay is very negligible.
If there is some other issue, like a race condition that I am not aware of,
then that would indeed be a bug in my book. But I have used
PreferMountedVolumes=no for disk virtual autochanger without (so far)
encountering any such problems, just the swapping between drives. And for disk
virtual autochangers, I don't see that as a problem in and of itself.
Regards,
=========================================================================
Heitor Medrado de Faria - LPIC-III | ITIL-F
Faltam poucos dias - Treinamento Telepresencial Bacula:
http://www.bacula.com.br/?p=2174
61 2021-8260<tel:%2B55%2061%202021-8260> | 8268-4220<tel:%2B55%2061%208268-4220>
Site: www.bacula.com.br<http://www.bacula.com.br/> | Facebook:
heitor.faria<http://www.facebook.com/heitor.faria> | Gtalk:
heitorfa...@gmail.com<mailto:heitorfa...@gmail.com>
==========================================================================
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net<mailto:Bacula-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
