Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
Thanks for that tip. I didn't see anything in the joblog about retries so
assumed it wasn't doing that.
I added a runafter runscript to the cloud job.
Runscript {
RunsOnFailure = yes
RunsOnClient = no
FailJobOnError = no
RunsWhen = "After"
Console = "cloud storage=cloud-sd allpools allfrompool upload"
}
Does that look sensible to you?
Chris
On Thu, 30 Nov 2023, 15:56 Bill Arlofski via Bacula-users, <
bacula-users@lists.sourceforge.net> wrote:
> On 11/30/23 03:29, Chris Wilkinson wrote:
> > I use Backblaze B2. It is S3 compliant but about the lowest cost I could
> find, certainly a fraction of the price of AWS.
> >
> > It works mostly OK except for errors of the kind in the log below. The
> "no tomes" error is from B2 when there are no upload
> > slots available temporarily. B2 says that the user application is
> expected to retry the transfer if this occurs. AKAIK the S3
> > driver doesn’t retry, just gives up. There is a bconsole 'cloud' command
> to upload volumes to cloud manually. A listing of
> > the cloud volumes shows that the errored volume in the log wasn't
> uploaded but I could do it manually.
> >
> > Because of this I do not delete the cache after backup, the downside
> being I have to provision the disc space.
> >
> > Chris
>
> Hello Chris, Bacula S3 by default re-tries 10 times to upload cloud volume
> parts. There is currently an internal feature
> request here at Bacula Systems to allow this to be configured, along with
> a feature request to configure the time between
> attempts. :)
>
> Additionally, it is recommended to have a Bacula Admin job which runs
> `cloud upload` commands to make sure that any parts
> that had failed to upload (ie: due to temporary networking issues during
> the backup job, etc)
>
>
> Hope this helps,
> Bill
>
> --
> Bill Arlofski
> w...@protonmail.com
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
On 11/30/23 03:29, Chris Wilkinson wrote: I use Backblaze B2. It is S3 compliant but about the lowest cost I could find, certainly a fraction of the price of AWS. It works mostly OK except for errors of the kind in the log below. The "no tomes" error is from B2 when there are no upload slots available temporarily. B2 says that the user application is expected to retry the transfer if this occurs. AKAIK the S3 driver doesn’t retry, just gives up. There is a bconsole 'cloud' command to upload volumes to cloud manually. A listing of the cloud volumes shows that the errored volume in the log wasn't uploaded but I could do it manually. Because of this I do not delete the cache after backup, the downside being I have to provision the disc space. Chris Hello Chris, Bacula S3 by default re-tries 10 times to upload cloud volume parts. There is currently an internal feature request here at Bacula Systems to allow this to be configured, along with a feature request to configure the time between attempts. :) Additionally, it is recommended to have a Bacula Admin job which runs `cloud upload` commands to make sure that any parts that had failed to upload (ie: due to temporary networking issues during the backup job, etc) Hope this helps, Bill -- Bill Arlofski w...@protonmail.com signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
Não sei se isso resolve para o seu caso, mas no meu caso resolveu.
Autochanger {
Name = Bacula-SD1
Device = Bacula-SD1-Dev1, Bacula-SD1-Dev2
Changer Command = ""
Changer Device = /dev/null
}
Device {
Name = Bacula-SD1-Dev1
Media Type = Diretorio-Local
Archive Device = /tmp/bacula
*LabelMedia = yes; # lets Bacula label unlabeled media*
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5
}
José Ferreira Neto
Profissional TI
Músico cristão membro da Assembleia de Deus.
*Faça contato comigo | Visite meu site*
Emails: netto@gmail.com | netto.li...@gmail.com
Lattes: https://t.co/O3fjQb6Zfb
Twitter's: @netto_info
MSN:
netto@gmail.com
Perfil: http://fb.com/nettux
LinkedIn: https://www.linkedin.com/in/nettux/
Em qua., 29 de nov. de 2023 às 12:10, Josh Fisher via Bacula-users <
bacula-users@lists.sourceforge.net> escreveu:
>
> On 11/29/23 05:47, MylesDearBusiness via Bacula-users wrote:
> >
> > Hello, Bacula experts.
> >
> > Due to message length limitations of this mailing list, I have been
> > unable to post the majority of necessary details, which is why I was
> > using my github gist system to store, apologies for the confusion or
> > inconvenience this caused. I just thought it would be more confusing
> > to break up the details into multiple messages.
> >
> > The latest after following up on some of Bill's suggestions, I added a
> > second device in my File Changer and now bconsole shows I am being
> > asked to execute the "label" command, which is failing.
> >
> > As a reminder, I'm running bacula-dir under user "bacula" (which does
> > not have access to the storage mount /mnt/my_backup).
> > I'm running bacula-sd and bacula-fd under user "backupuser" which has
> > sole permission to re ad/write files under this mount.
> >
>
> Is SELinux or AppArmor enabled? That could block writes even if Unix
> permissions are correct.
>
>
> > Please see
> > https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b for
> > more, I just added a new comment with additional details.
> >
> > Thanks,
> >
> >
> >
> >
> >
> >
> > ___
> > Bacula-users mailing list
> > Bacula-users@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
Thanks, Chris. My company is looking to align with companies who are not in the 5/9/14 eyes alliances, hence I'm now looking at Koofr (Slovenia). Backblaze is domiciled in the USA. Best, On 2023-11-30 5:29 a.m., Chris Wilkinson wrote: > I use Backblaze B2. It is S3 compliant but about the lowest cost I could > find, certainly a fraction of the price of AWS. > > It works mostly OK except for errors of the kind in the log below. The "no > tomes" error is from B2 when there are no upload slots available temporarily. > B2 says that the user application is expected to retry the transfer if this > occurs. AKAIK the S3 driver doesn’t retry, just gives up. There is a bconsole > 'cloud' command to upload volumes to cloud manually. A listing of the cloud > volumes shows that the errored volume in the log wasn't uploaded but I could > do it manually. > > Because of this I do not delete the cache after backup, the downside being I > have to provision the disc space. > > Chris > > 29-Nov 00:20 raspberrypi-dir JobId 3617: Start Backup JobId 3617, > Job=pi-cloud-docs.2023-11-29_00.20.00_30 > 29-Nov 00:20 raspberrypi-dir JobId 3617: Max configured use duration=10,800 > sec. exceeded. Marking Volume "usb16tb-cloud-incr4767" as Used. > 29-Nov 00:20 raspberrypi-dir JobId 3617: Created new > Volume="usb16tb-cloud-incr4768", Pool="usb16tb-cloud-incr", MediaType="Cloud" > in catalog. > 29-Nov 00:20 raspberrypi-dir JobId 3617: Using Device "B2Bucket" to write. > 29-Nov 00:20 raspberrypi-sd JobId 3617: Labeled new Volume > "usb16tb-cloud-incr4768" on Cloud device "B2Bucket" (/mnt/usb16tb/cache). > 29-Nov 00:20 raspberrypi-sd JobId 3617: Wrote label to prelabeled Volume > "usb16tb-cloud-incr4768" on Cloud device "B2Bucket" (/mnt/usb16tb/cache) > 29-Nov 00:20 raspberrypi-sd JobId 3617: Using S3 cloud driver > Host=s3.eu-central-003.backblazeb2.com Bucket=bacula01 > 29-Nov 00:20 raspberrypi-sd JobId 3617: Elapsed time=00:00:01, Transfer > rate=9.620 M Bytes/second > 29-Nov 00:20 raspberrypi-sd JobId 3617: Cloud Upload transfers: > > 29-Nov 00:20 raspberrypi-sd JobId 3617: Error: usb16tb-cloud-incr4768/part.1 > state=error size=257 B duration=0s msg=S3_put_object ERR=no tomes available > CURL Effective URL: > https://s3.eu-central-003.backblazeb2.com/bacula01/usb16tb-cloud-incr4768/part.1 > CURL Effective URL: > https://s3.eu-central-003.backblazeb2.com/bacula01/usb16tb-cloud-incr4768/part.1 > > 29-Nov 00:20 raspberrypi-sd JobId 3617: Error: usb16tb-cloud-incr4768/part.2 > state=error size=9.628 MB duration=0s msg=S3_put_object ERR=no tomes > available CURL Effective URL: > https://s3.eu-central-003.backblazeb2.com/bacula01/usb16tb-cloud-incr4768/part.2 > CURL Effective URL: > https://s3.eu-central-003.backblazeb2.com/bacula01/usb16tb-cloud-incr4768/part.2 > > 29-Nov 00:20 raspberrypi-sd JobId 3617: Sending spooled attrs to the > Director. Despooling 3,019 bytes ... > 29-Nov 00:20 raspberrypi-dir JobId 3617: Bacula raspberrypi-dir 11.0.6 > (10Mar22): > Build OS: aarch64-unknown-linux-gnu debian 11.3 > JobId: 3617 > > On Thu, 30 Nov 2023, 03:47 MylesDearBusiness via Bacula-users, > wrote: > >> It appears I'm out of luck. >> >> Apprently, curlftpfs no longer supports the open+read+write operation >> that Bacula requires, so it looks like I'll have to move to a different >> storage provider. >> >> https://sourceforge.net/p/curlftpfs/discussion/542750/thread/53e47b71/ >> >> Thanks for all the help so far, much appreciated. >> >> >> >> On 2023-11-29 5:47 a.m., MylesDearBusiness wrote: >>> >>> Hello, Bacula experts. >>> >>> Due to message length limitations of this mailing list, I have been >>> unable to post the majority of necessary details, which is why I was >>> using my github gist system to store, apologies for the confusion or >>> inconvenience this caused. I just thought it would be more confusing >>> to break up the details into multiple messages. >>> >>> The latest after following up on some of Bill's suggestions, I added a >>> second device in my File Changer and now bconsole shows I am being >>> asked to execute the "label" command, which is failing. >>> >>> As a reminder, I'm running bacula-dir under user "bacula" (which does >>> not have access to the storage mount /mnt/my_backup). >>> I'm running bacula-sd and bacula-fd under user "backupuser" which has >>> sole permission to read/write files under this mount. >>> >>> Please see >>> https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b for >>> more, I just added a new comment with additional details. >>> >>> Thanks, >>> >>> >>> >>> >> >> ___ >> Bacula-users mailing list >> Bacula-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bacula-users___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
I use Backblaze B2. It is S3 compliant but about the lowest cost I could find, certainly a fraction of the price of AWS. It works mostly OK except for errors of the kind in the log below. The "no tomes" error is from B2 when there are no upload slots available temporarily. B2 says that the user application is expected to retry the transfer if this occurs. AKAIK the S3 driver doesn’t retry, just gives up. There is a bconsole 'cloud' command to upload volumes to cloud manually. A listing of the cloud volumes shows that the errored volume in the log wasn't uploaded but I could do it manually. Because of this I do not delete the cache after backup, the downside being I have to provision the disc space. Chris 29-Nov 00:20 raspberrypi-dir JobId 3617: Start Backup JobId 3617, Job=pi-cloud-docs.2023-11-29_00.20.00_30 29-Nov 00:20 raspberrypi-dir JobId 3617: Max configured use duration=10,800 sec. exceeded. Marking Volume "usb16tb-cloud-incr4767" as Used. 29-Nov 00:20 raspberrypi-dir JobId 3617: Created new Volume="usb16tb-cloud-incr4768", Pool="usb16tb-cloud-incr", MediaType="Cloud" in catalog. 29-Nov 00:20 raspberrypi-dir JobId 3617: Using Device "B2Bucket" to write. 29-Nov 00:20 raspberrypi-sd JobId 3617: Labeled new Volume "usb16tb-cloud-incr4768" on Cloud device "B2Bucket" (/mnt/usb16tb/cache). 29-Nov 00:20 raspberrypi-sd JobId 3617: Wrote label to prelabeled Volume "usb16tb-cloud-incr4768" on Cloud device "B2Bucket" (/mnt/usb16tb/cache) 29-Nov 00:20 raspberrypi-sd JobId 3617: Using S3 cloud driver Host= s3.eu-central-003.backblazeb2.com Bucket=bacula01 29-Nov 00:20 raspberrypi-sd JobId 3617: Elapsed time=00:00:01, Transfer rate=9.620 M Bytes/second 29-Nov 00:20 raspberrypi-sd JobId 3617: Cloud Upload transfers: 29-Nov 00:20 raspberrypi-sd JobId 3617: Error: usb16tb-cloud-incr4768/part.1 state=error size=257 B duration=0s msg=S3_put_object ERR=no tomes available CURL Effective URL: https://s3.eu-central-003.backblazeb2.com/bacula01/usb16tb-cloud-incr4768/part.1 CURL Effective URL: https://s3.eu-central-003.backblazeb2.com/bacula01/usb16tb-cloud-incr4768/part.1 29-Nov 00:20 raspberrypi-sd JobId 3617: Error: usb16tb-cloud-incr4768/part.2 state=error size=9.628 MB duration=0s msg=S3_put_object ERR=no tomes available CURL Effective URL: https://s3.eu-central-003.backblazeb2.com/bacula01/usb16tb-cloud-incr4768/part.2 CURL Effective URL: https://s3.eu-central-003.backblazeb2.com/bacula01/usb16tb-cloud-incr4768/part.2 29-Nov 00:20 raspberrypi-sd JobId 3617: Sending spooled attrs to the Director. Despooling 3,019 bytes ... 29-Nov 00:20 raspberrypi-dir JobId 3617: Bacula raspberrypi-dir 11.0.6 (10Mar22): Build OS: aarch64-unknown-linux-gnu debian 11.3 JobId: 3617 On Thu, 30 Nov 2023, 03:47 MylesDearBusiness via Bacula-users, < bacula-users@lists.sourceforge.net> wrote: > It appears I'm out of luck. > > Apprently, curlftpfs no longer supports the open+read+write operation > that Bacula requires, so it looks like I'll have to move to a different > storage provider. > > https://sourceforge.net/p/curlftpfs/discussion/542750/thread/53e47b71/ > > Thanks for all the help so far, much appreciated. > > > > > On 2023-11-29 5:47 a.m., MylesDearBusiness wrote: > > > > Hello, Bacula experts. > > > > Due to message length limitations of this mailing list, I have been > > unable to post the majority of necessary details, which is why I was > > using my github gist system to store, apologies for the confusion or > > inconvenience this caused. I just thought it would be more confusing > > to break up the details into multiple messages. > > > > The latest after following up on some of Bill's suggestions, I added a > > second device in my File Changer and now bconsole shows I am being > > asked to execute the "label" command, which is failing. > > > > As a reminder, I'm running bacula-dir under user "bacula" (which does > > not have access to the storage mount /mnt/my_backup). > > I'm running bacula-sd and bacula-fd under user "backupuser" which has > > sole permission to read/write files under this mount. > > > > Please see > > https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b for > > more, I just added a new comment with additional details. > > > > Thanks, > > > > > > > > > > > > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
It appears I'm out of luck. Apprently, curlftpfs no longer supports the open+read+write operation that Bacula requires, so it looks like I'll have to move to a different storage provider. https://sourceforge.net/p/curlftpfs/discussion/542750/thread/53e47b71/ Thanks for all the help so far, much appreciated. On 2023-11-29 5:47 a.m., MylesDearBusiness wrote: > > Hello, Bacula experts. > > Due to message length limitations of this mailing list, I have been > unable to post the majority of necessary details, which is why I was > using my github gist system to store, apologies for the confusion or > inconvenience this caused. I just thought it would be more confusing > to break up the details into multiple messages. > > The latest after following up on some of Bill's suggestions, I added a > second device in my File Changer and now bconsole shows I am being > asked to execute the "label" command, which is failing. > > As a reminder, I'm running bacula-dir under user "bacula" (which does > not have access to the storage mount /mnt/my_backup). > I'm running bacula-sd and bacula-fd under user "backupuser" which has > sole permission to read/write files under this mount. > > Please see > https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b for > more, I just added a new comment with additional details. > > Thanks, > > > > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
On 11/29/23 05:47, MylesDearBusiness via Bacula-users wrote: Hello, Bacula experts. Due to message length limitations of this mailing list, I have been unable to post the majority of necessary details, which is why I was using my github gist system to store, apologies for the confusion or inconvenience this caused. I just thought it would be more confusing to break up the details into multiple messages. The latest after following up on some of Bill's suggestions, I added a second device in my File Changer and now bconsole shows I am being asked to execute the "label" command, which is failing. As a reminder, I'm running bacula-dir under user "bacula" (which does not have access to the storage mount /mnt/my_backup). I'm running bacula-sd and bacula-fd under user "backupuser" which has sole permission to re ad/write files under this mount. Is SELinux or AppArmor enabled? That could block writes even if Unix permissions are correct. Please see https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b for more, I just added a new comment with additional details. Thanks, ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
I notice in your config you have: /mnt/my_backup/backup/bacula/archive But the logs keep showing: /mnt/khapbackup/backup/bacula/archive Have you made a change to the sd conf and not restarted the SD? Or how about creating a symlink from /mnt/khapbackup to /mnt/my_backup ? - Michel From: MylesDearBusiness via Bacula-users Sent: Wednesday, November 29, 2023 2:48 AM To: bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume" Hello, Bacula experts. Due to message length limitations of this mailing list, I have been unable to post the majority of necessary details, which is why I was using my github gist system to store, apologies for the confusion or inconvenience this caused. I just thought it would be more confusing to break up the details into multiple messages. The latest after following up on some of Bill's suggestions, I added a second device in my File Changer and now bconsole shows I am being asked to execute the "label" command, which is failing. As a reminder, I'm running bacula-dir under user "bacula" (which does not have access to the storage mount /mnt/my_backup). I'm running bacula-sd and bacula-fd under user "backupuser" which has sole permission to read/write files under this mount. Please see https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b for more, I just added a new comment with additional details. Thanks, ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
Hello, Bacula experts. Due to message length limitations of this mailing list, I have been unable to post the majority of necessary details, which is why I was using my github gist system to store, apologies for the confusion or inconvenience this caused. I just thought it would be more confusing to break up the details into multiple messages. The latest after following up on some of Bill's suggestions, I added a second device in my File Changer and now bconsole shows I am being asked to execute the "label" command, which is failing. As a reminder, I'm running bacula-dir under user "bacula" (which does not have access to the storage mount /mnt/my_backup). I'm running bacula-sd and bacula-fd under user "backupuser" which has sole permission to read/write files under this mount. Please see https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8bfor more, I just added a new comment with additional details. Thanks, ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
On 11/28/23 13:27, MylesDearBusiness via Bacula-users wrote: > I'm not quite understanding where Vol-0014 and other similarly named volumes are coming from. I just noticed this statement You have `LabelFormat = Vol-` in your Pool called `File` The Director uses this as a template when it needs to create a new volume in the catalog and then tells the SD to create the same named file volume on disk. And, once you add that `LabelMedia = yes` into the SD's devices (remember to create several of them), then the SD is allowed to create the volumes on disk. Hope this helps to clarify things, Bill -- Bill Arlofski w...@protonmail.com signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
On 11/28/23 13:27, MylesDearBusiness via Bacula-users wrote:
Thanks for the reply, Bill.
First of all, Bacula seems to be looking for a different volume name
every time, Vol-0014 doesn't match any folder I created (and the storage
space, minus a few boilerplate files and directories, was empty
initially as I would expect a newly configured cloud storage service to
be).
Hello Miles,
To be clear, you are not trying to backup to an S3 "cloud" bucket, right? When I see the word cloud, I need to be sure
because when using Bacula cloud storage plugins, instead of creating a single file volume on disk for each volume in the
catalog, Bacula will create a directory for each volume, and then write parts of the cloud volume under that volume's
directory (ie: part.1, part.2...part.n)
Having said this, I see no "Cloud {}" resources in your pasted configuration so
I think we are OK. :)
I'm not quite understanding where Vol-0014 and other similarly
named volumes are coming from. All I know is that I have a curlftpfs
based userspace mounting daemon running that presents remote storage
using a familiar file-system-based integration and that I'm trying to
point Bacula to back up into it.
We'll get there. :)
I originally created the local directory /mnt/my_backup as a directory
owned by backupuser:backupuser and then under user backupuser I mounted
my curlftpfs remote storage space under that directory.
Only backupuser has the rights to read from and write to this directory.
As shown in my gist copied again below, I'm only running the Director
under the bacula username.
Yes. I see this and this is all fine. Most people don't bother, but it is perfectly fine to edit the systemd unit files and
change the user the Bacula daemons run as. :)
I'm running the SD and FD processes under the backupuser username
because that's the username I gave sole permission to access the storage
mount.
Yep. Still fine.
I also added more detail to the gist link to try to address some of your
questions :
https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b
More help is needed and help already given is much appreciated,
Thanks,
OK, I think I spotted at the issue.
I see what is more than likely the culprit:
In your SD Device called `FileChgr1-Dev1` you are missing an important parameter which allows the SD to create new file
volumes when needed.
8<
LabelMedia = yes
8<
The other piece of this puzzle which allows Bacula to create new file volumes
as necessary is to have:
8<
LabelFormat =
8<
...in the Pool (The pool is `File` in this case). This is OK in your
configuration, as I see:
8<
Label Format = "Vol-"
8<
I also see that you already have two other important settings in the `File`
Pool:
- MaximumVolumeBytes
- MaximumVolumes
But keep in mind 90G file volumes with a maximum of 100 of them in the pool and a volume retention of 1 year, you will
probably want/need to play with these numbers a bit depending on your amount of available storage. I personally prefer 10GB
file volumes, but people use any number of different sizes - this will depend on your environment.
I think once you make these changes things should start looking up.
I would then cancel any jobs you have running/stuck:
In bconsole:
* cancel (then follow the prompts)
Then list the volumes:
* list media pool=File
Then delete delete them from the catalog:
* delete yes volume= (where is the volume's name)
Then, reload the Director:
* reload
Then restart the SD, and try another test.
If it still fails please show:
* list joblog jobid= (where is the job id of a recent job sitting
waiting to mount a volume)
* list pools
* list media
Then, please show from a shell (as root):
# find /mnt -ls (The `-ls` is important.)
Please just post everything in the mailing list, It is easier for me to follow
this way. :)
Additionally, once you get this working, you will want to have more than one of these Devices in the SD's Autochanger. This
way you can run multiple concurrent jobs to different devices, and you will alway have a device ro restore with when backups
are running - especially if you add a few and set `ReadOnly = yes` :)
Good luck,
Bill
--
Bill Arlofski
w...@protonmail.com
signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
Myles, Just a brief note: Bacula uses "virtual tape" volumes, which are files that bacula mounts internally and writes backup data to. If a given volume is unreachable/unusable/full/etc bacula will attempt to create another similar volume in the designated location. Again, the volume will be a file, which will contain the files Bacula has backed up. Bacula volumes are written to using Bacula's own format. In the event of a disaster (loss of your bacula catalog database), you can use the bacula bscan tool to scan the volume(s) (or tapes if you use actual magnetic media tape), which will rebuild a catalog database. Are you sure bacula has write permissions for the destination directory, and are you sure bacula is using the user you think it is? I haven't heard of a bacula director and storage daemon using different user credentials before. Doesn't mean it won't supported, but I haven't heard of that. What happens when you temporarily set the target directory to be accessible by any user? (chmod -R 777 /mnt/my_backups) Does the find command + grep locate any of the volume names given by bacula? (find /|grep -i volume name here) Robert Gerber 402-237-8692 r...@craeon.net On Tue, Nov 28, 2023, 2:29 PM MylesDearBusiness via Bacula-users < bacula-users@lists.sourceforge.net> wrote: > Thanks for the reply, Bill. > > First of all, Bacula seems to be looking for a different volume name > every time, Vol-0014 doesn't match any folder I created (and the storage > space, minus a few boilerplate files and directories, was empty > initially as I would expect a newly configured cloud storage service to > be). I'm not quite understanding where Vol-0014 and other similarly > named volumes are coming from. All I know is that I have a curlftpfs > based userspace mounting daemon running that presents remote storage > using a familiar file-system-based integration and that I'm trying to > point Bacula to back up into it. > > I originally created the local directory /mnt/my_backup as a directory > owned by backupuser:backupuser and then under user backupuser I mounted > my curlftpfs remote storage space under that directory. > > Only backupuser has the rights to read from and write to this directory. > > As shown in my gist copied again below, I'm only running the Director > under the bacula username. > > I'm running the SD and FD processes under the backupuser username > because that's the username I gave sole permission to access the storage > mount. > > I also added more detail to the gist link to try to address some of your > questions : > > https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b > > More help is needed and help already given is much appreciated, > > Thanks, > > > > > On 2023-11-27 7:18 p.m., Bill Arlofski via Bacula-users wrote: > > On 11/27/23 15:26, MylesDearBusiness via Bacula-users wrote: > >> Hello, Bacula experts. > >> > >> My cloud provider offers only curlftpfs based storage, which I have > enabled and secured. I created a username “backupuser” > >> and a system service that launches the service, effectively mounting > the remote storage to /mnt/my_backup and running the > >> userspace curlftpfs command as the user “backupuser”. This user has > demonstrable read/write privileges in the file system as > >> expected. So far so good. > >> > >> I installed bacula on my Ubuntu 22.04.3 LTS server and got bacula-dir, > bacula-fd, and bacula-sd all running. > >> > >> In summary, when I try to run my backup job I get an error: > >> > >> Device is BLOCKED waiting for mount of volume > >> > >> ChatGPT4 doesn’t know nearly as much as this august body, I’m hoping a > kindly member may be able to give me a hand up. > >> > >> > >> More details: > https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b > >> > >>Thanks, > >> > >> > > > > What does this show?: > > > > # ls -la /mnt/my_backup/backup/bacula/archive > > > > > > Is there a Bacula File volume named `Vol-0014` in there? > > Is it rw for the `bacula` user which the SD (normally) runs as? > > > > If it is not there, but you know where it is, them you must move it > there, and set the ownership to `bacula:disk` and the > > permissions to allow the bacula user to, well... Read and write to it. :) > > > > It is just a guess, but is it possible that this Bacula file volume > lives under a directory which you mounted > > `/mnt/khapbackup/backup/bacula/archive` on top of, so it exists, but is > not visible currently? This is just a guess, but it > > is also a common mistake. :) > > > > If you truly do not know where this Bacula file volume is, then you need > to delete it from the catalog so the director no > > longer thinks it is accessible: > > > > * delete yes volume=Vol-0014 > > > > Might want to run the bconsole `query` command and select option 14 to > be sure there are no jobs on it that you might need. > > > > Once deleted from the catalog, the Director should select a new volume > (or create a new one if `LabelFormat =
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
Thanks for the reply, Bill. First of all, Bacula seems to be looking for a different volume name every time, Vol-0014 doesn't match any folder I created (and the storage space, minus a few boilerplate files and directories, was empty initially as I would expect a newly configured cloud storage service to be). I'm not quite understanding where Vol-0014 and other similarly named volumes are coming from. All I know is that I have a curlftpfs based userspace mounting daemon running that presents remote storage using a familiar file-system-based integration and that I'm trying to point Bacula to back up into it. I originally created the local directory /mnt/my_backup as a directory owned by backupuser:backupuser and then under user backupuser I mounted my curlftpfs remote storage space under that directory. Only backupuser has the rights to read from and write to this directory. As shown in my gist copied again below, I'm only running the Director under the bacula username. I'm running the SD and FD processes under the backupuser username because that's the username I gave sole permission to access the storage mount. I also added more detail to the gist link to try to address some of your questions : https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b More help is needed and help already given is much appreciated, Thanks, On 2023-11-27 7:18 p.m., Bill Arlofski via Bacula-users wrote: > On 11/27/23 15:26, MylesDearBusiness via Bacula-users wrote: >> Hello, Bacula experts. >> >> My cloud provider offers only curlftpfs based storage, which I have enabled >> and secured. I created a username “backupuser” >> and a system service that launches the service, effectively mounting the >> remote storage to /mnt/my_backup and running the >> userspace curlftpfs command as the user “backupuser”. This user has >> demonstrable read/write privileges in the file system as >> expected. So far so good. >> >> I installed bacula on my Ubuntu 22.04.3 LTS server and got bacula-dir, >> bacula-fd, and bacula-sd all running. >> >> In summary, when I try to run my backup job I get an error: >> >> Device is BLOCKED waiting for mount of volume >> >> ChatGPT4 doesn’t know nearly as much as this august body, I’m hoping a >> kindly member may be able to give me a hand up. >> >> >> More details:https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b >> >> Thanks, >> >> > > What does this show?: > > # ls -la /mnt/my_backup/backup/bacula/archive > > > Is there a Bacula File volume named `Vol-0014` in there? > Is it rw for the `bacula` user which the SD (normally) runs as? > > If it is not there, but you know where it is, them you must move it there, > and set the ownership to `bacula:disk` and the > permissions to allow the bacula user to, well... Read and write to it. :) > > It is just a guess, but is it possible that this Bacula file volume lives > under a directory which you mounted > `/mnt/khapbackup/backup/bacula/archive` on top of, so it exists, but is not > visible currently? This is just a guess, but it > is also a common mistake. :) > > If you truly do not know where this Bacula file volume is, then you need to > delete it from the catalog so the director no > longer thinks it is accessible: > > * delete yes volume=Vol-0014 > > Might want to run the bconsole `query` command and select option 14 to be > sure there are no jobs on it that you might need. > > Once deleted from the catalog, the Director should select a new volume (or > create a new one if `LabelFormat = yes` is set in > the Pool `File` and `LabelMedia = yes` is set in the SD's Drive devices > writing to this directory. > > Alternately, you can just disable the volume in the catalog: > > * update volume=Vol-0014 volstatus=Disabled > > > Also, consider that the ownership and permissions that you set on the mount > point for the user `backupuser` will not allow > the SD running as the user `bacula` to read/write there - unless you dis > something like make the bacula user a member of the > backupuser's group, and you gave the group read/write permissions into that > directory tree. > > > Hope this helps! > Bill > > -- > Bill Arlofski > w...@protonmail.com > > ___ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
Hello Myles, BTW, after I sent my first reply I saw in your Github posting that you had modified the Bacula SD systemd unit files to have the SD run as the other user, so I suspect this volume is either somewhere else or it has been "mounted over" as I mentioned previously. :) Hope this helps! Bill -- Bill Arlofski w...@protonmail.com signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
On 11/27/23 15:26, MylesDearBusiness via Bacula-users wrote: Hello, Bacula experts. My cloud provider offers only curlftpfs based storage, which I have enabled and secured. I created a username “backupuser” and a system service that launches the service, effectively mounting the remote storage to /mnt/my_backup and running the userspace curlftpfs command as the user “backupuser”. This user has demonstrable read/write privileges in the file system as expected. So far so good. I installed bacula on my Ubuntu 22.04.3 LTS server and got bacula-dir, bacula-fd, and bacula-sd all running. In summary, when I try to run my backup job I get an error: Device is BLOCKED waiting for mount of volume ChatGPT4 doesn’t know nearly as much as this august body, I’m hoping a kindly member may be able to give me a hand up. More details:https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b Thanks, What does this show?: # ls -la /mnt/khapbackup/backup/bacula/archive Is there a Bacula File volume named `Vol-0014` in there? Is it rw for the `bacula` user which the SD (normally) runs as? If it is not there, but you know where it is, them you must move it there, and set the ownership to `bacula:disk` and the permissions to allow the bacula user to, well... Read and write to it. :) It is just a guess, but is it possible that this Bacula file volume lives under a directory which you mounted `/mnt/khapbackup/backup/bacula/archive` on top of, so it exists, but is not visible currently? This is just a guess, but it is also a common mistake. :) If you truly do not know where this Bacula file volume is, then you need to delete it from the catalog so the director no longer thinks it is accessible: * delete yes volume=Vol-0014 Might want to run the bconsole `query` command and select option 14 to be sure there are no jobs on it that you might need. Once deleted from the catalog, the Director should select a new volume (or create a new one if `LabelFormat = yes` is set in the Pool `File` and `LabelMedia = yes` is set in the SD's Drive devices writing to this directory. Alternately, you can just disable the volume in the catalog: * update volume=Vol-0014 volstatus=Disabled Also, consider that the ownership and permissions that you set on the mount point for the user `backupuser` will not allow the SD running as the user `bacula` to read/write there - unless you dis something like make the bacula user a member of the backupuser's group, and you gave the group read/write permissions into that directory tree. Hope this helps! Bill -- Bill Arlofski w...@protonmail.com signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Having difficulty mounting curlftpfs on bacula : "Device is BLOCKED waiting for mount of volume"
Hello, Bacula experts. My cloud provider offers only curlftpfs based storage, which I have enabled and secured. I created a username “backupuser” and a system service that launches the service, effectively mounting the remote storage to /mnt/my_backup and running the userspace curlftpfs command as the user “backupuser”. This user has demonstrable read/write privileges in the file system as expected. So far so good. I installed bacula on my Ubuntu 22.04.3 LTS server and got bacula-dir, bacula-fd, and bacula-sd all running. In summary, when I try to run my backup job I get an error: Device is BLOCKED waiting for mount of volume ChatGPT4 doesn’t know nearly as much as this august body, I’m hoping a kindly member may be able to give me a hand up. More details: https://gist.github.com/mdear/99ed7d56fd5611216ce08ecff6244c8b Thanks, 11 Malcolm Place, OTTAWA, ON K2G 4H7, CANADA OFFICE: +1 613 416 8525 Canadian softphone (preferred) : +1 613 703 4152 Canadian cell : +1 613 858 2120 Sent from [Mail](https://go.microsoft.com/fwlink/?LinkId=550986) for Windows >___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
