[Bacula-users] Cloud storage and bacula 11
Has anyone been able to use cloud storage reliably with Bacula 11? I am trying
to convert from using volumes on local disk to using minio as a cloud service
without much luck.
The odd thing is, it will work for a time. But if bacula-sd or bacula-dir
restarts, I end up in a state where the device status shows things like:
> Device Cloud (S3 Plugin): "cloudDrive-01" (/tmp/cloudDrive-01) is not open.
> Device is BLOCKED waiting for mount of volume "rpi-0002",
> Pool: rpi
> Media type: CloudType
> Drive 0 is not loaded.
and I don't see any obvious way to force it to mount the cloud storage into
that device node -- short of running s3fs to mount the volume as if I was doing
a restore. I would think that would not be needed since this mount request is
from a backup.
Restores are another problem, as I have yet to get one to work reliably with
cloud media. Maybe I just need to tune my settings a bit more, but doing a
mount of a volume with lots of small files via. s3fs is painful at best, and
doesn't work at all in the worst case. Having to mount that under a specifc
directory when I'm running bacula-sd in a docker container is even more
painful, since it requires reconfiguing and restarting bacula-sd any time I
want to restore something.
Here's a sample of my config (with passwords/hostnames obsfucated of course):
> Device {
> Name = cloudDrive-01
> Device Type = Cloud
> Media Type = CloudType
> Archive Device = /tmp/cloudDrive-01
> AutomaticMount = yes
> AlwaysOpen = no
> RemovableMedia = yes
> Label Media = yes
> Random Access = yes
> Autochanger = yes
> Maximum Part Size = 50MB
> Maximum File Size = 50MB
> Cloud = minio-nas
> }
> Autochanger {
> Name = CloudAutoChangerS3
> Changer Device = /dev/null
> Changer Command = ""
> Device = cloudDrive-01, cloudDrive-02, cloudDrive-03, cloudDrive-04
> }
> Cloud {
> Name = "minio-nas"
> Driver = S3
> HostName = "nas.xxx.com:9000"
> BucketName = "bacula"
> AccessKey = "bacula"
> SecretKey = "xx"
> Protocol = HTTP
> UriStyle = "Path"
> Truncate Cache = "AfterUpload"
> Upload = "EachPart"
> Region = us-xxx-1
> MaximumUploadBandwidth = 100MB/s
> }
>
Dan Rich
http://www.employees.org/~drich/
"Step up to red alert!" "Are you sure, sir?
It means changing the bulb in the sign..."
- Red Dwarf (BBC)
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Client Behind NAT
Hello Josh (everyone else too),
I can confirm that if the FD --> DIR connection is opened, then the Director
does use this socket to communicate to the FD.
However, the "Connecting to Client..." message does not change, and incorrectly
(IMHO) reports that it is making an outbound
connection to the IP:port specified in the Director's resource for this Client:
8<
*s client=stinky-fd
Connecting to Client stinky-fd at 10.1.1.222:9102
8<
I did an `estimate` and then ran a job. Packet traces confirm that the
connection(s) created by the client are used and the
Director does not actually call out to it. A nice feature request would be to
change this Connecting message to something like:
8<
*s client=stinky-fd
Connecting to Client stinky-fd on inbound connection opened by Client
8<
Interestingly, if the Client's connection into the Director is down (ie: kill
the FD), then the Director does actually make
the attempt to connect to the C
lient on it's defined IP:port, which of course fails. :)
I think this is also incorrect behavior, or at least it is not behaving as
documented, or in the way I would expect/want it
to for Clients behind NAT when we know the Director will never be able to make
the connection.
This is all nice information and it proves this feature is (mostly) working as
documented, but now we still need to solve
Wanderlei's issue. :)
I am guessing one or more of a few things are the possible culprit:
- Port forwarding at the firewall is not working as expected
- The `Address = ` in the Director{} section of the FD is not correct
- The FD has not been restarted (I have seen systemctl restart bacula-fd not
always work)
@Wanderlei, I would recommend to do a tcpdump on the Director when things are
quiet (ie: no jobs running) to see if this
inbound connection from the client is actually making it to the Director
through your firewall:
First stop the FD.
Then start a tcp
dump session as root on the Director:
8<
# tcpdump -i any tcp port 9101 or 9102 -vvv -w bacula.dump
8<
Then, start the FD. The "Got #" message should increment. If it does not, we
have our answer. If it does, do a 'status
client=' in bconsole for this client. The "Got #" should increment some...
Kill the tcpdump and open the dump file in Wireshark to see what was happening.
You can even start the FD in foreground and debug mode to see what it is doing
and look for its outbound connection
attempt(s) to the Director to confirm it is trying to do the right thing:
8<
# killall bacula-fd
# /path/to/bacula-fd -f -d100 -c /path/to/bacula-fd.conf
8<
Let us know what you find!
Best regards,
Bill
--
Bill Arlofski
w...@protonmail.com
signature.asc
Description: OpenPGP digital signature
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bad caps from SD: auth cram-md5
On 10/18/22 04:59, Kees Bakker via Bacula-users wrote: Hi, Recently I upgraded one system to CentOS 9 stream which has bacula-client 11.0.1 I'm using the same PKI config as before (CentOS 7 with bacula-client 5.2.13) The director and the storage server have bacula 9.4.2. Other clients (Ubuntu and CentOS8) have version 9.4.2 and 9.0.6). All these clients work OK. However the 11.0.1 client gives the following error: Fatal error: hello.c:191 Bad caps from SD: auth cram-md5 Do I need to change anything in the config to avoid this error? -- Kees Yes. Downgrade the CentOS 9 Stream Bacula FD to version 9.4.2 (same as Director), or lower. Of course the other option is to upgrade the Director/SD server to Bacula 11.0.1 or higher. The general rule with Bacula daemon versions is (DIR = SDs) >= FDs Hope this helps! Bill -- Bill Arlofski w...@protonmail.com signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Client Behind NAT
Hello Josh! Maybe. However, it depends on what the client is doing. If the client goes into sleep-mode, it will not re-establish the connection after the ReconnectionTime. In that case, the Director would not have an active socket and so may well try to establish one. This does not seem to be the case. From the documentation, we have: 8< When AllowFDConnections is set to true, the Director will accept incoming connections from the Client and will keep the socket open for a future use. The Director will no longer use the Address to contact the File Daemon. This configuration is useful if the Director cannot contact the FileDaemon directly. 8< So it seems that when `AllowFDConnections` is enabled for a client, the Director should never attempt an outgoing connection to it. If the Client is sleeping, or off, then any commands and hence any jobs for this Client should just fail. At least this is how it appears it should work according t o the documentation. :) I am having issues with Virtualbox on my local system here so I was not able to test last night and capture packets etc, but I just remembered I have a spare laptop I can use as the remote FD instead on one of my VMs, so I should have some results soon. :) Best regards, Bill -- Bill Arlofski w...@protonmail.com signature.asc Description: OpenPGP digital signature ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Bad caps from SD: auth cram-md5
Hi, Recently I upgraded one system to CentOS 9 stream which has bacula-client 11.0.1 I'm using the same PKI config as before (CentOS 7 with bacula-client 5.2.13) The director and the storage server have bacula 9.4.2. Other clients (Ubuntu and CentOS8) have version 9.4.2 and 9.0.6). All these clients work OK. However the 11.0.1 client gives the following error: Fatal error: hello.c:191 Bad caps from SD: auth cram-md5 Do I need to change anything in the config to avoid this error? -- Kees ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Client Behind NAT
On 10/17/22 20:09, Bill Arlofski via Bacula-users wrote: On 10/17/22 13:14, Rodrigo Reimberg via Bacula-users wrote: > Telnet from DIR to FD on port 9102 not ok, I’m using ConnectToDirector parameter. Hello Rodrigo, Of course this will not work. The client is behind NAT, and the Director cannot connect to it on port 9102 (or any other port :). As you know, with the 'ConnectToDirector' feature enabled, the FD calls into the Dir on port 9101 (default). There is no requirement to set any connection Schedule. The Client will remain connected until the `ReconnectionTime` is reached (default 40 mins), at which point the connection will be dropped and immediately re-established. Per the documentation, this FD -> DIR connection *should be* used for all communications between the Director and this client: 8< ConnectToDirector = When the ConnectToDirector directive is set to true, the Client will contact the Director according to the rules. The connection initiated by the Client will be then used by the Director to start jobs or issue bconsole commands. 8< I just ran some tests, and it looks like there is a bug. Maybe. However, it depends on what the client is doing. If the client goes into sleep-mode, it will not re-establish the connection after the ReconnectionTime. In that case, the Director would not have an active socket and so may well try to establish one. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
