Hello, Please reply to bacula-users list, so all can help you or benefit from our discussion.
pon., 11 mar 2024 o 02:38 Ismael Matos <iabma...@gmail.com> napisał(a): > Hello Radosław. > > *This is the console output from a freshly restarted system.* > *From the bacula trace, I've found that the docker plugin uses a > variation of "docker ps -a" command.* > *While the 2 docker services are up automatically after the system > restart, but the "docker ps -a" command says different. * > > **** > im@imcserv24:~$ /usr/bin/docker ps -a --no-trunc=true > Cannot connect to the Docker daemon at > unix:///home/im/.docker/desktop/docker.sock. Is the docker daemon running? > What user bacula-fd is running on? Is it "im"? If not then this test is useless. You need to check it on the user which runs bacula-fd. In most cases it is a "root". im@imcserv24:~$ sudo systemctl status docker* > [sudo] password for im: > ● docker.service - Docker Application Container Engine > Loaded: loaded (/lib/systemd/system/docker.service; enabled; preset: > enabled) > Active: active (running) since Mon 2024-03-11 00:05:37 GMT; 1min 24s > ago > TriggeredBy: ● docker.socket > Docs: https://docs.docker.com > Main PID: 1674 (dockerd) > Tasks: 10 > Memory: 105.4M > CPU: 715ms > CGroup: /system.slice/docker.service > └─1674 /usr/bin/dockerd -H fd:// > --containerd=/run/containerd/containerd.sock > > Mar 11 00:05:36 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:36.092629876Z" level=info msg="Starting up" > Mar 11 00:05:36 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:36.108840311Z" level=info msg="detected 127.0.0.53 > nameserver, ass> > Mar 11 00:05:36 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:36.435263475Z" level=info msg="[graphdriver] using > prior storage d> > Mar 11 00:05:36 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:36.440764468Z" level=info msg="Loading containers: > start." > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.107855184Z" level=info msg="Default bridge > (docker0) is assigne> > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.199167737Z" level=info msg="Loading containers: > done." > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.269123107Z" level=info msg="Docker daemon" > commit=061aa95 conta> > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.270738445Z" level=info msg="Daemon has completed > initialization" > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.342865821Z" level=info msg="API listen on > /run/docker.sock" > Mar 11 00:05:37 imcserv24.imc systemd[1]: Started docker.service - Docker > Application Container Engine. > > ● docker.socket - Docker Socket for the API > Loaded: loaded (/lib/systemd/system/docker.socket; enabled; preset: > enabled) > Active: active (running) since Mon 2024-03-11 00:05:28 GMT; 1min 33s > ago > Triggers: ● docker.service > Listen: /run/docker.sock (Stream) > Tasks: 0 (limit: 19048) > Memory: 0B > CPU: 1ms > CGroup: /system.slice/docker.socket > > Mar 11 00:05:28 imcserv24.imc systemd[1]: Starting docker.socket - Docker > Socket for the API... > Mar 11 00:05:28 imcserv24.imc systemd[1]: Listening on docker.socket - > Docker Socket for the API. > **** > > *Just after the previous command: * > > **** > im@imcserv24:~$ /usr/bin/docker ps -a --no-trunc=true > Cannot connect to the Docker daemon at > unix:///home/im/.docker/desktop/docker.sock. Is the docker daemon running? > **** > > *After a few minutes. Still the same results:* > > **** > im@imcserv24:~$ /usr/bin/docker ps -a --no-trunc=true > Cannot connect to the Docker daemon at > unix:///home/im/.docker/desktop/docker.sock. Is the docker daemon running? > im@imcserv24:~$ sudo systemctl status docker* > ● docker.service - Docker Application Container Engine > Loaded: loaded (/lib/systemd/system/docker.service; enabled; preset: > enabled) > Active: active (running) since Mon 2024-03-11 00:05:37 GMT; 5min ago > TriggeredBy: ● docker.socket > Docs: https://docs.docker.com > Main PID: 1674 (dockerd) > Tasks: 10 > Memory: 104.0M > CPU: 799ms > CGroup: /system.slice/docker.service > └─1674 /usr/bin/dockerd -H fd:// > --containerd=/run/containerd/containerd.sock > > Mar 11 00:05:36 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:36.092629876Z" level=info msg="Starting up" > Mar 11 00:05:36 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:36.108840311Z" level=info msg="detected 127.0.0.53 > nameserver, ass> > Mar 11 00:05:36 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:36.435263475Z" level=info msg="[graphdriver] using > prior storage d> > Mar 11 00:05:36 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:36.440764468Z" level=info msg="Loading containers: > start." > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.107855184Z" level=info msg="Default bridge > (docker0) is assigne> > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.199167737Z" level=info msg="Loading containers: > done." > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.269123107Z" level=info msg="Docker daemon" > commit=061aa95 conta> > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.270738445Z" level=info msg="Daemon has completed > initialization" > Mar 11 00:05:37 imcserv24.imc dockerd[1674]: > time="2024-03-11T00:05:37.342865821Z" level=info msg="API listen on > /run/docker.sock" > Mar 11 00:05:37 imcserv24.imc systemd[1]: Started docker.service - Docker > Application Container Engine. > > ● docker.socket - Docker Socket for the API > Loaded: loaded (/lib/systemd/system/docker.socket; enabled; preset: > enabled) > Active: active (running) since Mon 2024-03-11 00:05:28 GMT; 5min ago > Triggers: ● docker.service > Listen: /run/docker.sock (Stream) > Tasks: 0 (limit: 19048) > Memory: 0B > CPU: 1ms > CGroup: /system.slice/docker.socket > > Mar 11 00:05:28 imcserv24.imc systemd[1]: Starting docker.socket - Docker > Socket for the API... > Mar 11 00:05:28 imcserv24.imc systemd[1]: Listening on docker.socket - > Docker Socket for the API. > **** > > *AFTER I started the Docker Desktop application:* > > **** > im@imcserv24:~$ /usr/bin/docker ps -a --no-trunc=true > CONTAINER ID IMAGE > COMMAND > CREATED STATUS PORTS > NAMES > c4dc2eed2779f33837fad3baed64d08256d3590eb9ff8825143934bbb0cd538d > quay.io/kiegroup/business-central-workbench-showcase:latest > "./start_business-central-wb.sh" 10 days ago Exited (255) 57 seconds > ago 0.0.0.0:8091->8001/tcp, 0.0.0.0:8090->8080/tcp frosty_pike > im@imcserv24:~$ > **** > > *Then I ran the backup job, and I got the same message:* > > IMCSERV24 JobId 2141: Error: docker: No Docker containers or objects to > backup found. > > Back to the bacula-fd trace: > > **** > IMCSERV24: dkcommctx.c:1693-2141 dkcommctx: get_all_list_from_docker > finish. > IMCSERV24: dkcommctx.c:1635-2141 dkcommctx: get_all_list_from_docker called > IMCSERV24: dkcommctx.c:364-2141 dkcommctx: Executing: /usr/bin/docker > image ls --no-trunc=true --format "{{.ID}}\t{{.Repository}}\t{{.T > ag}}\t{{.Size}}\t{{.CreatedAt}}" > IMCSERV24: dkcommctx.c:384-2141 dkcommctx: Command executed at PID=8331 > IMCSERV24: dkcommctx.c:1685-2141 dkcommctx: get_all_list_from_docker no > container found. > **** > > *BUT, executing the command manually:* > > **** > im@imcserv24:~$ /usr/bin/docker image ls --no-trunc=true --format > "{{.ID}}\t{{.Repository}}\t{{.Tag}}\t{{.Size}}\t{{.CreatedAt}}" > sha256:588095f0287918cf7aa28cc2bddf833d34ce18f027ed8e151eb8600c6519238a > quay.io/kiegroup/business-central-workbench-showcase latest1.06GB > 2023-08-14 11:24:01 +0100 BST > **** > > *BUUUUT, executing again as ROOT, I get nothing!* > > If your bacula-fd is running as "root" user (99,99% probability) then this is the cause of your issue. You need to check why. > **** > im@imcserv24:~$ sudo /usr/bin/docker image ls --no-trunc=true --format > "{{.ID}}\t{{.Repository}}\t{{.Tag}}\t{{.Size}}\t{{.CreatedAt}}" > im@imcserv24:~$ > **** > > Today this is NOT a problem for me, but I guess that would impact others > (or not). > > *Do they check all their container's backups, one by one?* > > Please let me know if I can be of assistance in testing any further this > situation. > First of all, you configured Bacula Docker plugin to backup all, even empty list of containers. So this is an expected behavior. If you need to get job error then you need to add "abortonerror" plugin parameter. Radek > > Many thanks again. > > Cheers, > > Ismael > > On Sat, 9 Mar 2024 at 12:11, Radosław Korzeniewski < > rados...@korzeniewski.net> wrote: > >> Hello, >> >> sob., 9 mar 2024 o 12:38 Ismael Matos <iabma...@gmail.com> napisał(a): >> >>> Hello all, >>> >>> First, thanks to Radosław Korzeniewski for helping me with the BPIPE >>> plugin. It's working now with the changes of a few things within the >>> scripts and the .conf file, as per his suggestions.. >>> >> >> No problem. >> >> >>> >>> Now, it is the Docker plugin. I've been using Docker Desktop for a low >>> priority application and got a container/image running on: >>> - Ubuntu 23.10 >>> - Docker Engine: 25.0.3 >>> - Compose: v2.24.5-desktop.1 >>> - Credential Helper: v0.8.1 >>> - Kubernetes: v1.29.1 >>> - Bacula: 13.0.4, including the 2 modules of Docker plugin. >>> - The relevant section of the bacula-dir file: >>> Include { >>> Plugin = "docker:" >>> Options { >>> Compression = "Gzip" >>> Signature = "Md5" >>> } >>> } >>> >>> When I run a Full backup, I get: >>> IMCSERV24 JobId 2131: Error: docker: No Docker containers or objects to >>> backup found. >>> >> >> Bacula Docker Plugin (AFAIK) uses docker command to manage backup/restore >> jobs. This error means the plugin was unable to list containers or images >> on your installation. >> You can set debug in bconsole: >> * setdebug level=500 trace=1 client=IMCSERV24 >> run backup/estimate job and review trace log created at >> $WORKINGDIR/*.trace. >> In this log you should get info why the plugin is unable to list docker. >> >> But, when I run an Incremental Backup, I also get: >>> IMCSERV24 JobId 2129: Error: docker: Unsupported backup level. Doing >>> FULL backup. >>> >> >> Yes, the docker plugin does not support Incremental level backup for >> containers. >> >> >>> Since the container is actually running >>> >> >> show me >> >> >>> and the Docker plugin starts, I wonder if there is some access >>> authorization issue to some resource; >>> >> >> Plugin needs the permissions to execute docker command, which needs >> access to docker.sock. It could be the main cause of this issue. >> >> >>> Bacula-fd is running as root, and some Docker processes are running as >>> root, while other processes are running with my user (I guess those >>> processes are the Docker Desktop front-end). >>> >> >> It has nothing to do with docker processes, but the permissions to >> execute docker commands and access to docker.sock. >> >> >>> Still on the Docker, what files can be safely excluded during the >>> backups? Suggestions and recomendations are welcome >>> >> >> It does not work that way. Docker uses a very different way to execute >> processes and access the data. >> You can backup docker containers as docker images, no single file backup >> possible. >> If you need (why?) to backup single files in docker containers then add >> bacula file daemon to docker container and do backups (silly idea). >> Additionally you can backup docker persistent volumes contents as simple >> tar files. >> >> I hope it helps. >> -- >> Radosław Korzeniewski >> rados...@korzeniewski.net >> > -- Radosław Korzeniewski rados...@korzeniewski.net
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
