Hello Ziga, If you are using the Orwcle Gateway there is no reason on Earth to use the S3 Driver. It will only mess your system, since I believe even in the SD startup it tries to connect to the bucket. If you want smaller volumes, just tune the Maximum Volume Bytes and maybe Jobs (e.g. one).
Regards, -- MSc Heitor Faria CEO Bacula LatAm mobile1: + 1 909 655-8971 mobile2: + 55 61 98268-4220 América Latina [ http://bacula.lat/] -------- Original Message -------- From: Žiga Žvan <ziga.z...@cetrtapot.si> Sent: Monday, December 28, 2020 04:07 PM To: Heitor Faria <hei...@bacula.com.br>,bacula-users@lists.sourceforge.net Subject: Re: [Bacula-users] bacula-sd - file driver - cloud resource >Hello Heitor, > >I am aware of that. I'm using Oracle Storage Gateway to upload data to >cloud (to S3 bucket). For bacula-sd, backup destination is ordinary >filesystem mounted over nfs. However I'm using file driver with cloud >resource (in order to have volume broken down in file parts and >therefore optimize upload amount). > >Everything worked until bacula decided to reuse old volume. I'm getting >strange error (cannot download Volume). In my opinion this is some sort >of bug, because data is available on nfs destination (see example output >from cloudcache and backup folder bellow). > >Could you check my configuration (bellow)? Could you confirm that this >is a bug and I did not misconfigure something? >Regards, >Ziga > > >On 28.12.2020 14:46, Heitor Faria wrote: >> >> Hello Ziga, >> >> The Bacula Community S3 Plugin was built for Amazon S3 protocol. >> I read that Oracle built some sort of S3 emulator for its cloud, but >> since Bacula Community is a free software, it comes without warranties >> or support. >> You could consider running the bacula-sd in debug mode to troubleshoot >> the problem. The Driver is not working properly. >> Bacula Systems already developed a specific plugin for Oracle Object >> Storages, available only in the Enterprise edition. >> >> Regards, >> -- >> MSc Heitor Faria >> CEO Bacula LatAm >> mobile1: + 1 909 655-8971 >> mobile2: + 55 61 98268-4220 >> >> América Latina >> [ http://bacula.lat/] >> >> >> >> -------- Original Message -------- >> From: Žiga Žvan <ziga.z...@cetrtapot.si> >> Sent: Monday, December 28, 2020 08:22 AM >> To: bacula-users@lists.sourceforge.net >> Subject: Re: [Bacula-users] bacula-sd - file driver - cloud resource >> >> Hi, >> What am I doing wrong? I would be grateful for a hint, why there is no >> reply to my emails. >> I believe that my problems are a bit complex for an email. Is there any >> way to discuss this in different way? >> Who should I contact in case that I would like to support development of >> oracle cloud plugin? >> >> Regards, >> Ziga >> >> On 21.12.2020 20:38, Žiga Žvan wrote: >> > Hello, >> > I'm using file driver with cloud resource. Bacula was able to backup >> > data in this way until it wrote data to new volumes. Now, after >> > retention period, I'm getting error: Fatal error: cloud_dev.c:983 >> > Unable to download Volume (see output below). Data on cloud path looks >> > ok but data in local cache contains only part.1 without any data. >> > >> > Is this expected? >> > Has anybody tested this scenario? >> > Should I avoid file driver in production environment? >> > >> > Regards, >> > Ziga >> > >> > >> > [root@bacula db-01-weekly-vol-0365]# ls -la >> > /mnt/ocisg/bacula/backup/db-01-weekly-vol-0365 >> > total 0 >> > drwxr-----. 2 bacula disk 0 Oct 24 07:45 . >> > drwxr-xr-x. 2 bacula bacula 0 Dec 18 23:38 .. >> > -rw-r--r--. 1 bacula disk 256 Oct 24 07:43 part.1 >> > -rw-r--r--. 1 bacula disk 999935992 Oct 24 07:44 part.2 >> > -rw-r--r--. 1 bacula disk 999935993 Oct 24 07:44 part.3 >> > -rw-r--r--. 1 bacula disk 381771773 Oct 24 07:45 part.4 >> > >> > [root@bacula db-01-weekly-vol-0365]# ls -la >> > /storage/bacula/cloudcache/db-01-weekly-vol-0365 >> > total 20 >> > drwxr-----. 2 bacula disk 28 Dec 11 23:10 . >> > drwxr-xr-x. 344 bacula bacula 16384 Dec 18 23:26 .. >> > -rw-r--r--. 1 bacula disk 0 Dec 11 23:10 part.1 >> > >> > SD config (autochanger) >> > >> > Device { >> > Name = FSOciCloudStandard2 >> > Device type = Cloud >> > Cloud = OracleViaStorageGateway >> > Maximum Part Size = 1000 MB >> > Media Type = File1 >> > Archive Device = /storage/bacula/cloudcache >> > LabelMedia = yes; # lets Bacula label unlabeled >> media >> > Random Access = Yes; >> > AutomaticMount = yes; # when device opened, read it >> > RemovableMedia = no; >> > AlwaysOpen = no; >> > Autochanger = yes; >> > } >> > ... >> > Device { >> > Name = FSOciCloudStandard4 >> > Device type = Cloud >> > Cloud = OracleViaStorageGateway >> > Maximum Part Size = 1000 MB >> > Media Type = File1 >> > Archive Device = /storage/bacula/cloudcache >> > LabelMedia = yes; # lets Bacula label unlabeled >> media >> > Random Access = Yes; >> > AutomaticMount = yes; # when device opened, read it >> > RemovableMedia = no; >> > AlwaysOpen = no; >> > Autochanger = yes; >> > } >> > >> > Cloud { >> > Name = OracleViaStorageGateway >> > Driver = "File" >> > HostName = "/mnt/ocisg/bacula/backup" >> > BucketName = "DummyBucket" >> > AccessKey = "DummyAccessKey" >> > SecretKey = "DummySecretKey" >> > Protocol = HTTPS >> > UriStyle = VirtualHost >> > Truncate Cache = AtEndOfJob >> > } >> > >> > >> > 21-Dec 19:14 bacula-dir JobId 2073: Start Backup JobId 2073, >> > Job=db-01-backup.2020-12-21_19.14.14_48 >> > 21-Dec 19:14 bacula-dir JobId 2073: Using Device "FSOciCloudStandard2" >> > to write. >> > 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable >> > to download Volume="db-01-weekly-vol-0365" label. >> > 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable >> > to download Volume="db-01-weekly-vol-0365" label. >> > 21-Dec 19:14 db-01.prod.kr.cetrtapot.si JobId 2073: Fatal error: >> > job.c:3013 Bad response from SD to Append Data command. Wanted 3000 OK >> > data >> > , got len=25 msg="3903 Error append data: " >> > 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable >> > to download Volume="db-01-weekly-vol-0365" label. >> > 21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud >> > device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume >> > "db-01-weekly-vol-0365" failed: ERR= >> > 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable >> > to download Volume="db-01-weekly-vol-0365" label. >> > 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: cloud_dev.c:983 Unable >> > to download Volume="db-01-weekly-vol-0365" label. >> > 21-Dec 19:14 bacula-sd JobId 2073: Warning: label.c:398 Open Cloud >> > device "FSOciCloudStandard2" (/storage/bacula/cloudcache) Volume >> > "db-01-weekly-vol-0365" failed: ERR= >> > 21-Dec 19:14 bacula-sd JobId 2073: Marking Volume >> > "db-01-weekly-vol-0365" in Error in Catalog. >> > 21-Dec 19:14 bacula-sd JobId 2073: Fatal error: Job 2073 canceled. >> > >> > >> > On 06.12.2020 20:52, Žiga Žvan wrote: >> >> Dear all, >> >> I'm using bacula 9.6.5 in a production for a month now. I'm >> >> experiencing random backup failures from my clients. Specific hosts >> >> report errors like the outputs attached. The same host is able to >> >> perform backup at some other time. The error is more often at large >> >> backups (more errors at full backups than incremental, more errors at >> >> hosts with large data sets). >> >> >> >> I have tried to implement heartbeat interval >> >> >> (https://www.bacula.org/9.6.x-manuals/en/main/Client_File_daemon_Configur.html#SECTION002210000000000000000) >> >> >> >> but there is no improvement. >> >> The error occures also on hosts in the same zone as bacula server (no >> >> router/firewall in between). >> >> Storage deamon is installed on the same server as bacula director. >> >> I'm using File cloud driver (backup to local disk via cloud resource). >> >> >> >> Could you please suggest a solution or a way to troubleshoot this >> >> further? >> >> Thx! >> >> >> >> Regards,Ziga Zvan >> >> >> >> Backup from linux hosts (on 05-dec 3 hosts failed, 20 hosts completed >> >> without error): >> >> 05-Dec 03:26 bacula-dir JobId 1721: Fatal error: Network error with >> >> FD during Backup: ERR=Connection reset by peer >> >> 05-Dec 03:27 bacula-dir JobId 1721: Fatal error: No Job status >> >> returned from FD. >> >> 05-Dec 03:27 bacula-dir JobId 1721: Error: Bacula bacula-dir 9.6.5 >> >> (11Jun20): >> >> >> >> Backup from windows hosts (on 05-dec 2 hosts failed, 5 hosts >> >> completed without error): >> >> 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: Error: >> >> lib/bsock.c:383 Write error sending 57172 bytes to Storage >> >> daemon:192.168.66.35:9103: ERR=Input/output error >> >> 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: Fatal error: >> >> filed/backup.c:848 Network send error to SD. ERR=Input/output error >> >> 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer >> >> (BackupComplete): "Task Scheduler Writer", State: 0x1 (VSS_WS_STABLE) >> >> 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer >> >> (BackupComplete): "VSS Metadata Store Writer", State: 0x1 >> >> (VSS_WS_STABLE) >> >> 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer >> >> (BackupComplete): "Performance Counters Writer", State: 0x1 >> >> (VSS_WS_STABLE) >> >> 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer >> >> (BackupComplete): "System Writer", State: 0x1 (VSS_WS_STABLE) >> >> 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer >> >> (BackupComplete): "ASR Writer", State: 0x1 (VSS_WS_STABLE) >> >> 05-Dec 00:40 iwhost01.kranj.cetrtapot.si-fd JobId 1726: VSS Writer >> >> (BackupComplete): "Shadow Copy Optimization Writer", State: 0x1 >> >> (VSS_WS_STABLE) >> >> 05-Dec 01:01 bacula-dir JobId 1726: Error: bsock.c:551 Read error >> >> from Client: >> >> iwhost01.kranj.cetrtapot.si-fd:iwhost01.kranj.cetrtapot.si:9102: >> >> ERR=Connection timed out >> >> 05-Dec 01:01 bacula-dir JobId 1726: Fatal error: Network error with >> >> FD during Backup: ERR=Connection timed out >> >> 05-Dec 01:02 bacula-dir JobId 1726: Fatal error: No Job status >> >> returned from FD. >> >> 05-Dec 01:02 bacula-dir JobId 1726: Error: Bacula bacula-dir 9.6.5 >> >> (11Jun20): >> >> >> >> Similar output from 21-Nov: >> >> 21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error: >> >> lib/bsock.c:383 Write error sending 4 bytes to Storage >> >> daemon:192.168.66.35:9103: ERR=Input/output error >> >> 21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error: >> >> lib/bsock.c:271 Socket has errors=1 on call to Storage >> >> daemon:192.168.66.35:9103 >> >> 21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Error: >> >> lib/bsock.c:271 Socket has errors=1 on call to Storage >> >> daemon:192.168.66.35:9103 >> >> 21-Nov 05:30 dc1.kranj.cetrtapot.si-fd JobId 1393: Fatal error: >> >> filed/backup.c:607 Network send error to SD. ERR=Input/output error >> >> 21-Nov 05:49 bacula-dir JobId 1393: Fatal error: Network error with >> >> FD during Backup: ERR=Connection timed out >> >> >> >> >> >> >> >> >> >> _______________________________________________ >> >> Bacula-users mailing list >> >> Bacula-users@lists.sourceforge.net >> >> https://lists.sourceforge.net/lists/listinfo/bacula-users >> > >> > >> > _______________________________________________ >> > Bacula-users mailing list >> > Bacula-users@lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/bacula-users >> >> >> _______________________________________________ >> Bacula-users mailing list >> Bacula-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bacula-users >>
_______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
