Re: [Bacula-users] Compilation Errors of Bacula 9.6.3 on Ubuntu 18.04
> On Mon, 11 May 2020 10:52:23 -0400, Josh Fisher said: > > This is a multi-part message in MIME format. > --===2027621642792716715== > Content-Type: multipart/alternative; > boundary="70CDB63220E9E5545F246201" > Content-Language: en-US > > This is a multi-part message in MIME format. > --70CDB63220E9E5545F246201 > Content-Type: text/plain; charset=utf-8; format=flowed > Content-Transfer-Encoding: 8bit > > > On 5/11/2020 7:38 AM, Kern Sibbald wrote: > > > > Hello Sven, > > > > I share your concerns, but here are a few mitigating factors: > > > > 1. I am not aware of any other C/C++ S3 library that will do the same > > job and is well maintained. I have to admit I have not looked > > recently, so any suggestions would be welcome. > > > > I'm not aware of other C libraries either, but why not use AWS CLI > high-level commands in much the same way that tape autochanger script > commands are implemented? AWS CLI is actively maintained by Amazon and > so shouldn't have the maintenance issues. These are simple mv, cp, ls, > etc. commands somewhat equivalent to their Unix command namesakes. They > shouldn't change even when/if Amazon changes the underlying S3 protocols. There is also the Amazon's own SDK: https://aws.amazon.com/sdk-for-cpp/ > > > > 2. Bacula Systems actively uses libs3 with its customers, and any > > corrections that they make will also be in the Bacula community libs3 > > git repo (under the same GNU Lesser V3 license) > > > > 3. Amazon (or other s3 supplier's) tools can be used to restore Bacula > > S3 Volumes to disk, and once that is done, Bacula can read those > > volumes regardless of what S3 library it is linked with. This is > > because when the Volumes are on disk, Bacula reads them as standard OS > > files. libs3 is only used to move files from a local disk to and from > > the S3 cloud. > > > > Best regards, > > > > Kern > > > > On 5/10/20 3:24 PM, Sven Hartge wrote: > >> On 10.05.20 15:01, Kern Sibbald wrote: > >> > >>> I agree with Sven, libs3 is a big disaster. It works well but the > >>> author abandoned it, and many things have changed since then. For the > >>> moment, we have a version that works with AWS (don't expect it to work > >>> with a number of other S3 implementations, which are not compatible with > >>> AWS). > >> Adding to that: Other than the horrendous possible security flaws > >> present in libs3 (try to compile with a recent GCC and see for yourself) > >> the nature of anything cloud-bases is inherently volatile. > >> > >> The AWS-API may change at any given moment (and it has in the past), > >> making libs3 incompatible without updates. > >> > >> And without an upstream author implementing those changes, your backups > >> are more or less gone. > >> > >> My very pessimistic view on the situation is: Don't use any backup > >> solution using libs3 if you value your data. > >> > >> But: YMMV. > >> > >> Grüße, > >> Sven. > >> > >> > >> > >> ___ > >> Bacula-users mailing list > >> Bacula-users@lists.sourceforge.net > >> https://lists.sourceforge.net/lists/listinfo/bacula-users > > > > > > ___ > > Bacula-users mailing list > > Bacula-users@lists.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bacula-users > ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Compilation Errors of Bacula 9.6.3 on Ubuntu 18.04
On 5/11/2020 7:38 AM, Kern Sibbald wrote: Hello Sven, I share your concerns, but here are a few mitigating factors: 1. I am not aware of any other C/C++ S3 library that will do the same job and is well maintained. I have to admit I have not looked recently, so any suggestions would be welcome. I'm not aware of other C libraries either, but why not use AWS CLI high-level commands in much the same way that tape autochanger script commands are implemented? AWS CLI is actively maintained by Amazon and so shouldn't have the maintenance issues. These are simple mv, cp, ls, etc. commands somewhat equivalent to their Unix command namesakes. They shouldn't change even when/if Amazon changes the underlying S3 protocols. 2. Bacula Systems actively uses libs3 with its customers, and any corrections that they make will also be in the Bacula community libs3 git repo (under the same GNU Lesser V3 license) 3. Amazon (or other s3 supplier's) tools can be used to restore Bacula S3 Volumes to disk, and once that is done, Bacula can read those volumes regardless of what S3 library it is linked with. This is because when the Volumes are on disk, Bacula reads them as standard OS files. libs3 is only used to move files from a local disk to and from the S3 cloud. Best regards, Kern On 5/10/20 3:24 PM, Sven Hartge wrote: On 10.05.20 15:01, Kern Sibbald wrote: I agree with Sven, libs3 is a big disaster. It works well but the author abandoned it, and many things have changed since then. For the moment, we have a version that works with AWS (don't expect it to work with a number of other S3 implementations, which are not compatible with AWS). Adding to that: Other than the horrendous possible security flaws present in libs3 (try to compile with a recent GCC and see for yourself) the nature of anything cloud-bases is inherently volatile. The AWS-API may change at any given moment (and it has in the past), making libs3 incompatible without updates. And without an upstream author implementing those changes, your backups are more or less gone. My very pessimistic view on the situation is: Don't use any backup solution using libs3 if you value your data. But: YMMV. Grüße, Sven. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Compilation Errors of Bacula 9.6.3 on Ubuntu 18.04
Hello Sven, I share your concerns, but here are a few mitigating factors: 1. I am not aware of any other C/C++ S3 library that will do the same job and is well maintained. I have to admit I have not looked recently, so any suggestions would be welcome. 2. Bacula Systems actively uses libs3 with its customers, and any corrections that they make will also be in the Bacula community libs3 git repo (under the same GNU Lesser V3 license) 3. Amazon (or other s3 supplier's) tools can be used to restore Bacula S3 Volumes to disk, and once that is done, Bacula can read those volumes regardless of what S3 library it is linked with. This is because when the Volumes are on disk, Bacula reads them as standard OS files. libs3 is only used to move files from a local disk to and from the S3 cloud. Best regards, Kern On 5/10/20 3:24 PM, Sven Hartge wrote: On 10.05.20 15:01, Kern Sibbald wrote: I agree with Sven, libs3 is a big disaster. It works well but the author abandoned it, and many things have changed since then. For the moment, we have a version that works with AWS (don't expect it to work with a number of other S3 implementations, which are not compatible with AWS). Adding to that: Other than the horrendous possible security flaws present in libs3 (try to compile with a recent GCC and see for yourself) the nature of anything cloud-bases is inherently volatile. The AWS-API may change at any given moment (and it has in the past), making libs3 incompatible without updates. And without an upstream author implementing those changes, your backups are more or less gone. My very pessimistic view on the situation is: Don't use any backup solution using libs3 if you value your data. But: YMMV. Grüße, Sven. ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Compilation Errors of Bacula 9.6.3 on Ubuntu 18.04
Hello,
niedz., 10 maj 2020 o 00:01 r0...@nxlplyx.com
napisał(a):
> Hi Radoslaw,
>
> Thank you for the reply. I disabled AFS in the configuration and avoided
> the AFS variable compilation errors.
>
> Unfortunately, I am now getting S3 driver compilation errors.
>
> It is the S3 modules that I need. This may help explain why I have not
> been able to get the S3 module of Bacula 9.6.3 to work in for Ubuntu 18.04.
>
> Can anyone help?
>
What s3 library do you use during Bacula build? You have to use the
library provided by Bacula project. Others won't work.
best regards
> -
>
> LT Compiling tape_dev.c
> LT Compiling vtape_dev.c
> Making libbacsd.la ...
> /home/user1/dirNFS/sw/bacula_9.6.3/bacula-9.6.3/libtool --silent --tag=CXX
> --mode=link /usr/bin/g++-o libbacsd.la \
>acquire.lo ansi_label.lo askdir.lo autochanger.lo block.lo
> block_util.lo butil.lo dev.lo device.lo ebcdic.lo init_dev.lo label.lo
> lock.lo match_bsr.lo mount.lo null_dev.lo os.lo parse_bsr.lo read.lo
> read_records.lo record_read.lo record_util.lo record_write.lo reserve.lo
> scan.lo sd_plugins.lo spool.lo tape_alert.lo tape_worm.lo vol_mgr.lo
> wait.lo fifo_dev.lo file_dev.lo tape_dev.lo vtape_dev.lo \
> -export-dynamic -rpath /opt/bacula/lib -release 9.6.3
> LT Compiling cloud_dev.c
> LT Compiling cloud_parts.c
> LT Compiling cloud_transfer_mgr.c
> LT Compiling s3_driver.c
> s3_driver.c: In member function 'virtual bool s3_driver::init(JCR*,
> cloud_dev*, DEVRES*)':
> s3_driver.c:561:10: error: 'S3BucketContext {aka struct S3BucketContext}'
> has no member named 'authRegion'
> s3ctx.authRegion = cloud->region;
> ^~
> Makefile:211: recipe for target 's3_driver.lo' failed
> make[1]: *** [s3_driver.lo] Error 1
> make[1]: Leaving directory
> '/home/user1/dirNFS/sw/bacula_9.6.3/bacula-9.6.3/src/stored'
>
>
> == Error in
> /home/user1/dirNFS/sw/bacula_9.6.3/bacula-9.6.3/src/stored ==
>
> -
>
>
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
--
Radosław Korzeniewski
rados...@korzeniewski.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula Errors Running Differential Backup Job with Accurate Set - MYSQL Query Logs
Hello,
>From differential backup logs we have the following queries:
(...)
2020-05-09T23:03:40.103092Z 9 Query SELECT FileSetId,CreateTime
FROM FileSet WHERE FileSet='Brinkley Files' AND MD5='d+/In7JKOTR1B71g2+hR0A'
2020-05-09T23:03:40.108266Z 9 Query SELECT StartTime, Job FROM
Job WHERE JobStatus IN ('T','W') AND Type='B' AND Level='F' AND
Name='Brinkley Online Backup' AND ClientId=3 AND FileSetId=5 ORDER BY
StartTime DESC LIMIT 1
2020-05-09T23:03:40.109686Z 9 Query SELECT StartTime, Job FROM
Job WHERE JobStatus IN ('T','W') AND Type='B' AND Level='F' AND
Name='Brinkley Online Backup' AND ClientId=3 AND FileSetId=5 ORDER BY
StartTime DESC LIMIT 1
(...)
which proves that Bacula searched for previous proper Full backup
availability to make a proper Differential backup. As we cannot find
information about "no prior Full backup found" in your Differential job
logs, it allows us to suspect that the above queries returned something
about Full job availability.
There are no query results available in your logs so we cannot confirm if
Bacula found something or not. If you can recreate the issue again then you
can check what is going on with your Catalog database.
I hope it helps.
pon., 11 maj 2020 o 09:14 Michael Williams napisał(a):
> I’m not able to send the MYSQL logs to the mailing list as it includes
> full file paths, which is a privacy issue. The backup logs have already
> been sent to the mailing list.
>
> On 11/05/2020, at 7:09 PM, Radosław Korzeniewski <
> rados...@korzeniewski.net> wrote:
>
> Hello, You should send these logs to bacula-users mailing lists and not
> only to me.
>
> Thanks.
> Radek
>
> niedz., 10 maj 2020 o 02:59 Michael Williams
> napisał(a):
>
>> Hi Radosław,
>>
>> Please find attached the MYSQL query log (from MYSQL general log) for
>> running the two backups, both full and differential. I have sent these
>> separately as they contain full file names and paths etc, so not something
>> I want all over the internet!
>>
>> Appreciate your help.
>>
>> Regards,
>> Michael
>>
>
>
> --
> Radosław Korzeniewski
> rados...@korzeniewski.net
>
>
best regards
--
Radosław Korzeniewski
rados...@korzeniewski.net
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
