[Bacula-users] Anyone using AWS tape gateway or other similar service?
Hi All, I've posted a couple of questions here in the last few months about our 12-month test of switching Bacula to using virtual tapes, drives and changers through the AWS Storage Gateway. We are about 10 months into our test and after overcoming a bunch of hurdles, things seem to be going very well. However, I wanted to ask if anyone else has or is using the AWS Storage Gateway (or a similar, cloud setup) to manage backups? What concerns do you have? What issues have you run into? I ask because I value this community's expertise and also because there are very few others who have publically documented a similar test in their infrastructure. One remaining concern which I have (not related to Bacula at all) is the time it would take to conduct a complete system restore due to needing to transfer the data from AWS to our local. Even with their fast connection, a 150 GB transfer could take about 24 hours. Any thoughts on this or other cloud-based tape backup solutions using Bacula? --- Luke Salsich -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula driver misconceptions
> On Aug 23, 2017, at 11:27 AM, Phil Stracchino wrote: > > On 08/23/17 11:06, Dimitri Maziuk wrote: >> On 2017-08-22 21:26, Heitor Faria wrote: >> >>> It usually cost a new tape library, a new set of tapes and coffee. >>> It is not that hard to migrate legacy backups if you use a backup rotation >>> strategy such as GFS and works in a common company with 1-5 maximum >>> montly/yearly retention backups. >> >> I repeat, how many times have *you* done it, how many tapes, and how >> much did it cost you? In hardware, your time, and energy bills? > > > I don't have the luxury of a controlled environment for my backup > system. One of the things I've learned as a result is that I greatly > extend the life of my tape drive by *only powering it on* when I'm > preparing to run monthly full backups. (This also, practically > speaking, rules out internal drives, period.) If you want to get a long > service life out of a modern tape drive you practically need to have a > clean-room for it. Requiring a controlled environment to get a useful > service life is another hidden cost factor in LTO tape, and the higher > the track densities go, the more sensitive the drives are going to be to > environmental factors. FWIW, I do the same, mostly for noise and power considerations. The tape libraries sit about three feet from my desk. Mine is for home use. Not a common situation. > I'm REALLY hoping that by the time I need to upgrade from my LTO4 drive, > there is a better and more robust alternative available. I'm hoping my LTO4 lasts at least 3 more years. -- Dan Langille - BSDCan / PGCon d...@langille.org -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Error 6 - authorization to Baculum error
Hello Jorge, Thanks for your feedback about no auth problem on Baculum 9.0.3. Also thanks for the bug report about broken links. I fixed it. Best regards. Marcin Haba (gani) On 25 August 2017 at 15:52, jorge.dejesus wrote: > Hi to all > > Marcin thank you for the help, I have analyzed the HTTP requests using > Apache's log forensic and dump env variables but no sucess the proxy just > add the generic proxy headers (e.g: X-Forward-For) and the headers between > localhost and proxy are the same. > > Anyway, followed your suggestion and install baculum 9.0.3 and everything is > OK, I am even using the HTTP auth on the proxy side. > > The only problem I had was link to the message.po that was broken and was > giving a gui full of @@ > > command>/usr/share/baculum/htdocs/protected/API/Lang/en/messages.mo -> > /usr/share/locale/en/LC_MESSAGES/baculum-api.mo (boken link) > > Thank you for the help > > Jorge de Jesus > > > > > > On 25-08-17 08:07, Marcin Haba wrote: >> >> Hello Jorge, >> >> I think that your diagnoze of the problem is correct. Something is >> lost when the request goes throught proxy. Baculum uses the server >> environment variables that are probably lost. You can try to pass them >> by the proxy. To do that you need to know what is lost, so you can try >> to print $_SERVER superglobal array on both sides and compare for >> differences. Example script content: >> >> > print_r($_SERVER); >> ?> >> >> I remember one thread where was lost 'HTTPS' variable: >> http://www.backupcentral.com/forum/19/271649 >> >> I also encourage to update to 9.0 version with new features as the API >> and the Bacula config module. >> >> I hope it helps. Please let know about the result. >> >> Good luck. >> >> Best regards. >> Marcin >> >> On 24 August 2017 at 16:53, jorge.dejesus >> wrote: >>> >>> Hi to all >>> >>> >>> I am trying to install baculum and I am getting the following at the end >>> of >>> the wizard >>> >>> Error 6 - authorization to Baculum error >>> >>> >>> Installed: >>> >>> baculum:7.4.4-1 >>> >>> Ubuntu 16.04.3 LTS >>> >>> >>> I am trying to run baculum behind an apache2 proxy, basically the apache2 >>> proxy supports HTTPS and redirects to baculum on port 9095, configuration >>> of >>> proxy: >>> >>> https://pastebin.com/3An0XZnB >>> >>> (Nothing special just a normal ProxyPass to the IP of the VM) >>> >>> >>> On the VM running baculum, the config is as follow >>> >>> https://pastebin.com/ZmgBgrB9 >>> >>> More or less the default config from baculum >>> >>> >>> When I try to access baculum using the url on the apache proxy e.g: >>> >>> https://bak.almoneda.net >>> >>> I get the error: Error 6 - authorization to Baculum error , the system >>> asks >>> for the auth relm and gives the error >>> >>> If on the VM I use lynx >>> >>> http://localhost:9095 >>> >>> I can see baculum website without problems >>> >>> >>> I have tried to run the http basic auth on the proxy side inside >> *> >>> with the same errors. Any tips on what could be the problem, it seems >>> something in the headers of the proxy since on the VM and localhost >>> everything is working >>> >>> I saw the following thread: >>> >>> http://bacula.10910.n7.nabble.com/Baculum-Authentication-Error-td85968.html >>> The error was solved by upgrading to a new version (not done) >>> >>> Any tips on what can I do to fix it or how to debug (still can try a new >>> version of baculum) >>> >>> Jorge >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> Check out the vibrant tech community on one of the world's most >>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >>> ___ >>> Bacula-users mailing list >>> Bacula-users@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/bacula-users >>> >> >> > -- "Greater love hath no man than this, that a man lay down his life for his friends." Jesus Christ "Większej miłości nikt nie ma nad tę, jak gdy kto życie swoje kładzie za przyjaciół swoich." Jezus Chrystus -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula driver misconceptions
On 2017-08-25 06:52, Josh Fisher wrote: 1 "copy" disk, and 10 "magazines" -- or 5 raid=1 "magazines". It's certainly doable but I'll have to swap the disks twice as often I don't quite understand. It will take twice the disk space whether using RAID1 or rsync. There is only one copy disk. I assume that once a "magazine" disk if full it's not going to fail while it's sitting there idle. (That was proven wrong, by the way, once or twice over the years.) The copy disk keeps a copy of the "currently active" magazine, it's only using "+1" space. If you only have one magazine drive that's twice, I have 10. Dima -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Error 6 - authorization to Baculum error
Hi to all Marcin thank you for the help, I have analyzed the HTTP requests using Apache's log forensic and dump env variables but no sucess the proxy just add the generic proxy headers (e.g: X-Forward-For) and the headers between localhost and proxy are the same. Anyway, followed your suggestion and install baculum 9.0.3 and everything is OK, I am even using the HTTP auth on the proxy side. The only problem I had was link to the message.po that was broken and was giving a gui full of @@ command>/usr/share/baculum/htdocs/protected/API/Lang/en/messages.mo -> /usr/share/locale/en/LC_MESSAGES/baculum-api.mo (boken link) Thank you for the help Jorge de Jesus On 25-08-17 08:07, Marcin Haba wrote: Hello Jorge, I think that your diagnoze of the problem is correct. Something is lost when the request goes throught proxy. Baculum uses the server environment variables that are probably lost. You can try to pass them by the proxy. To do that you need to know what is lost, so you can try to print $_SERVER superglobal array on both sides and compare for differences. Example script content: I remember one thread where was lost 'HTTPS' variable: http://www.backupcentral.com/forum/19/271649 I also encourage to update to 9.0 version with new features as the API and the Bacula config module. I hope it helps. Please let know about the result. Good luck. Best regards. Marcin On 24 August 2017 at 16:53, jorge.dejesus wrote: Hi to all I am trying to install baculum and I am getting the following at the end of the wizard Error 6 - authorization to Baculum error Installed: baculum:7.4.4-1 Ubuntu 16.04.3 LTS I am trying to run baculum behind an apache2 proxy, basically the apache2 proxy supports HTTPS and redirects to baculum on port 9095, configuration of proxy: https://pastebin.com/3An0XZnB (Nothing special just a normal ProxyPass to the IP of the VM) On the VM running baculum, the config is as follow https://pastebin.com/ZmgBgrB9 More or less the default config from baculum When I try to access baculum using the url on the apache proxy e.g: https://bak.almoneda.net I get the error: Error 6 - authorization to Baculum error , the system asks for the auth relm and gives the error If on the VM I use lynx http://localhost:9095 I can see baculum website without problems I have tried to run the http basic auth on the proxy side inside with the same errors. Any tips on what could be the problem, it seems something in the headers of the proxy since on the VM and localhost everything is working I saw the following thread: http://bacula.10910.n7.nabble.com/Baculum-Authentication-Error-td85968.html The error was solved by upgrading to a new version (not done) Any tips on what can I do to fix it or how to debug (still can try a new version of baculum) Jorge -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Bacula driver misconceptions
On 8/24/2017 10:49 AM, Dimitri Maziuk wrote: On 2017-08-24 09:24, Josh Fisher wrote: As for writing the same job to two disks simultaneously, I don't think that will be very easy to accomplish. It would, I believe, require a new Device type driver in Bacula that atomically reserves two Device resources at a time. Why not use Copy jobs? Or pair the removable drives into RAID1 arrays? Amanda had RAIT option since forever: you just tell it to use 2 tape devices. It is very possible. I just think the devs feel it is very low priority due to md raid. Copy jobs aren't quite for that, and with, say, 2U 12-3.5" bay chassis I'm looking at a spool ssd, 1 "copy" disk, and 10 "magazines" -- or 5 raid=1 "magazines". It's certainly doable but I'll have to swap the disks twice as often, that is less than perfect. I might actually do it in the next iteration, but the truth is our hdds don't fail that often so at this point it ain't really broken, just suboptimal. I don't quite understand. It will take twice the disk space whether using RAID1 or rsync. I can see why a Copy job may not be wanted, since it would additionally increase the catalog size and use a different volume. Someday I'll read up on fuse, too, I wonder how much trickery one can program into that. Dima -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
