Google cloud works and works well enough indeed. All of them work... but a non-block technique is capped at 5TB by almost all providers, and getting to there is complicated and can DOUBLE your storage cost over one month [only] if you cannot make temporaries without being charged. (looking at you, Wasabi!!).
However either A or B or C for Google specifically... A) the desired block size must be kept automatically small (it varies but ~40MB or smaller buffer or so for a 4GB system) ... and each DLE "tape" must be limited in size B) the biggest block size can be used to store 5TB objects [max == 512MB] but the curl buffers will take ~2.5GB and must be hardcoded [currently] in the build. Its too much for many systems. C) the biggest block size can be used but google cannot FAIL EVEN ONCE ... or the cloud upload cannot be restarted and the DLE fails basically. This doesn't succeed often on the way to 5TB. D) the biggest block size can be used but Multi-Part must be turned off and a second DLE and later gets to be very very very slow to add on Option D has NO limits to backups, but it is what needs the O(log N) check for single-stored blocks. This currently does a O(N) check against earlier blocks to check the cloud storage total after file #1 every time. Verrry slow at only 1000 per xaction. > -----Original Message----- > From: Stefan G. Weichinger <[email protected]> > Sent: Monday, April 4, 2022 1:40 AM > To: Chris Hassell <[email protected]>; AMANDA users <amanda- > [email protected]> > Subject: Re: using Google Cloud for virtual tapes > > WARNING: This email originated from outside of BETSOL. Do not click links or > open attachments unless you recognize the sender and know the content is safe. > > > Am 29.03.22 um 15:34 schrieb Chris Hassell: > > Google Cloud is somewhat difficult because they don't fully the support > Amazon S3 operations. One cannot upload blocks and "CopyPart" them into a > larger object. Wasabi and S3 and others can do that. > > > > There needs to be a simple overhaul of the "millions of blocks" upload > technique (non-multipart backups) so that it can be done without O(n) checks > for > every DLE. > > So is it usable? Or do I have to do some combo of local vtapes and amvaulting > them into the cloud, maybe? Confidentiality Notice | The information transmitted by this email is intended only for the person or entity to which it is addressed. This email may contain proprietary, business-confidential and/or privileged material. If you are not the intended recipient of this message, be aware that any use, review, re-transmission, distribution, reproduction or any action taken in reliance upon this message is strictly prohibited. If you received this in error, please contact the sender and delete the material from all computers.
