Re: recommended remote backup service?
2009/5/13 Hetz Ben Hamo het...@gmail.com Amazon S3? A bit late to the party but this morning I got an announcement from Amazon about a new import/export service, which could save on initial backup upload and allow exporting of backed-up data over media. It's in limited beta in their US DC but eventually expected to be available also in Europe. http://aws.amazon.com/importexport/ Cheers, --Amos ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: recommended remote backup service?
Hi, Take a look at my company service S3rsync http://www.s3rsync.com/ it let you Rsync to Amazon S3 storage with out the need to bother dealing with Ec2 machine. Rami Hetz Ben Hamo wrote: Amazon S3? You can create tarballs (either a new snapshot everyday or just diffs) and upload them to Amazon S3. There are many FUSE implementations of their protocol so you can use your own tools for copying/uploading etc.. Thanks, Hetz On Wed, May 13, 2009 at 2:23 PM, Dvir Volk dvir...@gmail.com wrote: Hi, I need to find a new, secure and very reliable remote backup service for my employer's office server. This will be used to backup mainly stuff like SVN dumps, TRAC database, etc. 10-20 gigs should be more than enough, and ssh/rsync/sftp etc. scriptable access is a must. any recommendations? Thanks, Dvir ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: recommended remote backup service?
Amazon S3? You can create tarballs (either a new snapshot everyday or just diffs) and upload them to Amazon S3. There are many FUSE implementations of their protocol so you can use your own tools for copying/uploading etc.. Thanks, Hetz On Wed, May 13, 2009 at 2:23 PM, Dvir Volk dvir...@gmail.com wrote: Hi, I need to find a new, secure and very reliable remote backup service for my employer's office server. This will be used to backup mainly stuff like SVN dumps, TRAC database, etc. 10-20 gigs should be more than enough, and ssh/rsync/sftp etc. scriptable access is a must. any recommendations? Thanks, Dvir ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: recommended remote backup service?
Do they give you any kind of guarantee regarding security/encryption and data redundancy? This is the company's core IP, all documentation and code. They'd rather pay a bit more and sleep better at night. On Wed, May 13, 2009 at 2:41 PM, Hetz Ben Hamo het...@gmail.com wrote: Amazon S3? You can create tarballs (either a new snapshot everyday or just diffs) and upload them to Amazon S3. There are many FUSE implementations of their protocol so you can use your own tools for copying/uploading etc.. Thanks, Hetz On Wed, May 13, 2009 at 2:23 PM, Dvir Volk dvir...@gmail.com wrote: Hi, I need to find a new, secure and very reliable remote backup service for my employer's office server. This will be used to backup mainly stuff like SVN dumps, TRAC database, etc. 10-20 gigs should be more than enough, and ssh/rsync/sftp etc. scriptable access is a must. any recommendations? Thanks, Dvir ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: recommended remote backup service?
Well, the authentication is secured. Without keys, no access... If you really want go crazy, then go ahead, and create an EC2 instance with your favorite distro installed, connect either EBS storage or S3 storage to this virtual machine, and then you can do whatever you want regarding encryption, security etc. The storage is backed up IIRC. Thanks, Hetz On Wed, May 13, 2009 at 3:26 PM, Dvir Volk dvir...@gmail.com wrote: Do they give you any kind of guarantee regarding security/encryption and data redundancy? This is the company's core IP, all documentation and code. They'd rather pay a bit more and sleep better at night. On Wed, May 13, 2009 at 2:41 PM, Hetz Ben Hamo het...@gmail.com wrote: Amazon S3? You can create tarballs (either a new snapshot everyday or just diffs) and upload them to Amazon S3. There are many FUSE implementations of their protocol so you can use your own tools for copying/uploading etc.. Thanks, Hetz On Wed, May 13, 2009 at 2:23 PM, Dvir Volk dvir...@gmail.com wrote: Hi, I need to find a new, secure and very reliable remote backup service for my employer's office server. This will be used to backup mainly stuff like SVN dumps, TRAC database, etc. 10-20 gigs should be more than enough, and ssh/rsync/sftp etc. scriptable access is a must. any recommendations? Thanks, Dvir ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: recommended remote backup service?
On Wed, May 13, 2009 at 3:38 PM, Hetz Ben Hamo het...@gmail.com wrote: Well, the authentication is secured. Without keys, no access... In that case you had better be good at keeping backup copies of the keys and depending upon whose data it is making sure they know exactly where to get them if they need to. I occasionally see postings from people who have encrypted data belonging to long dead family members that they have no way of retrieving. You also don't want to be disturbed on vacation when a file gets corrupted and no one has the key to the backup, or if they do have it, knows that they do. Geoff. -- Geoffrey S. Mendelson N3OWJ/4X1GM Jerusalem, Israel ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: recommended remote backup service?
You can always get your personal key that Amazon issues to you on their page (after login of course). Hetz On Wed, May 13, 2009 at 3:45 PM, Geoffrey Mendelson geoffreymendel...@gmail.com wrote: On Wed, May 13, 2009 at 3:38 PM, Hetz Ben Hamo het...@gmail.com wrote: Well, the authentication is secured. Without keys, no access... In that case you had better be good at keeping backup copies of the keys and depending upon whose data it is making sure they know exactly where to get them if they need to. I occasionally see postings from people who have encrypted data belonging to long dead family members that they have no way of retrieving. You also don't want to be disturbed on vacation when a file gets corrupted and no one has the key to the backup, or if they do have it, knows that they do. Geoff. -- Geoffrey S. Mendelson N3OWJ/4X1GM Jerusalem, Israel -- Skepticism is the lazy person's default position. my blog (hebrew): http://benhamo.org ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
RE: recommended remote backup service?
Hi, No direct experience, but I've heard good things about these guys: http://www.lingnu.com/en/backups.html as well as these http://www.rsync.net/ YMMV, of course. Rony -Original Message- From: linux-il-boun...@cs.huji.ac.il [mailto:linux-il-boun...@cs.huji.ac.il] On Behalf Of Dvir Volk Sent: Wednesday, May 13, 2009 2:24 PM To: linux-il Subject: recommended remote backup service? Hi, I need to find a new, secure and very reliable remote backup service for my employer's office server. This will be used to backup mainly stuff like SVN dumps, TRAC database, etc. 10-20 gigs should be more than enough, and ssh/rsync/sftp etc. scriptable access is a must. any recommendations? Thanks, Dvir ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: recommended remote backup service?
Dvir Volk wrote: Hi, I need to find a new, secure and very reliable remote backup service for my employer's office server. This will be used to backup mainly stuff like SVN dumps, TRAC database, etc. 10-20 gigs should be more than enough, and ssh/rsync/sftp etc. scriptable access is a must. any recommendations? We use duplicity to backup similar stuff to Amazon S3. Duplicity does all the heavy lifting of compression, delta, encryption with GPG. Amazon S3 just needs to be there for you to grab the file (even if someone grabs the files they are GPG encrypted). If you trust Amazon S3 to be there when you need the files, I highly recommend it. Gilad ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: recommended remote backup service?
Dvir Volk wrote: Hi, I need to find a new, secure and very reliable remote backup service for my employer's office server. This will be used to backup mainly stuff like SVN dumps, TRAC database, etc. 10-20 gigs should be more than enough, and ssh/rsync/sftp etc. scriptable access is a must. any recommendations? Thanks, Dvir Full disclosure - I have a personal AND business interest in this matter. Details later on. Feel free to question my objectivity. Depending on the size of the individual files you are going to encrypt and your upstream bandwidth, I suggest you be wary of using duplicity. It offers quite good security (unless you intend to lose your encryption key, in which case don't bother backing up), but they do the delta calculation on the client side, and push an encrypted version of the delta. A restore operation involves restoring the last full backup you made, and then restoring all incremental backups done since then. They must ALL be available, or data will be lost. With 10-20GB of data to back up, a full image upload may well be an impossible task to perform on a regular basis. Over a 1.5/150 ADSL link pushing 1GB of data will take you about 18 hours, which means that a 20GB backup will take you a whole week or more (depending on how well the data compresses) to upload if you dedicate your line to it. It follows that having a weekly snapshot is pretty much out of the question. I have not looked into duplicity much, yet, but it stands to reason it needs a copy of your entire data locally in order for it to have something to compare against. Deleting this data means that you cannot perform the incremental data. plug Or you can use rsyncrypto (http://rsyncrypto.lingnu.com). It's open source, it's fairly mature technology, and it is entirely scriptable. It encrypts the entire file locally, but in a rsync friendly way, so that you can use the standard rsync in order to push the file remotely. This means that any snapshot is a full snapshot, in that they can be deleted in arbitrary order without jeopardizing your data. A local copy, in this case, is also saved between runs, but it can be recreated from the session keys, so it should be seen as more of a cache than a functional aspect of the program. The session keys are 68 bytes per file, and take no space at all (and they can be recovered from the actual encrypted data, of course) This tool was created specifically for your use scenario, so if you find it lacking, I would really like to know in what way, so I can make it better. double plug My company, Lingnu open source consulting, is running a backup service that uses the technologies you mention as their technology. Our end user nice gui option is not as mature as we'd like, but for your intended use (scriptable working) the service is available today, and you would not be the first customer to use it. Being as it is that the technology is open source, you are, of course, free to use it with any other service that supports rsync. rsync.net and Amazon's S3 were mentioned, and I have heard good things about both. They are, both, also likely to be cheaper than our service. What we offer is a service that specifically aims at your precise use scenario (small businesses, fully encrypted data with a key that is not shared with us, using rsyncrypto as the base technology). This may not amount to much today, but we have plans for the future :-). If you decide that these are not compelling enough reasons to go with us, but would like to know more about the future plans when they are no longer future, drop me an email and I'll keep you posted (I promise - no spam). /double plug /plug Whatever you do, I whole heartily join the recommendation to keep your encryption key safe. I have some clients that chose to allow me to keep their key for them, and others that chose to keep the key themselves. The later group almost invariably had trouble getting to their key (in some cases - twice!). With rsyncrypto, if you did not lose your session keys, you can still recover without pushing the entire backup over the net from scratch, but this is still not a nice situation to be in. Shachar -- Shachar Shemesh Lingnu Open Source Consulting Ltd. http://www.lingnu.com ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
