Re: using bittorrent for backup of personal files
On Sb, 10 mar 12, 19:48:21, Mark Neidorff wrote: Aren't you trying to do something that BT doesn't do? BT is good when you have lots of people who want to download a file and lots of people have the file. Then transmitting the file is broken up among many people. In your case only one person has the file(s) and you want to copy them to a few other people. Seems to me like you are trying to use the wrong tool. No, this is what BitTorrent was designed for. Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: using bittorrent for backup of personal files
On Sun, Mar 11, 2012 at 11:00:48AM +0200, Andrei POPESCU wrote: On Sb, 10 mar 12, 19:48:21, Mark Neidorff wrote: Aren't you trying to do something that BT doesn't do? BT is good when you have lots of people who want to download a file and lots of people have the file. Then transmitting the file is broken up among many people. In your case only one person has the file(s) and you want to copy them to a few other people. Seems to me like you are trying to use the wrong tool. No, this is what BitTorrent was designed for. I agree with Andrei :) I'm trying to transmit files to 3 or 5 other computers. Not hundreds, like BT is famous for, but I still think this is appropriate use. It will certainly save me some bandwidth over rsync'ing. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120311174046.gb20...@aurora.owens.net
Re: using bittorrent for backup of personal files
On Fri, Mar 09, 2012 at 02:04:46AM +0200, Andrei POPESCU wrote: On Jo, 08 mar 12, 18:56:19, Rob Owens wrote: I'm a bit of a novice, so my terminology may be off. But by public I mean a torrent that I upload to any public tracker, like thepiratebay or something. Anyone could download my torrent, not that they would know to look for it. You don't need a public tracker, you can either set up your own tracker or just use DHT. For those following along, I've had some success testing on my LAN. Set up a tracker on machine1: bttrack --port 6969 --dfile dstate --logfile bttrack.log Create a torrent and reference the above tracker. I did it both with transmission-gtk and the following command line: btmakemetafile http://machine1:6969/announce somefile or btmakemetafile http://machine1:6969/announce somedirectory Seed the torrent with one machine, and download the torrent on another machine. I also had success downloading the torrent from the same user account that was seeding it, but using a different bittorrent client (and saving to a different directory). Some of the torrents take a long time to start downloading (like an hour or more). Some of them start right away. One torrent ran overnight and never started downloading. Then I paused and resumed it, and it started right up. Any ideas why this might happen? -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120311180818.gc20...@aurora.owens.net
Re: using bittorrent for backup of personal files
Le 09/03/2012 00:46, Rob Owens a écrit : On Wed, Mar 07, 2012 at 09:53:31AM -0800, Kelly Clowers wrote: [...] I don't know what BackupPC does internally, but I would just use rsync over ssh, with appropriate options. It ought to be pretty robust I have BackupPC configured to use rsync over ssh. It works well, but its main problems are: 1) It typically maxes out my internet connection. Plain old rsync would do this too, unless there is a throttling option that I don't know about. Rtorrent, which I use, has a throttling option. Of course rsync has a throttling option : Read the rsync's man : --bwlimit=KBPS limit I/O bandwidth; KBytes per second You can tell backuppc to use additional rsync options. [...] -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f5b6132.2090...@nuagelibre.org
Re: using bittorrent for backup of personal files
On Sat, Mar 10, 2012 at 03:12:02PM +0100, Gilles Mocellin wrote: Le 09/03/2012 00:46, Rob Owens a écrit : On Wed, Mar 07, 2012 at 09:53:31AM -0800, Kelly Clowers wrote: [...] I don't know what BackupPC does internally, but I would just use rsync over ssh, with appropriate options. It ought to be pretty robust I have BackupPC configured to use rsync over ssh. It works well, but its main problems are: 1) It typically maxes out my internet connection. Plain old rsync would do this too, unless there is a throttling option that I don't know about. Rtorrent, which I use, has a throttling option. Of course rsync has a throttling option : Read the rsync's man : --bwlimit=KBPS limit I/O bandwidth; KBytes per second Thanks for that. I didn't know it existed. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120310150448.gb9...@aurora.owens.net
Re: using bittorrent for backup of personal files
On Tuesday 06 March 2012 7:18:49 pm Rob Owens wrote: I'm considering using bittorrent to back up large files such as pictures and home movies. I am the admin for several of my family members' computers. The idea would be to back up my files onto their machines, then eventually back their stuff up in the same manner, resulting in several off-site backups for each of us. Aren't you trying to do something that BT doesn't do? BT is good when you have lots of people who want to download a file and lots of people have the file. Then transmitting the file is broken up among many people. In your case only one person has the file(s) and you want to copy them to a few other people. Seems to me like you are trying to use the wrong tool. Mark -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201203101948.21135.m...@neidorff.com
Re: using bittorrent for backup of personal files
On Ma, 06 mar 12, 19:18:49, Rob Owens wrote: I'm considering using bittorrent to back up large files such as pictures and home movies. I am the admin for several of my family members' computers. The idea would be to back up my files onto their machines, then eventually back their stuff up in the same manner, resulting in several off-site backups for each of us. From what I know of the BitTorrent protocol it is beneficial only if the data needs to be transfered to more than 1 site, where the remote site(s) are not sharing the same internet connection, otherwise it doesn't bring any benefit over ftp/rsync/etc. It's not obvious from your message if this is the case. I want to keep this data private. What are my options, besides a VPN? I hesitate to use a public torrent even on encrypted data, because the computers of tomorrow may easily crack today's encryption using brute force. I'm not sure what you consider to be a public torrent, but in my very un-informed opinion, encrypted torrents should be quite secure already, just don't use a public tracker or distribute the DHT key. Yes, there is a bit of security-by-obscurity here, but I wouldn't worry about it unless it was really sensitive data. My data is sorted in directories by year. If I make torrents for each year, most of the data will be static. But how should I handle the current year's data? Can I update the torrent file without forcing a re-download of all the current year's data? Not sure, but even if you create a new torrent each time you add more data, all clients I have tried so far will not re-download, but they will re-check the hash on all existing data. Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: using bittorrent for backup of personal files
On 8/03/2012 9:24 PM, Andrei POPESCU wrote: On Ma, 06 mar 12, 19:18:49, Rob Owens wrote: I'm considering using bittorrent to back up large files such as pictures and home movies. I am the admin for several of my family members' computers. The idea would be to back up my files onto their machines, then eventually back their stuff up in the same manner, resulting in several off-site backups for each of us. From what I know of the BitTorrent protocol it is beneficial only if the data needs to be transfered to more than 1 site, where the remote site(s) are not sharing the same internet connection, otherwise it doesn't bring any benefit over ftp/rsync/etc. It's not obvious from your message if this is the case. A bit torrent client that is still downloading content can and does serve whatever it has already downloaded to other clients. With multiple machines in various locations involved, BT might be a good solution. As one piece [chunk] of the data gets to two or three machines, then the fourth or fifth machine in the group can maximize downstream links. A shared drop box or better still, wuala storage might be an option too, but then you have to worry about how much data needs to be in the shared storage. Wuala at least encrypts everything and if you forget your password, then the data is not even good for you. They don't store your password, they cannot do password resets and they also cannot decrypt your data at all -- but again you have limits on storage space. A couple of other ways to deal with the issue, at least for historical data, is to swap HDDs or memory cards often or even keep sending new HDDs or memory cards -- they are quite cheap. For the current and changing folders, probably rsync is worth using. Cheers -- Kind Regards AndrewM -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4f58ad7c.60...@affinityvision.com.au
Re: using bittorrent for backup of personal files
On Wed, Mar 07, 2012 at 06:18:40PM +0100, 0xAAA wrote: On Tue, Mar 06, 2012 at 07:18:49PM -0500, Rob Owens wrote: I already use BackupPC successfully for offsite backups between family members. Very large files are sometimes a problem, however, which is why I'm considering bittorrent to supplement BackupPC. Thats the problem with a very slow bandwidth (I have only 250 kbps) which makes a backup of more than 1500 GB stuff nearly impossible! The p2p torrent network has - in this situation - no advantage! The upload with ftp or http has the same speed. p2p networks are designed to provide a constant download-speed. But upload is the problem of your dsl provider. p2p is more about sharing things. It was not built to backup large files. This is the task of ftp or maybe nfs What I like about bittorrent is that it is very easy to throttle the upload and download speeds. That makes a big difference in internet usability (at least in my experience). Rsync (as done with BackupPC) tends to max out your connection speed, making web browsing slow. I want to keep this data private. What are my options, besides a VPN? I hesitate to use a public torrent even on encrypted data, because the computers of tomorrow may easily crack today's encryption using brute force. One possible solution to your privacy problem is that you can encrypt your data at your local site and load the crypted stuff up to the p2p server. Furter details can be found in gpg(1). Yeah, but I don't like the idea of allowing my encrypted data to be downloaded by anybody. The data will essentially be available forever if I use a publicly-available torrent. Someday my encryption may end up being crackable. I'm not dealing with extremely sensitive data, just family photos and stuff. But I still would rather not rely on gpg to keep my data secure *forever*. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120308233736.gb27...@aurora.owens.net
Re: using bittorrent for backup of personal files
On Wed, Mar 07, 2012 at 09:53:31AM -0800, Kelly Clowers wrote: On Tue, Mar 6, 2012 at 16:18, Rob Owens row...@ptd.net wrote: I want to keep this data private. What are my options, besides a VPN? I hesitate to use a public torrent even on encrypted data, because the computers of tomorrow may easily crack today's encryption using brute force. Not sure off the top of my head what encryption BT uses, but if it is any good, it shouldn't be any easier to break than a VPN's encryption... I have read that BT's encryption was not intended so much for security, but more to help prevent ISPs from throttling your torrent downloads. I should look into it more, though. I don't know what BackupPC does internally, but I would just use rsync over ssh, with appropriate options. It ought to be pretty robust I have BackupPC configured to use rsync over ssh. It works well, but its main problems are: 1) It typically maxes out my internet connection. Plain old rsync would do this too, unless there is a throttling option that I don't know about. Rtorrent, which I use, has a throttling option. 2) If the transfer is interrupted, BackupPC allows you to pick up where you started just once. This is a simplified explanation, and there are workarounds, and it only applies to the first full backup. But that first full backup is sometimes huge and could take days or weeks. Throw in a user or two who dual-boots to Windows, and that complicates the issue even further. Thanks for all the ideas, everyone. Keep 'em coming. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120308234656.gc27...@aurora.owens.net
Re: using bittorrent for backup of personal files
On Thu, Mar 08, 2012 at 12:24:10PM +0200, Andrei POPESCU wrote: On Ma, 06 mar 12, 19:18:49, Rob Owens wrote: I'm considering using bittorrent to back up large files such as pictures and home movies. I am the admin for several of my family members' computers. The idea would be to back up my files onto their machines, then eventually back their stuff up in the same manner, resulting in several off-site backups for each of us. From what I know of the BitTorrent protocol it is beneficial only if the data needs to be transfered to more than 1 site, where the remote site(s) are not sharing the same internet connection, otherwise it doesn't bring any benefit over ftp/rsync/etc. It's not obvious from your message if this is the case. I intend to transfer to more than 1 site. Both for the purposes of backup, and for sharing the files with my family members in several different households. I want to keep this data private. What are my options, besides a VPN? I hesitate to use a public torrent even on encrypted data, because the computers of tomorrow may easily crack today's encryption using brute force. I'm not sure what you consider to be a public torrent, but in my very un-informed opinion, encrypted torrents should be quite secure already, just don't use a public tracker or distribute the DHT key. Yes, there is a bit of security-by-obscurity here, but I wouldn't worry about it unless it was really sensitive data. I'm a bit of a novice, so my terminology may be off. But by public I mean a torrent that I upload to any public tracker, like thepiratebay or something. Anyone could download my torrent, not that they would know to look for it. My data is sorted in directories by year. If I make torrents for each year, most of the data will be static. But how should I handle the current year's data? Can I update the torrent file without forcing a re-download of all the current year's data? Not sure, but even if you create a new torrent each time you add more data, all clients I have tried so far will not re-download, but they will re-check the hash on all existing data. I'll have to experiment with that. I haven't ever created my own torrent, but I guess I could figure this one out by creating a small torrent with dummy data, and then modifying it. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120308235619.gd27...@aurora.owens.net
Re: using bittorrent for backup of personal files
On Jo, 08 mar 12, 18:56:19, Rob Owens wrote: I'm a bit of a novice, so my terminology may be off. But by public I mean a torrent that I upload to any public tracker, like thepiratebay or something. Anyone could download my torrent, not that they would know to look for it. You don't need a public tracker, you can either set up your own tracker or just use DHT. Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: using bittorrent for backup of personal files
On 9 March 2012 07:46, Rob Owens row...@ptd.net wrote: 1) It typically maxes out my internet connection. Plain old rsync would do this too, unless there is a throttling option that I don't know about. Rtorrent, which I use, has a throttling option. Slightly tangential, but do you know about trickle? Trickle is a voluntary, cooperative bandwidth shaper. it works entirely in userland and is very easy to use. I've only ever used it for simple things, so I don't know if you can change throttling on-the-fly, but you might want to look at it. Cheers, Jason -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/ca+zd3ffgbzih6rqgam+qukekb57srr4e90xsx07kfna+qao...@mail.gmail.com
Re: using bittorrent for backup of personal files
On Tue, Mar 06, 2012 at 07:18:49PM -0500, Rob Owens wrote: Thanks for any advice you may have. http://cyber.law.harvard.edu/rss/bitTorrent.html Sorry if this way off track, but it may spur some ideas? -- Religion is excellent stuff for keeping common people quiet. -- Napoleon Bonaparte -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120307145246.GB4136@tal
Re: using bittorrent for backup of personal files
On Tue, Mar 06, 2012 at 07:18:49PM -0500, Rob Owens wrote: I'm considering using bittorrent to back up large files such as pictures and home movies. I am the admin for several of my family members' computers. The idea would be to back up my files onto their machines, then eventually back their stuff up in the same manner, resulting in several off-site backups for each of us. This idea sounds interesting! I have a lot of data at home, but I have no personal backup server or something like that, I only copy important stuff from one harddisk to another disk. I already use BackupPC successfully for offsite backups between family members. Very large files are sometimes a problem, however, which is why I'm considering bittorrent to supplement BackupPC. Thats the problem with a very slow bandwidth (I have only 250 kbps) which makes a backup of more than 1500 GB stuff nearly impossible! The p2p torrent network has - in this situation - no advantage! The upload with ftp or http has the same speed. p2p networks are designed to provide a constant download-speed. But upload is the problem of your dsl provider. p2p is more about sharing things. It was not built to backup large files. This is the task of ftp or maybe nfs I want to keep this data private. What are my options, besides a VPN? I hesitate to use a public torrent even on encrypted data, because the computers of tomorrow may easily crack today's encryption using brute force. One possible solution to your privacy problem is that you can encrypt your data at your local site and load the crypted stuff up to the p2p server. Furter details can be found in gpg(1). -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120307171840.ga2...@online.de
Re: using bittorrent for backup of personal files
On Tue, Mar 6, 2012 at 16:18, Rob Owens row...@ptd.net wrote: I'm considering using bittorrent to back up large files such as pictures and home movies. I am the admin for several of my family members' computers. The idea would be to back up my files onto their machines, then eventually back their stuff up in the same manner, resulting in several off-site backups for each of us. I already use BackupPC successfully for offsite backups between family members. Very large files are sometimes a problem, however, which is why I'm considering bittorrent to supplement BackupPC. I want to keep this data private. What are my options, besides a VPN? I hesitate to use a public torrent even on encrypted data, because the computers of tomorrow may easily crack today's encryption using brute force. Not sure off the top of my head what encryption BT uses, but if it is any good, it shouldn't be any easier to break than a VPN's encryption... Has anybody used bittorrent over VPN? Searching for those terms just gives me hits on VPN services that can be used to keep the MPAA and RIAA out of your face. I know you can do it, I do not know how well. I have heard of people running it over TOR, which is kind of crazy, TOR is slow enough as is. I can't think of any reason it wouldn't work over say, OpenVPN. My data is sorted in directories by year. If I make torrents for each year, most of the data will be static. But how should I handle the current year's data? Can I update the torrent file without forcing a re-download of all the current year's data? I doubt it. Probably not that hard to incorporate that into the spec, but without it being there, it is unlikely. I don't know what BackupPC does internally, but I would just use rsync over ssh, with appropriate options. It ought to be pretty robust Cheers, Kelly Clowers -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAFoWM=_pas5qlsvwm+hrkkntykrr6ue9h+7cthezk5bgpb2...@mail.gmail.com
using bittorrent for backup of personal files
I'm considering using bittorrent to back up large files such as pictures and home movies. I am the admin for several of my family members' computers. The idea would be to back up my files onto their machines, then eventually back their stuff up in the same manner, resulting in several off-site backups for each of us. I already use BackupPC successfully for offsite backups between family members. Very large files are sometimes a problem, however, which is why I'm considering bittorrent to supplement BackupPC. I want to keep this data private. What are my options, besides a VPN? I hesitate to use a public torrent even on encrypted data, because the computers of tomorrow may easily crack today's encryption using brute force. Has anybody used bittorrent over VPN? Searching for those terms just gives me hits on VPN services that can be used to keep the MPAA and RIAA out of your face. My data is sorted in directories by year. If I make torrents for each year, most of the data will be static. But how should I handle the current year's data? Can I update the torrent file without forcing a re-download of all the current year's data? Thanks for any advice you may have. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120307001849.ga14...@aurora.owens.net
