Re: Backups - was Re: LVM
On 15/06/10 14:31, Tom Furie wrote: On Tue, Jun 15, 2010 at 01:02:47PM +0100, Alan Chandler wrote: On 15/06/10 12:44, Alan Chandler wrote: The real magic command is "cp -alf" which essentially merges a shorter term store with a longer term one, making new entries where the shorter store has a file that isn't in the longer term store, and overwriting it where the shorter term store has a file with the same name as the longer one. I should have added, it does this virtually instantaneously because it is not moving anything, just dealing with hard links. Links are no way to handle back-ups. If the data goes bad, that just means there's multiple places you can't get at it from. You misunderstand me. Links are not used to avoid having two copies of everything, but just a way to merge either a daily snapshot into a weekly one, or a weekly snapshot into a monthly one. The magic is that it does it almost instantenously. My thought process was about how many backup copies do I need of something that changes daily - like a database (and its backup dump). I keep a separate version of the file for every day for a week. I then keep a single version for each week the following 5 weeks and then turn it into one version every month for 6 months - and from there is sits in a queue for archiving to dvd. This allows me to go back in time. Say I screwed up and corrupted data and didn't notice for a month. The best I can do is get a backup around a week of that occurrence not one of the daily ones. If I didn't notice for 6 weeks, then I would only have a monthly snapshot to choose from. -- Alan Chandler http://www.chandlerfamily.org.uk -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c193950.8080...@chandlerfamily.org.uk
Re: Backups - was: Re: LVM
On 06/16/2010 05:45 AM, Stan Hoeppner wrote: Ron Johnson put forth on 6/15/2010 10:59 AM: I wrote a script that only backs up our data directories (including much of /home) into a bunch of tarballs, excluding "junk" folders like caches, thumbnails, trash, etc, and compressing most but not stuff like image and OOo document directories. What you using for compression here Ron? gzip or bzip2? or ?? bzip2 in the form of "tar cfj". -- Seek truth from facts. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c19320f.1050...@cox.net
Re: Backups - was: Re: LVM
On Wednesday 16 June 2010 14:14:22 Camaleón wrote: > Many people send the replies to me directly and I am not sure > whether if they are full aware of that (intentionally off-list) or this > is just the famous Gmail's webmail "non-reply-to-list-but-sender" > error :-) I forgot that Gmail does that. ;-) I use Gmail for several reasons, not least for the archives. But I actually download via POP3 and do my reading and writing of emails in KMail. Which has a reply-to-list function that I had to override. Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201006161430.52237.lisi.re...@gmail.com
Re: Backups - was: Re: LVM
On Wednesday 16 June 2010 14:14:22 Camaleón wrote: > but I asked you why you were so reluctant to use > differential backups on her computer. I couldn't understand "why" because > today backup tasks are just "point-and-click", I mean, they are easier to > achieve than any image generation of the whole disk, so they are suitable > even for many non-savvy users :-) Yes - I was thinking of doing most backup myself, and just leaving her to do back-up on the fastest changing data. But I'll certainly look at this - after all, I was asking for ideas! Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201006161425.47987.lisi.re...@gmail.com
Re: Backups - was: Re: LVM
On Wed, 16 Jun 2010 10:05:56 +0100, Lisi wrote: > On Wednesday 16 June 2010 08:41:03 Camaleón wrote: >> El 2010-06-15 a las 22:58 +0100, Lisi escribió: >> >> (resending to the list) > > Sorry. I debated whether to send it to you or the list, and decided > that it was OT for the list since I was commenting on a specific > sentence of your that wasn't strictly germane to the subject of the > thread. No problem. Many people send the replies to me directly and I am not sure whether if they are full aware of that (intentionally off-list) or this is just the famous Gmail's webmail "non-reply-to-list-but-sender" error :-) >> > On Tuesday 15 June 2010 19:44:33 Camaleón wrote: >> I thought we were just taking/focusing about backup strategies and not >> about other limitations derived by the type of user. > > We were. Which is why I thought that a reply to your: > >> > > But it's "her" backup and "her" data. She should care about how to >> > > do things like these, whatever place she is (home, university, >> > > work...). > > was OT on the list. Was maybe OT... but I asked you why you were so reluctant to use differential backups on her computer. I couldn't understand "why" because today backup tasks are just "point-and-click", I mean, they are easier to achieve than any image generation of the whole disk, so they are suitable even for many non-savvy users :-) Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pan.2010.06.16.13.14...@gmail.com
Re: Backups - was: Re: LVM
Ron Johnson put forth on 6/15/2010 10:59 AM: > I wrote a script that only backs up our data directories (including much > of /home) into a bunch of tarballs, excluding "junk" folders like > caches, thumbnails, trash, etc, and compressing most but not stuff like > image and OOo document directories. What you using for compression here Ron? gzip or bzip2? or ?? -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c18ab37.8010...@hardwarefreak.com
Re: Backups - was: Re: LVM
On Wednesday 16 June 2010 08:41:03 Camaleón wrote: > El 2010-06-15 a las 22:58 +0100, Lisi escribió: > > (resending to the list) Sorry. I debated whether to send it to you or the list, and decided that it was OT for the list since I was commenting on a specific sentence of your that wasn't strictly germane to the subject of the thread. > > On Tuesday 15 June 2010 19:44:33 Camaleón wrote: > I thought we were just taking/focusing about backup strategies and not > about other limitations derived by the type of user. We were. Which is why I thought that a reply to your: > > > But it's "her" backup and "her" data. She should care about how to do > > > things like these, whatever place she is (home, university, work...). was OT on the list. But you clearly thought that we were talking about an adult, and I felt that you were a little hard on her as a result! We were all teenagers once. Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201006161005.56270.lisi.re...@gmail.com
Re: Backups - was: Re: LVM
El 2010-06-15 a las 22:58 +0100, Lisi escribió: (resending to the list) > On Tuesday 15 June 2010 19:44:33 Camaleón wrote: > > But it's "her" backup and "her" data. She should care about how to do > > things like these, whatever place she is (home, university, work...). > > I was forgetting that "school" means different things in American and British > English. We are talking about a young teenager, who is currently more > interested in fashion than the internal workings of a computer, especially > after some dire "IT" lessons at school (=institution for children). Okay, sorry for misunderstanding :-) I thought we were just taking/focusing about backup strategies and not about other limitations derived by the type of user. > She _was_ interested in the workings of computers, and she will I am sure be > so again. For now her expressed attitude is: "When I am at school the IT > department does it (my note - but very limitedly), when I am at home with > you, you do it, and when I am in Japan Daddy does it. Why do I need to do > it?" Now I see. Then you should definitely care about her backup. She is so young for taking these things by "motu-propio" (by her own), unless she is a power techie girl :-) > But she will do it for herself the first time she wants it done and neither I > nor her father is available. I have found that a good general ploy is "Of > course I'll come and help you do it. Just let me finish what I am doing at > the moment." Mmmm, then you could split the backup strategy in two separate tasks: Task 1/ Personal data backup can be triggered by the lady (by means of a simple script in the desktop). That way you are teaching her about good managing of the computer and to be responsible. This can be run once a week. If you know beforehand the computer is "on" on specific hours, it can be also automated. Task 2/ Full backup (disk image). This is a very slow task that requires time and plenty of free space so you can make a snapshot of the disk on a monthly basis (or two times in a month). > But _I_ want a backup of her hard drive because I would have to sort out the > mess if the hard drive were to die. :-( Yes, it's a valid point. And Clonezilla does a good job here (full disk imaging or partition imaging), although other people use rsync. But I'm afraid you'll need more than DVD discs to get this done flawlessly (external USB hard disk or small NAS would be better) :-) Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100616074103.gb4...@stt008.linux.site
Re: Backups - was: Re: LVM
On Tue, Jun 15, 2010 at 06:49:25PM +0100, Lisi wrote: > On Tuesday 15 June 2010 18:06:51 Camaleón wrote: > > On Tue, 15 Jun 2010 15:48:04 +0100, Lisi wrote: > > > On Tuesday 15 June 2010 14:40:44 Camaleón wrote: > > >> I would differentiate between "backup" data and "archived" data. > > > > (...) > > > > > Thanks for this. I was originally responding to Andrew's saying: > > > > > > There are many many ways to make take backups beyond having a disk big > > > enough to hold the data. > > > > > > > > > I can think of very few - and was interested in what he was thinking of. > > > Incremental/differential backups are not really practical, since she > > > will be at school. > > > > Why not practical? Just curious O:-) > > Because I shan't have hold of the computer for long enough or often enough! > > > > A periodic dd (or Clonezilla?) of the whole drive > > > and more frequent updates of her personal data (of which I understand > > > that there is not much) would be the optimum, but a trifle pricey, so I > > > am still looking at alternative possibilities. > > > > The main drawback I see for "dd" or "clonezilla" is that they are very > > "slowness". It takes much time (and space!) to make a full copy (or > > image) of the disk and so not very practical because at last the user > > stops doing the backup on a regular basis :-( > > The user isn't going to do the backup on a (frequent) regular basis anyway. > What I am hoping is to be able to dd (or Clonezilla or something) the drive > periodically and take a snapshot of the state of the machine at that point. > That will catch all the slow moving/changing files and facilitate a simple > restoration if needed. With luck, her personal stuff will fit on a CD or > two. Or, since we are anyway assuming that I shall be able to find the > money, I may get her a DVD RW. That she might do reasonably often. > > Another possibility that I haven't yet explored is to get a NAS or something > and back all of our machines up to it. > Have you investigated BackupPC? It's a pretty slick program, available in the Debian repos. It can use rsync, tar, or smb as its transfer mechanism. Rsync of course is best for over the internet backups, because it minimizes bandwidth usage. It pools common files, so the storage requirements per machine go down as you back up more and more machines. (If you both have a copy of the same JPEG on your machines, BackupPC will only keep one copy, but it knows that it belongs to both of you). It runs automatically, but not through cron. It will retry backups if the host machine (the one it's trying to back up) is disconnected from the network. The remote user's machine should run sshd and wait for BackupPC to connect. Alternatively, the remote user's machine can ssh to the BackupPC machine and set up a tunnel using reverse port forwarding. Or you could use a VPN. I use BackupPC to back up all my own computers, plus my parents', my sister's, and a friend's. Most of them probably don't even remember that they are being backed up, because it requires no interaction on their part. -Rob -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100615234114.gc24...@aurora.owens.net
Re: an apology - was Re: Backups - was: Re: LVM
On 06/15/2010 05:02 PM, Lisi wrote: Sorry for duplicate everybody. The list has been rejecting my emails with my usual SMTP set up. This means that I may think that something has gone when it hasn't or vice versa. Something like that happened to me a year or so ago when my ISP got much stricter about emails sent to it via Postfix instead of directly via an MUA. -- Seek truth from facts. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c17fa90.10...@cox.net
an apology - was Re: Backups - was: Re: LVM
Sorry for duplicate everybody. The list has been rejecting my emails with my usual SMTP set up. This means that I may think that something has gone when it hasn't or vice versa. Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201006152302.06968.lisi.re...@gmail.com
Re: Backups - was: Re: LVM
On Tuesday 15 June 2010 19:09:28 Ron Johnson wrote: > *Teach* her. She's in Uni, correct? Thus, she should be > responsible enough to take care of her own data by sticking in a USB > drive and running a script. Two people divided by a common language No, she's at school, where school in this case means an establishment for children from 3 to 18. In American terms (so far as I can work it out) she is in grade 8. And yes, she needs to learn to administer her own computer. And will. But _I_ don't want her to lose her journal, and I want to minimise the length of time it would take me to restore her (elderly) laptop to its present state! Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201006152023.53102.lisi.re...@gmail.com
Re: Backups - was: Re: LVM
On Tuesday 15 June 2010 19:09:28 Ron Johnson wrote: > *Teach* her. She's in Uni, correct? Thus, she should be > responsible enough to take care of her own data by sticking in a USB > drive and running a script. Two people divided by a common language No, she's at school, where school in this case means an establishment for children from 3 to 18. In American terms (so far as I can work it out) she is in grade 8. And yes, she needs to learn to administer her own computer. And will. But _I_ don't want her to lose her journal, and I want to minimise the length of time it would take me to restore her (elderly) laptop to its present state! Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201006152022.05722.lisi.re...@gmail.com
Re: Backups - was: Re: LVM
On 06/15/2010 01:18 PM, Andrew Sackville-West wrote: [snip] I think the OP said something like: I have 1.3 TB and it's too big to backup. This of course is patently ridiculous. meh. Right. You're a fool to buy a Lexus if you can't afford the (way more than bare legal minimum) auto insurance. -- Seek truth from facts. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c17caca.5060...@cox.net
Re: Backups - was: Re: LVM
On Tue, 15 Jun 2010 18:49:25 +0100, Lisi wrote: > On Tuesday 15 June 2010 18:06:51 Camaleón wrote: >> Why not practical? Just curious O:-) > > Because I shan't have hold of the computer for long enough or often > enough! But it's "her" backup and "her" data. She should care about how to do things like these, whatever place she is (home, university, work...). >> The main drawback I see for "dd" or "clonezilla" is that they are very >> "slowness". It takes much time (and space!) to make a full copy (or >> image) of the disk and so not very practical because at last the user >> stops doing the backup on a regular basis :-( > > The user isn't going to do the backup on a (frequent) regular basis > anyway. What I am hoping is to be able to dd (or Clonezilla or > something) the drive periodically and take a snapshot of the state of > the machine at that point. That will catch all the slow moving/changing > files and facilitate a simple restoration if needed. Oh, but image restoration is not "that" easy, there are many things to tweak, I mean, is not just clicking "restore" button and you're done :-) And full images need huge space! > With luck, her personal stuff will fit on a CD or two. Or, since we > are anyway assuming that I shall be able to find the money, I may get > her a DVD RW. > That she might do reasonably often. I would avoid DVD media as much as possible, at least I find it not suitable as "primary" backup (remember "RW" media has a limit of number of writings!). They are not very easy to manage (it requires a dedicated program to record the files), it's space-limited (compared to hard disks), they are not cheap (byte/€) and reading and writing operations are very, I mean, *very* slow :-) > Another possibility that I haven't yet explored is to get a NAS or > something and back all of our machines up to it. The best bet I see here is an USB case and a separate hard disk (min. 500 GiB, and as data space needings increases, the disk can be replaced very easily). There you can leave the full system images (clonezilla or "dd" based) as well as normal data backups that she can also copy into DVD at her convenience. Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pan.2010.06.15.18.44...@gmail.com
Re: Backups - was: Re: LVM
On Tue, Jun 15, 2010 at 09:27:47PM +1000, Alexander Samad wrote: > On Tue, Jun 15, 2010 at 8:32 PM, Lisi wrote: > > Please excuse the thread breaking. I have suddenly been being rejected by > > the > > list server and am sending for the third time. I hope that the list server > > is now happy with my SMTP settings. > > > > On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote: > >> There are many many ways to make take backups beyond having a disk big > >> enough to hold the data. > > > > Would you feel inclined to elaborate? I'm trying to solve this problem for > > my > > granddaughter's large HDD, and am not keen to have to buy a 300GB external > > drive. Tar would still require a fairly large medium. :-( see my other response to Ron Johnson. Basically, I was complaining, in a poorly worded way, that there is no such thing as too much data to backup. I think this is essentially what the OP was arguing. And just because the OP doesn't have an extra 1.3TB of hard disk lying around doesn't mean they can't take backups: stacks of DVD's, tape drives, online disk space etc Also, I did not mean to imply that you can fit all your data into less space than the data takes (barring compression... but that fits your data into the amount of space your data takes... so to speak). > > It all depends. Questions to ask > > How much total data is there to backup > How much data changes on each change > how many backups do I want to keep. > How easy do I want to make my restores > > > I use rdiff-backup remote diff backup package to a backup server, I > usually keep about 30 backups, figure if I haven't noticed in a month > then I probably don't really need it :) I have plenty of space on my > backup server (you could use a NAS box - one of those ones that takes > 2 disks so that you can raid1). > > I backup the system and the valuable data > > and for the really valuable data I send to 2 remote sites - its all > automated so I don't have to worry about it (sends me emails when it > has problems). took me a while to write/setup the system. This is fairly similar to what I do. I use rdiff-backup or rsnapshot to make hourly, daily, weekly or monthly backups of critical data with the time frequency varying depending on what the data is and how often it changes. This is all run via cron on the backup server. For certain really critical data (financial records in particular) I also tarball it up, encrypt it to myself and then send it to a server in another state that I have some spare space on. Again a cron job does it all automatically. Easy peasy once it's set up. > > I realise i will need a debian DVD to recover a machine, install base > and then just restore the system backup set and go from there. It's really a question of whether to backup the *machine* or the critical data. I think unless you need real high-availability, backing up a *machine* is really probably too much. At least for the general home user. In this day and age of really good installers, it's just easier to reinstall and then restore the data you need. For this purpose I keep backups of /home, and /etc and that's about it. I do keep backups of the MBR too, having once lost an MBR due to stupidity on my part... A signature.asc Description: Digital signature
Re: Backups - was: Re: LVM
On Tue, Jun 15, 2010 at 10:59:46AM -0500, Ron Johnson wrote: > On 06/15/2010 09:48 AM, Lisi wrote: > > > >Thanks for this. I was originally responding to Andrew's saying: > > > >There are many many ways to make take backups beyond having a disk big > >enough to hold the data. > > > > > >I can think of very few - and was interested in what he was thinking of. > >Incremental/differential backups are not really practical, since she will be > >at school. A periodic dd (or Clonezilla?) of the whole drive and more > >frequent updates of her personal data (of which I understand that there is > >not much) would be the optimum, but a trifle pricey, so I am still looking at > >alternative possibilities. > > > > I wrote a script that only backs up our data directories (including > much of /home) into a bunch of tarballs, excluding "junk" folders > like caches, thumbnails, trash, etc, and compressing most but not > stuff like image and OOo document directories. > > Each backup goes in a separate, dated directory. > > For huge binary directories (like uncompressible video and audio), I > simply do a "cp -vau" from the "live" tree to the backup tree. > > The bottom line, though, is that *yes*, you *do* need enough disk > space for the backup data. Yeah, my choice of words was unfortunate. What I really meant was something along the lines of: The inability to find a 1.3TB external disk it not a reason not to take backups. If the data needs backing up, then there are solutions besides one big honking disk to copy it onto. Tape drives, big stacks of DVD-R/RW's, arrays of smaller disks, leased disk space onlines somewhere, etc. I think the OP said something like: I have 1.3 TB and it's too big to backup. This of course is patently ridiculous. meh. To address Lisi's issue, I would suggest a cronjob that checks for network connectivity and then if it's got network, runs rsnapshot (or rdiff-backup) over passwordless ssh to a server somewhere. That would be fairly lightweight, once the initial copy is made. And it would be secure and easy in the longrun. If the user needs local backup, then a usb drive with rsnapshot would be reasonable. It creates duplicate filetrees at each snapshot, but uses hardlinks for unchanged files to keep the total size from ballooning out of control. I think it's pretty slick because it maintains some of the size control of differential backup but also makes access to the complete filetree at a given time a snap. very much my .02 A signature.asc Description: Digital signature
Re: Backups - was: Re: LVM
On 06/15/2010 12:49 PM, Lisi wrote: On Tuesday 15 June 2010 18:06:51 Camaleón wrote: On Tue, 15 Jun 2010 15:48:04 +0100, Lisi wrote: On Tuesday 15 June 2010 14:40:44 Camaleón wrote: I would differentiate between "backup" data and "archived" data. (...) Thanks for this. I was originally responding to Andrew's saying: There are many many ways to make take backups beyond having a disk big enough to hold the data. I can think of very few - and was interested in what he was thinking of. Incremental/differential backups are not really practical, since she will be at school. Why not practical? Just curious O:-) Because I shan't have hold of the computer for long enough or often enough! *Teach* her. She's in Uni, correct? Thus, she should be responsible enough to take care of her own data by sticking in a USB drive and running a script. A periodic dd (or Clonezilla?) of the whole drive and more frequent updates of her personal data (of which I understand that there is not much) would be the optimum, but a trifle pricey, so I am still looking at alternative possibilities. The main drawback I see for "dd" or "clonezilla" is that they are very "slowness". It takes much time (and space!) to make a full copy (or image) of the disk and so not very practical because at last the user stops doing the backup on a regular basis :-( The user isn't going to do the backup on a (frequent) regular basis anyway. What I am hoping is to be able to dd (or Clonezilla or something) the drive periodically and take a snapshot of the state of the machine at that point. That will catch all the slow moving/changing files and facilitate a simple restoration if needed. With luck, her personal stuff will fit on a CD or two. Or, since we are anyway assuming that I shall be able to find the money, I may get her a DVD RW. That she might do reasonably often. Another possibility that I haven't yet explored is to get a NAS or something and back all of our machines up to it. NFS and a multi-drive external USB/Firewire enclosure is all that's needed. -- Seek truth from facts. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c17c1d8.7090...@cox.net
Re: Backups - was: Re: LVM
On Tuesday 15 June 2010 18:06:51 Camaleón wrote: > On Tue, 15 Jun 2010 15:48:04 +0100, Lisi wrote: > > On Tuesday 15 June 2010 14:40:44 Camaleón wrote: > >> I would differentiate between "backup" data and "archived" data. > > (...) > > > Thanks for this. I was originally responding to Andrew's saying: > > > > There are many many ways to make take backups beyond having a disk big > > enough to hold the data. > > > > > > I can think of very few - and was interested in what he was thinking of. > > Incremental/differential backups are not really practical, since she > > will be at school. > > Why not practical? Just curious O:-) Because I shan't have hold of the computer for long enough or often enough! > > A periodic dd (or Clonezilla?) of the whole drive > > and more frequent updates of her personal data (of which I understand > > that there is not much) would be the optimum, but a trifle pricey, so I > > am still looking at alternative possibilities. > > The main drawback I see for "dd" or "clonezilla" is that they are very > "slowness". It takes much time (and space!) to make a full copy (or > image) of the disk and so not very practical because at last the user > stops doing the backup on a regular basis :-( The user isn't going to do the backup on a (frequent) regular basis anyway. What I am hoping is to be able to dd (or Clonezilla or something) the drive periodically and take a snapshot of the state of the machine at that point. That will catch all the slow moving/changing files and facilitate a simple restoration if needed. With luck, her personal stuff will fit on a CD or two. Or, since we are anyway assuming that I shall be able to find the money, I may get her a DVD RW. That she might do reasonably often. Another possibility that I haven't yet explored is to get a NAS or something and back all of our machines up to it. Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201006151849.25505.lisi.re...@gmail.com
Re: Backups - was: Re: LVM
Ron Johnson wrote: On 06/15/2010 05:32 AM, Lisi wrote: On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote: There are many many ways to make take backups beyond having a disk big enough to hold the data. Would you feel inclined to elaborate? I'm trying to solve this problem for my granddaughter's large HDD, and am not keen to have to buy a 300GB external drive. Tar would still require a fairly large medium. :-( Certainly not a name-brand pre-built model! However, empty external enclosures can be had for less than $20 at all the regular mail order places. Just slip a "big" drive in there and bob's your uncle! Indeed, I use 2 of them and just bought another one because the fan on one of the units is having a hard time of it. Unfortunately you only find out about this after you have had the unit for a while. These are USB enclosures, one for an ATA disk and one for an SATA disk. Curious, smartctl can read the ATA one but not the SATA one. You just pop in the drive and plug in the unit and away you go: Debian's recent kernels find the units. I am running 2.6.34 on a partition of the SATA enclosure right now. Hugo -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/hv8co0$a1...@dough.gmane.org
Re: Backups - was: Re: LVM
On Tue, 15 Jun 2010 15:48:04 +0100, Lisi wrote: > On Tuesday 15 June 2010 14:40:44 Camaleón wrote: >> I would differentiate between "backup" data and "archived" data. (...) > Thanks for this. I was originally responding to Andrew's saying: > > There are many many ways to make take backups beyond having a disk big > enough to hold the data. > > > I can think of very few - and was interested in what he was thinking of. > Incremental/differential backups are not really practical, since she > will be at school. Why not practical? Just curious O:-) > A periodic dd (or Clonezilla?) of the whole drive > and more frequent updates of her personal data (of which I understand > that there is not much) would be the optimum, but a trifle pricey, so I > am still looking at alternative possibilities. The main drawback I see for "dd" or "clonezilla" is that they are very "slowness". It takes much time (and space!) to make a full copy (or image) of the disk and so not very practical because at last the user stops doing the backup on a regular basis :-( Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pan.2010.06.15.17.06...@gmail.com
Re: Backups - was: Re: LVM
On 06/15/2010 09:48 AM, Lisi wrote: Thanks for this. I was originally responding to Andrew's saying: There are many many ways to make take backups beyond having a disk big enough to hold the data. I can think of very few - and was interested in what he was thinking of. Incremental/differential backups are not really practical, since she will be at school. A periodic dd (or Clonezilla?) of the whole drive and more frequent updates of her personal data (of which I understand that there is not much) would be the optimum, but a trifle pricey, so I am still looking at alternative possibilities. I wrote a script that only backs up our data directories (including much of /home) into a bunch of tarballs, excluding "junk" folders like caches, thumbnails, trash, etc, and compressing most but not stuff like image and OOo document directories. Each backup goes in a separate, dated directory. For huge binary directories (like uncompressible video and audio), I simply do a "cp -vau" from the "live" tree to the backup tree. The bottom line, though, is that *yes*, you *do* need enough disk space for the backup data. -- Seek truth from facts. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c17a372.2090...@cox.net
Re: Backups - was: Re: LVM
On 06/15/2010 05:32 AM, Lisi wrote: On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote: There are many many ways to make take backups beyond having a disk big enough to hold the data. Would you feel inclined to elaborate? I'm trying to solve this problem for my granddaughter's large HDD, and am not keen to have to buy a 300GB external drive. Tar would still require a fairly large medium. :-( Certainly not a name-brand pre-built model! However, empty external enclosures can be had for less than $20 at all the regular mail order places. Just slip a "big" drive in there and bob's your uncle! -- "There is usually only a limited amount of damage that can be done by dull or stupid people. For creating a truly monumental disaster, you need people with high IQs." Thomas Sowell -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c17a0db.9050...@cox.net
Re: Backups - was: Re: LVM
On Tuesday 15 June 2010 14:40:44 Camaleón wrote: > On Tue, 15 Jun 2010 11:32:15 +0100, Lisi wrote: > > On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote: > >> There are many many ways to make take backups beyond having a disk big > >> enough to hold the data. > > > > Would you feel inclined to elaborate? I'm trying to solve this problem > > for my granddaughter's large HDD, and am not keen to have to buy a 300GB > > external drive. Tar would still require a fairly large medium. :-( > > I would differentiate between "backup" data and "archived" data. > > By "backup" I see a copy of the current files in the system and as per > "archived" data I understand it as several snapshots of the data taken in > different days and so holding different data. > > Backup usually takes less space than archival, but sometimes archival is > necessary (a "must have" in a company). > > The most common procedure for a user's POV in order to get a data backup > is by using a "differential" backup with some kind of compression. The > first copy of the data will take all the files the user has selected to > be backed up but the rest of the times the copy is only > "differential" (only new or modified files are selected to be copied). > > This way (by using a differential backup strategy) you need less space in > the medium (the first copy is big, but the rest of the differential > copies are of small size and so the copy procedure is very quick). > > There are also those called "incremental" backups, but I find it a bit > more complex to manage that "differential" ones, as per data restoration: > with a differential backup yo only need the first big file and the last > differential copy, but in order to restore from an incremental backup you > need the first big file plus "all" the incremental ones). > > As per the programs to make backups... I still use "tar" (:-P) but > "rsync" is said to be one of the most mentioned/preferred for this task. Thanks for this. I was originally responding to Andrew's saying: There are many many ways to make take backups beyond having a disk big enough to hold the data. I can think of very few - and was interested in what he was thinking of. Incremental/differential backups are not really practical, since she will be at school. A periodic dd (or Clonezilla?) of the whole drive and more frequent updates of her personal data (of which I understand that there is not much) would be the optimum, but a trifle pricey, so I am still looking at alternative possibilities. Lisi -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201006151548.04355.lisi.re...@gmail.com
Re: Backups - was: Re: LVM
On Tue, 15 Jun 2010 11:32:15 +0100, Lisi wrote: > On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote: >> There are many many ways to make take backups beyond having a disk big >> enough to hold the data. > > Would you feel inclined to elaborate? I'm trying to solve this problem > for my granddaughter's large HDD, and am not keen to have to buy a 300GB > external drive. Tar would still require a fairly large medium. :-( I would differentiate between "backup" data and "archived" data. By "backup" I see a copy of the current files in the system and as per "archived" data I understand it as several snapshots of the data taken in different days and so holding different data. Backup usually takes less space than archival, but sometimes archival is necessary (a "must have" in a company). The most common procedure for a user's POV in order to get a data backup is by using a "differential" backup with some kind of compression. The first copy of the data will take all the files the user has selected to be backed up but the rest of the times the copy is only "differential" (only new or modified files are selected to be copied). This way (by using a differential backup strategy) you need less space in the medium (the first copy is big, but the rest of the differential copies are of small size and so the copy procedure is very quick). There are also those called "incremental" backups, but I find it a bit more complex to manage that "differential" ones, as per data restoration: with a differential backup yo only need the first big file and the last differential copy, but in order to restore from an incremental backup you need the first big file plus "all" the incremental ones). As per the programs to make backups... I still use "tar" (:-P) but "rsync" is said to be one of the most mentioned/preferred for this task. Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pan.2010.06.15.13.40...@gmail.com
Re: Backups - was Re: LVM
On Tue, Jun 15, 2010 at 01:02:47PM +0100, Alan Chandler wrote: > On 15/06/10 12:44, Alan Chandler wrote: > >> The real magic command is "cp -alf" which essentially merges a shorter >> term store with a longer term one, making new entries where the shorter >> store has a file that isn't in the longer term store, and overwriting it >> where the shorter term store has a file with the same name as the longer >> one. > > I should have added, it does this virtually instantaneously because it > is not moving anything, just dealing with hard links. Links are no way to handle back-ups. If the data goes bad, that just means there's multiple places you can't get at it from. Cheers, Tom -- What we cannot speak about we must pass over in silence. -- Wittgenstein signature.asc Description: Digital signature
Re: Backups - was Re: LVM
On 15/06/10 12:44, Alan Chandler wrote: The real magic command is "cp -alf" which essentially merges a shorter term store with a longer term one, making new entries where the shorter store has a file that isn't in the longer term store, and overwriting it where the shorter term store has a file with the same name as the longer one. I should have added, it does this virtually instantaneously because it is not moving anything, just dealing with hard links. -- Alan Chandler http://www.chandlerfamily.org.uk -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c176be7.7090...@chandlerfamily.org.uk
Re: Backups - was Re: LVM
On 15/06/10 11:36, Lisi wrote: On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote: There are many many ways to make take backups beyond having a disk big enough to hold the data. Would you feel inclined to elaborate? I'm trying to solve this problem for my granddaughter's large HDD, and am not keen to have to buy a 300GB external drive. Tar would still require a fairly large medium, would it not. :-( Lisi I find it interesting that the discussion is about the size of the media on which the data is stored rather than the size of the dataset(s) itself. I am not by any means running a professional shop, but at home I do have desktops and laptops to take care of, and I do have a major external site that I look after and some smaller stuff that I am starting to do professionally. I have been using approximately the same approach for several years, but I have just started to formalize it by documenting what I do in a private tiddly wiki (see http://www.tiddlywiki.org) that I can refer to and keep up to date (previously it was jottings in a notebook that I used to keep this sort of information in). I realise that I have to compromise because of the limited storage capacity I have in all locations (although my two 1TB disks I hope to take delivery of in the next few days is going to ease some worries :-) ), so I define some of my storage as single disk and some as software raid 1 (using mdadm). I first define the set of key data stores that I wish to protect and for each one indicate their size, whether they need to be operationally protected by being on raid and how frequent/recent a backup of that I need. For frequency I generally work on daily, weekly and monthly (because of the ability to drop a backup script in /etc/cron.daily etc) although I also have datasets that are accessed by Americans (I am in the UK) so I also have /etc/cron.d entries that specify 9-10am times as opposed to the middle of the night for the /etc/cron.daily and similar. I use that to develop a backup plan - where I can copy the datasets locally on the same machine, (mainly I tend to do that with database dumps in the first instance), across between two machines, or offsite from machines across the internet. For convenience I tend to use rsync -a sometimes with the --delete option sometimes not. Internally between machines I like to have rsyncd running, defining the local datasets with a module in rsyncd.conf - and specifically my wifes Windows XP desktop, and my Windows 7 laptop have rsync running as a service so I can back these machines up from linux. Once I know what my requirements are for dataset storages including the operational copy and the backups, I then define what I need in terms of raid, non-raid, and an approximation of the volumes. I tend to use LVM on raid or on a single disk. I try and avoid using LVM to span volumes, although in the past it has been necessary and it DID bite me. (I was using PVMOVE to move data off a disk I wanted to release. PVMOVE seems to be VERY sensitive to other things happening as well, and I stupidly got bored with waiting for it to finish and started another task on the same volume group. It screwed up and lost the data from that - I can't remember if it was just the LV that I was trying to move off the PV or whether it was the entire volume group) One key store is my longer term archive. I wrote about that in my 2005 blog entry http://www.chandlerfamily.org.uk/blog/2005/apr/backup-and-archiving-home (this store is both on raid, and backed up on another machine - sadly I can't afford to back it up off site as well). The real magic command is "cp -alf" which essentially merges a shorter term store with a longer term one, making new entries where the shorter store has a file that isn't in the longer term store, and overwriting it where the shorter term store has a file with the same name as the longer one. -- Alan Chandler http://www.chandlerfamily.org.uk -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c176795.4090...@chandlerfamily.org.uk
Re: Backups - was: Re: LVM
On Tue, Jun 15, 2010 at 8:32 PM, Lisi wrote: > Please excuse the thread breaking. I have suddenly been being rejected by the > list server and am sending for the third time. I hope that the list server > is now happy with my SMTP settings. > > On Tuesday 15 June 2010 01:25:56 Andrew Sackville-West wrote: >> There are many many ways to make take backups beyond having a disk big >> enough to hold the data. > > Would you feel inclined to elaborate? I'm trying to solve this problem for my > granddaughter's large HDD, and am not keen to have to buy a 300GB external > drive. Tar would still require a fairly large medium. :-( It all depends. Questions to ask How much total data is there to backup How much data changes on each change how many backups do I want to keep. How easy do I want to make my restores I use rdiff-backup remote diff backup package to a backup server, I usually keep about 30 backups, figure if I haven't noticed in a month then I probably don't really need it :) I have plenty of space on my backup server (you could use a NAS box - one of those ones that takes 2 disks so that you can raid1). I backup the system and the valuable data and for the really valuable data I send to 2 remote sites - its all automated so I don't have to worry about it (sends me emails when it has problems). took me a while to write/setup the system. I realise i will need a debian DVD to recover a machine, install base and then just restore the system backup set and go from there. One thing to remember also RAID is not a backup solution. > > Lisi > > > -- > To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org > with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org > Archive: http://lists.debian.org/201006151132.15778.lisi.re...@gmail.com > > -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktinmibpej51z5raiq-ipofihbym6tpar1cwjo...@mail.gmail.com
