Re: [gentoo-user] How to build a time machine on Gentoo
Let me add some text I wrote for another place but that explains how things would work: 1) The OpenSource definition http://www.opensource.org/docs/definition.php section 9 makes it very clear that an OSS license must not restrict other software and must not prevent to bundle different works under different licenses on one medium. 2) given the fact that the GPL is an aproved OSS licensse, it obviously complies with the OSS definition. 3) as a result, any GPL interpretation that is based on the assumption that a separate distribution would fix problems is wrong. There is a simple rule: - If you modify a GPLd work by your own, so that all you add was written by you for this modification only, then you created a "derivative work" and you need to put your modifications under GPL. - If you add another independent work to a GPLd work, you create a so called "collective work". This is permitted by the GPL. In this case, the GPL only applies to the GPLd part and the license for the other work applies to the other work. You need to respect the sum of all claims from all licenses in this case. Such a collective work can only be distributed if the claims from the licenses are not contradicting. If one license e.g. permits redistribution on Mondays only and the other permits redistribution on Wednesdays only, you cannot publish the collective work. ZFS is an independent work with respect to the Linx kernel. It was not written for or with the Linux kernel. - If like to you add ZFS to the Linux kernel, you first need to create a derivative work from ZFS and another derivative work from the Linux kernel in order to allow both to work together. You later create a collective work from the combination of the derivative works mentioned before. The modification in the ZFS code (in case they appear in files that come with ZFS) need to be put under CDDL, the modifications in the Linux kernel need to be put under GPL. Note: the GPLv3 tries to disallow most collective works, so be careful with GPLv3. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] How to build a time machine on Gentoo
Alan McKinnon wrote: > > ZFS has a very free license. This was the reason, why it could be ported to > > the BSDs. So why do you believe there is a "license hurdle"? > > You appear to not fully understand the licenses. Well, I of course fully understand the licenses. It may however be that you are missinformed because you have been listening to the wrong people. > Remember that the Linux kernel is GPL-2 and it's modules are considered > derivative works. The GPL-2 license demands that all derivative works be > either GPL-2 licensed or 100% compatible with the GPL-2. This is a claim that in conflict with the US copyright law. Check out: http://www.osscc.net/en/gpl.html There are verious statements from various lawyers that explain this and even give evidence for their claims. The GPL tries to redefine the definition for the term "derivative work" but this is forbidden by US Copyright law title 17 section 106. See my other posting for more information. > ZFS is licensed CCDL which although free and liberal, is not GPL-2 > compatible. > It is BSD-compatible which is why the BSDs can (and some do) ship it. > > The ZFS license is thus not a mere hurdle, it is an un-overcomeable barrier > in The only hurdle is in the brain of some Linux developers. I know of not a single lawyer that could claim such incompatibility and gives evidence for his statements. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] How to build a time machine on Gentoo
Apparently, though unproven, at 20:49 on Wednesday 18 August 2010, Joerg Schilling did opine thusly: > > remember it was running on one of the BSD's, too, since it's a matter of > > licensing that is the hurdle of greatest height. I've only played with > > BTRFS on my dev box and the simple workout I gave it did not tax it in > > any way--it worked okay. > > ZFS has a very free license. This was the reason, why it could be ported to > the BSDs. So why do you believe there is a "license hurdle"? You appear to not fully understand the licenses. Remember that the Linux kernel is GPL-2 and it's modules are considered derivative works. The GPL-2 license demands that all derivative works be either GPL-2 licensed or 100% compatible with the GPL-2. ZFS is licensed CCDL which although free and liberal, is not GPL-2 compatible. It is BSD-compatible which is why the BSDs can (and some do) ship it. The ZFS license is thus not a mere hurdle, it is an un-overcomeable barrier in it's current form. If Oracle were to re-license it then the problem could be solved, but few in this game hold any hope of that ever happening. But all of this has been hashed to death many many many times here and in other places - to the point where it is now conclusive. Google will reveal the entire discussion in all it's painful detail. Start with lkml. Let's not rehash it here again. Please, I beg of you. Let us not do that. -- alan dot mckinnon at gmail dot com
Re: [gentoo-user] How to build a time machine on Gentoo
Nganon wrote: > > Not really. ZFS is only available on Solaris right now. I seem to > > > remember it was running on one of the BSD's, too, since it's a matter of > > > licensing that is the hurdle of greatest height. I've only played with > > > BTRFS on my dev box and the simple workout I gave it did not tax it in > > > any way--it worked okay. > > > > ZFS has a very free license. This was the reason, why it could be ported to > > the > > BSDs. So why do you believe there is a "license hurdle"? > > > > > Because ZFS is licensed with Sun CDDL, which is incompatible with GNU GPL, > so it cant be distributed with Linux kernel. That's why it is ported to > FUSE. The CDDL is as incompatible to the GPL as the BSD license. Both do not allow to change the license of the code and for this reason, a code combination cannot happen as a "derivative work". If you like to create a derivative work from a GPL program and BSD code from another person, you would need to declare the code from another person to be _your_ "modification". This is something that would violate the Copyright law. The GPL however permits code combinations as "collective work". As the GPL then is only valir for the GPL part of the whole, this does not cause "incompatibilities". > Also note: btrfs now is three years old. ZFS was started aprox. 10 years > > ago. > > For this reason, btrfs is expected to need another 7 years to readh the > > level > > of stability currently seen with ZFS. > > > > > ZFS was announced on 2004. So approximately six year, not ten. Besides, > things in > computer world do not always work that linearly, you know. In 2002, the ZFS developers have already been forced to have their homedirectories on ZFS. In 2004, ZFS was already under test in universities. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] How to build a time machine on Gentoo
On 08/18/2010 11:49 AM, Joerg Schilling wrote: > Bill Longman wrote: > >> On 08/18/2010 11:03 AM, Nganon wrote: >>> Clear now, thanks. >>> >>> >>> If you want a robust filesystem, look into ZFS/BTRFS. >>> >>> >>> AFAIK ZFS is unmaintained and BTRFS is not stable, am I wrong? > > Why do you believe ZFS is unmaintained? That's Nganon's comment. I'll let him answer. > >> Not really. ZFS is only available on Solaris right now. I seem to >> remember it was running on one of the BSD's, too, since it's a matter of >> licensing that is the hurdle of greatest height. I've only played with >> BTRFS on my dev box and the simple workout I gave it did not tax it in >> any way--it worked okay. > > ZFS has a very free license. This was the reason, why it could be ported to > the > BSDs. So why do you believe there is a "license hurdle"? Only for getting it to run on Linux. The CDDL doesn't play well with GPL. > Also note: btrfs now is three years old. ZFS was started aprox. 10 years ago. > For this reason, btrfs is expected to need another 7 years to readh the level > of stability currently seen with ZFS. Might take even less!
Re: [gentoo-user] How to build a time machine on Gentoo
On 18 August 2010 21:49, Joerg Schilling < joerg.schill...@fokus.fraunhofer.de> wrote: > Bill Longman wrote: > > > On 08/18/2010 11:03 AM, Nganon wrote: > > > Clear now, thanks. > > > > > > > > > If you want a robust filesystem, look into ZFS/BTRFS. > > > > > > > > > AFAIK ZFS is unmaintained and BTRFS is not stable, am I wrong? > > Why do you believe ZFS is unmaintained? > > I was unsure, thats why I asked if I was wrong. My bad. > Not really. ZFS is only available on Solaris right now. I seem to > > remember it was running on one of the BSD's, too, since it's a matter of > > licensing that is the hurdle of greatest height. I've only played with > > BTRFS on my dev box and the simple workout I gave it did not tax it in > > any way--it worked okay. > > ZFS has a very free license. This was the reason, why it could be ported to > the > BSDs. So why do you believe there is a "license hurdle"? > > Because ZFS is licensed with Sun CDDL, which is incompatible with GNU GPL, so it cant be distributed with Linux kernel. That's why it is ported to FUSE. Also note: btrfs now is three years old. ZFS was started aprox. 10 years > ago. > For this reason, btrfs is expected to need another 7 years to readh the > level > of stability currently seen with ZFS. > > ZFS was announced on 2004. So approximately six year, not ten. Besides, things in computer world do not always work that linearly, you know.
Re: [gentoo-user] How to build a time machine on Gentoo
On 18 August 2010 21:37, Bill Longman wrote: > On 08/18/2010 11:03 AM, Nganon wrote: > > Clear now, thanks. > > > > > > If you want a robust filesystem, look into ZFS/BTRFS. > > > > > > AFAIK ZFS is unmaintained and BTRFS is not stable, am I wrong? > > Not really. ZFS is only available on Solaris right now. I seem to > remember it was running on one of the BSD's, too, since it's a matter of > licensing that is the hurdle of greatest height. I've only played with > BTRFS on my dev box and the simple workout I gave it did not tax it in > any way--it worked okay. > > Right, remembered after reading the 'licensing', it was only available through FUSE. I just checked on BTRFS. wiki.kernel.org says BTRFS is stable as of 2.6.29 but still masked on Gentoo. I will read more about it later and decide whether to convert.
Re: [gentoo-user] How to build a time machine on Gentoo
Bill Longman wrote: > On 08/18/2010 11:03 AM, Nganon wrote: > > Clear now, thanks. > > > > > > If you want a robust filesystem, look into ZFS/BTRFS. > > > > > > AFAIK ZFS is unmaintained and BTRFS is not stable, am I wrong? Why do you believe ZFS is unmaintained? > Not really. ZFS is only available on Solaris right now. I seem to > remember it was running on one of the BSD's, too, since it's a matter of > licensing that is the hurdle of greatest height. I've only played with > BTRFS on my dev box and the simple workout I gave it did not tax it in > any way--it worked okay. ZFS has a very free license. This was the reason, why it could be ported to the BSDs. So why do you believe there is a "license hurdle"? Also note: btrfs now is three years old. ZFS was started aprox. 10 years ago. For this reason, btrfs is expected to need another 7 years to readh the level of stability currently seen with ZFS. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: [gentoo-user] How to build a time machine on Gentoo
On 08/18/2010 11:03 AM, Nganon wrote: > Clear now, thanks. > > > If you want a robust filesystem, look into ZFS/BTRFS. > > > AFAIK ZFS is unmaintained and BTRFS is not stable, am I wrong? Not really. ZFS is only available on Solaris right now. I seem to remember it was running on one of the BSD's, too, since it's a matter of licensing that is the hurdle of greatest height. I've only played with BTRFS on my dev box and the simple workout I gave it did not tax it in any way--it worked okay.
Re: [gentoo-user] How to build a time machine on Gentoo
On 18 August 2010 17:53, Bill Longman wrote: > On 08/18/2010 04:53 AM, Nganon wrote: > > I did not know that. I was thinking of, in couple of months, buying a > > notebook > > with two HDDs with RAID1 installed and using the usb drive as a backup > > destination. So if RAID got corruped, the backups, made since then, > > would be > > useless? How would you resolve it? > > The ONLY thing RAID will save you from is hardware failure. > > Copying data from one disk to another is not RAID. If you use dd to copy > from one corrupt filesystem to another, you have two corrupt filesystems. > > Clear now, thanks. > If you want a robust filesystem, look into ZFS/BTRFS. > > AFAIK ZFS is unmaintained and BTRFS is not stable, am I wrong?
Re: [gentoo-user] How to build a time machine on Gentoo
On 18 August 2010 14:59, William Kenworthy wrote: > On Wed, 2010-08-18 at 14:09 +0300, Nganon wrote: > > > > > > On 17 August 2010 22:34, Enrico Weigelt wrote: > > For things I'd like to keep an history (eg. /etc) I'm using > > git, and > > pushing the repo to a remote server (denying non-fastfoward > > updates > > there, so an theorectical highjacker cannot destroy my > > history) > > > > > > Using git for /etc is a great idea. > > Thanks. > > > Another option is: > * app-backup/dirvish > Latest version available: 1.2.1 > Latest version installed: 1.2.1 > Size of downloaded files: 47 kB > Homepage:http://www.dirvish.org/ > Description: Dirvish is a fast, disk based, rotating network > backup system. > License: OSL-2.0 > > > Works by first creating a copy (--init) and then hard-linking subsequent > versions of files/directories back to the original original if its > identical. If a file is changed/new, it is copied instead of linked so > actual space usage quickly stabilises even with a varying number of > versions. Backup over the network (this is how I have configured mine) > uses rsync over ssh with keys and is "pull" from a cron job on the > backup server or manual on demand (i.e., server initiated). > > Version management is by a reasonably sophisticated date of version > scheme where by running "dirvish-expire" deletes out of date versions > (runs in a cron job). The smart part is that once the last hard link to > file is deleted, its gone, otherwise its kept in the remaining > versions :) > > Restore is a simple matter of identifying the version you want and > copying it back - Ive restored individual files through to complete > systems after total disk failure. > > Can do includes/excludes, whole systems or just directories such as /etc > and can be easily automated. > > Doesnt use compression, but most backup regimes (every day for a weekly > rota + a Sunday kept for 6 months) stabilise at about 2x the original > (gross) copy size, no matter how many copies with average changes > between versions. Though large scale changes such as an "emerge -e > world" will take more as it will generate new copies of most files. > > Downside is it will hammer the destination file system - reiserfs3 works > well, ext2/ext3 have been hopeless everytime I've tried - mass > corruption. The file system will need a large number of inodes (for > links) if there are an excessive number of files x versions - again > reiserfs3 scores well here. > > Highly recommended! > > BillK > > > > Thanks. It sound just it is made just for this. It even call itself 'time machine'. Obviously compression is left out by using links but it sounds kind of overwhelming to me. I don't have a reiserfs partition and cannot afford to have one at the mo..
Re: [gentoo-user] How to build a time machine on Gentoo
On 08/18/2010 04:53 AM, Nganon wrote: > I did not know that. I was thinking of, in couple of months, buying a > notebook > with two HDDs with RAID1 installed and using the usb drive as a backup > destination. So if RAID got corruped, the backups, made since then, > would be > useless? How would you resolve it? The ONLY thing RAID will save you from is hardware failure. Copying data from one disk to another is not RAID. If you use dd to copy from one corrupt filesystem to another, you have two corrupt filesystems. If you want a robust filesystem, look into ZFS/BTRFS.
Re: [gentoo-user] How to build a time machine on Gentoo
On Wed, 2010-08-18 at 14:09 +0300, Nganon wrote: > > > On 17 August 2010 22:34, Enrico Weigelt wrote: > For things I'd like to keep an history (eg. /etc) I'm using > git, and > pushing the repo to a remote server (denying non-fastfoward > updates > there, so an theorectical highjacker cannot destroy my > history) > > > Using git for /etc is a great idea. > Thanks. > Another option is: * app-backup/dirvish Latest version available: 1.2.1 Latest version installed: 1.2.1 Size of downloaded files: 47 kB Homepage:http://www.dirvish.org/ Description: Dirvish is a fast, disk based, rotating network backup system. License: OSL-2.0 Works by first creating a copy (--init) and then hard-linking subsequent versions of files/directories back to the original original if its identical. If a file is changed/new, it is copied instead of linked so actual space usage quickly stabilises even with a varying number of versions. Backup over the network (this is how I have configured mine) uses rsync over ssh with keys and is "pull" from a cron job on the backup server or manual on demand (i.e., server initiated). Version management is by a reasonably sophisticated date of version scheme where by running "dirvish-expire" deletes out of date versions (runs in a cron job). The smart part is that once the last hard link to file is deleted, its gone, otherwise its kept in the remaining versions :) Restore is a simple matter of identifying the version you want and copying it back - Ive restored individual files through to complete systems after total disk failure. Can do includes/excludes, whole systems or just directories such as /etc and can be easily automated. Doesnt use compression, but most backup regimes (every day for a weekly rota + a Sunday kept for 6 months) stabilise at about 2x the original (gross) copy size, no matter how many copies with average changes between versions. Though large scale changes such as an "emerge -e world" will take more as it will generate new copies of most files. Downside is it will hammer the destination file system - reiserfs3 works well, ext2/ext3 have been hopeless everytime I've tried - mass corruption. The file system will need a large number of inodes (for links) if there are an excessive number of files x versions - again reiserfs3 scores well here. Highly recommended! BillK
Re: [gentoo-user] How to build a time machine on Gentoo
On 18 August 2010 14:34, Neil Bothwick wrote: > On Tue, 17 Aug 2010 14:14:27 +0200, Maximilian Bräutigam wrote: > > > You should backup all in / except > > /tmp/* > > /sys/* > > /proc/* > > /lost+found/* > > /dev/* > > That backs up a lot of stuff that isn't needed. As long as you have /etc > and /var/lib you can recreate the system. Depending on space vs. time, > you may prefer not to backup the gigabytes in /usr that can be recreated > by portage (although saving /usr/local is a good idea). > > Thanks a lot for the valuable advice. I have a dozen of scripts in /usr/local/bin that I forgot about. > > By the way, since a new hdd of one TB is pretty cheap, think about > > running your gentoo in a software RAID. Guides: > > RAID is not an alternative to backups, a corrupted filesystem on a RAID is > just as corrupted as if it were on a single disk, you just get extra > copies of the corruption. > > I did not know that. I was thinking of, in couple of months, buying a notebook with two HDDs with RAID1 installed and using the usb drive as a backup destination. So if RAID got corruped, the backups, made since then, would be useless? How would you resolve it? -- > Neil Bothwick > > Despite the cost of living, have you noticed how it remains so popular? >
Re: [gentoo-user] How to build a time machine on Gentoo
On Tue, 17 Aug 2010 14:14:27 +0200, Maximilian Bräutigam wrote: > You should backup all in / except > /tmp/* > /sys/* > /proc/* > /lost+found/* > /dev/* That backs up a lot of stuff that isn't needed. As long as you have /etc and /var/lib you can recreate the system. Depending on space vs. time, you may prefer not to backup the gigabytes in /usr that can be recreated by portage (although saving /usr/local is a good idea). > By the way, since a new hdd of one TB is pretty cheap, think about > running your gentoo in a software RAID. Guides: RAID is not an alternative to backups, a corrupted filesystem on a RAID is just as corrupted as if it were on a single disk, you just get extra copies of the corruption. -- Neil Bothwick Despite the cost of living, have you noticed how it remains so popular? signature.asc Description: PGP signature
Re: [gentoo-user] How to build a time machine on Gentoo
On 17 August 2010 22:34, Enrico Weigelt wrote: > For things I'd like to keep an history (eg. /etc) I'm using git, and > pushing the repo to a remote server (denying non-fastfoward updates > there, so an theorectical highjacker cannot destroy my history) > Using git for /etc is a great idea. Thanks.
Re: [gentoo-user] How to build a time machine on Gentoo
2010/8/17 Maximilian Bräutigam > You should backup all in / except > /tmp/* > /sys/* > /proc/* > /lost+found/* > /dev/* > > Distfiles are saved outside the root and I can afford to rebuild world. My main concern was losing (gentoo) config files, speaking of which, I remembered to back up /usr/src/linux/.config, and user folders. I have no solution how to bzip or gzip your backups or how to make a dvd > backup, but I use "app-backup/rsnapshot" which uses rsync but implements > an intelligent rotating system that is done daily, weekly, monthly, > yearly according to your config. Of course you should store the backup > on another physical hdd. > > By the way, since a new hdd of one TB is pretty cheap, think about > running your gentoo in a software RAID. Guides: > > http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml > http://en.gentoo-wiki.com/wiki/Software_RAID_Install > Thanks for the advices. I am adding rsnapshot to my list.
Re: [gentoo-user] How to build a time machine on Gentoo
* Nganon wrote: Hi, > My first post on the list. I thought I would start with something that I > started > to think of as 'essential' after losing 90GB of data. Now I have two main > questions in mind: what to and how to back up on gentoo most efficiently. I'm using a little script like that: #!/bin/bash cd /var/backup/blackwidow || exit 1 DATE=`date +%F-%H-%M-%S` rsync --exclude-from /scr/etc/backup-blackwidow.exclude \ -avz blackwidow:/ /var/backup/blackwidow/ROOT \ --backup \ "--backup-dir=/var/backup/blackwidow/BACKUP-$DATE"\ --delete \ --delete-excluded \ --delay-updates \ --progress\ (of course with a carefully maintained exclude file ;-p) This doesnt make a real rotating backup, but stores all files that get overwritten into their own directory (named by the current date). And from time to time, I'm cleaning up the backup volume and look for things that I still might need. For things I'd like to keep an history (eg. /etc) I'm using git, and pushing the repo to a remote server (denying non-fastfoward updates there, so an theorectical highjacker cannot destroy my history) For really long-term backups with bigger content (that might be too big for git), you could also give venti+vac (from plan9port) a try. I'm using it for storing larger media files (videos, etc) in my MediaCloud platform. > 1. Apart from users' home directories and the followings, what should be > backed > up on a gentoo machine? > /etc/portage/ > /root > /var/lib/portage > ...? Depends on how long the recovery may take. For example, if you can afford recompiling world (or have another compatible image somewhere else), you can exclude everything where packages are installed (bindirs, libdirs, /usr/share, etc, etc) - assuming everything's installed by portage. > Though I can find enough space on the external drives, I don't trust them > any more. See above..sigh..(No I recovered about one third of it with > testdisk/photorec > which names them as file01 file2.. and half them are zero sized.. > which > quite justifies my agony) Make multiple copies on different media (eg. different servers). cu -- -- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weig...@metux.de mobile: +49 151 27565287 icq: 210169427 skype: nekrad666 -- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme --
Re: [gentoo-user] How to build a time machine on Gentoo
Maximilian Bräutigam writes: > You should backup all in / except > /tmp/* > /sys/* > /proc/* > /lost+found/* > /dev/* Attention here, you need at least the null and console entries in /dev, or the system will not come up. I also have tty and tty1 in there, I think those were neede for tuxonice. Wonko
Re: [gentoo-user] How to build a time machine on Gentoo
Hi Am 16.08.2010 01:11, schrieb Nganon: > Hello all, > > My first post on the list. I thought I would start with something that I > started > to think of as 'essential' after losing 90GB of data. Now I have two main > questions in mind: what to and how to back up on gentoo most efficiently. > > 1. Apart from users' home directories and the followings, what should be > backed > up on a gentoo machine? > /etc/portage/ > /root > /var/lib/portage > ...? You should backup all in / except /tmp/* /sys/* /proc/* /lost+found/* /dev/* I have no solution how to bzip or gzip your backups or how to make a dvd backup, but I use "app-backup/rsnapshot" which uses rsync but implements an intelligent rotating system that is done daily, weekly, monthly, yearly according to your config. Of course you should store the backup on another physical hdd. By the way, since a new hdd of one TB is pretty cheap, think about running your gentoo in a software RAID. Guides: http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml http://en.gentoo-wiki.com/wiki/Software_RAID_Install Kind regards, der Max > 2. Erm..okay, I am gonna say, what magic I want and then ask your way. > I first started making gzipped tar balls as follows: > > tar czpf /media/backups/userA-`date +%Y.%m.%d`.tgz -X userA-excludelist /etc > > But these can get huge especially for home dirs. I also want safe dvd > copies. > Though I can find enough space on the external drives, I don't trust them > any more. See above..sigh..(No I recovered about one third of it with > testdisk/photorec > which names them as file01 file2.. and half them are zero > sized.. which > quite justifies my agony) > > Here is what I wanna do. I want to have only one big backup for, say, > userA-2010.08.07.tgz and other small backup tars containing only the > files/folders that were modified since last update, 2010.08.07, as > userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz, > userA-diff-2010.08.28.tgz > etc. Now if I want to take the userA back to the future, 2010.08.21, I > want to > do it by first extracting the huge tar userA-2010.08.07.tgz and then the > tiny > backup userA-diff-2010-08-21.tgz. > > But the thing is I don't know how to do this. I am hoping maybe you can > tell me > how to of it as well as a better way of doing backups. > > By the way, since I want dvd backups as well, and I want to use +rw dvds > so I > can overwrite old backup after a while, what is best way of ensuring the > integrity and safety of them. Is it a good idea to use truecrypt > containers? Or > nothing tops signing and encrypting with gpg? > > Thanks for any comment in advance. >
Re: [gentoo-user] How to build a time machine on Gentoo
On 17 August 2010 02:53, Thomas Yao wrote: > On Tue, Aug 17, 2010 at 5:37 AM, Mick wrote: > > Not sure if it's in an overlay, but I don't think it's in portage. > > > > Run eix -l backup and see how many back up tools and scripts pop up. > > > > I have been using tar, star and rsync. They all work and they can all > make > > incremental back ups. You'll find that a lot of the other 'smart' back > up > > applications are based on these anyway. > > A disk archive utility called dar is a great staff to backup dir trees and > files > > Looks like the -A switch is exactly what I am looking for: This way you can make a first full backup, then many differential backup, each taking as reference the last backup made. Thanks! -- > @ghosTM55 > Mechanism, not policy > >
Re: [gentoo-user] How to build a time machine on Gentoo
On 17 August 2010 00:37, Mick wrote: > On Monday 16 August 2010 11:30:36 Nganon wrote: > > On 16 August 2010 11:36, Marco wrote: > > > On Mon, Aug 16, 2010 at 1:11 AM, Nganon > > > < > nganon%2bgen...@gmail.com >> > > > > > > wrote: > > > > [...] > > > > Here is what I wanna do. I want to have only one big backup for, say, > > > > userA-2010.08.07.tgz and other small backup tars containing only the > > > > files/folders that were modified since last update, 2010.08.07, as > > > > userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz, > > > > > > > > userA-diff-2010.08.28.tgz > > > > > > > > etc. Now if I want to take the userA back to the future, 2010.08.21, > I > > > > > > want > > > > > > > to > > > > do it by first extracting the huge tar userA-2010.08.07.tgz and then > > > > the tiny > > > > backup userA-diff-2010-08-21.tgz. > > > > > > backup2l can do exactly what you want: > > > > > > http://backup2l.sourceforge.net/ > > > > Nice one indeed. Exactly does what I want. It is also good that the > backups > > can be use without the program itself. > > > > It does not seem to be updated since 2009 but I will give it a try. > > Not sure if it's in an overlay, but I don't think it's in portage. > > Its not in the protage, I found a ebuild proposal at bugs.gentoo but it didnt even make it to sunrise. Run eix -l backup and see how many back up tools and scripts pop up. > > I know there are dozens of them in the portage. searching for backup does not answer me 'you can do that by these packages'. Perhaps if NLP is one day implemented in portage, usage of gentoo user lists will reduce by half. I have been using tar, star and rsync. They all work and they can all make > incremental back ups. You'll find that a lot of the other 'smart' back up > applications are based on these anyway. > > I will take a look at star, thanks. HTH. > -- > Regards, > Mick >
Re: [gentoo-user] How to build a time machine on Gentoo
On Tue, Aug 17, 2010 at 5:37 AM, Mick wrote: > Not sure if it's in an overlay, but I don't think it's in portage. > > Run eix -l backup and see how many back up tools and scripts pop up. > > I have been using tar, star and rsync. They all work and they can all make > incremental back ups. You'll find that a lot of the other 'smart' back up > applications are based on these anyway. A disk archive utility called dar is a great staff to backup dir trees and files -- @ghosTM55 Mechanism, not policy
Re: [gentoo-user] How to build a time machine on Gentoo
On Monday 16 August 2010 11:30:36 Nganon wrote: > On 16 August 2010 11:36, Marco wrote: > > On Mon, Aug 16, 2010 at 1:11 AM, Nganon > > > > > > > wrote: > > > [...] > > > Here is what I wanna do. I want to have only one big backup for, say, > > > userA-2010.08.07.tgz and other small backup tars containing only the > > > files/folders that were modified since last update, 2010.08.07, as > > > userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz, > > > > > > userA-diff-2010.08.28.tgz > > > > > > etc. Now if I want to take the userA back to the future, 2010.08.21, I > > > > want > > > > > to > > > do it by first extracting the huge tar userA-2010.08.07.tgz and then > > > the tiny > > > backup userA-diff-2010-08-21.tgz. > > > > backup2l can do exactly what you want: > > > > http://backup2l.sourceforge.net/ > > Nice one indeed. Exactly does what I want. It is also good that the backups > can be use without the program itself. > > It does not seem to be updated since 2009 but I will give it a try. Not sure if it's in an overlay, but I don't think it's in portage. Run eix -l backup and see how many back up tools and scripts pop up. I have been using tar, star and rsync. They all work and they can all make incremental back ups. You'll find that a lot of the other 'smart' back up applications are based on these anyway. HTH. -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] How to build a time machine on Gentoo
On 16 August 2010 11:36, Marco wrote: > On Mon, Aug 16, 2010 at 1:11 AM, Nganon > > > wrote: > > [...] > > Here is what I wanna do. I want to have only one big backup for, say, > > userA-2010.08.07.tgz and other small backup tars containing only the > > files/folders that were modified since last update, 2010.08.07, as > > userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz, > > userA-diff-2010.08.28.tgz > > etc. Now if I want to take the userA back to the future, 2010.08.21, I > want > > to > > do it by first extracting the huge tar userA-2010.08.07.tgz and then the > > tiny > > backup userA-diff-2010-08-21.tgz. > > backup2l can do exactly what you want: > > http://backup2l.sourceforge.net/ > > Nice one indeed. Exactly does what I want. It is also good that the backups can be use without the program itself. It does not seem to be updated since 2009 but I will give it a try. Thanks
Re: [gentoo-user] How to build a time machine on Gentoo
On 16 August 2010 03:15, Alex Schuster wrote: > Nganon > writes: > > > Hello all, > > > > My first post on the list. I thought I would start with something > > that I started > > to think of as 'essential' after losing 90GB of data. Now I have two > > main questions in mind: what to and how to back up on gentoo most > > efficiently. > > > > 1. Apart from users' home directories and the followings, what should > > be backed > > up on a gentoo machine? > > /etc/portage/ > > /root > > /var/lib/portage > > ...? > > Wouldn't it be easier to just backup the whole / directory? > Excluding /home, /usr/portage and /var/tmp/portage? > > Yes but that would not solve my huge backup file problem, would it? > > 2. Erm..okay, I am gonna say, what magic I want and then ask your way. > > I first started making gzipped tar balls as follows: > > > > tar czpf /media/backups/userA-`date +%Y.%m.%d`.tgz -X > > userA-excludelist /etc > > > > But these can get huge especially for home dirs. I also want safe dvd > > copies. > > Though I can find enough space on the external drives, I don't trust > > them any more. See above..sigh..(No I recovered about one third of it > > with testdisk/photorec > > which names them as file01 file2.. and half them are zero > > sized.. which > > quite justifies my agony) > > Okay, but I don't trust DVDs. Although DVD-RAM is quite safe I heard. > But external disks are flexible, offer more space, and if you want more > security, just use yet another drive, so you are safe even if your main > drive and a backup drive fails. > > I did not give a thought to DVD-RAM before. Will give it a try. Thanks. I suggest you have a look at rdiff-backup. It gives you a 1:1 copy of > the source directory, but also does incremental backups, which are > stored (in compressed form) in an additional folder in the destination > directory. I would use this at least for things like /etc, where I > sometimes might want to retrieve an old version of a file. Similar to > your approach with big tar files and small ones containing the > increments. > > I just emerged and tried it. Seems like incremental backups was what I was looking for. But from what I see, it is mirroring the src to dist and storing the metadata/stats of increments as archived. Maybe there is an option to archive increments along side their metadata as well. I will keep playing with it. I use a script for my backups, which I mentioned here on 2010-05-07, > subject 'Snackup'. It optionally creates LVM snapshots so I can make > backups from the running system, even if the source directory is > altered during the backup. This works on LVM only, though, and also > allows the volume to be LUKS-encrypted. It does backups by rdiff-backup, > rsync, cp, tar or dd. It may be overkill when not using the LVM > features, but still I suggest to use some script for backups, so one > does not always have to remember the backup commands. When I want to > update my backup, I enter something like 'snackup boot root home src', > and the script backs up my boot, root and home partition in the > background, and creates tar files each directory in /usr/src. > > I found the thread and script. I am gonna take a look at it. Once I decide on how to backup, I am gonna cron a script for it. > > By the way, since I want dvd backups as well, and I want to use +rw > > dvds so I can overwrite old backup after a while, what is best way of > > ensuring the integrity and safety of them. Is it a good idea to use > > truecrypt containers? Or nothing tops signing and encrypting with gpg? > > I'd use DVD-RAM. The media is a little more expensive, but AFAIK they > were made with long-time backups in mind. And access is much easier, > you just copy the files as to an external drive, no need to burn ISOs. > >Wonko > > Thanks for the advises.
Re: [gentoo-user] How to build a time machine on Gentoo
On Mon, Aug 16, 2010 at 1:11 AM, Nganon wrote: > [...] > Here is what I wanna do. I want to have only one big backup for, say, > userA-2010.08.07.tgz and other small backup tars containing only the > files/folders that were modified since last update, 2010.08.07, as > userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz, > userA-diff-2010.08.28.tgz > etc. Now if I want to take the userA back to the future, 2010.08.21, I want > to > do it by first extracting the huge tar userA-2010.08.07.tgz and then the > tiny > backup userA-diff-2010-08-21.tgz. backup2l can do exactly what you want: http://backup2l.sourceforge.net/ HTH! -- Marco
Re: [gentoo-user] How to build a time machine on Gentoo
Nganon writes: > Hello all, > > My first post on the list. I thought I would start with something > that I started > to think of as 'essential' after losing 90GB of data. Now I have two > main questions in mind: what to and how to back up on gentoo most > efficiently. > > 1. Apart from users' home directories and the followings, what should > be backed > up on a gentoo machine? > /etc/portage/ > /root > /var/lib/portage > ...? Wouldn't it be easier to just backup the whole / directory? Excluding /home, /usr/portage and /var/tmp/portage? > 2. Erm..okay, I am gonna say, what magic I want and then ask your way. > I first started making gzipped tar balls as follows: > > tar czpf /media/backups/userA-`date +%Y.%m.%d`.tgz -X > userA-excludelist /etc > > But these can get huge especially for home dirs. I also want safe dvd > copies. > Though I can find enough space on the external drives, I don't trust > them any more. See above..sigh..(No I recovered about one third of it > with testdisk/photorec > which names them as file01 file2.. and half them are zero > sized.. which > quite justifies my agony) Okay, but I don't trust DVDs. Although DVD-RAM is quite safe I heard. But external disks are flexible, offer more space, and if you want more security, just use yet another drive, so you are safe even if your main drive and a backup drive fails. I suggest you have a look at rdiff-backup. It gives you a 1:1 copy of the source directory, but also does incremental backups, which are stored (in compressed form) in an additional folder in the destination directory. I would use this at least for things like /etc, where I sometimes might want to retrieve an old version of a file. Similar to your approach with big tar files and small ones containing the increments. I use a script for my backups, which I mentioned here on 2010-05-07, subject 'Snackup'. It optionally creates LVM snapshots so I can make backups from the running system, even if the source directory is altered during the backup. This works on LVM only, though, and also allows the volume to be LUKS-encrypted. It does backups by rdiff-backup, rsync, cp, tar or dd. It may be overkill when not using the LVM features, but still I suggest to use some script for backups, so one does not always have to remember the backup commands. When I want to update my backup, I enter something like 'snackup boot root home src', and the script backs up my boot, root and home partition in the background, and creates tar files each directory in /usr/src. > By the way, since I want dvd backups as well, and I want to use +rw > dvds so I can overwrite old backup after a while, what is best way of > ensuring the integrity and safety of them. Is it a good idea to use > truecrypt containers? Or nothing tops signing and encrypting with gpg? I'd use DVD-RAM. The media is a little more expensive, but AFAIK they were made with long-time backups in mind. And access is much easier, you just copy the files as to an external drive, no need to burn ISOs. Wonko
[gentoo-user] How to build a time machine on Gentoo
Hello all, My first post on the list. I thought I would start with something that I started to think of as 'essential' after losing 90GB of data. Now I have two main questions in mind: what to and how to back up on gentoo most efficiently. 1. Apart from users' home directories and the followings, what should be backed up on a gentoo machine? /etc/portage/ /root /var/lib/portage ...? 2. Erm..okay, I am gonna say, what magic I want and then ask your way. I first started making gzipped tar balls as follows: tar czpf /media/backups/userA-`date +%Y.%m.%d`.tgz -X userA-excludelist /etc But these can get huge especially for home dirs. I also want safe dvd copies. Though I can find enough space on the external drives, I don't trust them any more. See above..sigh..(No I recovered about one third of it with testdisk/photorec which names them as file01 file2.. and half them are zero sized.. which quite justifies my agony) Here is what I wanna do. I want to have only one big backup for, say, userA-2010.08.07.tgz and other small backup tars containing only the files/folders that were modified since last update, 2010.08.07, as userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz, userA-diff-2010.08.28.tgz etc. Now if I want to take the userA back to the future, 2010.08.21, I want to do it by first extracting the huge tar userA-2010.08.07.tgz and then the tiny backup userA-diff-2010-08-21.tgz. But the thing is I don't know how to do this. I am hoping maybe you can tell me how to of it as well as a better way of doing backups. By the way, since I want dvd backups as well, and I want to use +rw dvds so I can overwrite old backup after a while, what is best way of ensuring the integrity and safety of them. Is it a good idea to use truecrypt containers? Or nothing tops signing and encrypting with gpg? Thanks for any comment in advance.
