Craig Barratt wrote: > >> On the other hand as I and others have mentioned before moving to a >> database would add the following advantages: > > I agree on all the points. If I was writing BackupPC today I > would seriously consider this approach. As Les points out, the > most important open question (other than reliability) is whether > the performance is adequate as the store expands to millions of > files (and 10^8, 10^9 or more file blocks). Of course, BackupPC > is relatively slow, so maybe the baseline expectation is already > sufficiently low.
And, you have to tackle the issue of ensuring that the filesystem and database are always kept in sync - or fixing things if they aren't. > I recently heard about lessfs, which runs on top of FUSE to provide > a file system that does block-level de-duplication. See: > > http://www.lessfs.com > https://sourceforge.net/project/showfiles.php?group_id=257120 > http://tokyocabinet.sourceforge.net/index.html > > The actual storage is several very large (sparse?) files on any > file system(s) of your choice. It should provide all the benefits > you expect: no issues of local limitations on hardlink counts, > meta-data etc, and the database files can be copied or rsynced. > I'm corresponding with the author to see if some additional useful > features could be added. > > Yes, taking this approach would require a very substantial rewrite. > BackupPC would become a lot simpler. But it also creates a significant > issue of backward compatibility. The only solution would be to provide > tools that import the old BackupPC store into a new one. That is > possible, but would likely be very slow. I see in the roadmap with the just-released version of opensolaris that block de-duplication is scheduled to be in ZFS in the next version, perhaps early next year. How hard would it be to simply make the links for pooling an option that you could disable if the filesystem handles it better - as you can already do with compression? I'm not sure why you would need any other format change - and if you had a tool to reconstruct the pool links you could switch back if you wanted - at some cost in time and CPU. If the ZFS snapshot send/receive also works well for replication, that would take care of almost all of backuppc's major issues (well, except for working nicely on Linux - but the price is right for opensolaris too...). -- Les Mikesell lesmikes...@gmail.com ------------------------------------------------------------------------------ OpenSolaris 2009.06 is a cutting edge operating system for enterprises looking to deploy the next generation of Solaris that includes the latest innovations from Sun and the OpenSource community. Download a copy and enjoy capabilities such as Networking, Storage and Virtualization. Go to: http://p.sf.net/sfu/opensolaris-get _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
