Les Mikesell wrote at about 15:23:41 -0500 on Monday, August 31, 2009: > Jeffrey J. Kosowsky wrote: > > > > > > > I see lots of advantage in keeping the database portion relatively > > > > small, fast, replicable, and moveable. Then you can keep and > > > > distribute the files themselves wherever you want them spread across > > > > one or more separate filesystems. Then the database portion is > > > > optimized for what a database does best and the file-storing > > > > portion can be optimized for what a filesystem stores best. And both > > > > parts are easily moveable, replicable and not dependent or limited by > > > > hardlinks or other filesystem-dependent functionality. > > > > > > But the parts aren't independent. How do you propose keeping them in > > > sync or fixing them when they inevitably differ? > > > > Perhaps analogously to the way BackupPC_Nightly now makes sure that > > pool is in synch. > > It just uses filesystem operations so it is as reliable as the > filesystem itself. > > > More generally, we would need to consider two things: > > 1. What are the normal ways in which the two could get out of synch > > and then address each of those cases > > Start with that copy you wanted to be able to make. If the files and > metadata are separate things, there will clearly be times they are out > of sync and lots of opportunity for them to stay that way.
First copying is not even my primary reason for proposing a database approach. Second, copying a filesystem also creates synchronization issues unless you can do some type of shadow copy which either requires you to unmount the filesystem or use something like ZFS. Third, how hard is it to 'halt' BackupPC for the infrequent times that you want to backup or move the Backup tree? > > > 2. If necessary, create a repair tool (similar to what I created for > > the current system) if something breaks in non-standard ways (e.g., > > due to crashes, disk failures, etc.) > > > > I guess I can't answer your question without knowing what use cases > > you are worried about. > > All of them. Filesystems have journal/check mechanisms. How would you > provide the equivalent? I am certainly not a database expert but I am aware enough to know that such integrity issues are dealt with all the time in real world databases where you have even more complex use cases of multiple simultaneous users across multiple servers, disks, data sets, applications, etc. Obviously, moving to a database version would require significant thought and work. Even the "simple" existing BackupPC implementation went through multiple iterations involving and eliminating various race and locking conditions. No one is saying this would be easy or that it is a short-term feature addition -- it may not even be worthwhile and maybe the existing limitations could be addressed otherwise. I am certainly not trying to argue that the current approach doesn't suit your needs. I am only suggesting that the current implementation seems to be difficult to extend to address other relatively common and standard backup requirements. We are all just trying to suggest potential alternative long-term development paths. My one and only point is that if the advantages of a database approach are compelling then I am confident that the nuts-and-bolts details are solvable. We are not exactly stretching the cutting edge of databases here and we shouldn't be afraid to discuss a potentially better solution because of generic bogeymen like "atomicity." Let's all keep an open mind, especially since we are all far from committing development resources to any path. ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
