On Friday 04 September 2009 00:59:28 Gavin McCullagh wrote: > Hi, > > On Thu, 03 Sep 2009, Kern Sibbald wrote: > > I suspect that all that is already possible with Bacula but may require a > > bit of scripting. In short, you need support from a Bacula specialist, > > and we don't do that on this list (bacula-devel). > > Fair enough. I asked for suggested approaches on -users a few weeks ago > but didn't get much back. Perhaps we need to put our hands in our pockets > > :-) > : > > What you are asking for is a bit specialized, so I doubt that it is > > something that would be directly implemented in Bacula, unless we saw > > that it was something a lot of users want, which means it would need to > > be a very clearly defined Feature Request, voted on, then someone would > > need to volunteer to write it. > > I hadn't really imagined it to be specialized (long term snapshots without > storing large numbers of full backups with lots of redundant files), but I > guess it must be as there don't seem to be many people popping up who want > it. I did get one positive response on the -users list and I know a lot of > people use the rsync + hard links approach (one drawback of which you > mention below). This proposal is the simplest way I can think of (so far) > to get the same kind of storage efficiency with Bacula's volume storage. > We mostly use disk storage rather than tapes just now, so perhaps it > doesn't make sense for tapes.
It sounds to me like the easiest way for you to accomplish what you want is to buy a decent tape drive. Tapes are very good for long term storage. If you buy a drive that can hold a Full backup of everything on one cassette, you have an ideal way of storing data relatively inexpensively long term. > > I can write up a draft feature request (perhaps trying to state it more > clearly) if that helps. If you guys really don't like the idea you can > always shoot it down anyway, there's no real need for a vote if it's really > not good. Should I use the descriptions in projects.gz as a template? If you want to submit a Feature Request, you should see: www.bacula.org -> Feature Requests > > An alternative description which sets out the potential space savings is > outlined at the end of this email. Apologies to those for whom this is > obvious. > > > As I say, the other solution is to ask on the bacula-users list or ask > > for professional help. > > > > Thanks for using Bacula. > > Many more thanks for creating it, we're very appreciative :-) > > > PS: Be careful with programs like BackupPC that create a lot of hard > > links. If you create enough of them, you will not be able to boot your > > system because fsck must keep track of all the hard linked files in > > memory, so if you have too many hard links and not enough memory, you may > > one day be stuck. The same problem exists with restoring hard links with > > Bacula. On a normal system that has a few hundred or even a hundred > > thousand, no problem, but if you start getting millions of them you may > > be in trouble. > > That's good to be aware of, many thanks. I know that attempting a "du" on > that tree of the filesystem is something that takes an unreasonably long > time right now, which may be an indicator that I need to look at this. Yes, that would worry me a lot. > > Gavin > > > If you suppose for the sake of argument that x Bytes of new data get > permanently added to the data set each month, that they don't get removed > (I'd guess this is pretty common on email servers) and that you keep > indefinite monthly snapshots (again, lots of people like long memories on > email servers), the total data consumed by the backups increases as > follows: > > total_data(month1) = x > total_data(month2) = 2x + x > total_data(month3) = 3x + 2x + x > total_data(month4) = 4x + 3x + 2x + x I don't think you have looked into the possibility of doing separate "long term" backups to tape. If you do it correctly, I am not convinced that the growth in data is as bad as you say. If you do a Full once a year then incrementals once a month, I think you will have a reasonable solution. > > Using the system I've outlined, I think the required storage scales > linearly. Using monthly full backups each time, you get an arithmetic > progression which gets progressively worse. After 12 months, the > arithmetic progression is at 78x, where the linear is at 12x. I have never understood your "system". From what I see you are proposing to make incrementals back in time -- that is something that I do not think is possible and with the information you supplied, I could not imagine any practical way to implement a working solution. It could be that I am missing something. I would suggest that you consider implementing a solution that does a special "archival" or "long term" storage backup as I noted above. The total incremental data it backs up over one year is not really excessive, and lends itself to a tape or disk based solution. > > Obviously that's a very simplified picture, but I think it reasonably > represents our situation. > Regards, Kern ------------------------------------------------------------------------------ Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
