Nordgren, Bryce L -FS <bnordgren <at> fs.fed.us> writes: > > Aside from the notification email, is there any way to go back and look > at : > > * Job elapsed time > * Compression method > * Compression ratio > Job elapsed time can be calculated by getting the info from the database see Job table.
* EndTime - StartTime (or RealEndTime - StartTime) Compression method is part of the fileset and not stored anywhere in the database. Compression ratio is something you can also calculate using two fields of the Job table (e.g. JobBytes and ReadBytes) > ? Using llist I can compute elapsed time from start and end, and list > tells me job bytes, but I can't find these other parameters. Since its > in the notification email, its not a huge deal. I just want to know if > there's a way I can go back and look these up after the fact... See above using a sql query either via the bconsole or via your database utility. > > > you could also try lz4, lz4hc and lzfast. > > They're on the list. I'm more or less just changing the compression > method between nightly incrementals to observe the impact. That takes > some time... :) > Indeed also when you have these vast amounts of data :-) > > What you can always do is split your fileset into chunks and run > > backups of each one in parallel. > > I see your point. My concern with this strategy is that it is likely to > cause more disk seeks on both the FD and SD end. I'll keep this in mind, > but I'd prefer a solution where the IO stays sequential and the > computation is parallelized. For the moment, I'll just pick the fastest > out of the box solution. > That all depends of course largely on the filesystem used. Filesystems like ZFS have no real problem when you have different separate sequential streams. For the SD you should use spooling so it doesn't start interleaving the data when you run multiple backup jobs. > In the long term tho, it seems like allowing the FD to fully utilize the > client machine would be a feature worthy of inclusion. Is there an > objection to entering this as a feature request on the bug tracker? There is already one for windows where the problem when using VSS is bigger as you can only have one backup job running. There are however some thing that are currently making this not something we can easily achieve and needs some rather serious redesign which most likely will occur some day. Currently the FD reads the data in chunks of 64 Kb and then compresses that data and then sends it to the SD. To get things to be parallel you probably need something like a pipeline which changes the whole backup process a bit e.g. : - thread that reads the data and pushes it onto a work queue - thread(s) that do compression and encryption and push it onto a next work queue. - thread that gets the data from the second work queue and sends it to the SD. The current setup however has single buffers for compression and encryption so it needs quite some work. Also the 64 Kb chunks are probably something that needs to change, and to get things send to the SD in order e.g. maybe some pipeline is faster then an other is also something to care about. We have one serious advantage and that is that the compression and encryption routines are nicely separated in the lib and with little effort they should be ok to used with multiple threads. That was one of the many refactoring jobs we performed when forking from Bacula. -- Marco van Wieringen [email protected] Bareos GmbH & Co. KG Phone: +49-221-63069389 http://www.bareos.com Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646 Komplementär: Bareos Verwaltungs-GmbH Geschäftsführer: Stephan Dühr, M. Außendorf, J. Steffens, P. Storz, M. v. Wieringen -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
