Tony Albers <tony.albers <at> gmail.com> writes: > > > > An alternative could be to rsync the files to the server running > bareos-fd That can work but still you should think about the things later on in this reply.
> > > 2014-05-12 8:27 GMT+02:00 Bruno Friedmann <friedmann.bruno <at> gmail.com>: > On Sunday 11 May 2014 23.05:37 PH wrote: > > Hi, > > > > i would like to know some recommendations howto effectively backup > > very high number of small files, around 100.000.000 to tape library > > using Bareos 13.2, it seems that process takes a very long time. The combination of NFS and lots of files will always be a problem as if you have a decent NFS server things will be highly synchronous which means without a SSD or other cache on the NFS server will give poor performance. With NFS it also depends a lot on your mount options how things will behave and also a lot on client and server OS. > > > > I have no problem with backup of 4TB with 3.000.000 small files. > > > > Please give me a idea, using some compression on network, or? Thank you. Compression won't help you much as the overhead is probably in the open/read/close overhead of processing all those small files. Things to enable are at least attribute spooling and data spooling as with a lot of small files you won't be able to push enough data to the storage daemon to keep the tape streaming. Attribute spooling also will help a lot as you then don't insert every file into the catalog when it is processed but at the end of your job in a batch insert. You could also setup things to backup to disk and then copy to tape using Copy Jobs (e.g. a DTDTT Disk To Disk To Tape) setup. > small files + nfs will never be a high-end throughput ... > If you're asking for that, I guess you're not able to install > a bareos-fd on the nfs server? The obvious solution is indeed run a FD on the NFS server and if that is not possible and you are looking for some sort of DR solution for your fileserver look into things like NDMP. -- Marco van Wieringen [email protected] Bareos GmbH & Co. KG Phone: +49-221-63069389 http://www.bareos.com Sitz der Gesellschaft: Köln | Amtsgericht Köln: HRA 29646 Komplementär: Bareos Verwaltungs-GmbH Geschäftsführer: Stephan Dühr, M. Außendorf, J. Steffens, P. Storz, M. v. Wieringen -- You received this message because you are subscribed to the Google Groups "bareos-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/d/optout.
