Hello, On 07/13/2012 08:05 AM, Josip Almasi wrote: > Hi, > > collegues ran into performance issues during mark/unmark. > We narrowed it down to hardlinks: mark and unmark both call set_extract, > which traverses entire directory tree for each hardlink. > > Let's have a look at hardlinks [1] on a casual RHEL 6 system: > > root@joe:~/tmp> find / ! -type d -links +1 -ls|sort -n>hardlinks.txt > root@joe:~/tmp> wc -l hardlinks.txt > 23298 hardlinks.txt > > Most of these are pyc & pyo, and yum things. > > On our test mail server, 392577 hardlinks in /var/spool/imap. > It's how cyrus lmtpd works, and it's good. > However, mark works for well over an hour. > But on a production mail server - 1459466 hardlinks in imap folders. > > Turning off hardlinks in bacula is not a solution, and turning of > hardlinks in cyrus is neither. > Bottom line, we can't sell bacula for mail servers. > > So, what can we do about it? > > A rblist could be used for lookup instead of dir tree traversal: The tree code was written a long time ago, and from what I remember, it already uses a red black tree structure, so I am not sure without looking at the code exactly what you want to do differently. The best I can tell, you want a separate hard-link list. > > 1) TREE_ROOT would contain hardlinks rblist member (tree.h) > 2) A comparator function needs to be implemented (ua_tree.c?) > It would compare items based on JobId and FileIndex. > 3) insert_tree_handler needs to maintain hardlink rblist (ua_tree.c) > Only first ones, marked with ok. > This would build up hardlinks rblist along with directory tree. > 4) set_extract would use that rblist for hardlink lookup (ua_tree.c) > 5) What about bookkeeping? > I see rblist internally allocates and frees, and free_tree would relase > it all, right? > > Did I miss something?
I am sorry, I cannot really say if that is the right thing to do. I am not even sure if you have nailed down the problem. It seems to me that Bacula keeps a "pointer" to the link, and the problem I have previously seen is that for each file that is a hard link, Bacula must do a SQL query to find the file -- when you start doing millions of SQL queries, things can get slow unless you are a DBA and do the appropriate DB tuning, which is not evident. So, are you going to implement this and submit it as a contribution? If so, please let me know and I will look at the code a bit more when I get back from vacation in August and tell you what I think. Hard links have always been a problem for Bacula -- and by the way, if you create too many of them you may not be able to boot your system since during the boot process (non-Bacula) from what I understand all the hard links found must be able to fit within the available boot memory. If not, are you asking us to implement something for free so that you can sell and make money off of Bacula? It might be possible for the Enterprise version, but right now, I am pretty overloaded doing bug fixes and other implementations for the community version. Best regards, Kern > > Regards... > > [1] > http://www.unix.com/unix-dummies-questions-answers/542-search-hardlinks-based-filename-via-find-command.html > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bacula-devel mailing list > Bacula-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-devel > ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
