Hi, collegues ran into performance issues during mark/unmark. We narrowed it down to hardlinks: mark and unmark both call set_extract, which traverses entire directory tree for each hardlink.
Let's have a look at hardlinks [1] on a casual RHEL 6 system: root@joe:~/tmp> find / ! -type d -links +1 -ls|sort -n>hardlinks.txt root@joe:~/tmp> wc -l hardlinks.txt 23298 hardlinks.txt Most of these are pyc & pyo, and yum things. On our test mail server, 392577 hardlinks in /var/spool/imap. It's how cyrus lmtpd works, and it's good. However, mark works for well over an hour. But on a production mail server - 1459466 hardlinks in imap folders. Turning off hardlinks in bacula is not a solution, and turning of hardlinks in cyrus is neither. Bottom line, we can't sell bacula for mail servers. So, what can we do about it? A rblist could be used for lookup instead of dir tree traversal: 1) TREE_ROOT would contain hardlinks rblist member (tree.h) 2) A comparator function needs to be implemented (ua_tree.c?) It would compare items based on JobId and FileIndex. 3) insert_tree_handler needs to maintain hardlink rblist (ua_tree.c) Only first ones, marked with ok. This would build up hardlinks rblist along with directory tree. 4) set_extract would use that rblist for hardlink lookup (ua_tree.c) 5) What about bookkeeping? I see rblist internally allocates and frees, and free_tree would relase it all, right? Did I miss something? Regards... [1] http://www.unix.com/unix-dummies-questions-answers/542-search-hardlinks-based-filename-via-find-command.html ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Bacula-devel mailing list Bacula-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-devel
