'Mingus Dew' wrote: > All, > I am running Bacula 5.0.1 on Solaris 10 x86. I'm currently running > MySQL 4.1.22 for the database server. I do plan on upgrading to a > compatible version of MySQL 5, but migrating to PostgreSQL isn't an > option at this time. > > I am trying to backup to tape a very large number of files for a > client. While the data size is manageable at around 2TB, the number of > files is incredibly large. >The first of the jobs had 27 million files and initially failed because > the batch table became "Full". I changed the myisam_data_pointer size > to a value of 6 in the config. > >This job was then able to run successfully and did not take too long. > > I have another job which has 42 million files. I'm not sure what that > equates to in rows that need to be inserted, but I can say that I've > not been able to successfully run the job, as it seems to hang for > over 30 hours in a "Dir inserting attributes" status. This causes > other jobs to backup in the queue and once canceled I have to restart > Bacula. > > I'm looking for way to boost performance of MySQL or Bacula (or both) > to get this job completed.
You *really* need to upgrade to MySQL 5 and change to InnoDB - there is no way in hell that MySQL 4 + MyISAM is going to perform decent in your situation. Solaris 10 is a Tier 1 platform for MySQL so the latest versions are always available from www.mysql.com in the native pkg format so there really is no excuse. We run our Bacula Catalog MySQl servers on Solaris (OpenSolaris) so perhaps I can give you some pointers. Our smallest Bacula DB is currently ~70 GB (381,230,610 rows). Since you are using Solaris 10 I assume that you are going to run MySQL off ZFS - in that case you need to adjust the ZFS recordsize for the filesystem that is going to hold your InnoDB datafiles to match the InnoDB block size. If you are using ZFS you should also consider getting yourself a fast SSD as a SLOG (or to disable the ZIL entirely if you dare) - all InnoDB writes to datafiles are O_SYNC and benefit *greatly* from an SSD in terms of write / transaction speed. If you have enough CPU power to spare you should try turning on compression for the ZFS filesystem holding the datafiles - it also can accelerate DB writes / reads but YMMV. Lastly, our InnoDB related configuration from my.cnf : # InnoDB options skip-innodb_doublewrite innodb_data_home_dir = /tank/db/ innodb_log_group_home_dir = /tank/logs/ innodb_support_xa = false innodb_file_per_table = true innodb_buffer_pool_size = 20G innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 128M innodb_log_file_size = 512M innodb_log_files_in_group = 2 innodb_max_dirty_pages_pct = 90 > >Thanks, >Shon >------------------------------------------------------------------------------ >Beautiful is writing same markup. Internet Explorer 9 supports >standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. >Spend less time writing and rewriting code and more time creating great >experiences on the web. Be a part of the beta today. >http://p.sf.net/sfu/beautyoftheweb >_______________________________________________ >Bacula-users mailing list >Bacula-users@lists.sourceforge.net >https://lists.sourceforge.net/lists/listinfo/bacula-users -- Med venlig hilsen / Best Regards Henrik Johansen hen...@scannet.dk Tlf. 75 53 35 00 ScanNet Group A/S ScanNet ------------------------------------------------------------------------------ Beautiful is writing same markup. Internet Explorer 9 supports standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3. Spend less time writing and rewriting code and more time creating great experiences on the web. Be a part of the beta today. http://p.sf.net/sfu/beautyoftheweb _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
