On Friday 01 September 2006 19:33, Peter Sjoberg wrote: > I wonder if it's anything I can do to improve the performance when > inserting the attributes. > > I'm backing up several servers but one specifically is about 86Gb data > and 1.6Million files. The data backup finish after a few hours (<6) but > then it (I assume) updates the database with all the attributes and that > takes >18h! > > Sysinfo: > The server is a Dual PIII 800MHz with 2G memory > The db is around 500mb > The spooled attributes file (I'm spooling attributes and data) is abotu > 480Mb. > mysql version: 4.0.18 > > During the db update the mysqld proces is using 100% of one cpu. > I did change /etc/my.cnf to the "large" template and made sure it has > thread_concurrency=4 but I guess the inserts are single threaded so it > still ends up running on one cpu. > > I looked a little on variables etc and it seems like the pounding is > SELECT and INSERT statments. > > Short of getting faster CPU or be more patient, what can I look > at/change to get it finished a little faster?
There is lots that can be done, but will take quite a bit of programming. I'll be happy to explain it, but only if you are 100.00000% sure you are going to work on it. I think I have explained this at least two times already, so it is almost certainly in the archives as well. It involves doing grouping the attributes and doing them in blocks of about 100 inside a transaction. > > -- > ------------------------------------------------------------------- > Techwiz, Peter Sjoberg PGP key (12F506C8) on keyserver & homepage > Key fingerprint = 3DC2 CEBA 1590 B41A 3780 955A DB42 02BB 12F5 06C8 > mailto:peters AT techwiz.ca http://www.techwiz.ca/~peters > > > ------------------------------------------------------------------------- > Using Tomcat but need to do more? Need to support web services, security? > Get stuff done quickly with pre-integrated technology to make your job easier > Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 > _______________________________________________ > Bacula-users mailing list > Bacula-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bacula-users > ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
