Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-14 Thread ebollengier
Hello, Graham Keeling wrote: Hello, I now believe that the 'taking hours' problem that I was having was down to having additional indexes on my File table, as Eric suggested. I am using mysql-5.0.45. I had these indexes: JobId JobId, PathId, FilenameId PathId FilenameId Now

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-13 Thread Graham Keeling
Hello, I now believe that the 'taking hours' problem that I was having was down to having additional indexes on my File table, as Eric suggested. I am using mysql-5.0.45. I had these indexes: JobId JobId, PathId, FilenameId PathId FilenameId Now I have these indexes: JobId JobId, PathId,

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-09 Thread Graham Keeling
On Thu, Apr 08, 2010 at 12:29:05PM -0700, ebollengier wrote: Graham Keeling wrote: On Thu, Apr 08, 2010 at 07:44:14AM -0700, ebollengier wrote: Hello, Graham Keeling wrote: Hello, I'm still waiting for my test database to fill up with Eric's data (actually,

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-09 Thread ebollengier
Graham Keeling wrote: I don't understand this at all. If you cannot trust the JobIds or FileIds in the File table, then the postgres query is also broken. The postgres query doesn't even mention JobTDate. In fact, the postgres query is using StartTime to do the ordering. And

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-09 Thread Graham Keeling
On Fri, Apr 09, 2010 at 05:27:44AM -0700, ebollengier wrote: I'm really thinking that the problem is on the MySQL side (bad version perhaps), or on your modifications (my tests shows that with a FilenameId, PathId index, results are 10 times slower than with the default indexes) What

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-08 Thread Graham Keeling
On Wed, Apr 07, 2010 at 09:58:51AM -0700, ebollengier wrote: ebollengier wrote: Graham Keeling wrote: On Wed, Apr 07, 2010 at 08:22:09AM -0700, ebollengier wrote: I tweaked my test to compare both queries, and it shows no difference with and without base job part... If you

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-08 Thread Graham Keeling
Hello, I'm still waiting for my test database to fill up with Eric's data (actually, it's full now, but generating the right indexes is taking lots of time). But, I have another proposed solution, better than the last one I made. My previous solution was still taking a very very long time for

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-08 Thread ebollengier
Hello, Graham Keeling wrote: Hello, I'm still waiting for my test database to fill up with Eric's data (actually, it's full now, but generating the right indexes is taking lots of time). But, I have another proposed solution, better than the last one I made. My previous solution

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-08 Thread Graham Keeling
On Thu, Apr 08, 2010 at 07:44:14AM -0700, ebollengier wrote: Hello, Graham Keeling wrote: Hello, I'm still waiting for my test database to fill up with Eric's data (actually, it's full now, but generating the right indexes is taking lots of time). But, I have another

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-08 Thread ebollengier
Graham Keeling wrote: On Thu, Apr 08, 2010 at 07:44:14AM -0700, ebollengier wrote: Hello, Graham Keeling wrote: Hello, I'm still waiting for my test database to fill up with Eric's data (actually, it's full now, but generating the right indexes is taking lots of time).

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread Graham Keeling
On Tue, Apr 06, 2010 at 09:01:13AM -0700, ebollengier wrote: Hello Graham, Hello, thanks for your reply. Graham Keeling wrote: Hello, I'm using bacula-5.0.1. I have a 2.33GHz CPU with 2G of RAM. I am using MySQL. I had a VirtualFull scheduled for my client. My log says the

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread ebollengier
Graham Keeling wrote: On Tue, Apr 06, 2010 at 09:01:13AM -0700, ebollengier wrote: Hello Graham, Hello, thanks for your reply. Graham Keeling wrote: Hello, I'm using bacula-5.0.1. I have a 2.33GHz CPU with 2G of RAM. I am using MySQL. I had a VirtualFull scheduled for my

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread Martin Simmons
On Wed, 7 Apr 2010 12:40:24 +0100, Graham Keeling said: On Tue, Apr 06, 2010 at 09:01:13AM -0700, ebollengier wrote: Hello Graham, Hello, thanks for your reply. Graham Keeling wrote: Hello, I'm using bacula-5.0.1. I have a 2.33GHz CPU with 2G of RAM. I am using MySQL.

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread Graham Keeling
On Wed, Apr 07, 2010 at 02:51:42PM +0100, Martin Simmons wrote: Does it still run quickly if keep that Job.JobId IN clause but use the numbers returned by SELECT DISTINCT BaseJobId FROM BaseFiles WHERE JobId IN (22,23,31,34,42,48,52) in place of the the nested select? In my case,

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread Graham Keeling
Since: a) I am not using Base jobs b) I am currently stuck with using MySQL c) There is not a 'proper' fix yet I am going to use the attached patch as a temporary solution to the problem. Index: src/cats/sql_cmds.c === RCS file:

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread ebollengier
ebollengier wrote: Graham Keeling wrote: On Tue, Apr 06, 2010 at 09:01:13AM -0700, ebollengier wrote: Hello Graham, Hello, thanks for your reply. Graham Keeling wrote: Is there anything I can do that would speed it up? Perhaps even more importantly, what was it doing for

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread Graham Keeling
On Wed, Apr 07, 2010 at 08:22:09AM -0700, ebollengier wrote: I tweaked my test to compare both queries, and it shows no difference with and without base job part... If you want to test queries on your side with your data, you can download my tool (accurate-test.pl) on

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread ebollengier
Graham Keeling wrote: On Wed, Apr 07, 2010 at 08:22:09AM -0700, ebollengier wrote: I tweaked my test to compare both queries, and it shows no difference with and without base job part... If you want to test queries on your side with your data, you can download my tool (accurate-test.pl)

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-07 Thread ebollengier
ebollengier wrote: Graham Keeling wrote: On Wed, Apr 07, 2010 at 08:22:09AM -0700, ebollengier wrote: I tweaked my test to compare both queries, and it shows no difference with and without base job part... If you want to test queries on your side with your data, you can download my

[Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-06 Thread Graham Keeling
Hello, I'm using bacula-5.0.1. I have a 2.33GHz CPU with 2G of RAM. I am using MySQL. I had a VirtualFull scheduled for my client. My log says the following: Apr 4 18:56:02 Start Virtual Backup JobId 56, Job=Linux:cvs.2010-04-04_18.56.00_03 Apr 4 18:56:02 This Job is not an Accurate backup

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-06 Thread Graham Keeling
On Tue, Apr 06, 2010 at 12:59:49PM +0200, Matija Nalis wrote: On Tue, Apr 06, 2010 at 11:13:44AM +0100, Graham Keeling wrote: I'm using bacula-5.0.1. At about Apr 6 10:00, I logged into mysql, and ran 'show full processlist;', which gave the following. | 3033 | root | localhost |

Re: [Bacula-users] VirtualFull mysql query blocks other jobs for a long time

2010-04-06 Thread ebollengier
Hello Graham, Graham Keeling wrote: Hello, I'm using bacula-5.0.1. I have a 2.33GHz CPU with 2G of RAM. I am using MySQL. I had a VirtualFull scheduled for my client. My log says the following: Apr 4 18:56:02 Start Virtual Backup JobId 56, Job=Linux:cvs.2010-04-04_18.56.00_03