Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Kern Sibbald
On Friday 16 September 2005 17:51, Russell Howe wrote: > Kern Sibbald wrote: > > However, given some of the recent discussions, I'm not 100% what is the > > best. I suspect that forgetting about the FileId key, something like: > > > >index JobId > >index JobId, FilenameId, PathId > > > > wo

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Alan Brown
On Fri, 16 Sep 2005, Kern Sibbald wrote: For the File table in MySQL it was: PRIMARY KEY(FileId), INDEX (JobId, PathId, FilenameId) For PostgreSQL it was: primary key (fileid) create index file_jobid_idx on file (jobid); create index file_fp_idx on file (filenameid, pathid); The dif

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Russell Howe
Kern Sibbald wrote: > However, given some of the recent discussions, I'm not 100% what is the best. > > I suspect that forgetting about the FileId key, something like: > >index JobId >index JobId, FilenameId, PathId > > would work the best for both databases (SQLite too). I wouldn't

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Kern Sibbald
On Friday 16 September 2005 17:30, Alan Brown wrote: > On Fri, 16 Sep 2005, Kern Sibbald wrote: > > Gee, it looks like I really need to document checking/adding/deleting > > indexes to the manual so that users can easily check and correct any > > problems. This would probably be a nice mini-projec

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Alan Brown
On Fri, 16 Sep 2005, Kern Sibbald wrote: Gee, it looks like I really need to document checking/adding/deleting indexes to the manual so that users can easily check and correct any problems. This would probably be a nice mini-project for a script jockey. Probably. What are the recommended inde

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Kern Sibbald
On Friday 16 September 2005 16:57, Alan Brown wrote: > On Fri, 16 Sep 2005, Kern Sibbald wrote: > >> But not on upgrades from previous versions, right? > > > > On most upgrades, the index should not be lost, because generally an > > upgrade adds new columns or new tables. > > But does an uipgrade f

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Alan Brown
On Fri, 16 Sep 2005, Kern Sibbald wrote: But not on upgrades from previous versions, right? On most upgrades, the index should not be lost, because generally an upgrade adds new columns or new tables. But does an uipgrade form previous version of bacula database format add new indexing too?

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Daniel Weuthen
> On Fri, 16 Sep 2005, Kern Sibbald wrote: > > Are you running on 1.36.3? If so, this is strange since the default > > table setup for 1.36.3 includes an index on JobId in the File table. > > But not on upgrades from previous versions, right? debian sarge version is 1.36.2, that is the version

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Kern Sibbald
On Friday 16 September 2005 16:31, Alan Brown wrote: > On Fri, 16 Sep 2005, Kern Sibbald wrote: > > Are you running on 1.36.3? If so, this is strange since the default > > table setup for 1.36.3 includes an index on JobId in the File table. > > But not on upgrades from previous versions, right?

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Alan Brown
On Fri, 16 Sep 2005, Kern Sibbald wrote: Are you running on 1.36.3? If so, this is strange since the default table setup for 1.36.3 includes an index on JobId in the File table. But not on upgrades from previous versions, right? --- SF.N

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Daniel Weuthen
> Are you running on 1.36.3? If so, this is strange since the default table > setup for 1.36.3 includes an index on JobId in the File table. looks so: Connecting to Director papa-bacula:9101 1000 OK: papa-bacula-dir Version: 1.36.3 (22 April 2005) Enter a period to cancel a command. *ver papa-b

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Alan Brown
On Fri, 16 Sep 2005, Daniel Weuthen wrote: This is surprising because I always understood that a multiple index (JobId, PathId, FilenameId) was equivalent (actually better) than one only on JobId. It should be Could you tell me what DB you are using MySQL, ...? mysql Ver 12.22 Distrib

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Dan Langille
On 16 Sep 2005 at 14:37, Kern Sibbald wrote: > On Friday 16 September 2005 14:02, Daniel Weuthen wrote: > > > > > I had the same problems with Jobs which have more than 68 files > > > > > and mysql as database. After putting an index on JobId in the File > > > > > table I got an enormous speed

R: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Fabio Paracchini
-Messaggio originale- Da: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Per conto di Daniel Weuthen Inviato: venerdì 16 settembre 2005 14.03 A: bacula-users@lists.sourceforge.net Oggetto: Re: [Bacula-users] how to speed up directory tree building? > > > I had the same problems

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Kern Sibbald
On Friday 16 September 2005 14:02, Daniel Weuthen wrote: > > > > I had the same problems with Jobs which have more than 68 files > > > > and mysql as database. After putting an index on JobId in the File > > > > table I got an enormous speed up: > > > > > > I just tried this and the speedup is

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Daniel Weuthen
> > > I had the same problems with Jobs which have more than 68 files and > > > mysql as database. After putting an index on JobId in the File table I > > > got an enormous speed up: > > > > I just tried this and the speedup is amazing. Getting to selection of > > individual old files for a use

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Kern Sibbald
On Friday 16 September 2005 12:47, Alan Brown wrote: > On Thu, 15 Sep 2005, Daniel Weuthen wrote: > > I had the same problems with Jobs which have more than 68 files and > > mysql as database. After putting an index on JobId in the File table I > > got an enormous speed up: > > I just tried thi

Re: [Bacula-users] how to speed up directory tree building?

2005-09-16 Thread Alan Brown
On Thu, 15 Sep 2005, Daniel Weuthen wrote: I had the same problems with Jobs which have more than 68 files and mysql as database. After putting an index on JobId in the File table I got an enormous speed up: I just tried this and the speedup is amazing. Getting to selection of individual

Re: [Bacula-users] how to speed up directory tree building?

2005-09-15 Thread Daniel Weuthen
I had the same problems with Jobs which have more than 68 files and mysql as database. After putting an index on JobId in the File table I got an enormous speed up: without index: # Time: 050915 15:12:42 # [EMAIL PROTECTED]: bacula[bacula] @ localhost [127.0.0.1] # Query_time: 1564 Lock_ti

Re: [Bacula-users] how to speed up directory tree building?

2005-07-12 Thread Alan Brown
On Tue, 21 Jun 2005, Kern Sibbald wrote: You didn't mention how many files/job you have. If it is more than about 500K then I can understand the problem. Some of my backup sets (usually 1TB partitions) contain upwards of 4 million files(*) :-) Some ideas: - Split your jobs to keep t

Re: [Bacula-users] how to speed up directory tree building?

2005-06-23 Thread Sebastian Stark
On Tuesday 21 June 2005 22:25, Kern Sibbald wrote: > > *estimate job="Backup lech system" > > Connecting to Client lech-fd at lech:9102 > > 2000 OK estimate files=38800 bytes=2,149,638,554 > > > > The building of the directory tree took about 10 minutes. > > No memory tree is built for the estimate

Re: AW: AW: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Kern Sibbald
t- > Von: Arno Lehmann [mailto:[EMAIL PROTECTED] > Gesendet: Dienstag, 21. Juni 2005 13:07 > An: Jürgen Kuri > Cc: Sebastian Stark; bacula-users@lists.sourceforge.net > Betreff: Re: AW: [Bacula-users] how to speed up directory tree building? > > > Hello, > > Jürgen

Re: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Kern Sibbald
On Tuesday 21 June 2005 18:59, Peter Eriksson wrote: > At 12:13 2005-06-21, Kern Sibbald wrote: > >On Tuesday 21 June 2005 10:31, Sebastian Stark wrote: > > > Is there a way to speed up the creation of the directory tree when > > > restoring files? For some clients this takes more than an hour for

Re: AW: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Arno Lehmann
rag von Arno Lehmann Gesendet: Dienstag, 21. Juni 2005 10:43 An: Sebastian Stark Cc: bacula-users@lists.sourceforge.net Betreff: Re: [Bacula-users] how to speed up directory tree building? Hello, Sebastian Stark wrote: Is there a way to speed up the creation of the directory tree when restoring fil

Re: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Kern Sibbald
On Tuesday 21 June 2005 13:12, Sebastian Stark wrote: > On Tuesday 21 June 2005 12:13, Kern Sibbald wrote: > > On Tuesday 21 June 2005 10:31, Sebastian Stark wrote: > > > Is there a way to speed up the creation of the directory tree when > > > restoring files? For some clients this takes more than

Re: AW: AW: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Arno Lehmann
Jürgen, Jürgen Kuri wrote: Is c't planning an article about bacula? In fact I was thinking about such a project... This is a misunderstanding I am in fact not the author of the mentioned computer magazine. Sorry, I wrote faster than I thought. Arno -- IT-Service Lehmann

AW: AW: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Jürgen Kuri
05 13:07 An: Jürgen Kuri Cc: Sebastian Stark; bacula-users@lists.sourceforge.net Betreff: Re: AW: [Bacula-users] how to speed up directory tree building? Hello, Jürgen Kuri wrote: > Hello, > > I observed the very same phenomenon but I run a PostgreSql instead of a MySql > database.

Re: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Peter Eriksson
At 12:13 2005-06-21, Kern Sibbald wrote: On Tuesday 21 June 2005 10:31, Sebastian Stark wrote: > Is there a way to speed up the creation of the directory tree when > restoring files? For some clients this takes more than an hour for us. > > Our MySQL catalog has grown quite large (~5G) and I thi

Re: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Sebastian Stark
On Tuesday 21 June 2005 12:13, Kern Sibbald wrote: > On Tuesday 21 June 2005 10:31, Sebastian Stark wrote: > > Is there a way to speed up the creation of the directory tree when > > restoring files? For some clients this takes more than an hour for us. > > > > Our MySQL catalog has grown quite larg

AW: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Jürgen Kuri
10:43 An: Sebastian Stark Cc: bacula-users@lists.sourceforge.net Betreff: Re: [Bacula-users] how to speed up directory tree building? Hello, Sebastian Stark wrote: > Is there a way to speed up the creation of the directory tree when restoring > files? For some clients this takes more than an hour for us.

Re: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Kern Sibbald
On Tuesday 21 June 2005 10:31, Sebastian Stark wrote: > Is there a way to speed up the creation of the directory tree when > restoring files? For some clients this takes more than an hour for us. > > Our MySQL catalog has grown quite large (~5G) and I think this is the > reason. But maybe there's a

Re: [Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Arno Lehmann
Hello, Sebastian Stark wrote: Is there a way to speed up the creation of the directory tree when restoring files? For some clients this takes more than an hour for us. Our MySQL catalog has grown quite large (~5G) and I think this is the reason. But maybe there's another way to speed this up

[Bacula-users] how to speed up directory tree building?

2005-06-21 Thread Sebastian Stark
Is there a way to speed up the creation of the directory tree when restoring files? For some clients this takes more than an hour for us. Our MySQL catalog has grown quite large (~5G) and I think this is the reason. But maybe there's another way to speed this up other than splitting up the cat