Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-09-02 Thread Marc Cousin
> Actually, it is worse than what you describe above. The problem is that I > did not consider in my proposal what happens in an Incremental backup. As > proposed, it simply will not work. > > > Maybe it would be easier to add a parentid in the Path table. > > This is probably a nice solution tha

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-09-02 Thread demerphq
On 8/26/07, Dan Langille <[EMAIL PROTECTED]> wrote: > On 26 Aug 2007 at 13:28, demerphq wrote: > > On 8/26/07, Dan Langille <[EMAIL PROTECTED]> wrote: > > > On 26 Aug 2007 at 0:52, demerphq wrote: > > [...] > > > > Select all and only children of a single node in a single query: > > > > select C.*

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-09-02 Thread Kern Sibbald
On Thursday 30 August 2007 20:44, Marc Cousin wrote: > > > If we separate the current File data into Dirs and Files, first, we > > > have reduced the amount of data that we need to look through to find > > > the directory tree for a job by about a factor of 100 on most typical > > > Unix systems.

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-09-02 Thread Kern Sibbald
On Thursday 30 August 2007 20:44, Marc Cousin wrote: > > > If we separate the current File data into Dirs and Files, first, we > > > have reduced the amount of data that we need to look through to find > > > the directory tree for a job by about a factor of 100 on most typical > > > Unix systems.

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-30 Thread Marc Cousin
> > If we separate the current File data into Dirs and Files, first, we have > > reduced the amount of data that we need to look through to find the > > directory tree for a job by about a factor of 100 on most typical Unix > > systems. That is already good. Then, in the Dirs table we can have th

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Cousin Marc
Le Tuesday 28 August 2007 16:12:56 Kern Sibbald, vous avez écrit : > On Tuesday 28 August 2007 13:30, Cousin Marc wrote: > > Le Tuesday 28 August 2007 11:57:28 Kern Sibbald, vous avez écrit : > > > On Tuesday 28 August 2007 10:41, BOLLENGIER Eric wrote: > > > > > > For the visibility flag, it may n

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Kern Sibbald
On Tuesday 28 August 2007 13:30, Cousin Marc wrote: > Le Tuesday 28 August 2007 11:57:28 Kern Sibbald, vous avez écrit : > > On Tuesday 28 August 2007 10:41, BOLLENGIER Eric wrote: > > > > > For the visibility flag, it may not be that easy : a directory may > > > > > be visible even if it's not in

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Cousin Marc
Le Tuesday 28 August 2007 11:57:28 Kern Sibbald, vous avez écrit : > On Tuesday 28 August 2007 10:41, BOLLENGIER Eric wrote: > > > > For the visibility flag, it may not be that easy : a directory may be > > > > visible even if it's not in a backup. For instance, /home if > > > > /home/marc is backu

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Cousin Marc
Le Tuesday 28 August 2007 10:24:03 Kern Sibbald, vous avez écrit : > On Sunday 26 August 2007 09:17, Marc Cousin wrote: > > On Sunday 26 August 2007 07:43:25 Kern Sibbald wrote: > > > Hello Marc, > > > > > > I don't yet understand the details of how you use these tables, but it > > > seems to me th

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Kern Sibbald
On Tuesday 28 August 2007 10:41, BOLLENGIER Eric wrote: > > > For the visibility flag, it may not be that easy : a directory may be > > > visible even if it's not in a backup. For instance, /home if /home/marc > > > is backuped should be displayed, so we add an entry from /home in > > > pathvisibil

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread BOLLENGIER Eric
> > For the visibility flag, it may not be that easy : a directory may be > > visible even if it's not in a backup. For instance, /home if /home/marc > > is backuped should be displayed, so we add an entry from /home in > > pathvisibility for the job where /home/marc is backed up. > > Isn't the vis

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-28 Thread Kern Sibbald
On Sunday 26 August 2007 09:17, Marc Cousin wrote: > On Sunday 26 August 2007 07:43:25 Kern Sibbald wrote: > > Hello Marc, > > > > I don't yet understand the details of how you use these tables, but it > > seems to me that we could modify the Bacula table structure as follows: > > > > - Split the F

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread Marc Cousin
> If you don't need to grab a subtree then yes, PP is fine. If you do > its not ideal. I guess i misunderstood the problem, i thought you > wanted to get "all descendants of a given node". If not, well, > apologies for the noise. :-) > > Yves Yes, we just want direct children... The purpose is to

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread Dan Langille
On 26 Aug 2007 at 13:28, demerphq wrote: > On 8/26/07, Dan Langille <[EMAIL PROTECTED]> wrote: > > On 26 Aug 2007 at 0:52, demerphq wrote: > [...] > > > > Can you explain the pros and cons over something like a parent-child > > > > relationship? That each, each file entry has a link to the parent

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread demerphq
On 8/26/07, Marc Cousin <[EMAIL PROTECTED]> wrote: > On Sunday 26 August 2007 00:52:33 demerphq wrote: [snip] > > The idea is fairly straight forward, each record has a Left and Right > > value, any children will have left and right values such that P.Left > > <= C.left and C.right <=P.right. An in

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread demerphq
On 8/26/07, Dan Langille <[EMAIL PROTECTED]> wrote: > On 26 Aug 2007 at 0:52, demerphq wrote: [...] > > > Can you explain the pros and cons over something like a parent-child > > > relationship? That each, each file entry has a link to the parent > > > directory. This would be a self referential

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread Marc Cousin
On Sunday 26 August 2007 00:52:33 demerphq wrote: > On 8/25/07, Dan Langille <[EMAIL PROTECTED]> wrote: > > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > > Hello Marc, > > > > > > > > Thanks for the information. I think this is something

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-26 Thread Marc Cousin
On Sunday 26 August 2007 07:43:25 Kern Sibbald wrote: > Hello Marc, > > I don't yet understand the details of how you use these tables, but it > seems to me that we could modify the Bacula table structure as follows: > > - Split the File table into two new tables: > - Files containing only file

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Kern Sibbald
Hello Marc, I don't yet understand the details of how you use these tables, but it seems to me that we could modify the Bacula table structure as follows: - Split the File table into two new tables: - Files containing only file entries - Dirs containing the equivalent of a current File e

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Dan Langille
On 26 Aug 2007 at 0:52, demerphq wrote: > On 8/25/07, Dan Langille <[EMAIL PROTECTED]> wrote: > > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > > Hello Marc, > > > > > > > > Thanks for the information. I think this is something that

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread demerphq
On 8/25/07, Dan Langille <[EMAIL PROTECTED]> wrote: > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > Hello Marc, > > > > > > Thanks for the information. I think this is something that is probably > > > well worth the effort to put in the

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Marc Cousin
On Saturday 25 August 2007 22:25:00 Dan Langille wrote: > On 25 Aug 2007 at 22:07, Marc Cousin wrote: > > On Saturday 25 August 2007 21:34:25 Dan Langille wrote: > > > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > > > Hello Marc, > > >

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Dan Langille
On 25 Aug 2007 at 22:07, Marc Cousin wrote: > On Saturday 25 August 2007 21:34:25 Dan Langille wrote: > > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > > On Saturday 25 August 2007 10:31:04 you wrote: > > > > Hello Marc, > > > > > > > > Thanks for the information. I think this is something

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Marc Cousin
On Saturday 25 August 2007 21:34:25 Dan Langille wrote: > On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > > On Saturday 25 August 2007 10:31:04 you wrote: > > > Hello Marc, > > > > > > Thanks for the information. I think this is something that is probably > > > well worth the effort to put in th

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Dan Langille
On 25 Aug 2007 at 10:51, Eric Bollengier wrote: > On Saturday 25 August 2007 10:31:04 you wrote: > > Hello Marc, > > > > Thanks for the information. I think this is something that is probably > > well worth the effort to put in the core Bacula code. I'm going to study > > it a bit to see how it

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Marc Cousin
On Saturday 25 August 2007 17:48:22 Dirk Bartley wrote: > On Sat, 2007-08-25 at 09:43 +0200, Marc Cousin wrote: > > On Saturday 25 August 2007 09:03:49 Kern Sibbald wrote: > > > Hello Marc, > > > > > > Now that we have a number of "browsers" and users are doing a lot more > > > queries, it might be

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Dirk Bartley
On Sat, 2007-08-25 at 09:43 +0200, Marc Cousin wrote: > On Saturday 25 August 2007 09:03:49 Kern Sibbald wrote: > > Hello Marc, > > > > Now that we have a number of "browsers" and users are doing a lot more > > queries, it might be a good idea to consider re-organizing a few of the > > Bacula table

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Kern Sibbald
Thanks, Kern On Saturday 25 August 2007 10:51, Eric Bollengier wrote: > On Saturday 25 August 2007 10:31:04 you wrote: > > Hello Marc, > > > > Thanks for the information. I think this is something that is probably > > well worth the effort to put in the core Bacula code. I'm going to study > >

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Eric Bollengier
> For example, this is how we list files in a directory : > > SELECT File.FilenameId, listfiles.id, listfiles.Name, File.LStat, > File.JobId FROM > (SELECT Filename.Name, max(File.FileId) as id >FROM File, Filename >WHERE File.FilenameId = Filename.FilenameId > AND Fi

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Eric Bollengier
On Saturday 25 August 2007 10:31:04 you wrote: > Hello Marc, > > Thanks for the information. I think this is something that is probably > well worth the effort to put in the core Bacula code. I'm going to study > it a bit to see how it might be organized into a project. Code like this > might al

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Kern Sibbald
Hello Marc, Thanks for the information. I think this is something that is probably well worth the effort to put in the core Bacula code. I'm going to study it a bit to see how it might be organized into a project. Code like this might also help a lot for the simple restores that Bacula curre

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Marc Cousin
On Saturday 25 August 2007 09:03:49 Kern Sibbald wrote: > Hello Marc, > > Now that we have a number of "browsers" and users are doing a lot more > queries, it might be a good idea to consider re-organizing a few of the > Bacula tables to improve the efficiency of the queries (and also perhaps > bac

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-25 Thread Kern Sibbald
Hello Marc, Now that we have a number of "browsers" and users are doing a lot more queries, it might be a good idea to consider re-organizing a few of the Bacula tables to improve the efficiency of the queries (and also perhaps backups and restores). Please see more below... On Friday 24 Augu

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-24 Thread Marc Cousin
On Thursday 23 August 2007 04:02:07 Dirk Bartley wrote: > On Wed, 2007-08-22 at 10:11 -0400, Dan Langille wrote: > > On 21 Aug 2007 at 20:07, Dirk Bartley wrote: > > > On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > > There are basically three queries. One to get a list of > > > > >

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-23 Thread Dan Langille
On 22 Aug 2007 at 22:02, Dirk Bartley wrote: > On Wed, 2007-08-22 at 10:11 -0400, Dan Langille wrote: > > If you can put your database on a website somewhere, I'll download it > > and try it out and work on the query that way. > > I will respectfully choose not to do that at this time. I fea

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-23 Thread Kern Sibbald
On Thursday 23 August 2007 04:02, Dirk Bartley wrote: > On Wed, 2007-08-22 at 10:11 -0400, Dan Langille wrote: > > On 21 Aug 2007 at 20:07, Dirk Bartley wrote: > > > On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > > There are basically three queries. One to get a list of > > > > > d

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-22 Thread Dirk Bartley
On Wed, 2007-08-22 at 10:11 -0400, Dan Langille wrote: > On 21 Aug 2007 at 20:07, Dirk Bartley wrote: > > > On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > > > > > There are basically three queries. One to get a list of directories, > > > > another to get a list of files and one t

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-22 Thread Dan Langille
On 21 Aug 2007 at 20:07, Dirk Bartley wrote: > On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > > > There are basically three queries. One to get a list of directories, > > > another to get a list of files and one to get a list of versions for a > > > specific file. > > > > > > 1.

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-22 Thread Kern Sibbald
On Sunday 19 August 2007 18:35, Dirk Bartley wrote: > Hello > > Now that 2.2.0 is released, I've had the opportunity to use bat in my > work environment. It worked generally as hoped with one major > exception. The catalog browser. slow slow slow. Yes, in some cases, this is my experience too.

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-21 Thread Dirk Bartley
On Mon, 2007-08-20 at 08:04 -0400, Dan Langille wrote: > > > > There are basically three queries. One to get a list of directories, > > another to get a list of files and one to get a list of versions for a > > specific file. > > > > 1. To get a list of directories given a list of jobs. Here is

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-20 Thread Cousin Marc
Hi, I'm interfering in the discussion as I've worked with Eric on brestore (doing the SQL part), doing something looking a lot like what you're doing now. My comments are inline Le Monday 20 August 2007 03:47:15 Dirk Bartley, vous avez écrit : > On Sun, 2007-08-19 at 13:06 -0400, Dan Langille w

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-20 Thread Dan Langille
On 19 Aug 2007 at 21:47, Dirk Bartley wrote: > On Sun, 2007-08-19 at 13:06 -0400, Dan Langille wrote: > > On 19 Aug 2007 at 12:35, Dirk Bartley wrote: > >=20 > > > There is one alternative index to the above. The index of the two > > > colums (FilenameId, JobId) is really only needed for the cond

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-19 Thread Dirk Bartley
On Sun, 2007-08-19 at 13:06 -0400, Dan Langille wrote: > On 19 Aug 2007 at 12:35, Dirk Bartley wrote: > > > There is one alternative index to the above. The index of the two > > colums (FilenameId, JobId) is really only needed for the condition of > > FilenameId = (SELECT FilenameId FROM Filename

Re: [Bacula-devel] Query changes in the catalog browser and indexes

2007-08-19 Thread Dan Langille
On 19 Aug 2007 at 12:35, Dirk Bartley wrote: > There is one alternative index to the above. The index of the two > colums (FilenameId, JobId) is really only needed for the condition of > FilenameId = (SELECT FilenameId FROM Filename WHERE name=''). A > subquery can't be used when creating an ind

[Bacula-devel] Query changes in the catalog browser and indexes

2007-08-19 Thread Dirk Bartley
Hello Now that 2.2.0 is released, I've had the opportunity to use bat in my work environment. It worked generally as hoped with one major exception. The catalog browser. slow slow slow. Problem #1. I never set up my director to do pruning and my postgres database dumps had become 8Gb. So admin