------------------------------------- When replying, please edit your Subject line so it is more specific than "Re: Contents of Bacula-devel digest..."
Today's Topics: 1. Re: AIX 5.3 problems compiling dev.c from stored (Kern Sibbald) 2. Re: scheduler display misbehaviour in 2.5.42 (Kern Sibbald) 3. Re: confused about day-spec (Kern Sibbald) 4. performance regression? (James Harper) 5. Re: performance regression? (Frank Sweetser) ---------------------------------------------------------------------- Message: 1 Date: Sat, 4 Apr 2009 09:56:52 +0200 From: Kern Sibbald <[email protected]> Subject: Re: [Bacula-devel] AIX 5.3 problems compiling dev.c from stored To: [email protected] Cc: "Pugh, Jeffrey" <[email protected]>, Jack Griffin <[email protected]> Message-ID: <[email protected]> Content-Type: text/plain; charset="iso-8859-1" On Monday 30 March 2009 18:38:06 Pugh, Jeffrey wrote: > Hello, > > I've been trying to compile bacula-2.4.4 on AIX 5.3 ML 6. After some > digging, everything appears to compile ok now except stored, erroring > off in dev.c. > The error is : > > dev.c In member function 'void DEVICE::open_tape_device(DCR*, int)': > 347: error: aggregate 'mtop mt_com' has incomplete type and cannot be > defined. > > Followed by a lot of failures that appear related to this. > > I'm using gcc 4.2.4 and gmake. > > Any help, hints, pointers etc. would be much appreciated. It aooks like the AIX definition of mtop is different from what it is on Solaris and Linux systems (not surprising) or more likely, the definition of mtop on AIX is in a different location (we use mtio.h or sys/mtio.h whichever is detected by ./configure). I doubt that anyone has built anything but an AIX Client, so you are probably doing something new. To make Bacula function with tapes requires an OS that has good, mature tape drivers (the case for AIX) but also one that supports the Berkeley tape I/O conventions (BSD) rather than the SysV behavoir. This latter point may be a problem for AIX, which is SysV based if I remember right. On the programming side, unless you want professional support (in which case, please contact Jack Griffin off-list), you are pretty much on your own since we rely on contributions for the AIX client support, though maybe someone on the bacula-users list can help. On the testing side (providing you can build it), the Tape testing chapter of the manual has a lot of good details. Hope the above helps. Best regards, Kern ------------------------------ Hi Kern, Thanks for the response. After a more careful review of the configure output, for some reason it missed picking up mtio.h from include/sys. I made a copy of mtio.h and put it beside bacula.h in the src directory, and added a #include "mtio.h" in dev.c and it successfully compiled. I'll be moving all this stuff to another machine with a tape library already configured, so we'll see what happens from there. I'll keep Jack's email handy, we may need some professional support in the future. Hope some of this info may be of use to someone else in the future. Thanks again, and best regards, Jeff ------------------------------ Message: 2 Date: Sat, 4 Apr 2009 10:58:30 +0200 From: Kern Sibbald <[email protected]> Subject: Re: [Bacula-devel] scheduler display misbehaviour in 2.5.42 To: [email protected] Cc: Arno Lehmann <[email protected]> Message-ID: <[email protected]> Content-Type: text/plain; charset="iso-8859-15" I believe that this behavior is because Bacula has two "schedulers": 1. is the job scheduling function that starts jobs at the scheduled hour (as determined in the Run directive). 2. is the jobq, which is a sort of scheduler, which comes into play after the job is started but before it is run. The jobq -- short for Job queue -- is responsible for allocating resources (tape drives) for the job, rescheduling it if it failed, holding it if the priority is not right, ... Currently the status command only shows jobs that will be scheduled within a period of time (48 hours by default if I remember right) or jobs that are running. Jobs that are being held in the job queue are not listed. Regards, Kern On Sunday 29 March 2009 22:44:38 Arno Lehmann wrote: > Hi, > > 29.03.2009 22:21, Kern Sibbald wrote: > > On Sunday 29 March 2009 21:38:29 Arno Lehmann wrote: > >> Hi, > >> > >> I'm currently testing 2.5.42 (svn as of Mar 24, I believe) and have > >> the following set in the defaults for all jobs: > >> > >> Allow Duplicate Jobs = No > >> Allow Mixed Priority = Yes # doesn't apply as of now... > >> Max Run Sched Time = 3600 # 1 hour > >> Max Run Time = 1800 # 1/2 hour > >> Max Wait Time = 1200 # 20 min > >> > >> All jobs are run every 15 minutes with different levels. > > > > You talk about scheduling problems, but show only a JobDefs. I don't see > > the connection, and thus without a schedule resource and a Job resource, > > I cannot confirm anything. > > Well, as stated, the schedules *work* correctly, they just aren't > displayed. But ok: > > JobDefs { > Name = DefaultDemo > Type = Backup > Level = Incremental > Client = gnom-fd > FileSet = Demo1Set > Storage = File > Messages = Standard > Pool = Default > Priority = 10 > Accurate = Yes > Spool Data = Yes > Spool Attributes = Yes > Allow Duplicate Jobs = No > Allow Mixed Priority = Yes > Max Run Sched Time = 3600 > Max Run Time = 1800 > Max Wait Time = 1200 > } > > Job { > Name = Demo1 > JobDefs = DefaultDemo > Write Bootstrap = /var/demo-bacula/Demo1.bsr > Schedule = Demo > } > > Job { > Name = Demo2 > JobDefs = DefaultDemo > Client = neuelf-fd > FileSet = Demo2Set > Write Bootstrap = /var/demo-bacula/Demo2.bsr > Schedule = Demo > } > > Job { > Name = Demo3 > JobDefs = DefaultDemo > Client = balrog-fd > FileSet = Demo2Set > Write Bootstrap = /var/demo-bacula/Demo3.bsr > Schedule = Demo > } > > Job { > Name = DemoWin > JobDefs = DefaultDemo > Client = phoenix-fd > FileSet = DemoWinSet > Write Bootstrap = /var/demo-bacula/DemoWin.bsr > Schedule = Demo > Accurate = No > Pool = Tier1 > } > > Job { > Name = "BackupCatalog" > JobDefs = "DefaultJob" > Level = Full > FileSet="Catalog" > Schedule = "WeeklyCycleAfterBackup" > RunBeforeJob = "/usr/local/demo-bacula/etc/make_catalog_backup > demobacula demobacula '' neuelf" > RunAfterJob = "/usr/local/demo-bacula/etc/delete_catalog_backup" > Write Bootstrap = "/var/demo-bacula/BackupCatalog.bsr" > Priority = 11 # run after main backup > } > > Job { > Name = "MigratePhoenix" > Pool = Tier1 > Type = Migrate > Selection Type = SQLQuery > Selection Pattern = "SELECT Job.JobId FROM Job,Client,Pool WHERE > (Job.ClientId=Client.ClientId AND Client.Name='phoen > ix-fd' AND EndTime<DATE_SUB(NOW(), interval 1 day) AND > Job.JobStatus='T' AND Job.Type='B' AND Job.Level='F' AND Job.Poo > lId=Pool.PoolId AND Pool.Name='Tier1' AND Job.PriorJobId=0) ORDER BY > Job.EndTime ASC LIMIT 1;" > Messages = Standard > Client = phoenix-fd > FileSet = "Full Set" > Schedule = MigrationDemo > } > > Job { > Name = "CopyDemo3" > Pool = Default > Type = Copy > Selection Type = SQLQuery > Selection Pattern ="SELECT Job.JobId FROM Job,Client,Pool WHERE > (Job.ClientId=Client.ClientId AND Client.Name='balrog > -fd' AND EndTime>DATE_SUB(NOW(), interval 1 day) AND Job.JobStatus='T' > AND Job.Type='B' AND Job.Level='F' AND Job.PoolI > d=Pool.PoolId AND Pool.Name='Default') ORDER BY Job.EndTime DESC LIMIT 1;" > Messages = Standard > Client = balrog-fd > FileSet = "Full Set" > } > > FileSet { > Name = DemoWinSet > Include { > Options { > signature = SHA1 > } > File = "c:/testbackups/" > } > } > > FileSet { > Name = Demo1Set > Include { > Options { > signature = SHA1 > } > File = /usr/local/demo-bacula/stuff > } > } > > FileSet { > Name = Demo2Set > Include { > Options { > signature = SHA1 > } > File = /var/adm/backup > } > } > > FileSet { > Name = "Full Set" > Include { > Options { > signature = MD5 > } > File = / > } > Exclude { > File = /proc > File = /tmp > File = /.journal > File = /.fsck > } > } > > Schedule { > Name = "WeeklyCycle" > Run = Full 1st sun at 23:05 > Run = Differential 2nd-5th sun at 23:05 > Run = Incremental mon-sat at 23:05 > } > > Schedule { > Name = Demo > Run = Level=Full hourly at 0:30 > Run = Level=Differential hourly at 0:00 > Run = Level=Incremental hourly at 0:15 > Run = Level=Incremental hourly at 0:45 > } > > Schedule { > Name = MigrationDemo > Run = daily at 23:25 > Run = daily at 5:23 > Run = daily at 11:23 > Run = daily at 17:23 > } > Schedule { > Name = "WeeklyCycleAfterBackup" > Run = Full sun-sat at 23:10 > } > > FileSet { > Name = "Catalog" > Include { > Options { > signature = MD5 > } > File = /var/demo-bacula/demobacula.sql > } > } > > Client { > Name = gnom-fd > Address = gnom > FDPort = 59001 > Catalog = MyCatalog > Password = "it's" > File Retention = 5 days > Job Retention = 10 days > AutoPrune = yes > } > > Client { > Name = phoenix-fd > Address = phoenix > FDPort = 9102 > Catalog = MyCatalog > Password = "very" > File Retention = 5 days > Job Retention = 10 days > AutoPrune = Yes > } > > Client { > Name = neuelf-fd > Address = neuelf > FDPort = 9102 > Catalog = MyCatalog > Password = "secret" > File Retention = 5 days > Job Retention = 10 days > AutoPrune = Yes > } > > Client { > Name = balrog-fd > Address = balrog > Catalog = MyCatalog > Password = "indeed" > File Retention = 5 days > Job Retention = 10 days > AutoPrune = Yes > } > > Arno > > > Kern > > > >> DIR config excerpts: > >> > >> JobDefs { > >> Name = "DefaultJob" > >> Type = Backup > >> Level = Incremental > >> Client = gnom-fd > >> FileSet = "Full Set" > >> Schedule = "WeeklyCycle" > >> Storage = File > >> Messages = Standard > >> Pool = Default > >> Priority = 10 > >> Allow Duplicate Jobs = No > >> Allow Mixed Priority = Yes > >> Max Run Sched Time = 3600 > >> Max Run Time = 1800 > >> Max Wait Time = 1200 > >> } > >> > >> With these settings, the scheduler queue display is incorrect. See here: > >>> *autom > >>> Turn on or off? off > >>> *time > >>> 29-M?r-2009 21:29:03 > >>> *sta dir > >>> demo-dir Version: 2.5.42-b2 (16 March 2009) i686-pc-linux-gnu suse 11.1 > >>> Daemon started 29-M?r009 21:28, 0 Jobs run since started. > >> > >> DIR freshly started to make sure I didn't mess up anything earlier... > >> > >>> Heap: heap=270,336 smbytes=64,080 max_bytes=64,633 bufs=356 > >>> max_bufs=358 > >>> > >>> Scheduled Jobs: > >>> Level Type Pri Scheduled Name > >>> Volume > >>> ======================================================================= > >>>== ========== > >> > >> No jobs scheduled before the BackupCatalog, i.e. at 21:30, 21:45, > >> 22:00, etc.?! > >> > >>> Full Backup 11 29-M?r009 23:10 BackupCatalog > >>> FIL0537 Migrate 10 29-M?r009 23:25 MigratePhoenix > >>> Differential Backup 10 30-M?r009 00:00 Demo1 > >>> FIL0537 Differential Backup 10 30-M?r009 00:00 DemoWin > >>> FIL0258 Differential Backup 10 30-M?r009 00:00 Demo3 > >>> FIL0537 Differential Backup 10 30-M?r009 00:00 Demo2 > >>> FIL0537 Incremental Backup 10 30-M?r009 00:15 Demo1 > >>> FIL0537 Incremental Backup 10 30-M?r009 00:15 Demo2 > >>> FIL0537 Incremental Backup 10 30-M?r009 00:15 Demo3 > >>> FIL0537 Incremental Backup 10 30-M?r009 00:15 > >>> DemoWin FIL0258 Full Backup 10 30-M?r009 00:30 > >>> Demo1 FIL0537 Full Backup 10 30-M?r009 > >>> 00:30 Demo2 FIL0537 Full Backup 10 > >>> 30-M?r009 00:30 Demo3 FIL0537 Full Backup > >>> 10 30-M?r009 00:30 DemoWin FIL0258 Incremental Backup > >>> 10 30-M?r009 00:45 Demo1 FIL0537 Incremental > >>> Backup 10 30-M?r009 00:45 Demo2 FIL0537 Incremental > >>> Backup 10 30-M?r009 00:45 Demo3 FIL0537 > >>> Incremental Backup 10 30-M?r009 00:45 DemoWin > >>> FIL0258 Migrate 10 30-M?r009 05:23 MigratePhoenix > >>> Migrate 10 30-M?r009 11:23 MigratePhoenix > >>> Migrate 10 30-M?r009 17:23 MigratePhoenix > >>> ==== > >>> > >>> Running Jobs: > >>> Console connected at 29-M?r009 21:28 > >>> Console connected at 29-M?r009 21:28 > >>> No Jobs running. > >>> ==== > >>> > >>> Terminated Jobs: > >>> JobId Level Files Bytes Status Finished Name > >>> ==================================================================== > >>> 2596 Incr 10 255.5 K OK 29-M?r009 21:20 > >>> DemoWin.2009-03-29_17 2595 Full 10 255.5 K OK > >>> 29-M?r009 21:20 MigratePhoenix.2009-03-29_17 2659 Incr 0 > >>> 0 OK 29-M?r009 21:20 Demo3.2009-03-29_21 2658 Incr 0 > >>> 0 OK 29-M?r009 21:20 Demo2.2009-03-29_21 2656 Diff > >>> 0 0 Error 29-M?r009 21:20 DemoWin.2009-03-29_21 2657 > >>> Incr 64 2.447 K OK 29-M?r009 21:20 Demo1.2009-03-29_21 2660 > >>> Incr 0 0 OK 29-M?r009 21:21 > >>> DemoWin.2009-03-29_21 2600 Full 18 262.5 K OK > >>> 29-M?r009 21:21 DemoWin.2009-03-29_17 2604 Incr 0 0 > >>> OK 29-M?r009 21:22 DemoWin.2009-03-29_17 2608 Diff 0 > >>> 0 OK 29-M?r009 21:22 DemoWin.2009-03-29_18 > >>> > >>> ==== > >> > >> Wait a bit... > >> > >>> *time > >>> 29-M?r-2009 21:30:40 > >>> You have messages. > >>> *sta dir > >>> demo-dir Version: 2.5.42-b2 (16 March 2009) i686-pc-linux-gnu suse 11.1 > >>> Daemon started 29-M?r009 21:28, 1 Job run since started. > >>> Heap: heap=389,120 smbytes=113,893 max_bytes=124,942 bufs=514 > >>> max_bufs=549 > >>> > >>> Scheduled Jobs: > >>> Level Type Pri Scheduled Name > >>> Volume > >>> ======================================================================= > >>>== ========== > >> > >> Still nothing new scheduled. > >> > >>> Full Backup 11 29-M?r009 23:10 BackupCatalog > >>> FIL0537 Migrate 10 29-M?r009 23:25 MigratePhoenix > >>> Differential Backup 10 30-M?r009 00:00 Demo1 > >>> FIL0537 Differential Backup 10 30-M?r009 00:00 DemoWin > >>> FIL0258 Differential Backup 10 30-M?r009 00:00 Demo3 > >>> FIL0537 Differential Backup 10 30-M?r009 00:00 Demo2 > >>> FIL0537 Incremental Backup 10 30-M?r009 00:15 Demo1 > >>> FIL0537 Incremental Backup 10 30-M?r009 00:15 Demo2 > >>> FIL0537 Incremental Backup 10 30-M?r009 00:15 Demo3 > >>> FIL0537 Incremental Backup 10 30-M?r009 00:15 > >>> DemoWin FIL0258 Full Backup 10 30-M?r009 00:30 > >>> Demo1 FIL0537 Full Backup 10 30-M?r009 > >>> 00:30 Demo2 FIL0537 Full Backup 10 > >>> 30-M?r009 00:30 Demo3 FIL0537 Full Backup > >>> 10 30-M?r009 00:30 DemoWin FIL0258 Incremental Backup > >>> 10 30-M?r009 00:45 Demo1 FIL0537 Incremental > >>> Backup 10 30-M?r009 00:45 Demo2 FIL0537 Incremental > >>> Backup 10 30-M?r009 00:45 Demo3 FIL0537 > >>> Incremental Backup 10 30-M?r009 00:45 DemoWin > >>> FIL0258 Migrate 10 30-M?r009 05:23 MigratePhoenix > >>> Migrate 10 30-M?r009 11:23 MigratePhoenix > >>> Migrate 10 30-M?r009 17:23 MigratePhoenix > >>> ==== > >> > >> But: > >>> Running Jobs: > >>> Console connected at 29-M?r009 21:28 > >>> Console connected at 29-M?r009 21:28 > >>> JobId Level Name Status > >>> ====================================================================== > >>> 2662 Full Demo2.2009-03-29_21.30.00.05 is running > >>> 2663 Full Demo3.2009-03-29_21.30.00.06 is running > >>> 2664 Full DemoWin.2009-03-29_21.30.00.07 is waiting on Storage > >>> File ==== > >>> > >>> Terminated Jobs: > >>> JobId Level Files Bytes Status Finished Name > >>> ==================================================================== > >>> 2595 Full 10 255.5 K OK 29-M?r009 21:20 > >>> MigratePhoenix.2009-03-29_17 2659 Incr 0 0 OK > >>> 29-M?r009 21:20 Demo3.2009-03-29_21 2658 Incr 0 0 > >>> OK 29-M?r009 21:20 Demo2.2009-03-29_21 2656 Diff 0 0 > >>> Error 29-M?r009 21:20 DemoWin.2009-03-29_21 2657 Incr 64 > >>> 2.447 K OK 29-M?r009 21:20 Demo1.2009-03-29_21 2660 Incr 0 > >>> 0 OK 29-M?r009 21:21 DemoWin.2009-03-29_21 2600 Full 18 > >>> 262.5 K OK 29-M?r009 21:21 DemoWin.2009-03-29_17 2604 Incr > >>> 0 0 OK 29-M?r009 21:22 > >>> DemoWin.2009-03-29_17 2608 Diff 0 0 OK > >>> 29-M?r009 21:22 DemoWin.2009-03-29_18 2661 Full 1,017 65.49 K > >>> OK 29-M?r009 21:30 Demo1.2009-03-29_21 > >> > >> And this one is already finished... > >> > >> And I did not start any jobs manually, by the way :-) > >> > >> So, for some reason, something breaks the scheduler display, but the > >> scheduler itself works correctly. > >> > >> This might be related to the above mentioned settings as I did not > >> observe this behaviour before I inserted that to the job defaults. > >> > >> Can anybody confirm this? > >> > >> Arno > >> > >>> ==== > >>> *quit ------------------------------ Message: 3 Date: Sat, 4 Apr 2009 11:02:45 +0200 From: Kern Sibbald <[email protected]> Subject: Re: [Bacula-devel] confused about day-spec To: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset="iso-8859-1" On Monday 30 March 2009 16:01:06 Ulrich Leodolter wrote: > On Mon, 2009-03-30 at 14:49 +0100, Kern Sibbald wrote: > > On Monday 30 March 2009 15:25:27 Ulrich Leodolter wrote: > > > Hello, > > > > > > http://www.bacula.org/manuals/en/install/install/Configuring_Director.h > > >tml# SECTION00850000000000000000 > > > > > > ... > > > <day-spec> = <day> | <wday-keyword> | > > > <day-range> | <wday-range> | > > > <daily-keyword> > > > <day-spec> = <day> | <wday-keyword> | > > > <day> | <wday-range> | > > > <week-keyword> <wday-keyword> | > > > <week-keyword> <wday-range> > > > > > > Which one is the right one? > > > > They are both correct. Neither is totally complete by itself. > > Ok, but can't they merged into one? Yes, it just requires identifying the problem then finding someone to take the time to fix it. Done. Kern ------------------------------ Message: 4 Date: Sun, 5 Apr 2009 10:59:18 +1000 From: "James Harper" <[email protected]> Subject: [Bacula-devel] performance regression? To: <[email protected]> Message-ID: <aec6c66638c05b468b556ea548c1a77d0162c...@trantor> Content-Type: text/plain; charset="US-ASCII" Has anyone else noticed a performance regression in the last few weeks with the latest 2.5.x code? My backups are taking longer than they used to but I'm not sure what the cause is, it may well be completely not related to bacula... Thanks James ------------------------------ Message: 5 Date: Sat, 04 Apr 2009 21:39:07 -0400 From: Frank Sweetser <[email protected]> Subject: Re: [Bacula-devel] performance regression? To: James Harper <[email protected]> Cc: [email protected] Message-ID: <[email protected]> Content-Type: text/plain; charset=ISO-8859-1 James Harper wrote: > Has anyone else noticed a performance regression in the last few weeks > with the latest 2.5.x code? My backups are taking longer than they used > to but I'm not sure what the cause is, it may well be completely not > related to bacula... I didn't notice it before (all my tests run unattended), but I can see a recent jump in the test results on regress.bacula.org. On April 30, the 2.5.44 mysql test run on my test system took 57 minutes to complete: http://regress.bacula.org/index.php?project=bacula&date=2009-03-30 But on the 31st, it took 87 minutes: http://regress.bacula.org/index.php?project=bacula&date=2009-03-31 Does this look consistent with the jump in time you're seeing? -- Frank Sweetser fs at wpi.edu | For every problem, there is a solution that WPI Senior Network Engineer | is simple, elegant, and wrong. - HL Mencken GPG fingerprint = 6174 1257 129E 0D21 D8D4 E8A3 8E39 29E3 E2E8 8CEC ------------------------------ ------------------------------------------------------------------------ ------ ------------------------------ _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel End of Bacula-devel Digest, Vol 36, Issue 3 ******************************************* ------------------------------------------------------------------------------ _______________________________________________ Bacula-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/bacula-devel
