Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
I've already set both of these options... Last night i had a weird thing where instead of using the wednesday tape in the drive, it asked for the previous FRIDAY's tape which was still in append mode. A simple unmount/remount fixed it and the backup started again.. after waiting for user intervention for 12 hours. I've been getting quite a few failed backups because it seems so unpredictable! On fridays, I actually delete volumes out of the catalog, then re-label them, as this is the only way i've found works every time! Martin Simmons [EMAIL PROTECTED] 03/01/2006 18:29 On Tue, 03 Jan 2006 11:15:21 +, Beren Gamble [EMAIL PROTECTED] said: Beren I'm convinced that there's no solution in the current build. I think Beren there needs to be a new setting implemented which forces Bacula to Beren use the volume in the drive IF it is outside of the retention period. FWIW, it used to be options like Recycle Current Volume = yes Accept Any Volume = yes that made that work. __Martin Martin Simmons [EMAIL PROTECTED] 30/12/2005 23:11 On Fri, 30 Dec 2005 16:23:38 +0100 (CET), Gabriele Bulfon [EMAIL PROTECTED] said: Gabriele At last, I think I discovered the basic problem behind the Automatic Recycle/Purge/Prune so many people are facing (me included). Gabriele I decided to start from a scratch situation, with the latest configuration I came up with, using all the suggestions I received in this list. Gabriele As a first step, I just used bconsole to purge all my 7 weekly volumes, assuming that any previous job would have been deleted and removed from the database. Gabriele With a lot of surprise, I discovered that some jobs were still in the bconsole list. Don't know why. Gabriele So I decided to delete them by hand (delete job). Gabriele Finally I had a scratch situationI thought... Gabriele But I wanted to be sure. Gabriele So I stopped bacula daemons, ran psql on bacula database (I am using Postgres) and issued Gabriele some sql statements to examine the database status: Gabriele - a select count(*) from job was never returning(or probably returning after a long time) Gabriele - a select count(*) from file was never returining. Gabriele (or probably returning after an even longer time) Gabriele So I supposed that both tables should have been very very large and still containing hidden data! Gabriele Now, to get my hands on its data, I used pg_dump to dump all the bacula db on a flat file and see its content. Gabriele ..many minutes of dump300Mb of flat file was produced. Gabriele ...wow.and there, I found a lot of data inside the job and file tables. Gabriele So, I decided to drop the whole bacula db and recreate it. Gabriele After doing this, my latest configuration files work perfectly! Gabriele So, my last suggestion to anyone going crazy with their weekly backup, is to start from scratch Gabriele any time you need to change the bacula-dir.conf file, then add the pre-labaled volumes by hand. Gabriele Bacula won't prune or recycle some of your tapes, because old jobs are probably still in the db Gabriele in some hidden status. Gabriele I hope somone can clarify why this situation occurs. Beren Junk in the File and Path tables is OK, because the entries are never removed Beren and shouldn't affect recycling. Beren Junk in the Jobs table is more worrying though. However, some types of job Beren (e.g. Admin and Verify) are not directly associated with any media, so they Beren don't get pruned and shouldn't effect recycling either. Beren I think the JobMedia table is the one that affects Recycle/Purge/Prune most. Beren __Martin Beren --- Beren This SF.net email is sponsored by: Splunk Inc. Do you grep through log files Beren for problems? Stop! Download the new AJAX search engine that makes Beren searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! Beren http://ads.osdn.com/?ad_id=7637alloc_id=16865op=click Beren ___ Beren Bacula-users mailing list Beren Bacula-users@lists.sourceforge.net Beren https://lists.sourceforge.net/lists/listinfo/bacula-users Beren *** Beren Mail FROM London Borough of Harrow: Beren Unencrypted electronic mail is not secure and may not be authentic, in whole or in part. You are advised to check directly with the sender before acting upon any e-mail received. Beren The information contained in this message and any attachments is confidential and is intended for receipt by the above named addressee(s) only. If you have otherwise encountered this message please notify its originator via +44(0)20 8863 5611 at LONDON BOROUGH OF HARROW. The unauthorised use, disclosure, copying or alteration of this message
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
Hello, On 12/30/2005 4:23 PM, Gabriele Bulfon wrote: At last, I think I discovered the basic problem behind the Automatic Recycle/Purge/Prune so many people are facing (me included). I decided to start from a scratch situation, with the latest configuration I came up with, using all the suggestions I received in this list. As a first step, I just used bconsole to purge all my 7 weekly volumes, assuming that any previous job would have been deleted and removed from the database. With a lot of surprise, I discovered that some jobs were still in the bconsole list. Don't know why. Probably in status storage output... that data is stored by the SD (and FD, respectively) itself and has no equivalent in the catalog. For example, if you have an FD which is used by two DIRs, the status from that FD will always show the last jobs run via *both* DIRs. So I decided to delete them by hand (delete job). Finally I had a scratch situationI thought... But I wanted to be sure. So I stopped bacula daemons, ran psql on bacula database (I am using Postgres) and issued some sql statements to examine the database status: - a select count(*) from job was never returning(or probably returning after a long time) - a select count(*) from file was never returining. (or probably returning after an even longer time) So I supposed that both tables should have been very very large and still containing hidden data! Now, to get my hands on its data, I used pg_dump to dump all the bacula db on a flat file and see its content. ..many minutes of dump300Mb of flat file was produced. ...wow.and there, I found a lot of data inside the job and file tables. So, I decided to drop the whole bacula db and recreate it. After doing this, my latest configuration files work perfectly! Which makes me more sure than ever that even before, the problem wasn't in your configuration, but in your catalog data. I suspect that your original volume, fileset and client attributes weren't updated in the catalog after you changed the configuration files. Anyway, the remaining data in file table should have no impact on your setup because it simply stores a list of all file system names your Bacula installation encountered. The job table shouldn't matter in your case, too, although I'm not 100% sure there. So, my last suggestion to anyone going crazy with their weekly backup, is to start from scratch any time you need to change the bacula-dir.conf file, then add the pre-labaled volumes by hand. Bacula won't prune or recycle some of your tapes, because old jobs are probably still in the db in some hidden status. No, I don't think so. First, there's no such thing as a hidden status in *my* catalogs. And I also only use the supplied scripts to create the databases... second, once you purged all volumes, there are no job records pointing to the media in question left - that's what purging does. Until now, I never encountered a misbehaviour on Bacula concerning catalog management, and I'd really like to see some proof for such a thing. A database dump after the deletion and purging of all jobs or media would be very interesting, for example - if there are really some remains of jobs, relating to volumes that should be pruned by then, that's a bug that should be investigated in more depth. I hope somone can clarify why this situation occurs. I can't explain what you encountered, but I assume that, what you describe above, doesn't completely explain the diferent behaviour regarding recycling. Rather, I suspect that you fixed some volume, job or client attributes the hard way :-) Arno Thanks a lot. Gabriele. http://www.sonicle.com Gabriele Bulfon - Sonicle S.r.l. Tel +39 028246016 Int. 30 - Fax +39 028243880 Via Felice Cavallotti 16 - 20089, Rozzano - Milano - ITALY http://www.sonicle.com -- Da: Arno Lehmann [EMAIL PROTECTED] A: bacula-users@lists.sourceforge.net Data: 28 dicembre 2005 21.04.50 CET Oggetto: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Hello, On 12/28/2005 6:11 PM, Beren Gamble wrote: Hi Arno, Sorry I didn't reply to the list... That's me on autopilot :D - Hope you had a Merry Xmas! No problem... and thanks, I did have some nice days. I have it working perfectly, EXCEPT the only problem i've now got, is when Bacula looks for volumes to use, it chooses Recycled volumes over Purged volumes. In the hope that i'd be able to force Bacula to use the tape in my drive, I've manually set all of my volumes to Used.. even the ones that have never been used, but some of the volumes have never had a successful backup to them, so bacula sees that no jobs are associated with a volume, so it marks it Recycled, then of course, it chooses the Recycled volume to backup to. During this process
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
tmas days, partly due to you writing to me personally :-) anyway... On 12/23/2005 10:04 AM, Beren Gamble wrote: Arno Lehmann Writes- Well, only to repeat myself - it's not that Bacula is designed to overwrite valid data, instead it tries to protect that data. I fully appreciate this, but in cases where we have single tape units, I need a setting which overrides this and uses the tape that I supply. After all, if I put a tape in which is outside of the retention period, a retention period which "I" have set, I want it to be overwritten.. I don't want the data preserved. And it doesn't seem possible to do this wich Bacula. Hmm. I admit I still don't understand why anyone would want to overwrite a perfect backup volume, but using the right settings for recycling, maximum volume use duration and volume retention should allow what you want. Keep in mind that I haven't tried this, but allowing, for example 12 hour of volume use time and setting a volume retention of 11 hours should have the result you want: If, after, one day, the volume is still loaded it can be recycled, and thus re-used. Hope this helps, and I guess I'll need to start some experiments when I've got more time available... Arno Arno Lehmann [EMAIL PROTECTED] 23/12/2005 08:14 Hello, On 12/23/2005 2:19 AM, Doug Sampson wrote: FWIW. My experience with using Bacula on tape drives is similar to Beren's. When used on HD server, Bacula runs like a dream. But when used with a server using tape drives, I am seeing Bacula wanting to append to volumes that aren't full when I've specificaly requested that Bacula reuse the tape currently in the tape drive. Bacula doesn't ask for a different tape but appends to the tape inserted. Well, only to repeat myself - it's not that Bacula is designed to overwrite valid data, instead it tries to protect that data. I have been following this thread and making adjustments to the bacula-dir.conf file as I see fit. I've inserted the 'Volume Use Duration' parameter and am not seeing any changes that I expect. Volumes marked "Append" are still marked such as even after the Volume Use Duration time parameter has passed. I assume you didn't see the following sentence in the manual, Director configuration, in the section dealing with Volume Use Duration. "The use duration is checked and the Used status is set only at the end of a job that writes to the particular volume, which means that even though the use duration may have expired, the catalog entry will not be updated until the next job that uses this volume is run." Currently running 1.38.2. Currenly not running 1.38.3 :-( Arno ~Doug -Original Message- From: Beren Gamble [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 21, 2005 08:34 AM To: bacula-users@lists.sourceforge.net; [EMAIL PROTECTED] Subject: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Ok, maybe i'm not picking up a concept here in the documentation I've now got my Volume Use Duration at 23h, which works beautifully, marking the volume as Used. However, I just tried to run a backup job. All of my volumes in the pool were set as Used. (If VolUsen didn't mark them used, I did, just for uniformities' sake) But bacula then found a volume which had no jobs associated with it and decided to recycle that one instead of the volume that was in the drive (it definitely is past the 12 day retention period). Bacula is an absolute dream with my autochanger because tapes are only either marked as Full or, Purged or Recycle and I never remove a half used tape. But on the single tape systems, I still haven't figured out how to make bacula cope with rotating half filled volumes.. or volumes where all backups failed and no data was written. Kern Sibbald [EMAIL PROTECTED] 17/12/2005 10:56:45 I think the basic problem that you have had, and it seems like you don't quite understand though you have found a solution, is that recycling for Bacula requires applying two concepts: 1. the time Bacula can write on a Volume, 2. the time Bacula keeps that Volume before reusing it. Most people come with the pre-conceived idea that there is one "Recycle" time, when in fact Bacula is significantly more sophisticated than that. I've reviewed the chapter you refer to below, and if one reads the full chapter, all the information *is* there (at least IMO). I have added additional text that stresses the above two concept points a bit more as well as refer the user to the "Basic Volume Management" chapter that gives a detailed practical example of why you want to do. The Recycle chapter is complete in itself assuming you want to fill all your Volumes. If not, you must also read the Basic Volume Management chapter as well. Concerning your comment: "Can someone please update the documentation here http://www.bacula.org/rel-ma
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
On Fri, 30 Dec 2005 16:23:38 +0100 (CET), Gabriele Bulfon [EMAIL PROTECTED] said: Gabriele At last, I think I discovered the basic problem behind the Automatic Recycle/Purge/Prune so many people are facing (me included). Gabriele I decided to start from a scratch situation, with the latest configuration I came up with, using all the suggestions I received in this list. Gabriele As a first step, I just used bconsole to purge all my 7 weekly volumes, assuming that any previous job would have been deleted and removed from the database. Gabriele With a lot of surprise, I discovered that some jobs were still in the bconsole list. Don't know why. Gabriele So I decided to delete them by hand (delete job). Gabriele Finally I had a scratch situationI thought... Gabriele But I wanted to be sure. Gabriele So I stopped bacula daemons, ran psql on bacula database (I am using Postgres) and issued Gabriele some sql statements to examine the database status: Gabriele - a select count(*) from job was never returning(or probably returning after a long time) Gabriele - a select count(*) from file was never returining. Gabriele (or probably returning after an even longer time) Gabriele So I supposed that both tables should have been very very large and still containing hidden data! Gabriele Now, to get my hands on its data, I used pg_dump to dump all the bacula db on a flat file and see its content. Gabriele ..many minutes of dump300Mb of flat file was produced. Gabriele ...wow.and there, I found a lot of data inside the job and file tables. Gabriele So, I decided to drop the whole bacula db and recreate it. Gabriele After doing this, my latest configuration files work perfectly! Gabriele So, my last suggestion to anyone going crazy with their weekly backup, is to start from scratch Gabriele any time you need to change the bacula-dir.conf file, then add the pre-labaled volumes by hand. Gabriele Bacula won't prune or recycle some of your tapes, because old jobs are probably still in the db Gabriele in some hidden status. Gabriele I hope somone can clarify why this situation occurs. Junk in the File and Path tables is OK, because the entries are never removed and shouldn't affect recycling. Junk in the Jobs table is more worrying though. However, some types of job (e.g. Admin and Verify) are not directly associated with any media, so they don't get pruned and shouldn't effect recycling either. I think the JobMedia table is the one that affects Recycle/Purge/Prune most. __Martin --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637alloc_id=16865op=click ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
Hi Arno, Sorry I didn't reply to the list... That's me on autopilot :D - Hope you had a Merry Xmas! I have it working perfectly, EXCEPT the only problem i've now got, is when Bacula looks for volumes to use, it chooses Recycled volumes over Purged volumes. In the hope that i'd be able to force Bacula to use the tape in my drive, I've manually set all of my volumes to Used.. even the ones that have never been used, but some of the volumes have never had a successful backup to them, so bacula sees that no jobs are associated with a volume, so it marks it Recycled, then of course, it chooses the Recycled volume to backup to. During this process, Bacula marks the volume in my drive as Purged, as it should, but it ignores it. The empty volumes are the stick in the mud here. The reason there are empty volumes is because they may have failed any time within the last 2 weeks (we run a 2 week cycle). Thanks, Beren Arno Lehmann [EMAIL PROTECTED] 27/12/2005 22:17 Hello, it took me a while to answer, partly due to the christmas days, partly due to you writing to me personally :-) anyway... On 12/23/2005 10:04 AM, Beren Gamble wrote: Arno Lehmann Writes- Well, only to repeat myself - it's not that Bacula is designed to overwrite valid data, instead it tries to protect that data. I fully appreciate this, but in cases where we have single tape units, I need a setting which overrides this and uses the tape that I supply. After all, if I put a tape in which is outside of the retention period, a retention period which I have set, I want it to be overwritten.. I don't want the data preserved. And it doesn't seem possible to do this wich Bacula. Hmm. I admit I still don't understand why anyone would want to overwrite a perfect backup volume, but using the right settings for recycling, maximum volume use duration and volume retention should allow what you want. Keep in mind that I haven't tried this, but allowing, for example 12 hour of volume use time and setting a volume retention of 11 hours should have the result you want: If, after, one day, the volume is still loaded it can be recycled, and thus re-used. Hope this helps, and I guess I'll need to start some experiments when I've got more time available... Arno Arno Lehmann [EMAIL PROTECTED] 23/12/2005 08:14 Hello, On 12/23/2005 2:19 AM, Doug Sampson wrote: FWIW. My experience with using Bacula on tape drives is similar to Beren's. When used on HD server, Bacula runs like a dream. But when used with a server using tape drives, I am seeing Bacula wanting to append to volumes that aren't full when I've specificaly requested that Bacula reuse the tape currently in the tape drive. Bacula doesn't ask for a different tape but appends to the tape inserted. Well, only to repeat myself - it's not that Bacula is designed to overwrite valid data, instead it tries to protect that data. I have been following this thread and making adjustments to the bacula-dir.conf file as I see fit. I've inserted the 'Volume Use Duration' parameter and am not seeing any changes that I expect. Volumes marked Append are still marked such as even after the Volume Use Duration time parameter has passed. I assume you didn't see the following sentence in the manual, Director configuration, in the section dealing with Volume Use Duration. The use duration is checked and the Used status is set only at the end of a job that writes to the particular volume, which means that even though the use duration may have expired, the catalog entry will not be updated until the next job that uses this volume is run. Currently running 1.38.2. Currenly not running 1.38.3 :-( Arno ~Doug -Original Message- From: Beren Gamble [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 21, 2005 08:34 AM To: bacula-users@lists.sourceforge.net; [EMAIL PROTECTED] Subject: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Ok, maybe i'm not picking up a concept here in the documentation I've now got my Volume Use Duration at 23h, which works beautifully, marking the volume as Used. However, I just tried to run a backup job. All of my volumes in the pool were set as Used. (If VolUsen didn't mark them used, I did, just for uniformities' sake) But bacula then found a volume which had no jobs associated with it and decided to recycle that one instead of the volume that was in the drive (it definitely is past the 12 day retention period). Bacula is an absolute dream with my autochanger because tapes are only either marked as Full or, Purged or Recycle and I never remove a half used tape. But on the single tape systems, I still haven't figured out how to make bacula cope with rotating half filled volumes.. or volumes where all backups failed and no data was written. Kern Sibbald [EMAIL PROTECTED] 17/12/2005 10:56:45 I think the basic problem that you have had
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
with Volume Use Duration. The use duration is checked and the Used status is set only at the end of a job that writes to the particular volume, which means that even though the use duration may have expired, the catalog entry will not be updated until the next job that uses this volume is run. Currently running 1.38.2. Currenly not running 1.38.3 :-( Arno ~Doug -Original Message- From: Beren Gamble [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 21, 2005 08:34 AM To: bacula-users@lists.sourceforge.net; [EMAIL PROTECTED] Subject: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Ok, maybe i'm not picking up a concept here in the documentation I've now got my Volume Use Duration at 23h, which works beautifully, marking the volume as Used. However, I just tried to run a backup job. All of my volumes in the pool were set as Used. (If VolUsen didn't mark them used, I did, just for uniformities' sake) But bacula then found a volume which had no jobs associated with it and decided to recycle that one instead of the volume that was in the drive (it definitely is past the 12 day retention period). Bacula is an absolute dream with my autochanger because tapes are only either marked as Full or, Purged or Recycle and I never remove a half used tape. But on the single tape systems, I still haven't figured out how to make bacula cope with rotating half filled volumes.. or volumes where all backups failed and no data was written. Kern Sibbald [EMAIL PROTECTED] 17/12/2005 10:56:45 I think the basic problem that you have had, and it seems like you don't quite understand though you have found a solution, is that recycling for Bacula requires applying two concepts: 1. the time Bacula can write on a Volume, 2. the time Bacula keeps that Volume before reusing it. Most people come with the pre-conceived idea that there is one Recycle time, when in fact Bacula is significantly more sophisticated than that. I've reviewed the chapter you refer to below, and if one reads the full chapter, all the information *is* there (at least IMO). I have added additional text that stresses the above two concept points a bit more as well as refer the user to the Basic Volume Management chapter that gives a detailed practical example of why you want to do. The Recycle chapter is complete in itself assuming you want to fill all your Volumes. If not, you must also read the Basic Volume Management chapter as well. Concerning your comment: Can someone please update the documentation here http://www.bacula.org/rel-manual/Automatic_Volume_Recycling.html with this information, that should hopefully forestall this question coming up so much. I appreciate you pointing out where you got confused. However, this is a free software project, and I understand recycling, so I don't easily see why some people don't get the concepts or I would have already changed the doc. As a consequence it is much more helpful if you would submit a suggested change to the documentation that clarifies it for you. I suggest you look at the documentation in: http://www.bacula.org/dev-manual/Automatic_Volume_Recycling.html It is usually much more up to date. On Saturday 17 December 2005 07:00, Harondel J. Sibble wrote: On 16 Dec 2005 at 1:25, Arno Lehmann wrote: sohow does one force it to do it?! Apart from the configuration directives like prune oldest volume (or something) my advice is usually you don't. Philisophical disagreement, If I as the admin decide that, that is an appropriate thing to do, I should be able to do it in some manner. Reminds me of the unix philosophy, build a bunch of single purpose tools that you can chain together to do anything you want . Anyhow, on the how to do it, here's what I got from an unrelated list where I was ranting about the same problem the OP has as I had it for months also. What you need is this Volume Use Duration = added to your pool definition where the duration is equal to or less than your Volume Retention setting, in my case the relevant lines are Volume Retention = 6d Volume Use Duration = 6d With this setup, Bacula FINALLY works like I expect and how pretty much any other backup software I've used on Windows or Unix works. It will overwrite the tape in the drive, assuming 6 days has passed, it marks the tape as used and then recycles it, however it only does that at the beginning of the next job, so the recycling happens at the start of the current job. Can someone please update the documentation here http://www.bacula.org/rel-manual/Automatic_Volume_Recycling.html with this information, that should hopefully forestall this question coming up so much. From the messages on tonights job 16-Dec 21:30 fileserver-dir: Start Backup JobId 70, Job=Company-Fileserver_Full_Back 16-Dec 21:30
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
Hello, it took me a while to answer, partly due to the christmas days, partly due to you writing to me personally :-) anyway... On 12/23/2005 10:04 AM, Beren Gamble wrote: Arno Lehmann Writes- Well, only to repeat myself - it's not that Bacula is designed to overwrite valid data, instead it tries to protect that data. I fully appreciate this, but in cases where we have single tape units, I need a setting which overrides this and uses the tape that I supply. After all, if I put a tape in which is outside of the retention period, a retention period which I have set, I want it to be overwritten.. I don't want the data preserved. And it doesn't seem possible to do this wich Bacula. Hmm. I admit I still don't understand why anyone would want to overwrite a perfect backup volume, but using the right settings for recycling, maximum volume use duration and volume retention should allow what you want. Keep in mind that I haven't tried this, but allowing, for example 12 hour of volume use time and setting a volume retention of 11 hours should have the result you want: If, after, one day, the volume is still loaded it can be recycled, and thus re-used. Hope this helps, and I guess I'll need to start some experiments when I've got more time available... Arno Arno Lehmann [EMAIL PROTECTED] 23/12/2005 08:14 Hello, On 12/23/2005 2:19 AM, Doug Sampson wrote: FWIW. My experience with using Bacula on tape drives is similar to Beren's. When used on HD server, Bacula runs like a dream. But when used with a server using tape drives, I am seeing Bacula wanting to append to volumes that aren't full when I've specificaly requested that Bacula reuse the tape currently in the tape drive. Bacula doesn't ask for a different tape but appends to the tape inserted. Well, only to repeat myself - it's not that Bacula is designed to overwrite valid data, instead it tries to protect that data. I have been following this thread and making adjustments to the bacula-dir.conf file as I see fit. I've inserted the 'Volume Use Duration' parameter and am not seeing any changes that I expect. Volumes marked Append are still marked such as even after the Volume Use Duration time parameter has passed. I assume you didn't see the following sentence in the manual, Director configuration, in the section dealing with Volume Use Duration. The use duration is checked and the Used status is set only at the end of a job that writes to the particular volume, which means that even though the use duration may have expired, the catalog entry will not be updated until the next job that uses this volume is run. Currently running 1.38.2. Currenly not running 1.38.3 :-( Arno ~Doug -Original Message- From: Beren Gamble [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 21, 2005 08:34 AM To: bacula-users@lists.sourceforge.net; [EMAIL PROTECTED] Subject: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Ok, maybe i'm not picking up a concept here in the documentation I've now got my Volume Use Duration at 23h, which works beautifully, marking the volume as Used. However, I just tried to run a backup job. All of my volumes in the pool were set as Used. (If VolUsen didn't mark them used, I did, just for uniformities' sake) But bacula then found a volume which had no jobs associated with it and decided to recycle that one instead of the volume that was in the drive (it definitely is past the 12 day retention period). Bacula is an absolute dream with my autochanger because tapes are only either marked as Full or, Purged or Recycle and I never remove a half used tape. But on the single tape systems, I still haven't figured out how to make bacula cope with rotating half filled volumes.. or volumes where all backups failed and no data was written. Kern Sibbald [EMAIL PROTECTED] 17/12/2005 10:56:45 I think the basic problem that you have had, and it seems like you don't quite understand though you have found a solution, is that recycling for Bacula requires applying two concepts: 1. the time Bacula can write on a Volume, 2. the time Bacula keeps that Volume before reusing it. Most people come with the pre-conceived idea that there is one Recycle time, when in fact Bacula is significantly more sophisticated than that. I've reviewed the chapter you refer to below, and if one reads the full chapter, all the information *is* there (at least IMO). I have added additional text that stresses the above two concept points a bit more as well as refer the user to the Basic Volume Management chapter that gives a detailed practical example of why you want to do. The Recycle chapter is complete in itself assuming you want to fill all your Volumes. If not, you must also read the Basic Volume Management chapter as well. Concerning your comment: Can someone please update the documentation here http://www.bacula.org/rel
RE: [Bacula-users] Puring/Pruning doesn't seem to be happening.
FWIW. My experience with using Bacula on tape drives is similar to Beren's. When used on HD server, Bacula runs like a dream. But when used with a server using tape drives, I am seeing Bacula wanting to append to volumes that aren't full when I've specificaly requested that Bacula reuse the tape currently in the tape drive. Bacula doesn't ask for a different tape but appends to the tape inserted. I have been following this thread and making adjustments to the bacula-dir.conf file as I see fit. I've inserted the 'Volume Use Duration' parameter and am not seeing any changes that I expect. Volumes marked Append are still marked such as even after the Volume Use Duration time parameter has passed. Currently running 1.38.2. ~Doug -Original Message- From: Beren Gamble [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 21, 2005 08:34 AM To: bacula-users@lists.sourceforge.net; [EMAIL PROTECTED] Subject: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Ok, maybe i'm not picking up a concept here in the documentation I've now got my Volume Use Duration at 23h, which works beautifully, marking the volume as Used. However, I just tried to run a backup job. All of my volumes in the pool were set as Used. (If VolUsen didn't mark them used, I did, just for uniformities' sake) But bacula then found a volume which had no jobs associated with it and decided to recycle that one instead of the volume that was in the drive (it definitely is past the 12 day retention period). Bacula is an absolute dream with my autochanger because tapes are only either marked as Full or, Purged or Recycle and I never remove a half used tape. But on the single tape systems, I still haven't figured out how to make bacula cope with rotating half filled volumes.. or volumes where all backups failed and no data was written. Kern Sibbald [EMAIL PROTECTED] 17/12/2005 10:56:45 I think the basic problem that you have had, and it seems like you don't quite understand though you have found a solution, is that recycling for Bacula requires applying two concepts: 1. the time Bacula can write on a Volume, 2. the time Bacula keeps that Volume before reusing it. Most people come with the pre-conceived idea that there is one Recycle time, when in fact Bacula is significantly more sophisticated than that. I've reviewed the chapter you refer to below, and if one reads the full chapter, all the information *is* there (at least IMO). I have added additional text that stresses the above two concept points a bit more as well as refer the user to the Basic Volume Management chapter that gives a detailed practical example of why you want to do. The Recycle chapter is complete in itself assuming you want to fill all your Volumes. If not, you must also read the Basic Volume Management chapter as well. Concerning your comment: Can someone please update the documentation here http://www.bacula.org/rel-manual/Automatic_Volume_Recycling.html with this information, that should hopefully forestall this question coming up so much. I appreciate you pointing out where you got confused. However, this is a free software project, and I understand recycling, so I don't easily see why some people don't get the concepts or I would have already changed the doc. As a consequence it is much more helpful if you would submit a suggested change to the documentation that clarifies it for you. I suggest you look at the documentation in: http://www.bacula.org/dev-manual/Automatic_Volume_Recycling.html It is usually much more up to date. On Saturday 17 December 2005 07:00, Harondel J. Sibble wrote: On 16 Dec 2005 at 1:25, Arno Lehmann wrote: sohow does one force it to do it?! Apart from the configuration directives like prune oldest volume (or something) my advice is usually you don't. Philisophical disagreement, If I as the admin decide that, that is an appropriate thing to do, I should be able to do it in some manner. Reminds me of the unix philosophy, build a bunch of single purpose tools that you can chain together to do anything you want . Anyhow, on the how to do it, here's what I got from an unrelated list where I was ranting about the same problem the OP has as I had it for months also. What you need is this Volume Use Duration = added to your pool definition where the duration is equal to or less than your Volume Retention setting, in my case the relevant lines are Volume Retention = 6d Volume Use Duration = 6d With this setup, Bacula FINALLY works like I expect and how pretty much any other backup software I've used on Windows or Unix works. It will overwrite the tape in the drive, assuming 6 days has passed, it marks the tape as used and then recycles it, however it only does
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
Ok, maybe i'm not picking up a concept here in the documentation I've now got my Volume Use Duration at 23h, which works beautifully, marking the volume as Used. However, I just tried to run a backup job. All of my volumes in the pool were set as Used. (If VolUsen didn't mark them used, I did, just for uniformities' sake) But bacula then found a volume which had no jobs associated with it and decided to recycle that one instead of the volume that was in the drive (it definitely is past the 12 day retention period). Bacula is an absolute dream with my autochanger because tapes are only either marked as Full or, Purged or Recycle and I never remove a half used tape. But on the single tape systems, I still haven't figured out how to make bacula cope with rotating half filled volumes.. or volumes where all backups failed and no data was written. Kern Sibbald [EMAIL PROTECTED] 17/12/2005 10:56:45 I think the basic problem that you have had, and it seems like you don't quite understand though you have found a solution, is that recycling for Bacula requires applying two concepts: 1. the time Bacula can write on a Volume, 2. the time Bacula keeps that Volume before reusing it. Most people come with the pre-conceived idea that there is one Recycle time, when in fact Bacula is significantly more sophisticated than that. I've reviewed the chapter you refer to below, and if one reads the full chapter, all the information *is* there (at least IMO). I have added additional text that stresses the above two concept points a bit more as well as refer the user to the Basic Volume Management chapter that gives a detailed practical example of why you want to do. The Recycle chapter is complete in itself assuming you want to fill all your Volumes. If not, you must also read the Basic Volume Management chapter as well. Concerning your comment: Can someone please update the documentation here http://www.bacula.org/rel-manual/Automatic_Volume_Recycling.html with this information, that should hopefully forestall this question coming up so much. I appreciate you pointing out where you got confused. However, this is a free software project, and I understand recycling, so I don't easily see why some people don't get the concepts or I would have already changed the doc. As a consequence it is much more helpful if you would submit a suggested change to the documentation that clarifies it for you. I suggest you look at the documentation in: http://www.bacula.org/dev-manual/Automatic_Volume_Recycling.html It is usually much more up to date. On Saturday 17 December 2005 07:00, Harondel J. Sibble wrote: On 16 Dec 2005 at 1:25, Arno Lehmann wrote: sohow does one force it to do it?! Apart from the configuration directives like prune oldest volume (or something) my advice is usually you don't. Philisophical disagreement, If I as the admin decide that, that is an appropriate thing to do, I should be able to do it in some manner. Reminds me of the unix philosophy, build a bunch of single purpose tools that you can chain together to do anything you want . Anyhow, on the how to do it, here's what I got from an unrelated list where I was ranting about the same problem the OP has as I had it for months also. What you need is this Volume Use Duration = added to your pool definition where the duration is equal to or less than your Volume Retention setting, in my case the relevant lines are Volume Retention = 6d Volume Use Duration = 6d With this setup, Bacula FINALLY works like I expect and how pretty much any other backup software I've used on Windows or Unix works. It will overwrite the tape in the drive, assuming 6 days has passed, it marks the tape as used and then recycles it, however it only does that at the beginning of the next job, so the recycling happens at the start of the current job. Can someone please update the documentation here http://www.bacula.org/rel-manual/Automatic_Volume_Recycling.html with this information, that should hopefully forestall this question coming up so much. From the messages on tonights job 16-Dec 21:30 fileserver-dir: Start Backup JobId 70, Job=Company-Fileserver_Full_Back 16-Dec 21:30 fileserver-dir: Recycled current volume Friday3 16-Dec 21:30 fileserver-sd: Recycled volume Friday3 on device DDS-4 (/dev/ns This is the first week since the beginning of the year where I've not had to manually purge the tapes each day. Think: if the tape goes full after 3 jobs, and I have 4 each night, my last job will not be done. Or, in other words, you need enough space available. Yes, that makes perfect sense. No, this is,in my opinion, the best way to handle backups: Avoid overwriting valid data whenever possible. If I setup a rotation, I should be able as the administrator to decide what is the appropriate retention time Well, your assumption is not
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
On 17 Dec 2005 at 11:56, Kern Sibbald wrote: I appreciate you pointing out where you got confused. However, this is a free software project, and I understand recycling, so I don't easily see why some people don't get the concepts or I would have already changed the doc. As a consequence it is much more helpful if you would submit a suggested change to the documentation that clarifies it for you. Sure, gladly, will do it in the new year after the holidays or over the holidays if I get time. -- Harondel J. Sibble Sibble Computer Consulting Creating solutions for the small business and home computer user. [EMAIL PROTECTED] (use pgp keyid 0x3AD5C11D) http://www.pdscc.com (604) 739-3709 (voice/fax) (604) 686-2253 (pager) --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637alloc_id=16865op=click ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
Hello, On 12/16/2005 7:04 PM, Gabriele Bulfon wrote: I think you didn't get the problem. I have 4 jobs each day. I have a retention of 6 days. I have auto prune. I have the recycle flag on. Then a week passes, and the tape is not recycled. Maybe because Bacula wants to retain as much as possibleand yes, there's still space on the tape (but how does it knows?)... I knows because the tape is not full. Bacula considers a tape full when it gets the corresponding return code when writing. Simething like the error code for no space or end of media. So, even if it appends, I'll get another 4 jobs appended on the tapeokI'm safe this second week. And another week passes, and the tape is not recycled again (why?!). Ok it wants to ratain again as much as possibleand this time the space is limited. So...I get one job doneand the secondand then I it asks for another tape! But it's 2am! Nobody's gonna change the tape!...and jobs will be canceled later on (If I don't auto cancel them, no other future job will be run without my manual intervention of canceling). this is s bad! Why it didn't want to recyle?! Because, as I tried to point out in my earlier mail, you didn't configure it to work the way you want it to work. Once again: You have to set up the volumes to be used for a given time period, so Bacula knows when to mark a volume as used. Use the Volume Use Duration setting. Please read the manual section explaining this setting. It contains a lot of useful information for your needs. But then...this happens on monday, but not on tuesday (this week), but it happens on tuesday too next weekit's a mess!! Where's the problem?! And please, can you tell me what is the User defined maximum capacity? It defines the maximum amount of data to put on a volume. I never defined this. It worked for weeks on an LTO library, and now it stops at a certain amount of Gb saying that the maximum defined capacity is reached.where do I define this? I never did. What Information about the volume is in the catalog? The llist command can show you. Thanks. Gabriele. http://www.sonicle.com Gabriele Bulfon - Sonicle S.r.l. Tel +39 028246016 Int. 30 - Fax +39 028243880 Via Felice Cavallotti 16 - 20089, Rozzano - Milano - ITALY http://www.sonicle.com -- Da: Arno Lehmann [EMAIL PROTECTED] A: bacula-users@lists.sourceforge.net Data: 16 dicembre 2005 1.25.43 CET Oggetto: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Hello, On 12/16/2005 12:56 AM, Gabriele Bulfon wrote: sohow does one force it to do it?! Apart from the configuration directives like prune oldest volume (or something) my advice is usually you don't. When I find I have to force Bacula to anything, it's usually my fault or (more often these days, running a beta version :-) Bacula shows a bug. Think: if the tape goes full after 3 jobs, and I have 4 each night, my last job will not be done. Or, in other words, you need enough space available. And this is not acceptable. No, this is,in my opinion, the best way to handle backups: Avoid overwriting valid data whenever possible. If I tell bacula that the retention period is 12 days, I assume that 12 days later the tape will be recycled. Well, your assumption is not coherent with Bacula, then. Bacula treats a retention like keep the data for at least this time. Only after this time has passed consider recycling that volume. Thus, you should assume that your data will not be overwriten until 12 days have passed after the last job on a given volume is finished. After all...how does bacula knows the true capacity of a tape, until it goes EOT? Why would Bacula need that information? Wouldn't it be easier to have some kind of flag to tell Bacula : Ok, this tape is in the correct pool, look if it's recyclable, and if it is, don't even try to append. Just scratch and reuse. Hmm. Isn't that how Bacula works? If a volume is in the right pool, and it is flagged as recyclable, and automatic pruning for the jobs and volumes is on, AND the retention period has passed, the volume will be considered when Bacula looks for a recyclable volume. If you want to limit the time Bacula uses a volume, you can do so by setting the proper volume use time or number of jobs. And, while jobs can be pruned automatically after a job, the step of recycling a volume only happens when Bacula actually needs another volume. I see many users asking for this. And I still haven't found a way to do it. Always appending. No, appending as long as it's allowed. Either until the volumes is full, it's filled with as much data as you configured
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
Nobody's gonna change the tape!...and jobs will be canceled later on (If I don't auto cancel them, no other future job will be run without my manual intervention of canceling). this is s bad! Why it didn't want to recyle?! Because, as I tried to point out in my earlier mail, you didn't configure it to work the way you want it to work. Once again: You have to set up the volumes to be used for a given time period, so Bacula knows when to mark a volume as used. Use the "Volume Use Duration" setting. Please read the manual section explaining this setting. It contains a lot of useful information for your needs. But then...this happens on monday, but not on tuesday (this week), but it happens on tuesday too next weekit's a mess!! Where's the problem?! And please, can you tell me what is the "User defined maximum capacity"? It defines the maximum amount of data to put on a volume. I never defined this. It worked for weeks on an LTO library, and now it stops at a certain amount of Gb saying that the maximum defined capacity is reached.where do I define this? I never did. What Information about the volume is in the catalog? The llist command can show you. Thanks. Gabriele. http://www.sonicle.com Gabriele Bulfon - Sonicle S.r.l. Tel +39 028246016 Int. 30 - Fax +39 028243880 Via Felice Cavallotti 16 - 20089, Rozzano - Milano - ITALY http://www.sonicle.com -- Da: Arno Lehmann [EMAIL PROTECTED] A: bacula-users@lists.sourceforge.net Data: 16 dicembre 2005 1.25.43 CET Oggetto: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Hello, On 12/16/2005 12:56 AM, Gabriele Bulfon wrote: sohow does one force it to do it?! Apart from the configuration directives like "prune oldest volume" (or something) my advice is usually "you don't". When I find I have to force Bacula to anything, it's usually my fault or (more often these days, running a beta version :-) Bacula shows a bug. Think: if the tape goes full after 3 jobs, and I have 4 each night, my last job will not be done. Or, in other words, you need enough space available. And this is not acceptable. No, this is,in my opinion, the best way to handle backups: Avoid overwriting valid data whenever possible. If I tell bacula that the retention period is 12 days, I assume that 12 days later the tape will be recycled. Well, your assumption is not coherent with Bacula, then. Bacula treats a retention like "keep the data for at least this time. Only after this time has passed consider recycling that volume." Thus, you should assume that your data will not be overwriten until 12 days have passed after the last job on a given volume is finished. After all...how does bacula knows the true capacity of a tape, until it goes EOT? Why would Bacula need that information? Wouldn't it be easier to have some kind of flag to tell Bacula : "Ok, this tape is in the correct pool, look if it's recyclable, and if it is, don't even try to append. Just scratch and reuse". Hmm. Isn't that how Bacula works? If a volume is in the right pool, and it is flagged as recyclable, and automatic pruning for the jobs and volumes is on, AND the retention period has passed, the volume will be considered when Bacula looks for a recyclable volume. If you want to limit the time Bacula uses a volume, you can do so by setting the proper volume use time or number of jobs. And, while jobs can be pruned automatically after a job, the step of recycling a volume only happens when Bacula actually needs another volume. I see many users asking for this. And I still haven't found a way to do it. Always appending. No, appending as long as it's allowed. Either until the volumes is full, it's filled with as much data as you configured, with as many jobs as you set, or used for the time you told it to use the volume. But sometimes (some kind of randomly) it recycleswierdisn't it? Erm. No. I understand Bacula works in a way to ensure that backups are available as long as possible. When determining how to set it up, it's my duty to set the retention times and the recycling strategy. Knowing the amount of data I want to store, I have to make sure I've got enough space. If I set up a pool to have backups available for a certain time, and I've not got enough volumes, Bacula tells me so. Either I re-think my backup strategy and modify the setup, or I buy more tapes. I never had a situation where Bacula didn't recycle
[Bacula-users] Puring/Pruning doesn't seem to be happening.
I'm going crosseyed, it's probably something simple, but i've been looking at
it for too long.
This shows the media i'm going to use for tonight's backup. The problem is,
it's still set to append, shouldn't it have a VolStatus or Purged?
+-++---+-+--+--+-+--+---+---+-+
| MediaId | VolumeName | VolStatus | VolBytes| VolFiles | VolRetention
| Recycle | Slot | InChanger | MediaType | LastWritten |
+-++---+-+--+--+-+--+---+---+-+
| 6 | THURSDAY2 | Append| 99,017,877,864 | 102 |1,036,800
| 1 |0 | 1 | SDLT | 2005-12-02 00:22:38 |
Here's my retention stuff from bacula-dir.conf
Pool {
Name = Daily
Pool Type = Backup
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 12 days #
Accept Any Volume = yes # write on any volume in the pool
Recycle Current Volume = Yes
}
#==
Client {
Name = SERVER1-fd
Address = server1.harrow.gov.uk
FDPort = 9102
Catalog = MyCatalog
Password = x # password for FileDaemon
File Retention = 31 days# days
Job Retention = 2 months# months
AutoPrune = yes # Prune expired Jobs/Files
}
Thanks guys.. any help would be greatly appreciated.
Beren
***
Mail FROM London Borough of Harrow:
Unencrypted electronic mail is not secure and may not be authentic, in whole or
in part. You are advised to check directly with the sender before acting upon
any e-mail received.
The information contained in this message and any attachments is confidential
and is intended for receipt by the above named addressee(s) only. If you have
otherwise encountered this message please notify its originator via +44(0)20
8863 5611 at LONDON BOROUGH OF HARROW. The unauthorised use, disclosure,
copying or alteration of this message is strictly forbidden. The views
expressed within this message are those of the individual sender and not
necessarily those of Harrow Council.
Mail TO London Borough of Harrow:
London Borough of Harrow monitors all electronic mail it receives for Policy
compliance and to protect its systems including anti-spam and anti-virus
measures.
Electronic mail does not guarantee delivery, nor notification of non-delivery.
It is suggested you contact your intended recipient(s) by other means should
confirmation of receipt be important.
***
---
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37alloc_id865op=click
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
Hello, On 12/15/2005 5:01 PM, Beren Gamble wrote: I'm going crosseyed, it's probably something simple, but i've been looking at it for too long. This shows the media i'm going to use for tonight's backup. The problem is, it's still set to append, shouldn't it have a VolStatus or Purged? +-++---+-+--+--+-+--+---+---+-+ | MediaId | VolumeName | VolStatus | VolBytes| VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten | +-++---+-+--+--+-+--+---+---+-+ | 6 | THURSDAY2 | Append| 99,017,877,864 | 102 |1,036,800 | 1 |0 | 1 | SDLT | 2005-12-02 00:22:38 | Considering your retention period of 12 days, I guess that's what you're aiming at. And no, it should not yet be purged. Bacula only prunes jobs and volumes when it actually needs a new volume. In this case, I suppose that, if this evening there's no more space available or usable, Bacula will prune this volume and then recycle it. Arno PS: *** Mail FROM London Borough of Harrow: Unencrypted electronic mail is not secure and may not be authentic, in whole or in part. You are advised to check directly with the sender before acting upon any e-mail received. The information contained in this message and any attachments is confidential and is intended for receipt by the above named addressee(s) only. If you have otherwise encountered this message please notify its originator via +44(0)20 8863 5611 at LONDON BOROUGH OF HARROW. The unauthorised use, disclosure, copying or alteration of this message is strictly forbidden. The views expressed within this message are those of the individual sender and not necessarily those of Harrow Council. Mail TO London Borough of Harrow: London Borough of Harrow monitors all electronic mail it receives for Policy compliance and to protect its systems including anti-spam and anti-virus measures. If anybody at your institution thinks that this disclaimer would be helpful it would be nice to wrap the text as usual. Otherwise, please simply drop it. If I really honored this disclaimer, I wouldn't reply to you - I'm not the addressee of your mail, and we've got no contract, so how would I know I'm really the one this mail intended for? And, if you worry about confidentiality, you should cryptographically sign or encript your mail. Or, even better, not use E-Mail at all. Perhaps whoever is responsible for this stuff in your organisation can explain if there's any real reason you need to send this out probably thousands of times a day. Thanks, Arno Electronic mail does not guarantee delivery, nor notification of non-delivery. It is suggested you contact your intended recipient(s) by other means should confirmation of receipt be important. *** --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_idv37alloc_id865op=click ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- IT-Service Lehmann[EMAIL PROTECTED] Arno Lehmann http://www.its-lehmann.de --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_id=7637alloc_id=16865op=click ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
sohow does one force it to do it?!Think: if the tape goes full after 3 jobs, and I have 4 each night, my last job will not be done.And this is not acceptable.If I tell bacula that the retention period is 12 days, I assume that 12 days later the tape will be recycled.After all...how does bacula knows the true capacity of a tape, until it goes EOT?Wouldn't it be easier to have some kind of flag to tell Bacula : "Ok, this tape is in the correct pool, look if it's recyclable, and if it is, don't even try to append. Just scratch and reuse".I see many users asking for this. And I still haven't found a way to do it.Always appending. But sometimes (some kind of randomly) it recycleswierdisn't it? Gabriele Bulfon - Sonicle S.r.l. Tel +39 028246016 Int. 30 - Fax +39 028243880 Via Felice Cavallotti 16 - 20089, Rozzano - Milano - ITALY http://www.sonicle.com --Da: Arno Lehmann [EMAIL PROTECTED]A: bacula-users@lists.sourceforge.net Data: 15 dicembre 2005 21.12.01 CETOggetto: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.Hello, On 12/15/2005 5:01 PM, Beren Gamble wrote: I'm going crosseyed, it's probably something simple, but i've been looking at it for too long. This shows the media i'm going to use for tonight's backup. The problem is, it's still set to append, shouldn't it have a VolStatus or Purged? +-++---+-+--+--+-+--+---+---+-+ | MediaId | VolumeName | VolStatus | VolBytes| VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten | +-++---+-+--+--+-+--+---+---+-+ | 6 | THURSDAY2 | Append| 99,017,877,864 | 102 |1,036,800 | 1 |0 | 1 | SDLT | 2005-12-02 00:22:38 | Considering your retention period of 12 days, I guess that's what you're aiming at. And no, it should not yet be purged. Bacula only prunes jobs and volumes when it actually needs a new volume. In this case, I suppose that, if this evening there's no more space available or usable, Bacula will prune this volume and then recycle it. Arno PS: *** Mail FROM London Borough of Harrow: Unencrypted electronic mail is not secure and may not be authentic, in whole or in part. You are advised to check directly with the sender before acting upon any e-mail received. The information contained in this message and any attachments is confidential and is intended for receipt by the above named addressee(s) only. If you have otherwise encountered this message please notify its originator via +44(0)20 8863 5611 at LONDON BOROUGH OF HARROW. The unauthorised use, disclosure, copying or alteration of this message is strictly forbidden. The views expressed within this message are those of the individual sender and not necessarily those of Harrow Council. Mail TO London Borough of Harrow: London Borough of Harrow monitors all electronic mail it receives for Policy compliance and to protect its systems including anti-spam and anti-virus measures. If anybody at your institution thinks that this disclaimer would be helpful it would be nice to wrap the text as usual. Otherwise, please simply drop it. If I really honored this disclaimer, I wouldn't reply to you - I'm not the addressee of your mail, and we've got no contract, so how would I know I'm really the one this mail intended for? And, if you worry about confidentiality, you should cryptographically sign or encript your mail. Or, even better, not use E-Mail at all. Perhaps whoever is responsible for this stuff in your organisation can explain if there's any real reason you need to send this out probably thousands of times a day. Thanks, Arno Electronic mail does not guarantee delivery, nor notification of non-delivery. It is suggested you contact your intended recipient(s) by other means should confirmation of receipt be important. *** --- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://ads.osdn.com/?ad_idv37alloc_id865op=click ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users -- IT-Service Lehmann[EMAIL PROTECTED] Arno Lehmann http://www.its-lehmann.de
Re: [Bacula-users] Puring/Pruning doesn't seem to be happening.
Hello, On 12/16/2005 12:56 AM, Gabriele Bulfon wrote: sohow does one force it to do it?! Apart from the configuration directives like prune oldest volume (or something) my advice is usually you don't. When I find I have to force Bacula to anything, it's usually my fault or (more often these days, running a beta version :-) Bacula shows a bug. Think: if the tape goes full after 3 jobs, and I have 4 each night, my last job will not be done. Or, in other words, you need enough space available. And this is not acceptable. No, this is,in my opinion, the best way to handle backups: Avoid overwriting valid data whenever possible. If I tell bacula that the retention period is 12 days, I assume that 12 days later the tape will be recycled. Well, your assumption is not coherent with Bacula, then. Bacula treats a retention like keep the data for at least this time. Only after this time has passed consider recycling that volume. Thus, you should assume that your data will not be overwriten until 12 days have passed after the last job on a given volume is finished. After all...how does bacula knows the true capacity of a tape, until it goes EOT? Why would Bacula need that information? Wouldn't it be easier to have some kind of flag to tell Bacula : Ok, this tape is in the correct pool, look if it's recyclable, and if it is, don't even try to append. Just scratch and reuse. Hmm. Isn't that how Bacula works? If a volume is in the right pool, and it is flagged as recyclable, and automatic pruning for the jobs and volumes is on, AND the retention period has passed, the volume will be considered when Bacula looks for a recyclable volume. If you want to limit the time Bacula uses a volume, you can do so by setting the proper volume use time or number of jobs. And, while jobs can be pruned automatically after a job, the step of recycling a volume only happens when Bacula actually needs another volume. I see many users asking for this. And I still haven't found a way to do it. Always appending. No, appending as long as it's allowed. Either until the volumes is full, it's filled with as much data as you configured, with as many jobs as you set, or used for the time you told it to use the volume. But sometimes (some kind of randomly) it recycleswierdisn't it? Erm. No. I understand Bacula works in a way to ensure that backups are available as long as possible. When determining how to set it up, it's my duty to set the retention times and the recycling strategy. Knowing the amount of data I want to store, I have to make sure I've got enough space. If I set up a pool to have backups available for a certain time, and I've not got enough volumes, Bacula tells me so. Either I re-think my backup strategy and modify the setup, or I buy more tapes. I never had a situation where Bacula didn't recycle correctly, and even my 1-year-retention pools are cycling as I set them up. Of course, for short-term storage I need the space for a worst-case scenario. Arno http://www.sonicle.com Gabriele Bulfon - Sonicle S.r.l. Tel +39 028246016 Int. 30 - Fax +39 028243880 Via Felice Cavallotti 16 - 20089, Rozzano - Milano - ITALY http://www.sonicle.com -- Da: Arno Lehmann [EMAIL PROTECTED] A: bacula-users@lists.sourceforge.net Data: 15 dicembre 2005 21.12.01 CET Oggetto: Re: [Bacula-users] Puring/Pruning doesn't seem to be happening. Hello, On 12/15/2005 5:01 PM, Beren Gamble wrote: I'm going crosseyed, it's probably something simple, but i've been looking at it for too long. This shows the media i'm going to use for tonight's backup. The problem is, it's still set to append, shouldn't it have a VolStatus or Purged? +-++---+-+--+--+-+--+---+---+-+ | MediaId | VolumeName | VolStatus | VolBytes | VolFiles | VolRetention | Recycle | Slot | InChanger | MediaType | LastWritten | +-++---+-+--+--+-+--+---+---+-+ | 6 | THURSDAY2 | Append | 99,017,877,864 | 102 | 1,036,800 | 1 | 0 | 1 | SDLT | 2005-12-02 00:22:38 | Considering your retention period of 12 days, I guess that's what you're aiming at. And no, it should not yet be purged. Bacula only prunes jobs and volumes when it actually needs a new volume. In this case, I suppose that, if this evening there's no more space available or usable, Bacula will prune this volume and then recycle it. Arno PS: *** Mail FROM London Borough of Harrow: Unencrypted electronic mail
