Re: tapelist format
Jon,
Can you try the attached patch?
On 25/02/17 01:58 PM, Jon LaBadie wrote:
> I'm getting a lot of these messages in my nightly report
>
>planner: Last full dump of cyber.jgcomp.com:Home on tape DS1-084
> overwritten in 1 run.
>planner: Last full dump of cyber.jgcomp.com:Vault on tape DS1-079
> overwritten in 1 run.
>
> It made little sense as there are many full dumps or these
> various DLEs sprinkled back to tape DS1-000. And, although
> there were about 10 unused tapes remaining, I added a bunch
> more which had no effect on the "will be overwritten" messages.
>
> Then I looked at the tapelist and found 2 different formats.
> This is certainly due to my switching from CentOS packages
> (amanda 3.3.3.) to Zmanda pre-built binaries for RHEL (amanda
> 3.4.1).
>
> Previously labeled tapes have a 4 column layout but when the
> new amanda version uses a tape, it modifies that layout to a
> 7 column format adding "POOL", "STORAGE", and "CONFIG" columns.
> Now my tapelist has 4 contiguous groups.
>
> recently labeled unused 7 column layout
>previously labeled unused 4 column layout
>previously labeledused 7 column layout
>previously labeledused 4 column layout
>
> Why does the planner think it will overwrite the last full dumps?
>
> Would simply adding the additional 7 fields to the tapelist
> correct this error or would other things have to change also?
It should fix the issue.
>
> Would adding the extra columns adversely affect anything else?
no.
Jean-Louis
diff --git a/server-src/planner.c b/server-src/planner.c
index 3464810..4d2b55b 100644
--- a/server-src/planner.c
+++ b/server-src/planner.c
@@ -1291,6 +1291,8 @@ static int when_overwrite(
int nb_in_storage;
int runtapes;
storage_t *st;
+policy_s *po;
+intretention_tapes;
if ((tp = lookup_tapelabel(label)) == NULL)
return 1; /* "shouldn't happen", but trigger warning message */
@@ -1308,13 +1310,17 @@ static int when_overwrite(
}
if (!st)
return 1;
+po = lookup_policy(storage_get_policy(st));
+if (!po)
+ return 1;
+retention_tapes = policy_get_retention_tapes(po);
runtapes = storage_get_runtapes(st);
if (runtapes == 0) runtapes = 1;
-nb = tape_overwrite(tp);
-nb_in_storage = nb_tape_in_storage(tp->storage);
-if (conf_tapecycle+1 > nb_in_storage)
- nb += (conf_tapecycle+1 - nb_in_storage);
+nb = tape_overwrite(st, tp);
+nb_in_storage = nb_tape_in_storage(st);
+if (retention_tapes > nb_in_storage)
+ nb += (retention_tapes - nb_in_storage + 1);
tp->when_overwrite = (nb - 1) / runtapes;
if (tp->when_overwrite < 0)
diff --git a/server-src/tapefile.c b/server-src/tapefile.c
index 7e0d81f..63f8ebf 100644
--- a/server-src/tapefile.c
+++ b/server-src/tapefile.c
@@ -1175,14 +1175,21 @@ get_retention_type(
int
tape_overwrite(
+storage_t *st,
tape_t *tp)
{
tape_t *tp1;
int nb_tapes = 0;
for (tp1 = tp; tp1 != NULL; tp1 = tp1->next) {
- if (!tp1->retention && tp1->storage && tp->storage &&
- g_str_equal(tp->storage, tp1->storage)) {
+ if (!tp1->retention &&
+ (((!tp1->storage || !tp->storage) &&
+ match_labelstr(storage_get_labelstr(st),
+ storage_get_autolabel(st),
+ tp1->label, tp1->barcode, tp1->meta,
+ storage_name(st))) ||
+ (tp1->storage && tp->storage &&
+ g_str_equal(tp->storage, tp1->storage {
nb_tapes++;
}
}
@@ -1191,14 +1198,18 @@ tape_overwrite(
int
nb_tape_in_storage(
-char *storage_name)
+storage_t *st)
{
tape_t *tp;
int nb_tapes = 0;
-
+char *storage = storage_name(st);
for (tp = tape_list; tp != NULL; tp = tp->next) {
- if ((!storage_name && !tp->storage) ||
- (storage_name && tp->storage && g_str_equal(storage_name,tp->storage))) {
+ if (((!storage || !tp->storage) &&
+ match_labelstr(storage_get_labelstr(st),
+ storage_get_autolabel(st),
+ tp->label, tp->barcode, tp->meta,
+ storage_name(st))) ||
+ (storage && tp->storage && g_str_equal(storage, tp->storage))) {
nb_tapes++;
}
}
diff --git a/server-src/tapefile.h b/server-src/tapefile.h
index 4af86da..547f8ef 100644
--- a/server-src/tapefile.h
+++ b/server-src/tapefile.h
@@ -35,6 +35,7 @@
#define TAPEFILE_H
#include "amanda.h"
+#include "conffile.h"
typedef enum {
RETENTION_NO = 0,
@@ -110,7 +111,7 @@ int volume_is_reusable(const char *label);
int guess_runs_from_tapelist(void);
gchar **list_new_tapes(char *storage_n, int nb);
RetentionType get_retention_type(char *pool, char *label);
-int tape_overwrite(tape_t *tp);
-int nb_tape_in_storage(char *storage_name);
+int tape_overwrite(storage_t *st, tape_t *tp);
+int nb_tape_in_storage(storage_t *st);
#endif /* !TAPEFILE_H */
Re: tapelist format
On Mon, Feb 27, 2017 at 11:31:38AM -0500, Jean-Louis Martineau wrote:
> Jon,
>
> Can you try the attached patch?
I'm running the zmanda pre-builts and do not have a build
system set up here.
>
> On 25/02/17 01:58 PM, Jon LaBadie wrote:
> > I'm getting a lot of these messages in my nightly report
> >
> >planner: Last full dump of cyber.jgcomp.com:Home on tape DS1-084
> > overwritten in 1 run.
> >planner: Last full dump of cyber.jgcomp.com:Vault on tape DS1-079
> > overwritten in 1 run.
> >
> > It made little sense as there are many full dumps or these
> > various DLEs sprinkled back to tape DS1-000. And, although
> > there were about 10 unused tapes remaining, I added a bunch
> > more which had no effect on the "will be overwritten" messages.
> >
> > Then I looked at the tapelist and found 2 different formats.
> > This is certainly due to my switching from CentOS packages
> > (amanda 3.3.3.) to Zmanda pre-built binaries for RHEL (amanda
> > 3.4.1).
> >
> > Previously labeled tapes have a 4 column layout but when the
> > new amanda version uses a tape, it modifies that layout to a
> > 7 column format adding "POOL", "STORAGE", and "CONFIG" columns.
> > Now my tapelist has 4 contiguous groups.
> >
> > recently labeled unused 7 column layout
> >previously labeled unused 4 column layout
> >previously labeledused 7 column layout
> >previously labeledused 4 column layout
> >
> > Why does the planner think it will overwrite the last full dumps?
> >
> > Would simply adding the additional 7 fields to the tapelist
> > correct this error or would other things have to change also?
> It should fix the issue.
> >
> > Would adding the extra columns adversely affect anything else?
> no.
I didn't think so, I'll make the change and check tonights results.
Jon
>
> Jean-Louis
> diff --git a/server-src/planner.c b/server-src/planner.c
> index 3464810..4d2b55b 100644
> --- a/server-src/planner.c
> +++ b/server-src/planner.c
> @@ -1291,6 +1291,8 @@ static int when_overwrite(
> int nb_in_storage;
> int runtapes;
> storage_t *st;
> +policy_s *po;
> +intretention_tapes;
>
> if ((tp = lookup_tapelabel(label)) == NULL)
> return 1; /* "shouldn't happen", but trigger warning message */
> @@ -1308,13 +1310,17 @@ static int when_overwrite(
> }
> if (!st)
> return 1;
> +po = lookup_policy(storage_get_policy(st));
> +if (!po)
> + return 1;
> +retention_tapes = policy_get_retention_tapes(po);
> runtapes = storage_get_runtapes(st);
> if (runtapes == 0) runtapes = 1;
>
> -nb = tape_overwrite(tp);
> -nb_in_storage = nb_tape_in_storage(tp->storage);
> -if (conf_tapecycle+1 > nb_in_storage)
> - nb += (conf_tapecycle+1 - nb_in_storage);
> +nb = tape_overwrite(st, tp);
> +nb_in_storage = nb_tape_in_storage(st);
> +if (retention_tapes > nb_in_storage)
> + nb += (retention_tapes - nb_in_storage + 1);
>
> tp->when_overwrite = (nb - 1) / runtapes;
> if (tp->when_overwrite < 0)
> diff --git a/server-src/tapefile.c b/server-src/tapefile.c
> index 7e0d81f..63f8ebf 100644
> --- a/server-src/tapefile.c
> +++ b/server-src/tapefile.c
> @@ -1175,14 +1175,21 @@ get_retention_type(
>
> int
> tape_overwrite(
> +storage_t *st,
> tape_t *tp)
> {
> tape_t *tp1;
> int nb_tapes = 0;
>
> for (tp1 = tp; tp1 != NULL; tp1 = tp1->next) {
> - if (!tp1->retention && tp1->storage && tp->storage &&
> - g_str_equal(tp->storage, tp1->storage)) {
> + if (!tp1->retention &&
> + (((!tp1->storage || !tp->storage) &&
> + match_labelstr(storage_get_labelstr(st),
> + storage_get_autolabel(st),
> + tp1->label, tp1->barcode, tp1->meta,
> + storage_name(st))) ||
> + (tp1->storage && tp->storage &&
> + g_str_equal(tp->storage, tp1->storage {
> nb_tapes++;
> }
> }
> @@ -1191,14 +1198,18 @@ tape_overwrite(
>
> int
> nb_tape_in_storage(
> -char *storage_name)
> +storage_t *st)
> {
> tape_t *tp;
> int nb_tapes = 0;
> -
> +char *storage = storage_name(st);
> for (tp = tape_list; tp != NULL; tp = tp->next) {
> - if ((!storage_name && !tp->storage) ||
> - (storage_name && tp->storage &&
> g_str_equal(storage_name,tp->storage))) {
> + if (((!storage || !tp->storage) &&
> + match_labelstr(storage_get_labelstr(st),
> + storage_get_autolabel(st),
> + tp->label, tp->barcode, tp->meta,
> + storage_name(st))) ||
> + (storage && tp->storage && g_str_equal(storage, tp->storage))) {
> nb_tapes++;
> }
> }
> diff --git a/server-src/tapefile.h b/server-src/tapefile.h
> index 4af86da..547f8ef 100644
> --- a/server-src/tapefile.h
> +++ b/server-src/tapefile.h
> @@ -35,6 +35,
problem with taper not changing tape, 3.4.[12]
I've found what appears to be a bug in the 3.4 release of amanda: under some circumstances, amanda overwrites a tape that was just written. an example from a backup to tape: These dumps were to tape RAIDsSet6-35. The next 2 tapes Amanda expects to use are: RAIDsSet6-36, RAIDsSet6-37. taper: tape RAIDsSet6-35 kb 2077738050 fm 308 [OK] taper: tape RAIDsSet6-35 kb 356540480 fm 61 [OK] as you can see, I have runtapes=2, and amanda seems to know that, yet it has overwritten the first part of the dump with the second. looking at the tape I've verified that there are only 61 files on tape 6-35. I made a test case using virtual tapes, and found the same thing: These dumps were to tape TEST-04. The next 2 tapes Amanda expects to use are: TEST-05, TEST-06. taper: tape TEST-04 kb 614240 fm 1 [OK] taper: tape TEST-04 kb 350240 fm 1 [OK] and the size of TEST-04 is indeed around 350G. any thoughts as to what's going on? the amanda.conf and disklist are the same ones I used under 3.3.x, and I didn't see this problem then. I've just joined this list, so I'm not sure about sending attachments (like amanda.conf). -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
Re: problem with taper not changing tape, 3.4.[12]
Ned, Send as attachement * amanda.conf * amdump. * log..0 * taper..debug Jean-Louis On 27/02/17 03:07 PM, Ned Danieley wrote: > I've found what appears to be a bug in the 3.4 release of amanda: under > some circumstances, amanda overwrites a tape that was just written. an > example from a backup to tape: > > These dumps were to tape RAIDsSet6-35. > The next 2 tapes Amanda expects to use are: RAIDsSet6-36, RAIDsSet6-37. > > taper: tape RAIDsSet6-35 kb 2077738050 fm 308 [OK] > taper: tape RAIDsSet6-35 kb 356540480 fm 61 [OK] > > as you can see, I have runtapes=2, and amanda seems to know that, yet it has > overwritten the first part of the dump with the second. looking at the tape > I've verified that there are only 61 files on tape 6-35. > > I made a test case using virtual tapes, and found the same thing: > > These dumps were to tape TEST-04. > The next 2 tapes Amanda expects to use are: TEST-05, TEST-06. > > taper: tape TEST-04 kb 614240 fm 1 [OK] > taper: tape TEST-04 kb 350240 fm 1 [OK] > > and the size of TEST-04 is indeed around 350G. > > any thoughts as to what's going on? the amanda.conf and disklist are the > same ones I used under 3.3.x, and I didn't see this problem then. I've just > joined this list, so I'm not sure about sending attachments (like > amanda.conf). >
Amanda and OpenStack
Dear Mailing List I am just new to Amanda Community Edition and was wondering how easy it was to configure it to use OpenStack object storage. I notice an Amazon S3 device http://wiki.zmanda.com/index.php/How_To:Backup_to_Amazon_S3 but do not know how to change this to point to a particular swift storage server. Are there any examples I could look at to see how this works? Id be also interested in how Amanda determines if a file needs to be updated on the storage. Does it have its own internal DB or does it look inside the virtual tapes to do this? thanks for any assistance/pointers regards *---Simon Michnowicz * Senior Application Specialist, High-Performance Computing *Research Support Services - eSolutions* *Monash eResearch Centre* Monash University 15 Innovation Walk, Building 75, Clayton Campus Wellington Road, VIC 3800 Australia T: +61 3 9902 0794 M: +61 3 0418 302 046 E: simon.michnow...@monash.edu monash.edu
Re: Amanda and OpenStack
On Tue, Feb 28, 2017 at 05:39:17PM +1100, Simon Michnowicz wrote: > Dear Mailing List > I am just new to Amanda Community Edition and was wondering how easy it was > to configure it to use OpenStack object storage. I notice an Amazon S3 > device > http://wiki.zmanda.com/index.php/How_To:Backup_to_Amazon_S3 > but do not know how to change this to point to a particular swift storage > server. Are there any examples I could look at to see how this works? Id Sorry, no idea on this > be also interested in how Amanda determines if a file needs to be updated > on the storage. Does it have its own internal DB or does it look inside the > virtual tapes to do this? Amanda is a backup manager. It leaves mant details to the underlying backup program you elect to use. For most this would be tar or dump. These record when the last dump was done and check if any change has occured since that time. Most likely this is done by comparing the last dumpdate with the files "ctime" (change time). This would account for changes to the data or to the file's properties, eg. link count, permissions, etc. You can see a file's ctime with "ls -lc". Welcome to the world according to amanda. Jon -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
