Re: "vaulting run" amdump email report issues -- driver.c "nodump" option

2017-11-23 Thread Nathan Stratton Treadway 
On Thu, Nov 23, 2017 at 14:32:38 -0500, Jean-Louis Martineau wrote:
> I forgot to commit it.
> It is now committed.

Yep, looks good now.  Thanks again.

Nathan


Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: "vaulting run" amdump email report issues -- driver.c "nodump" option

2017-11-23 Thread Jean-Louis Martineau 
On 23/11/17 02:00 PM, Nathan Stratton Treadway wrote:
> On Wed, Nov 22, 2017 at 11:58:12 -0500, Jean-Louis Martineau wrote:
> > On 22/11/17 12:56 AM, Nathan Stratton Treadway wrote:
> > > On Mon, Nov 20, 2017 at 11:45:07 -0500, Jean-Louis Martineau wrote:
> > > > Nathan,
> > > >
> > > > I was too fast to write the first patch, and you are completely 
> right.
> > > >
> > > > I committed the attached patch,now amflush use the --no-dump 
> argument
> > > > and driver merge both 'nodump' and '--no-dump'.
> > >
> > > Yeah, looks good.
> > >
> > > I applied this patch and ran the test again, and "amdump --no-dump
> > > --no-flush" worked just as well as with the previous patch, but showed
> > > no "dumper lines" in the log.*.0 file.
> > >
> > > (Of course the email report did still have the unexpected RESULTS
> > > MISSING line for each DLE.)
> > I already committed a fix for that, use SVN or GIT.
>
> My next step will be to rebuild the latest version from git and use that
> for further tests, but while I'm working on that:
>
> The code used in the above test did include the Report-3.5.diff patch
> from Nov 17, which eliminated the RESULTS MISSING lines in the report
> when amdump is run with a BADHOST argument.
>
> Did you also push a fix that eliminates the RESULTS MISSING lines which
> were being generated when amdump is run with "--no-dump" instead of
> BADHOST? (This was the case I was referring to above, following up from
> my original discussion in an email from Sat, 18 Nov 2017 00:43:32
> -0500.)
>
> I looked through the git log (for the 3_5 branch) but didn't notice any
> changes that seemed to apply to that problem (either on the
> Report.pm/human.pm 
>  
> side, or on the planner.c side). If you did fix that
> case, could you point me to the commit date or commit id?

I forgot to commit it.
It is now committed.

Jean-Louis
>
> Thanks.
>
> Nathan
>
> 
> Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region
> Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ 
> 
> GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt 
>  
> ID: 1023D/ECFB6239
> Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail

Re: "vaulting run" amdump email report issues -- driver.c "nodump" option

2017-11-23 Thread Nathan Stratton Treadway 
On Wed, Nov 22, 2017 at 11:58:12 -0500, Jean-Louis Martineau wrote:
> On 22/11/17 12:56 AM, Nathan Stratton Treadway wrote:
> > On Mon, Nov 20, 2017 at 11:45:07 -0500, Jean-Louis Martineau wrote:
> > > Nathan,
> > >
> > > I was too fast to write the first patch, and you are completely right.
> > >
> > > I committed the attached patch,now amflush use the --no-dump argument
> > > and driver merge both 'nodump' and '--no-dump'.
> >
> > Yeah, looks good.
> >
> > I applied this patch and ran the test again, and "amdump --no-dump
> > --no-flush" worked just as well as with the previous patch, but showed
> > no "dumper lines" in the log.*.0 file.
> >
> > (Of course the email report did still have the unexpected RESULTS
> > MISSING line for each DLE.)
> I already committed a fix for that, use SVN or GIT.

My next step will be to rebuild the latest version from git and use that
for further tests, but while I'm working on that:

The code used in the above test did include the Report-3.5.diff patch
from Nov 17, which eliminated the RESULTS MISSING lines in the report
when amdump is run with a BADHOST argument.  

Did you also push a fix that eliminates the RESULTS MISSING lines which
were being generated when amdump is run with "--no-dump" instead of
BADHOST?  (This was the case I was referring to above, following up from
my original discussion in an email from Sat, 18 Nov 2017 00:43:32
-0500.)

I looked through the git log (for the 3_5 branch) but didn't notice any
changes that seemed to apply to that problem (either on the
Report.pm/human.pm side, or on the planner.c side).  If you did fix that
case, could you point me to the commit date or commit id?

Thanks.

Nathan


Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: "vaulting run" amdump email report issues -- driver.c "nodump" option

2017-11-22 Thread Jean-Louis Martineau 
On 22/11/17 12:56 AM, Nathan Stratton Treadway wrote:
> On Mon, Nov 20, 2017 at 11:45:07 -0500, Jean-Louis Martineau wrote:
> > Nathan,
> >
> > I was too fast to write the first patch, and you are completely right.
> >
> > I committed the attached patch,now amflush use the --no-dump argument
> > and driver merge both 'nodump' and '--no-dump'.
>
> Yeah, looks good.
>
> I applied this patch and ran the test again, and "amdump --no-dump
> --no-flush" worked just as well as with the previous patch, but showed
> no "dumper lines" in the log.*.0 file.
>
> (Of course the email report did still have the unexpected RESULTS
> MISSING line for each DLE.)
I already committed a fix for that, use SVN or GIT.

Jean-Louis
>
> I also tried the amflush command with this patch applied, and it too
> started up the driver and flushed files to the target storage without
> any trouble.
>
>
> Nathan
>
> 
> Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region
> Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ 
> 
> GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt 
>  
> ID: 1023D/ECFB6239
> Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail

Re: "vaulting run" amdump email report issues -- driver.c "nodump" option

2017-11-21 Thread Nathan Stratton Treadway 
On Mon, Nov 20, 2017 at 11:45:07 -0500, Jean-Louis Martineau wrote:
> Nathan,
> 
> I was too fast to write the first patch, and you are completely right.
> 
> I committed the attached patch,now amflush use the --no-dump argument 
> and driver merge both 'nodump' and '--no-dump'.

Yeah, looks good.

I applied this patch and ran the test again, and "amdump --no-dump
--no-flush" worked just as well as with the previous patch, but showed
no "dumper lines" in the log.*.0 file.

(Of course the email report did still have the unexpected RESULTS
MISSING line for each DLE.)

I also tried the amflush command with this patch applied, and it too
started up the driver and flushed files to the target storage without
any trouble.


Nathan


Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: "vaulting run" amdump email report issues -- driver.c "nodump" option

2017-11-20 Thread Jean-Louis Martineau 
Nathan,

I was too fast to write the first patch, and you are completely right.

I committed the attached patch,now amflush use the --no-dump argument 
and driver merge both 'nodump' and '--no-dump'.

Jean-Louis

On 18/11/17 01:11 AM, Nathan Stratton Treadway wrote:
> On Thu, Nov 16, 2017 at 10:15:28 -0500, Jean-Louis Martineau wrote:
> > Nathan,
> >
> > Try this patch.
> >
> > run: amdump --no-dump --no-flush CONFIG
> > Will do all scheduled vault to the vault-storage
>
> [...]
>
> > diff --git a/server-src/driver.c b/server-src/driver.c
> > index 351494a..fd01296 100644
> > --- a/server-src/driver.c
> > +++ b/server-src/driver.c
> > @@ -107,6 +107,10 @@ static int taper_started = 0;
> > static int nb_storage;
> > static cmddatas_t *cmddatas = NULL;
> >
> > +static int no_dump = FALSE;
> > +//static int no_flush = FALSE;
> > +static int no_vault = FALSE;
> > +
> > static int wait_children(int count);
> > static void wait_for_children(void);
> > static void allocate_bandwidth(netif_t *ip, unsigned long kps);
> > @@ -305,6 +309,30 @@ main(
> > }
> >
> > if (argc > 2) {
> > + if (g_str_equal(argv[2], "--no-dump")) {
> > + no_dump = TRUE;
> > + argv++;
> > + argc--;
> > + }
> > + }
>
> I noticed when reviewing the log file from my "amdump --no-dump" run
> that there were still "INFO dumper" lines in there, from when the
> dumpers start up at the beginning of the run and when they terminate at
> the end (with no activity in between).
>
> I wondered if the dumpers should be getting started at all when no_dump
> is true, and took a look at driver.c to see how the dumpers were getting
> kicked off -- and discovered that driver.c already implements a "nodump"
> option, and that indeed if that option is enabled, the dumper processes
> are not spawned.
>
> (I see that this "nodump" option is used by Amflush.pm 
> .)
>
> Anyway, I guess the question is whether it makes sense to merge the new
> --no-dump and original nodump options for driver.c, at least as far as
> having one internal boolean variable to handle both -- or, if not,
> if there are any places where the code currently checks for !nodump
> where !no_dump should be treated equivalently?
>
> Nathan
>
>
>
> 
> Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region
> Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ 
> 
> GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt 
>  
> ID: 1023D/ECFB6239
> Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail
diff --git a/perl/Amanda/Amflush.pm b/perl/Amanda/Amflush.pm
index f2f8aa2..8323733 100644
--- a/perl/Amanda/Amflush.pm
+++ b/perl/Amanda/Amflush.pm
@@ -317,10 +317,9 @@ sub run {
 	POSIX::close($driver_pipe);
 	POSIX::dup2(fileno($self->{'amdump_log'}), 1);
 	POSIX::dup2(fileno($self->{'amdump_log'}), 2);
-	debug("exec: " . join(' ', $driver, $self->{'config'}, "nodump", "--no-vault",
-   @log_filename, @config_overrides));
+	debug("exec: " . join(' ', $driver, $self->{'config'}, @log_filename, "--no-dump", "--no-vault", @config_overrides));
 	close($self->{'amdump_log'});
-	exec $driver, $self->{'config'}, "nodump", "--no-vault", @log_filename,
+	exec $driver, $self->{'config'}, @log_filename, "--no-dump", "--no-vault", 
 		  @config_overrides;
 	die "Could not exec $driver: $!";
 }
diff --git a/server-src/driver.c b/server-src/driver.c
index fd01296..1625d06 100644
--- a/server-src/driver.c
+++ b/server-src/driver.c
@@ -79,8 +79,6 @@ static off_t total_disksize;
 static char *dumper_program;
 static char *chunker_program;
 static int  inparallel;
-static int nodump = 0;
-static int novault = 0;
 static storage_t *storage;
 static int conf_max_dle_by_volume;
 static int conf_taperalgo;
@@ -274,22 +272,6 @@ main(
 set_config_overrides(cfg_ovr);
 argv0 = argv[0];
 
-if(argc > 2) {
-if(g_str_equal(argv[2], "nodump")) {
-nodump = 1;
-	argv++;
-	argc--;
-}
-}
-
-if(argc > 2) {
-if(g_str_equal(argv[2], "--no-vault")) {
-novault = 1;
-	argv++;
-	argc--;
-}
-}
-
 log_filename = NULL;
 if (argc > 3) {
 	if (g_str_equal(argv[2], "--log-filename")) {
@@ -393,7 +375,7 @@ main(
 
 /* check that we don't do many dump in a day and usetimestamps is off */
 if(strlen(driver_timestamp) == 8) {
-	if (!nodump) {
+	if (!no_dump) {
 	char *conf_logdir = config_dir_relative(getconf_str(CNF_LOGDIR));
 	char *logfile= g_strjoin(NULL, conf_logdir, "/log.",
 	 driver_timestamp, ".0", NULL);
@@ -533,7 +515,7 @@ main(
 nb_storage

Re: "vaulting run" amdump email report issues -- driver.c "nodump" option

2017-11-17 Thread Nathan Stratton Treadway 
On Thu, Nov 16, 2017 at 10:15:28 -0500, Jean-Louis Martineau wrote:
> Nathan,
> 
> Try this patch.
> 
> run: amdump --no-dump --no-flush CONFIG
> Will do all scheduled vault to the vault-storage

[...]

> diff --git a/server-src/driver.c b/server-src/driver.c
> index 351494a..fd01296 100644
> --- a/server-src/driver.c
> +++ b/server-src/driver.c
> @@ -107,6 +107,10 @@ static int taper_started = 0;
>  static int nb_storage;
>  static cmddatas_t *cmddatas = NULL;
>  
> +static int no_dump = FALSE;
> +//static int no_flush = FALSE;
> +static int no_vault = FALSE;
> +
>  static int wait_children(int count);
>  static void wait_for_children(void);
>  static void allocate_bandwidth(netif_t *ip, unsigned long kps);
> @@ -305,6 +309,30 @@ main(
>  }
>  
>  if (argc > 2) {
> + if (g_str_equal(argv[2], "--no-dump")) {
> + no_dump = TRUE;
> + argv++;
> + argc--;
> + }
> +}

I noticed when reviewing the log file from my "amdump --no-dump" run
that there were still "INFO dumper" lines in there, from when the
dumpers start up at the beginning of the run and when they terminate at
the end (with no activity in between).

I wondered if the dumpers should be getting started at all when no_dump
is true, and took a look at driver.c to see how the dumpers were getting
kicked off -- and discovered that driver.c already implements a "nodump"
option, and that indeed if that option is enabled, the dumper processes
are not spawned.

(I see that this "nodump" option is used by Amflush.pm.)

Anyway, I guess the question is whether it makes sense to merge the new
--no-dump and original nodump options for driver.c, at least as far as
having one internal boolean variable to handle both -- or, if not,
if there are any places where the code currently checks for !nodump
where !no_dump should be treated equivalently?

Nathan




Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239