Re: dumporder
Am 06.11.18 um 19:18 schrieb Chris Nighswonger: This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Did you succeed? ;-) I currently try to use amplot on Debian 11.5 and get a ps file where the margins don't match the text etc I also think of generating a png-file after every amdump run and mailing it as well.
Re: dumporder
On Tue, Nov 06, 2018 at 01:18:15PM -0500, Chris Nighswonger wrote: > This seems to work: > > amplot /var/backups/campus/log/amdump.1 > > Running under the amanda user. > > However, the issue now is the attempt to write the output to the X11 terminal: > > > gnuplot: unable to open display '' > gnuplot: X11 aborted. > > Not sure what all that's about. So I'm doing a bit of hacking on the > gnuplot script to have it write the results out to a png file. > You have the permissions on your X screen restrictive (as they should probably be). Thus if you logged in a nathan and ran a program that needs the screen it cannot. The "xhost" command controls the access permissions. jl -- Jon H. LaBadie j...@jgcomp.com 11226 South Shore Rd. (703) 787-0688 (H) Reston, VA 20190 (703) 935-6720 (C)
RE: dumporder
Yah, the plot is a very helpful tool. You can get it to work harder if you can enlarge the work area or add an additional work area. Also you might want to check your chunk size, tendency, at least when I started out, was for many smaller files, I increased the size of the chunks in the holding area and believe it help to improve performance as there were few files and fewer file creates/accesses/deletes later on. From: Chris Nighswonger Sent: Tuesday, November 6, 2018 2:38 PM To: Cuttler, Brian R (HEALTH) Cc: ned.danie...@duke.edu; amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Setting the output to postscript (-p) and then converting to pdf (ps2pdf) worked the trick. It looks like my holding disk maxes out. [20181106020001.jpg] Christopher Nighswonger Faculty Member Network & Systems Director Foundations Bible College & Seminary www.foundations.edu<https://protect2.fireeye.com/url?k=5fb1533cbfe0ffe5.5fb3aa09-e5969cc601a99a3a&u=http://www.foundations.edu> www.fbcradio.org<https://protect2.fireeye.com/url?k=7e89f7371429ff69.7e8b0e02-990af0384ddad8fd&u=http://www.fbcradio.org> - NOTICE: The information contained in this electronic mail message is intended only for the use of the intended recipient, and may also be protected by the Electronic Communications Privacy Act, 18 USC Sections 2510-2521. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please reply to the sender, and delete the original message. Thank you. On Tue, Nov 6, 2018 at 2:26 PM Cuttler, Brian R (HEALTH) mailto:brian.cutt...@health.ny.gov>> wrote: Xhost, and environmental variable DISPLAY, plus your output needs to be x11. You can also write a pdf file and print it, or view with an appropriate viewer. -Original Message- From: owner-amanda-us...@amanda.org<mailto:owner-amanda-us...@amanda.org> mailto:owner-amanda-us...@amanda.org>> On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 1:18 PM To: ned.danie...@duke.edu<mailto:ned.danie...@duke.edu> Cc: amanda-users@amanda.org<mailto:amanda-users@amanda.org> Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley mailto:ned.danie...@duke.edu>> wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my > system, I have to run amplot as user 'amanda'. that means that I have > to be in a directory where amanda has write permission, otherwise > title can't be generated. my home directory doesn't work, but a temp > dir that's chmod 777 does. > > -- > Ned Danieley (ned.danie...@duke.edu<mailto:ned.danie...@duke.edu>) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
RE: dumporder
Xhost, and environmental variable DISPLAY, plus your output needs to be x11. You can also write a pdf file and print it, or view with an appropriate viewer. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 1:18 PM To: ned.danie...@duke.edu Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my > system, I have to run amplot as user 'amanda'. that means that I have > to be in a directory where amanda has write permission, otherwise > title can't be generated. my home directory doesn't work, but a temp > dir that's chmod 777 does. > > -- > Ned Danieley (ned.danie...@duke.edu) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
Re: dumporder
On Tue, Nov 06, 2018 at 13:18:15 -0500, Chris Nighswonger wrote: > However, the issue now is the attempt to write the output to the X11 terminal: > > > gnuplot: unable to open display '' > gnuplot: X11 aborted. > > Not sure what all that's about. So I'm doing a bit of hacking on the > gnuplot script to have it write the results out to a png file. Not sure if this is more useful than getting it to write directly to a PNG file, but the amplot man page lists an existing "-p" option to generate a PostScript file in place of trying to display directly on the screen via X11. Nathan Nathan Stratton Treadway - natha...@ontko.com - Mid-Atlantic region Ray Ontko & Co. - Software consulting services - http://www.ontko.com/ GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt ID: 1023D/ECFB6239 Key fingerprint = 6AD8 485E 20B9 5C71 231C 0C32 15F3 ADCD ECFB 6239
Re: dumporder
This seems to work: amplot /var/backups/campus/log/amdump.1 Running under the amanda user. However, the issue now is the attempt to write the output to the X11 terminal: gnuplot: unable to open display '' gnuplot: X11 aborted. Not sure what all that's about. So I'm doing a bit of hacking on the gnuplot script to have it write the results out to a png file. Chris On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley wrote: > > On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > > Digging around a bit, it appears that it might be a reference to a > > file which is missing. From amplot.g, line 62 we see: > > > > # file title has the parameters that this program needs > > load 'title' > > plot"run_queue" title "Run Queue" with lines,\ > > "tape_queue" title "Tape Queue" with lines,\ > > "finished" title "Dumps Finished" with lines,\ > > "bandw_free" title "Bandwidth Allocated" with lines, \ > > "disk_alloc" title "%Disk Allocated" with lines, \ > > "tape_wait" title "%Tape Wait" with lines,\ > > "tape_idle" title "Taper Idle" with lines,\ > > "dump_idle" title "Dumpers Idle" with lines > > > > Where is a developer when you need one? :-P > > looks like the awk script is supposed to generate 'title'. on my system, I > have to run amplot as user 'amanda'. that means that I have to be in a > directory where amanda has write permission, otherwise title can't be > generated. my home directory doesn't work, but a temp dir that's chmod 777 > does. > > -- > Ned Danieley (ned.danie...@duke.edu) > Department of Biomedical Engineering > Box 90281, Duke University > Durham, NC 27708 (919) 660-5111 > > http://dilbert.com/strips/comic/2012-02-11/
AW: dumporder
Hello Chris, Just a guess without access to a amanda box currently: maybe the file is somewhere in the distribution and a) not installed correctly or b) is expected to be in the current directory, so a 'cd' to another directory might help. Regards, Ingo Originalnachricht Von: Chris Nighswonger Gesendet: Dienstag, 6. November 2018 17:51 An: brian.cutt...@health.ny.gov Cc: amanda-users@amanda.org Betreff: Re: dumporder Digging around a bit, it appears that it might be a reference to a file which is missing. From amplot.g, line 62 we see: # file title has the parameters that this program needs load 'title' plot "run_queue" title "Run Queue" with lines,\ "tape_queue" title "Tape Queue" with lines,\ "finished" title "Dumps Finished" with lines,\ "bandw_free" title "Bandwidth Allocated" with lines, \ "disk_alloc" title "%Disk Allocated" with lines, \ "tape_wait" title "%Tape Wait" with lines,\ "tape_idle" title "Taper Idle" with lines,\ "dump_idle" title "Dumpers Idle" with lines Where is a developer when you need one? :-P On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) wrote: > > It has been a while, title might be the org string from amanda.conf? > I'm sorry, it has been years since I ran it regularly. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:42 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > wrote: > > > > Chris, > > > > There is an amplot utility that I used to run a lot, it would show me what > > I was constrained on, often holding space, which if you are running a bunch > > of large dumps to start off with could constrain dumping later on. > > > > > > -Original Message- > > From: owner-amanda-us...@amanda.org On > > Behalf Of Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:02 AM > > To: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > wrote: > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > inparallel 15 > > dumporder "STs" > > > > Which resulted in a reduced time over what I have been seeing. Here are > > some stats from amstatus for this config. It looks like I could drop about > > half of the dumpers and still be fine. Any idea what causes the dumpers > > over the first five to be utilized less? > > > > dumper0 busy : 5:13:42 ( 97.98%) > > dumper1 busy : 1:12:09 ( 22.54%) > > dumper2 busy : 0:40:30 ( 12.65%) > > dumper3 busy : 0:33:44 ( 10.54%) > > dumper4 busy : 0:03:47 ( 1.19%) > > dumper5 busy : 0:37:32 ( 11.73%) > > dumper6 busy : 0:02:00 ( 0.62%) > > dumper7 busy : 0:00:57 ( 0.30%) > > dumper8 busy : 0:05:54 ( 1.85%) > > dumper9 busy : 0:04:38 ( 1.45%) > > dumper10 busy : 0:00:16 ( 0.08%) > > dumper11 busy : 0:01:39 ( 0.52%) > > dumper12 busy : 0:00:01 ( 0.01%) > > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) > > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) > > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) > > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) > > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) > > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) > > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) > > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) > > 5: 0:00:09 ( 22.36%) > > 4: 0:00:08 ( 18.89%) > > 1: 0:00:01 ( 2.37%) > > 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) > > 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) > > 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) > > 5: 0:00:02 ( 7.13%) > > 4: 0:00:01 ( 3.09%) > > 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) > > 12 dumpers busy : 0
Re: dumporder
On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote: > Digging around a bit, it appears that it might be a reference to a > file which is missing. From amplot.g, line 62 we see: > > # file title has the parameters that this program needs > load 'title' > plot"run_queue" title "Run Queue" with lines,\ > "tape_queue" title "Tape Queue" with lines,\ > "finished" title "Dumps Finished" with lines,\ > "bandw_free" title "Bandwidth Allocated" with lines, \ > "disk_alloc" title "%Disk Allocated" with lines, \ > "tape_wait" title "%Tape Wait" with lines,\ > "tape_idle" title "Taper Idle" with lines,\ > "dump_idle" title "Dumpers Idle" with lines > > Where is a developer when you need one? :-P looks like the awk script is supposed to generate 'title'. on my system, I have to run amplot as user 'amanda'. that means that I have to be in a directory where amanda has write permission, otherwise title can't be generated. my home directory doesn't work, but a temp dir that's chmod 777 does. -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
Re: dumporder
My bad here. I should have inferred this from the man page for amplot, but I've been so wound up in the debug logs that I forgot there are others. This seems to work: amplot /var/backups/campus/log/amdump.1 On Tue, Nov 6, 2018 at 11:54 AM Cuttler, Brian R (HEALTH) wrote: > > I think you need to provide the name of the amconfig. I believe it reads > amanda.conf. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:51 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > Digging around a bit, it appears that it might be a reference to a file which > is missing. From amplot.g, line 62 we see: > > # file title has the parameters that this program needs load 'title' > plot"run_queue" title "Run Queue" with lines,\ > "tape_queue" title "Tape Queue" with lines,\ > "finished" title "Dumps Finished" with lines,\ > "bandw_free" title "Bandwidth Allocated" with lines, \ > "disk_alloc" title "%Disk Allocated" with lines, \ > "tape_wait" title "%Tape Wait" with lines,\ > "tape_idle" title "Taper Idle" with lines,\ > "dump_idle" title "Dumpers Idle" with lines > > Where is a developer when you need one? :-P > > On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) > wrote: > > > > It has been a while, title might be the org string from amanda.conf? > > I'm sorry, it has been years since I ran it regularly. > > > > -Original Message- > > From: Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:42 AM > > To: Cuttler, Brian R (HEALTH) > > Cc: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > The amplot utility is new to me. Here is what it says when I attempt to run > > it: > > > > root@scriptor:~# amplot > > /var/log/amanda/server/campus/amdump.20181106020001.debug > > Displaying graph on the screen, for next graph : MISSING SPACE > > DECLARATION "title", line 62: Cannot open script file 'title' > > > > I have both gnuplot and gawk installed on the backup server. > > > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > > wrote: > > > > > > Chris, > > > > > > There is an amplot utility that I used to run a lot, it would show me > > > what I was constrained on, often holding space, which if you are running > > > a bunch of large dumps to start off with could constrain dumping later on. > > > > > > > > > -Original Message- > > > From: owner-amanda-us...@amanda.org > > > On Behalf Of Chris Nighswonger > > > Sent: Tuesday, November 6, 2018 11:02 AM > > > To: amanda-users@amanda.org > > > Subject: Re: dumporder > > > > > > ATTENTION: This email came from an external source. Do not open > > > attachments or click on links from unknown senders or unexpected emails. > > > > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > > wrote: > > > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > > > inparallel 15 > > > dumporder "STs" > > > > > > Which resulted in a reduced time over what I have been seeing. Here are > > > some stats from amstatus for this config. It looks like I could drop > > > about half of the dumpers and still be fine. Any idea what causes the > > > dumpers over the first five to be utilized less? > > > > > > dumper0 busy : 5:13:42 ( 97.98%) > > > dumper1 busy : 1:12:09 ( 22.54%) > > > dumper2 busy : 0:40:30 ( 12.65%) > > > dumper3 busy : 0:33:44 ( 10.54%) > > > dumper4 busy : 0:03:47 ( 1.19%) > > > dumper5 busy : 0:37:32 ( 11.73%) > > > dumper6 busy : 0:02:00 ( 0.62%) > > > dumper7 busy : 0:00:57 ( 0.30%)
Re: dumporder
On Tue, Nov 06, 2018 at 11:42:16AM -0500, Chris Nighswonger wrote: > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION > "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. I have the same problem. the script 'amplot.g' says # file title has the parameters that this program needs load 'title' is the file 'title' supposed to be part of the distribution? -- Ned Danieley (ned.danie...@duke.edu) Department of Biomedical Engineering Box 90281, Duke University Durham, NC 27708 (919) 660-5111 http://dilbert.com/strips/comic/2012-02-11/
RE: dumporder
I think you need to provide the name of the amconfig. I believe it reads amanda.conf. -Original Message- From: Chris Nighswonger Sent: Tuesday, November 6, 2018 11:51 AM To: Cuttler, Brian R (HEALTH) Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Digging around a bit, it appears that it might be a reference to a file which is missing. From amplot.g, line 62 we see: # file title has the parameters that this program needs load 'title' plot"run_queue" title "Run Queue" with lines,\ "tape_queue" title "Tape Queue" with lines,\ "finished" title "Dumps Finished" with lines,\ "bandw_free" title "Bandwidth Allocated" with lines, \ "disk_alloc" title "%Disk Allocated" with lines, \ "tape_wait" title "%Tape Wait" with lines,\ "tape_idle" title "Taper Idle" with lines,\ "dump_idle" title "Dumpers Idle" with lines Where is a developer when you need one? :-P On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) wrote: > > It has been a while, title might be the org string from amanda.conf? > I'm sorry, it has been years since I ran it regularly. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:42 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > wrote: > > > > Chris, > > > > There is an amplot utility that I used to run a lot, it would show me what > > I was constrained on, often holding space, which if you are running a bunch > > of large dumps to start off with could constrain dumping later on. > > > > > > -Original Message- > > From: owner-amanda-us...@amanda.org > > On Behalf Of Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:02 AM > > To: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > wrote: > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > inparallel 15 > > dumporder "STs" > > > > Which resulted in a reduced time over what I have been seeing. Here are > > some stats from amstatus for this config. It looks like I could drop about > > half of the dumpers and still be fine. Any idea what causes the dumpers > > over the first five to be utilized less? > > > > dumper0 busy : 5:13:42 ( 97.98%) > > dumper1 busy : 1:12:09 ( 22.54%) > > dumper2 busy : 0:40:30 ( 12.65%) > > dumper3 busy : 0:33:44 ( 10.54%) > > dumper4 busy : 0:03:47 ( 1.19%) > > dumper5 busy : 0:37:32 ( 11.73%) > > dumper6 busy : 0:02:00 ( 0.62%) > > dumper7 busy : 0:00:57 ( 0.30%) > > dumper8 busy : 0:05:54 ( 1.85%) > > dumper9 busy : 0:04:38 ( 1.45%) > > dumper10 busy : 0:00:16 ( 0.08%) > > dumper11 busy : 0:01:39 ( 0.52%) > > dumper12 busy : 0:00:01 ( 0.01%) > > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 > > (100.00%) > > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 > > (100.00%) > > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 > > (100.00%) > > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 > > (100.00%) > > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( > > 99.99%) > > 5 dumpers busy : 0:01:07
Re: dumporder
Digging around a bit, it appears that it might be a reference to a file which is missing. From amplot.g, line 62 we see: # file title has the parameters that this program needs load 'title' plot"run_queue" title "Run Queue" with lines,\ "tape_queue" title "Tape Queue" with lines,\ "finished" title "Dumps Finished" with lines,\ "bandw_free" title "Bandwidth Allocated" with lines, \ "disk_alloc" title "%Disk Allocated" with lines, \ "tape_wait" title "%Tape Wait" with lines,\ "tape_idle" title "Taper Idle" with lines,\ "dump_idle" title "Dumpers Idle" with lines Where is a developer when you need one? :-P On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) wrote: > > It has been a while, title might be the org string from amanda.conf? > I'm sorry, it has been years since I ran it regularly. > > -Original Message- > From: Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:42 AM > To: Cuttler, Brian R (HEALTH) > Cc: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > The amplot utility is new to me. Here is what it says when I attempt to run > it: > > root@scriptor:~# amplot > /var/log/amanda/server/campus/amdump.20181106020001.debug > Displaying graph on the screen, for next graph : MISSING SPACE > DECLARATION "title", line 62: Cannot open script file 'title' > > I have both gnuplot and gawk installed on the backup server. > > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) > wrote: > > > > Chris, > > > > There is an amplot utility that I used to run a lot, it would show me what > > I was constrained on, often holding space, which if you are running a bunch > > of large dumps to start off with could constrain dumping later on. > > > > > > -Original Message- > > From: owner-amanda-us...@amanda.org On > > Behalf Of Chris Nighswonger > > Sent: Tuesday, November 6, 2018 11:02 AM > > To: amanda-users@amanda.org > > Subject: Re: dumporder > > > > ATTENTION: This email came from an external source. Do not open attachments > > or click on links from unknown senders or unexpected emails. > > > > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > > wrote: > > > > > > Is there any wisdom available on optimization of dumporder? > > > > > > > After looking over the feedback from Brian and Austin and reviewing the > > actual sizes of the DLEs, I ended up with this sort of thing: > > > > inparallel 15 > > dumporder "STs" > > > > Which resulted in a reduced time over what I have been seeing. Here are > > some stats from amstatus for this config. It looks like I could drop about > > half of the dumpers and still be fine. Any idea what causes the dumpers > > over the first five to be utilized less? > > > > dumper0 busy : 5:13:42 ( 97.98%) > > dumper1 busy : 1:12:09 ( 22.54%) > > dumper2 busy : 0:40:30 ( 12.65%) > > dumper3 busy : 0:33:44 ( 10.54%) > > dumper4 busy : 0:03:47 ( 1.19%) > > dumper5 busy : 0:37:32 ( 11.73%) > > dumper6 busy : 0:02:00 ( 0.62%) > > dumper7 busy : 0:00:57 ( 0.30%) > > dumper8 busy : 0:05:54 ( 1.85%) > > dumper9 busy : 0:04:38 ( 1.45%) > > dumper10 busy : 0:00:16 ( 0.08%) > > dumper11 busy : 0:01:39 ( 0.52%) > > dumper12 busy : 0:00:01 ( 0.01%) > > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 > > (100.00%) > > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 > > (100.00%) > > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 > > (100.00%) > > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 > > (100.00%) > > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( > > 99.99%) > > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( > > 99.98%) > > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( > > 99.59%) > > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( > > 56.35%) > > 5: 0:00:09 ( > > 22.36%) > >
RE: dumporder
It has been a while, title might be the org string from amanda.conf? I'm sorry, it has been years since I ran it regularly. -Original Message- From: Chris Nighswonger Sent: Tuesday, November 6, 2018 11:42 AM To: Cuttler, Brian R (HEALTH) Cc: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. The amplot utility is new to me. Here is what it says when I attempt to run it: root@scriptor:~# amplot /var/log/amanda/server/campus/amdump.20181106020001.debug Displaying graph on the screen, for next graph : MISSING SPACE DECLARATION "title", line 62: Cannot open script file 'title' I have both gnuplot and gawk installed on the backup server. On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) wrote: > > Chris, > > There is an amplot utility that I used to run a lot, it would show me what I > was constrained on, often holding space, which if you are running a bunch of > large dumps to start off with could constrain dumping later on. > > > -Original Message- > From: owner-amanda-us...@amanda.org On > Behalf Of Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:02 AM > To: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > wrote: > > > > Is there any wisdom available on optimization of dumporder? > > > > After looking over the feedback from Brian and Austin and reviewing the > actual sizes of the DLEs, I ended up with this sort of thing: > > inparallel 15 > dumporder "STs" > > Which resulted in a reduced time over what I have been seeing. Here are some > stats from amstatus for this config. It looks like I could drop about half of > the dumpers and still be fine. Any idea what causes the dumpers over the > first five to be utilized less? > > dumper0 busy : 5:13:42 ( 97.98%) > dumper1 busy : 1:12:09 ( 22.54%) > dumper2 busy : 0:40:30 ( 12.65%) > dumper3 busy : 0:33:44 ( 10.54%) > dumper4 busy : 0:03:47 ( 1.19%) > dumper5 busy : 0:37:32 ( 11.73%) > dumper6 busy : 0:02:00 ( 0.62%) > dumper7 busy : 0:00:57 ( 0.30%) > dumper8 busy : 0:05:54 ( 1.85%) > dumper9 busy : 0:04:38 ( 1.45%) > dumper10 busy : 0:00:16 ( 0.08%) > dumper11 busy : 0:01:39 ( 0.52%) > dumper12 busy : 0:00:01 ( 0.01%) > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) > 5: 0:00:09 ( 22.36%) > 4: 0:00:08 ( 18.89%) > 1: 0:00:01 ( 2.37%) > 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) > 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) > 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) > 5: 0:00:02 ( 7.13%) > 4: 0:00:01 ( 3.09%) > 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) > 12 dumpers busy : 0:00:00 ( 0.00%) > 13 dumpers busy : 0:00:00 ( 0.00%) > > I am going to give things another week or so and then try breaking up some of > the very large DLEs in the config.
Re: dumporder
The amplot utility is new to me. Here is what it says when I attempt to run it: root@scriptor:~# amplot /var/log/amanda/server/campus/amdump.20181106020001.debug Displaying graph on the screen, for next graph : MISSING SPACE DECLARATION "title", line 62: Cannot open script file 'title' I have both gnuplot and gawk installed on the backup server. On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) wrote: > > Chris, > > There is an amplot utility that I used to run a lot, it would show me what I > was constrained on, often holding space, which if you are running a bunch of > large dumps to start off with could constrain dumping later on. > > > -Original Message- > From: owner-amanda-us...@amanda.org On Behalf > Of Chris Nighswonger > Sent: Tuesday, November 6, 2018 11:02 AM > To: amanda-users@amanda.org > Subject: Re: dumporder > > ATTENTION: This email came from an external source. Do not open attachments > or click on links from unknown senders or unexpected emails. > > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger > wrote: > > > > Is there any wisdom available on optimization of dumporder? > > > > After looking over the feedback from Brian and Austin and reviewing the > actual sizes of the DLEs, I ended up with this sort of thing: > > inparallel 15 > dumporder "STs" > > Which resulted in a reduced time over what I have been seeing. Here are some > stats from amstatus for this config. It looks like I could drop about half of > the dumpers and still be fine. Any idea what causes the dumpers over the > first five to be utilized less? > > dumper0 busy : 5:13:42 ( 97.98%) > dumper1 busy : 1:12:09 ( 22.54%) > dumper2 busy : 0:40:30 ( 12.65%) > dumper3 busy : 0:33:44 ( 10.54%) > dumper4 busy : 0:03:47 ( 1.19%) > dumper5 busy : 0:37:32 ( 11.73%) > dumper6 busy : 0:02:00 ( 0.62%) > dumper7 busy : 0:00:57 ( 0.30%) > dumper8 busy : 0:05:54 ( 1.85%) > dumper9 busy : 0:04:38 ( 1.45%) > dumper10 busy : 0:00:16 ( 0.08%) > dumper11 busy : 0:01:39 ( 0.52%) > dumper12 busy : 0:00:01 ( 0.01%) > 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) > 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) > 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) > 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) > 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) > 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) > 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) > 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) > 5: 0:00:09 ( 22.36%) > 4: 0:00:08 ( 18.89%) > 1: 0:00:01 ( 2.37%) > 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) > 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) > 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) > 5: 0:00:02 ( 7.13%) > 4: 0:00:01 ( 3.09%) > 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) > 12 dumpers busy : 0:00:00 ( 0.00%) > 13 dumpers busy : 0:00:00 ( 0.00%) > > I am going to give things another week or so and then try breaking up some of > the very large DLEs in the config.
RE: dumporder
Note on that - you want to maximize throughput, not maximize the number of running dumpers. More dumpers moving data is usually good, but more isn't always better. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Cuttler, Brian R (HEALTH) Sent: Tuesday, November 6, 2018 11:11 AM To: Chris Nighswonger ; amanda-users@amanda.org Subject: RE: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Chris, There is an amplot utility that I used to run a lot, it would show me what I was constrained on, often holding space, which if you are running a bunch of large dumps to start off with could constrain dumping later on. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 11:02 AM To: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
RE: dumporder
Chris, There is an amplot utility that I used to run a lot, it would show me what I was constrained on, often holding space, which if you are running a bunch of large dumps to start off with could constrain dumping later on. -Original Message- From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Tuesday, November 6, 2018 11:02 AM To: amanda-users@amanda.org Subject: Re: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
Re: dumporder
On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger wrote: > > Is there any wisdom available on optimization of dumporder? > After looking over the feedback from Brian and Austin and reviewing the actual sizes of the DLEs, I ended up with this sort of thing: inparallel 15 dumporder "STs" Which resulted in a reduced time over what I have been seeing. Here are some stats from amstatus for this config. It looks like I could drop about half of the dumpers and still be fine. Any idea what causes the dumpers over the first five to be utilized less? dumper0 busy : 5:13:42 ( 97.98%) dumper1 busy : 1:12:09 ( 22.54%) dumper2 busy : 0:40:30 ( 12.65%) dumper3 busy : 0:33:44 ( 10.54%) dumper4 busy : 0:03:47 ( 1.19%) dumper5 busy : 0:37:32 ( 11.73%) dumper6 busy : 0:02:00 ( 0.62%) dumper7 busy : 0:00:57 ( 0.30%) dumper8 busy : 0:05:54 ( 1.85%) dumper9 busy : 0:04:38 ( 1.45%) dumper10 busy : 0:00:16 ( 0.08%) dumper11 busy : 0:01:39 ( 0.52%) dumper12 busy : 0:00:01 ( 0.01%) 0 dumpers busy : 0:02:43 ( 0.85%) 0: 0:02:43 (100.00%) 1 dumper busy : 3:39:13 ( 68.47%) 0: 3:39:13 (100.00%) 2 dumpers busy : 0:23:10 ( 7.24%) 0: 0:23:10 (100.00%) 3 dumpers busy : 1:07:57 ( 21.22%) 0: 1:07:57 (100.00%) 4 dumpers busy : 0:02:09 ( 0.67%) 0: 0:02:09 ( 99.99%) 5 dumpers busy : 0:01:07 ( 0.35%) 0: 0:01:07 ( 99.98%) 6 dumpers busy : 0:00:03 ( 0.02%) 0: 0:00:03 ( 99.59%) 7 dumpers busy : 0:00:43 ( 0.22%) 0: 0:00:24 ( 56.35%) 5: 0:00:09 ( 22.36%) 4: 0:00:08 ( 18.89%) 1: 0:00:01 ( 2.37%) 8 dumpers busy : 0:00:20 ( 0.10%) 0: 0:00:19 ( 96.50%) 9 dumpers busy : 0:01:51 ( 0.58%) 0: 0:01:51 ( 99.98%) 10 dumpers busy : 0:00:41 ( 0.22%) 0: 0:00:37 ( 89.71%) 5: 0:00:02 ( 7.13%) 4: 0:00:01 ( 3.09%) 11 dumpers busy : 0:00:07 ( 0.04%) 5: 0:00:07 ( 99.54%) 12 dumpers busy : 0:00:00 ( 0.00%) 13 dumpers busy : 0:00:00 ( 0.00%) I am going to give things another week or so and then try breaking up some of the very large DLEs in the config.
Re: dumporder
On 11/5/2018 1:31 PM, Chris Nighswonger wrote: Is there any wisdom available on optimization of dumporder? This is personal experience only, but I find that in general, if all your dumps run at about the same speed and you don't have to worry about bandwidth, using something like the following generally gets reasonably good behavior: 'ssSS' In essence, it ensures that the smallest dumps complete fast, while still making sure the big ones get started early. Where I work, we've got a couple of slow systems, and I find that this works a bit better under those circumstances: 'ssST' Similar to the above, except it makes sure that the long backups get started early too (I would use 'ssTT', except that we only run one DLE at a time for any given host).
RE: dumporder
Depends on how many dumpers you are using, I like of like TSTSTSts, or something like that. Also very much depends on the size/length of the relative dumps, but I do like to kick off some of the longest ones early. Caveat – I haven’t done enough experimentation to know that what I’m doing is actually reasonable. From: owner-amanda-us...@amanda.org On Behalf Of Chris Nighswonger Sent: Monday, November 5, 2018 1:31 PM To: amanda-users Subject: dumporder ATTENTION: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails. Is there any wisdom available on optimization of dumporder? Kind regards, Chris
dumporder
Is there any wisdom available on optimization of dumporder? Kind regards, Chris
RE: dumporder parameter
Thanks very much. Makes perfect sense now. I did not expect that dump to bypass the holding disk, but I'm a bit low on holding disk space as things stand, so will look into that. Thanks for pointing that out, though. Much appreciated. Johan -Original Message- From: Paul Bijnens [mailto:[EMAIL PROTECTED] Sent: 23 July 2008 12:35 To: Johan Booysen Cc: amanda-users@amanda.org Subject: Re: dumporder parameter Johan Booysen wrote: > Another question: > > How can I verify that those DLE entries were in fact only dumped after > 05:00? In the Amanda email report I can see when the backups completed, > and can more or less work out from that when writing to tape started for > them. But I'd like to check when they actually started dumping. > > I checked through the logs in /var/log/amanda/server/daily, but the only > relevant thing that may be what I'm looking for is from > dumper.20080722235901.debug (my job started at 23:59, and I set up those > disklist entries with starttime 0500): > > 1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2 > feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0 > GNUTAR X X X > |;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list; > OPTIONS features=9ffe00;hostname=server2; > 1216790552.535954: dumper: name = 'server2' > > Any ideas on where I can find this information? > Each line there is tagged with a timestamp, in seconds since epoch. To convert to human readable format use (assuming you are now in GMT+1 $ TZ=UTC perl -le 'print scalar localtime(1216790552.349935)' Wed Jul 23 05:22:32 2008 (leave out the "TZ=UTC" on the local computer to have the time in local time there). btw, the word "PORT-DUMP" indicates that this dump bypassed holdingdisk. If that is what you were expected, then that's ok. Bypassing holdingdisk and using a fast tapedrive usually does not work very well together: the backup datastream cannot follow the tape streaming, resulting in frequent stop/rewind/restart of the tapedrive. This really wears out the hardware of the tape (and at the same time looses lots of capacity in those gaps). Typically LTO drives can fall back to about half the top speed without shoeshining the tape; below that speed the process will damage the drive within a year for daily operations. Keep an eye on that too. -- Paul
Re: dumporder parameter
Johan Booysen wrote: Another question: How can I verify that those DLE entries were in fact only dumped after 05:00? In the Amanda email report I can see when the backups completed, and can more or less work out from that when writing to tape started for them. But I'd like to check when they actually started dumping. I checked through the logs in /var/log/amanda/server/daily, but the only relevant thing that may be what I'm looking for is from dumper.20080722235901.debug (my job started at 23:59, and I set up those disklist entries with starttime 0500): 1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2 feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0 GNUTAR X X X |;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list; OPTIONS features=9ffe00;hostname=server2; 1216790552.535954: dumper: name = 'server2' Any ideas on where I can find this information? Each line there is tagged with a timestamp, in seconds since epoch. To convert to human readable format use (assuming you are now in GMT+1 $ TZ=UTC perl -le 'print scalar localtime(1216790552.349935)' Wed Jul 23 05:22:32 2008 (leave out the "TZ=UTC" on the local computer to have the time in local time there). btw, the word "PORT-DUMP" indicates that this dump bypassed holdingdisk. If that is what you were expected, then that's ok. Bypassing holdingdisk and using a fast tapedrive usually does not work very well together: the backup datastream cannot follow the tape streaming, resulting in frequent stop/rewind/restart of the tapedrive. This really wears out the hardware of the tape (and at the same time looses lots of capacity in those gaps). Typically LTO drives can fall back to about half the top speed without shoeshining the tape; below that speed the process will damage the drive within a year for daily operations. Keep an eye on that too. -- Paul
RE: dumporder parameter
Another question: How can I verify that those DLE entries were in fact only dumped after 05:00? In the Amanda email report I can see when the backups completed, and can more or less work out from that when writing to tape started for them. But I'd like to check when they actually started dumping. I checked through the logs in /var/log/amanda/server/daily, but the only relevant thing that may be what I'm looking for is from dumper.20080722235901.debug (my job started at 23:59, and I set up those disklist entries with starttime 0500): 1216790552.349935: dumper: getcmd: PORT-DUMP 00-6 11003 server2 feff9ffe07 /dir4/backups/backup1 NODEVICE 0 1970:1:1:0:0:0 GNUTAR X X X |;auth=BSD;compress-fast;index;exclude-list=/etc/amanda/exclude-list; OPTIONS features=9ffe00;hostname=server2; 1216790552.535954: dumper: name = 'server2' Any ideas on where I can find this information? Thank you. Johan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen Sent: 22 July 2008 17:07 To: amanda-users@amanda.org Subject: RE: dumporder parameter Hi, No worries - I thought maybe my question was obvious to the point of not being worthy a reply... :) Paul Bijns did point out to me that the starttime parameter would be the one to try, so I will do that. Here's what he said: The dumpers are not spread in order of the disklist. You cannot map the third letter to the third server. There is the disklist parameter "starttime": starttime int Default: none. Backups will not start until after this time of day. The value should be hh*100+mm, e.g. 6:30PM (18:30) would be entered as 1830. e.g. Server2/dir4/backups/backup1 { comp-user-tar dumpcycle 0 # force a full backup each time starttime 0700 } Server3 /dir5/backups/backup2 { comp-user-tar dumpcycle 0 starttime 0705 } So I've implemented his suggestions and will see what happens on the next Amanda run. Thanks! Johan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jon LaBadie Sent: 22 July 2008 15:10 To: amanda-users@amanda.org Subject: Re: dumporder parameter On Tue, Jul 22, 2008 at 09:37:17AM +0100, Johan Booysen wrote: > Hi, > > I got no reply so apologies if it was a stupid question. I'm now simply > using dumporder "Ssss", and Amanda seems to be dumping disklist entries > starting from largest to smallest, which will work just fine for me. > I'm doing level 0 backups each night, and know how long the largest DLE > takes, so can reasonably accurately work out when to start the backup > job. > > Anyways - I'll keep an eye on it just to make sure it keeps on doing > that. > John, I didn't reply because I've never used the feature. As no one else has, I'll suggest you look at the "starttime" parameter. If specified in a dumptype the dump of DLEs using that dumptype will not begin until after the indicated time. > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen > Sent: 18 July 2008 21:20 > To: amanda-users@amanda.org > Subject: dumporder parameter > > I have a question about the dumporder parameter in amanda.conf. > > I want to back up 4 servers. The volumes of data on disk are as > follows: > > Server1 > /dir1 298G > /dir2 74G > /dir3 66G > Server2 > /dir4/backups/backup1 57G > Server3 > /dir5/backups/backup2 28G > Server4 > /dir6/backups/backup3 9.5G > > My problem is that all the data to back up on Server2, Server3, and > Server4 isn't necessarily ready to be backed up until around 7am each > morning (this could possibly be moved to as early as 5am, but not by > much more than that). Server1 can be backed up at any time I wish > during the night. The sizes of the data to be backed up are fairly > static - I don't anticipate massive sudden increases (touch wood). I > also now have the capacity to force full backups each night. The data > on Servers 2, 3, and 4 will change completely each night anyway, so I > know I'll be backing up the amounts as above, even if I used Amanda the > classic way of level 0s and level 1s. > > How can I configure the dumporder parameter to first back up Server1 (I > know how long a level 0 of that will take to dump) before doing the > other servers? Then I can plan more accurately when to actually start > the backup job. > > Something like "SSsS", if I want to back up Server1 first, then Server2, > then S
Re: Another dumporder parameter question.
McGraw, Robert P wrote: When I backup I would like all my larger size files to be put on the first part of the tape and work down to the smaller size files toward the end of the tape. This I believe and hope will provide better tape usage. Yes, it does. If large size files are dumped last to tape there is a good chance that it will hit EOT and will then have to start over and a large chunk of tape will have been wasted and so will time. If smaller size files are sent at the end then if it does hit EOT not much tape is wasted. I have been using "SS". So my question: 1) dose Amanda get the size of all the backup files and then figure the order that it will write to tape? Amanda does an estimate of each DLE, and from those figures decides on which dumps to choose first (dumporder). 2) will Amanda get the size of the tape and then figure the order that it will write files to tape. When taper is free to write the next dumpimage from holdingdisk to tape, then the "taperalgo" parameter decides which image from holdingdisk (only those that are completely done) is the next onej to write to tape. 3) will Amanda do the dumps in the order that it needs to write to tape. No. Dumporder is independent of tapeorder. But tapeorder is influenced by the availability of finished dumps in the holdingdisk. Dumps that bypass holdingdisk are always done last, even when, and mostly because, they are (too) large (to fit in holdingdisk space). See: http://wiki.zmanda.com/index.php/How_To:Fill_tapes_to_100%25
Another dumporder parameter question.
When I backup I would like all my larger size files to be put on the first part of the tape and work down to the smaller size files toward the end of the tape. This I believe and hope will provide better tape usage. If large size files are dumped last to tape there is a good chance that it will hit EOT and will then have to start over and a large chunk of tape will have been wasted and so will time. If smaller size files are sent at the end then if it does hit EOT not much tape is wasted. I have been using "SS". So my question: 1) dose Amanda get the size of all the backup files and then figure the order that it will write to tape? 2) will Amanda get the size of the tape and then figure the order that it will write files to tape. 3) will Amanda do the dumps in the order that it needs to write to tape. Robert smime.p7s Description: S/MIME cryptographic signature
RE: dumporder parameter
Hi, No worries - I thought maybe my question was obvious to the point of not being worthy a reply... :) Paul Bijns did point out to me that the starttime parameter would be the one to try, so I will do that. Here's what he said: The dumpers are not spread in order of the disklist. You cannot map the third letter to the third server. There is the disklist parameter "starttime": starttime int Default: none. Backups will not start until after this time of day. The value should be hh*100+mm, e.g. 6:30PM (18:30) would be entered as 1830. e.g. Server2/dir4/backups/backup1 { comp-user-tar dumpcycle 0 # force a full backup each time starttime 0700 } Server3 /dir5/backups/backup2 { comp-user-tar dumpcycle 0 starttime 0705 } So I've implemented his suggestions and will see what happens on the next Amanda run. Thanks! Johan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jon LaBadie Sent: 22 July 2008 15:10 To: amanda-users@amanda.org Subject: Re: dumporder parameter On Tue, Jul 22, 2008 at 09:37:17AM +0100, Johan Booysen wrote: > Hi, > > I got no reply so apologies if it was a stupid question. I'm now simply > using dumporder "Ssss", and Amanda seems to be dumping disklist entries > starting from largest to smallest, which will work just fine for me. > I'm doing level 0 backups each night, and know how long the largest DLE > takes, so can reasonably accurately work out when to start the backup > job. > > Anyways - I'll keep an eye on it just to make sure it keeps on doing > that. > John, I didn't reply because I've never used the feature. As no one else has, I'll suggest you look at the "starttime" parameter. If specified in a dumptype the dump of DLEs using that dumptype will not begin until after the indicated time. > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen > Sent: 18 July 2008 21:20 > To: amanda-users@amanda.org > Subject: dumporder parameter > > I have a question about the dumporder parameter in amanda.conf. > > I want to back up 4 servers. The volumes of data on disk are as > follows: > > Server1 > /dir1 298G > /dir2 74G > /dir3 66G > Server2 > /dir4/backups/backup1 57G > Server3 > /dir5/backups/backup2 28G > Server4 > /dir6/backups/backup3 9.5G > > My problem is that all the data to back up on Server2, Server3, and > Server4 isn't necessarily ready to be backed up until around 7am each > morning (this could possibly be moved to as early as 5am, but not by > much more than that). Server1 can be backed up at any time I wish > during the night. The sizes of the data to be backed up are fairly > static - I don't anticipate massive sudden increases (touch wood). I > also now have the capacity to force full backups each night. The data > on Servers 2, 3, and 4 will change completely each night anyway, so I > know I'll be backing up the amounts as above, even if I used Amanda the > classic way of level 0s and level 1s. > > How can I configure the dumporder parameter to first back up Server1 (I > know how long a level 0 of that will take to dump) before doing the > other servers? Then I can plan more accurately when to actually start > the backup job. > > Something like "SSsS", if I want to back up Server1 first, then Server2, > then Server4, and then Server3 (Server3 takes the longest to create the > data to back up, so should come last)? > > No sooner had I gotten the backup server to work with two tape drives, > than one tape drive died. Now I have a fancy new tape drive with lots > of capacity, so I'm trying to consolidate backups that previously were > done seperately, i.e. previously Servers 2, 3, and 4 weren't part of the > Amanda backup routine at all. > > I'd appreciate any advice. > > Johan > >>> End of included message <<< -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 12027 Creekbend Drive (703) 787-0884 Reston, VA 20194 (703) 787-0922 (fax)
Re: dumporder parameter
On Tue, Jul 22, 2008 at 09:37:17AM +0100, Johan Booysen wrote: > Hi, > > I got no reply so apologies if it was a stupid question. I'm now simply > using dumporder "Ssss", and Amanda seems to be dumping disklist entries > starting from largest to smallest, which will work just fine for me. > I'm doing level 0 backups each night, and know how long the largest DLE > takes, so can reasonably accurately work out when to start the backup > job. > > Anyways - I'll keep an eye on it just to make sure it keeps on doing > that. > John, I didn't reply because I've never used the feature. As no one else has, I'll suggest you look at the "starttime" parameter. If specified in a dumptype the dump of DLEs using that dumptype will not begin until after the indicated time. > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen > Sent: 18 July 2008 21:20 > To: amanda-users@amanda.org > Subject: dumporder parameter > > I have a question about the dumporder parameter in amanda.conf. > > I want to back up 4 servers. The volumes of data on disk are as > follows: > > Server1 > /dir1 298G > /dir2 74G > /dir3 66G > Server2 > /dir4/backups/backup1 57G > Server3 > /dir5/backups/backup2 28G > Server4 > /dir6/backups/backup3 9.5G > > My problem is that all the data to back up on Server2, Server3, and > Server4 isn't necessarily ready to be backed up until around 7am each > morning (this could possibly be moved to as early as 5am, but not by > much more than that). Server1 can be backed up at any time I wish > during the night. The sizes of the data to be backed up are fairly > static - I don't anticipate massive sudden increases (touch wood). I > also now have the capacity to force full backups each night. The data > on Servers 2, 3, and 4 will change completely each night anyway, so I > know I'll be backing up the amounts as above, even if I used Amanda the > classic way of level 0s and level 1s. > > How can I configure the dumporder parameter to first back up Server1 (I > know how long a level 0 of that will take to dump) before doing the > other servers? Then I can plan more accurately when to actually start > the backup job. > > Something like "SSsS", if I want to back up Server1 first, then Server2, > then Server4, and then Server3 (Server3 takes the longest to create the > data to back up, so should come last)? > > No sooner had I gotten the backup server to work with two tape drives, > than one tape drive died. Now I have a fancy new tape drive with lots > of capacity, so I'm trying to consolidate backups that previously were > done seperately, i.e. previously Servers 2, 3, and 4 weren't part of the > Amanda backup routine at all. > > I'd appreciate any advice. > > Johan > >>> End of included message <<< -- Jon H. LaBadie [EMAIL PROTECTED] JG Computing 12027 Creekbend Drive (703) 787-0884 Reston, VA 20194 (703) 787-0922 (fax)
Re: dumporder parameter
On 2008-07-22 10:37, Johan Booysen wrote: Hi, I got no reply so apologies if it was a stupid question. I'm now simply using dumporder "Ssss", and Amanda seems to be dumping disklist entries starting from largest to smallest, which will work just fine for me. I'm doing level 0 backups each night, and know how long the largest DLE takes, so can reasonably accurately work out when to start the backup job. Correction: dumporder "Ssss" actually means that the first dumper will choose the the largest dump available from the queue, the second to the fourth dumper will chose the smallest from the queue. If you have only 1 dumper running (inparallel 1, or without holdingdisk, or client constrained) then, and only then the dumps go from large to small with that parameter. Use "" to be more certain about the largest to smallest. My problem is that all the data to back up on Server2, Server3, and Server4 isn't necessarily ready to be backed up until around 7am each morning (this could possibly be moved to as early as 5am, but not by much more than that). Server1 can be backed up at any time I wish during the night. The sizes of the data to be backed up are fairly static - I don't anticipate massive sudden increases (touch wood). I also now have the capacity to force full backups each night. The data on Servers 2, 3, and 4 will change completely each night anyway, so I know I'll be backing up the amounts as above, even if I used Amanda the classic way of level 0s and level 1s. > How can I configure the dumporder parameter to first back up Server1 (I know how long a level 0 of that will take to dump) before doing the other servers? Then I can plan more accurately when to actually start the backup job. Something like "SSsS", if I want to back up Server1 first, then Server2, then Server4, and then Server3 (Server3 takes the longest to create the data to back up, so should come last)? The dumpers are not spread in order of the disklist. You cannot map the third letter to the third server. There is the disklist parameter "starttime": starttime int Default: none. Backups will not start until after this time of day. The value should be hh*100+mm, e.g. 6:30PM (18:30) would be entered as 1830. e.g. Server2/dir4/backups/backup1 { comp-user-tar dumpcycle 0 # force a full backup each time starttime 0700 } Server3 /dir5/backups/backup2 { comp-user-tar dumpcycle 0 starttime 0705 } -- Paul Bijnens, xplanation Technology ServicesTel +32 16 397.511 Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax +32 16 397.512 http://www.xplanation.com/ email: [EMAIL PROTECTED] *** * I think I've got the hang of it now: exit, ^D, ^C, ^\, ^Z, ^Q, ^^, * * F6, quit, ZZ, :q, :q!, M-Z, ^X^C, logoff, logout, close, bye, /bye, * * stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt, abort, hangup, * * PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e, kill -1 $$, shutdown, * * init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... * * ... "Are you sure?" ... YES ... Phew ... I'm out * ***
RE: dumporder parameter
Hi, I got no reply so apologies if it was a stupid question. I'm now simply using dumporder "Ssss", and Amanda seems to be dumping disklist entries starting from largest to smallest, which will work just fine for me. I'm doing level 0 backups each night, and know how long the largest DLE takes, so can reasonably accurately work out when to start the backup job. Anyways - I'll keep an eye on it just to make sure it keeps on doing that. Regards, Johan -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Johan Booysen Sent: 18 July 2008 21:20 To: amanda-users@amanda.org Subject: dumporder parameter I have a question about the dumporder parameter in amanda.conf. I want to back up 4 servers. The volumes of data on disk are as follows: Server1 /dir1 298G /dir2 74G /dir3 66G Server2 /dir4/backups/backup1 57G Server3 /dir5/backups/backup2 28G Server4 /dir6/backups/backup3 9.5G My problem is that all the data to back up on Server2, Server3, and Server4 isn't necessarily ready to be backed up until around 7am each morning (this could possibly be moved to as early as 5am, but not by much more than that). Server1 can be backed up at any time I wish during the night. The sizes of the data to be backed up are fairly static - I don't anticipate massive sudden increases (touch wood). I also now have the capacity to force full backups each night. The data on Servers 2, 3, and 4 will change completely each night anyway, so I know I'll be backing up the amounts as above, even if I used Amanda the classic way of level 0s and level 1s. How can I configure the dumporder parameter to first back up Server1 (I know how long a level 0 of that will take to dump) before doing the other servers? Then I can plan more accurately when to actually start the backup job. Something like "SSsS", if I want to back up Server1 first, then Server2, then Server4, and then Server3 (Server3 takes the longest to create the data to back up, so should come last)? No sooner had I gotten the backup server to work with two tape drives, than one tape drive died. Now I have a fancy new tape drive with lots of capacity, so I'm trying to consolidate backups that previously were done seperately, i.e. previously Servers 2, 3, and 4 weren't part of the Amanda backup routine at all. I'd appreciate any advice. Johan
dumporder parameter
I have a question about the dumporder parameter in amanda.conf. I want to back up 4 servers. The volumes of data on disk are as follows: Server1 /dir1 298G /dir2 74G /dir3 66G Server2 /dir4/backups/backup1 57G Server3 /dir5/backups/backup2 28G Server4 /dir6/backups/backup3 9.5G My problem is that all the data to back up on Server2, Server3, and Server4 isn't necessarily ready to be backed up until around 7am each morning (this could possibly be moved to as early as 5am, but not by much more than that). Server1 can be backed up at any time I wish during the night. The sizes of the data to be backed up are fairly static - I don't anticipate massive sudden increases (touch wood). I also now have the capacity to force full backups each night. The data on Servers 2, 3, and 4 will change completely each night anyway, so I know I'll be backing up the amounts as above, even if I used Amanda the classic way of level 0s and level 1s. How can I configure the dumporder parameter to first back up Server1 (I know how long a level 0 of that will take to dump) before doing the other servers? Then I can plan more accurately when to actually start the backup job. Something like "SSsS", if I want to back up Server1 first, then Server2, then Server4, and then Server3 (Server3 takes the longest to create the data to back up, so should come last)? No sooner had I gotten the backup server to work with two tape drives, than one tape drive died. Now I have a fancy new tape drive with lots of capacity, so I'm trying to consolidate backups that previously were done seperately, i.e. previously Servers 2, 3, and 4 weren't part of the Amanda backup routine at all. I'd appreciate any advice. Johan
Re: Question about dumporder
Jean-Louis Martineau schrieb: bandwidth estimation is done using the previous backup of the DLE, You can get the 'dump rates' with: amadmin info Would be cool if this short info could be added to the dumporder option of the amanda.conf manpage. I didn't found this information. Thanks Regards Marc -- Marc Muehlfeld (Leitung Systemadministration) Zentrum fuer Humangenetik und Laboratoriumsmedizin Dr. Klein und Dr. Rost Lochhamer Str. 29 - D-82152 Martinsried Telefon: +49(0)89/895578-0 - Fax: +49(0)89/895578-78 http://www.medizinische-genetik.de
Re: Question about dumporder
bandwidth estimation is done using the previous backup of the DLE, You can get the 'dump rates' with: amadmin info Jean-Louis Marc Muehlfeld wrote: Hello, I have a question about the dumporder parameter. To what are the two bandwidth options (B and b) related? Client/Server connection (e. g. 1Gbit)? Or the real bandwith between server and client? I have a client that is connected by a 1Gbit LAN-card to it's subnet in our branch office. This subnet uses a 4 Mbit connection to exchange data with the subnet in our head office. The amanda server (in the head office subnet) is also connected by a 1Gbit LAN-card. I wanna do some optimizations to ensure that the DLE's of the branch office are backuped first, because a full dump of all DLE's is running a long time (because of the slow connection) and I just want to make sure, that the dump of the client is done as early as possible Regards Marc
Question about dumporder
Hello, I have a question about the dumporder parameter. To what are the two bandwidth options (B and b) related? Client/Server connection (e. g. 1Gbit)? Or the real bandwith between server and client? I have a client that is connected by a 1Gbit LAN-card to it's subnet in our branch office. This subnet uses a 4 Mbit connection to exchange data with the subnet in our head office. The amanda server (in the head office subnet) is also connected by a 1Gbit LAN-card. I wanna do some optimizations to ensure that the DLE's of the branch office are backuped first, because a full dump of all DLE's is running a long time (because of the slow connection) and I just want to make sure, that the dump of the client is done as early as possible Regards Marc -- Marc Muehlfeld (Leitung Systemadministration) Zentrum fuer Humangenetik und Laboratoriumsmedizin Dr. Klein und Dr. Rost Lochhamer Str. 29 - D-82152 Martinsried Telefon: +49(0)89/895578-0 - Fax: +49(0)89/895578-78 http://www.medizinische-genetik.de
Re: Good option for dumporder
On Tuesday 08 July 2003 14:46, Bob Zahn wrote: >This is in response to Gene's and Joshua's additional quesitons: > >1. I'm only running one Samba server and that's on the amanda host > server. > >2. Sorry, disklist looks like: >amanda1 //server1/C$ pc-tar 3 ge0 >amanda1 //server1/D$ pc-tar 4 ge0 >amanda1 //server2/C$ pc-tar 5 ge0 >amanda1 //server2/D$ pc-tar 6 ge0 >amanda1 //server3/C$ pc-tar 11 ge0 >amanda1 //server3/D$ pc-tar 12 ge0 >amanda1 //server4/C$ pc-tar 44 ge0 >amanda1 //server4/D$ pc-tar 45 ge0 >amanda1 //server5/C$ pc-tar 47 ge0 >amanda1 //server5/D$ pc-tar 48 ge0 >From amanda.conf >define dumptype pc-tar { >comment "Full dump of this filesystem always" >program "GNUTAR" >maxdumps 8 >holdingdisk yes >compress none >index yes >priority high >strategy noinc >dumpcycle 0 >} >Also, I have 2 holding disks, 20GB each with chunksize of 1GB. > >3. I thought that the general opinion of this listserv was that > cygwin was not a good option and samba was better for backing up M$ > servers. Have I got this backwards? I'm not the 'expert' in that arena, so I'll let those that have first hand knowledge testify. But I'd suspect that success or failure is more in the hands of cygwin than anything else. Personal opinion of course. My 'samba' bad experience was with linux, not windows, on the other end of the cableing. -- Cheers, Gene AMD [EMAIL PROTECTED] 320M [EMAIL PROTECTED] 512M 99.26% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are: Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: Good option for dumporder
On Tue, 8 Jul 2003 at 1:46pm, Bob Zahn wrote > This is in response to Gene's and Joshua's additional quesitons: > > 1. I'm only running one Samba server and that's on the amanda host server. You'll want to look at your dumprates before and after increasing maxdumps. 8 may lead to too much contention (disk and net), slowing you down disproportionately. As for samba vs. cygwin, I still use samba here, although I don't do whole disk dumps. I just get user data -- I'm never going to recover a whole system from tape only. The only way to know what works best for you is to try it out. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Re: Good option for dumporder
This is in response to Gene's and Joshua's additional quesitons: 1. I'm only running one Samba server and that's on the amanda host server. 2. Sorry, disklist looks like: amanda1 //server1/C$ pc-tar 3 ge0 amanda1 //server1/D$ pc-tar 4 ge0 amanda1 //server2/C$ pc-tar 5 ge0 amanda1 //server2/D$ pc-tar 6 ge0 amanda1 //server3/C$ pc-tar 11 ge0 amanda1 //server3/D$ pc-tar 12 ge0 amanda1 //server4/C$ pc-tar 44 ge0 amanda1 //server4/D$ pc-tar 45 ge0 amanda1 //server5/C$ pc-tar 47 ge0 amanda1 //server5/D$ pc-tar 48 ge0 >From amanda.conf define dumptype pc-tar { comment "Full dump of this filesystem always" program "GNUTAR" maxdumps 8 holdingdisk yes compress none index yes priority high strategy noinc dumpcycle 0 } Also, I have 2 holding disks, 20GB each with chunksize of 1GB. 3. I thought that the general opinion of this listserv was that cygwin was not a good option and samba was better for backing up M$ servers. Have I got this backwards? Bob... -- Original Message -- From: Gene Heskett <[EMAIL PROTECTED]> Reply-To: [EMAIL PROTECTED] Date: Tue, 8 Jul 2003 13:28:20 -0400 >On Tuesday 08 July 2003 12:21, Bob Zahn wrote: >>I'm trying to set up the options in amanda.conf to improve the >>performance (this equates to shortest time to backup) of these five >>Windows NT/2K servers: >> >>server1 C$ 2.19GB >>server1 D$ 35.6GB >>server2 C$ 1.58GB >>server2 D$ 12.9GB >>server3 C$ 2.18GB >>server3 D$ 11.6GB >>server4 C$ 1.3GB >>server4 D$ 20.4GB >>server5 C$ 2.97GB >>server5 D$ 13.7GB >> >>If you run the backup on these servers single threaded it takes >> about 10 hours to backup. I've changed inparallel to 8 and maxdumps >> to 8 also. The dumporder is "sssSsssS". In the latest test, the >> backup completed in 8 hours and 41 minutes. I'm using amanda with >> gnutar and Samba to do the backup. Does anyone have any ideas on >> how I may improve the performance of this backup? Thanks. Bob... > >You didn't post your disklist Bob. And while you can use samba, if >you can get cygwin installed on the m$ boxes so you can build and run >an amanda client on each box, then you have the possibility of doing >as show below. > >One of the things that will speed things up is to make sure that each >disklist entry that is in fact on a different hard drive has a >different, unique to that disk, 'spindle number', usually the last >item on each line of the disklist. This will allow amanda to run in >parallel, a client for each spindle, speeding things up considerably. > >I have no idea how this will play out in a samba environment however >as after the first dozen runs, I learned to avoid that at all costs. >This is due to the broken way windows and samba handle the files >dateing, which results in an incremental being made into a full in >most cases. Since that is never going to be fixed, its time to walk >away Jim, the horse is dead. > >That, combined with making the clients do the compression so that many >clients can all be working on the compression of their own stuff at >the same as the others, and a 10 hour backup can become a 2 hour >backup, depending of course on the speed of the tape drive itself. > >This is dependant on making cygwin run on your M$ boxes I think. >Somebody correct me please if this in in-accurate due to recent >progress on the windows front. My windows are all made of glass >here, a 100% linux house. :) > >>Robert Zahn UNIX Systems Administrator >>Oklahoma City Community College >> S. May Avenue >>Oklahoma City, Ok 73159 >>[EMAIL PROTECTED][EMAIL PROTECTED] >>-- > >-- >Cheers, Gene >AMD [EMAIL PROTECTED] 320M >[EMAIL PROTECTED] 512M >99.26% setiathome rank, not too shabby for a WV hillbilly >Yahoo.com attornies please note, additions to this message >by Gene Heskett are: >Copyright 2003 by Maurice Eugene Heskett, all rights reserved. > > -- Robert Zahn UNIX Systems Administrator Oklahoma City Community College S. May Avenue Oklahoma City, Ok 73159 [EMAIL PROTECTED][EMAIL PROTECTED] --
Re: Good option for dumporder
On Tuesday 08 July 2003 12:21, Bob Zahn wrote: >I'm trying to set up the options in amanda.conf to improve the >performance (this equates to shortest time to backup) of these five >Windows NT/2K servers: > >server1 C$ 2.19GB >server1 D$ 35.6GB >server2 C$ 1.58GB >server2 D$ 12.9GB >server3 C$ 2.18GB >server3 D$ 11.6GB >server4 C$ 1.3GB >server4 D$ 20.4GB >server5 C$ 2.97GB >server5 D$ 13.7GB > >If you run the backup on these servers single threaded it takes > about 10 hours to backup. I've changed inparallel to 8 and maxdumps > to 8 also. The dumporder is "sssSsssS". In the latest test, the > backup completed in 8 hours and 41 minutes. I'm using amanda with > gnutar and Samba to do the backup. Does anyone have any ideas on > how I may improve the performance of this backup? Thanks. Bob... You didn't post your disklist Bob. And while you can use samba, if you can get cygwin installed on the m$ boxes so you can build and run an amanda client on each box, then you have the possibility of doing as show below. One of the things that will speed things up is to make sure that each disklist entry that is in fact on a different hard drive has a different, unique to that disk, 'spindle number', usually the last item on each line of the disklist. This will allow amanda to run in parallel, a client for each spindle, speeding things up considerably. I have no idea how this will play out in a samba environment however as after the first dozen runs, I learned to avoid that at all costs. This is due to the broken way windows and samba handle the files dateing, which results in an incremental being made into a full in most cases. Since that is never going to be fixed, its time to walk away Jim, the horse is dead. That, combined with making the clients do the compression so that many clients can all be working on the compression of their own stuff at the same as the others, and a 10 hour backup can become a 2 hour backup, depending of course on the speed of the tape drive itself. This is dependant on making cygwin run on your M$ boxes I think. Somebody correct me please if this in in-accurate due to recent progress on the windows front. My windows are all made of glass here, a 100% linux house. :) >Robert Zahn UNIX Systems Administrator >Oklahoma City Community College > S. May Avenue >Oklahoma City, Ok 73159 >[EMAIL PROTECTED][EMAIL PROTECTED] >-- -- Cheers, Gene AMD [EMAIL PROTECTED] 320M [EMAIL PROTECTED] 512M 99.26% setiathome rank, not too shabby for a WV hillbilly Yahoo.com attornies please note, additions to this message by Gene Heskett are: Copyright 2003 by Maurice Eugene Heskett, all rights reserved.
Re: Good option for dumporder
On Tue, 8 Jul 2003 at 11:21am, Bob Zahn wrote > I'm trying to set up the options in amanda.conf to improve the > performance (this equates to shortest time to backup) of these five > Windows NT/2K servers: > > server1 C$ 2.19GB > server1 D$ 35.6GB > server2 C$ 1.58GB > server2 D$ 12.9GB > server3 C$ 2.18GB > server3 D$ 11.6GB > server4 C$ 1.3GB > server4 D$ 20.4GB > server5 C$ 2.97GB > server5 D$ 13.7GB How many samba servers are you using? > If you run the backup on these servers single threaded it takes about > 10 hours to backup. I've changed inparallel to 8 and maxdumps to 8 If you have one samba server, and maxdumps 8, you may see a slowdown of your dumprates offsetting your increased parallelism. How do your dumprates compare? -- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Good option for dumporder
I'm trying to set up the options in amanda.conf to improve the performance (this equates to shortest time to backup) of these five Windows NT/2K servers: server1 C$ 2.19GB server1 D$ 35.6GB server2 C$ 1.58GB server2 D$ 12.9GB server3 C$ 2.18GB server3 D$ 11.6GB server4 C$ 1.3GB server4 D$ 20.4GB server5 C$ 2.97GB server5 D$ 13.7GB If you run the backup on these servers single threaded it takes about 10 hours to backup. I've changed inparallel to 8 and maxdumps to 8 also. The dumporder is "sssSsssS". In the latest test, the backup completed in 8 hours and 41 minutes. I'm using amanda with gnutar and Samba to do the backup. Does anyone have any ideas on how I may improve the performance of this backup? Thanks. Bob... -- Robert Zahn UNIX Systems Administrator Oklahoma City Community College S. May Avenue Oklahoma City, Ok 73159 [EMAIL PROTECTED][EMAIL PROTECTED] --
Re: amanda-2.4.3b1 / dumporder warning
On Sat, Dec 08, 2001 at 01:51:21PM -0500, Jean-Louis Martineau wrote: > > I guess I should try this patch with the too short dumporder-string > > again, right? > > Right, it will work with a long dumporder-string but I will > appreciate if you can test my patch. Ok, tested with too short dumporder-string and no warnings in logfile. Thanks, Lipo -- "They that can give up essential liberty | Roland E. Lipovits to obtain a little temporary safety | Vienna, Austria deserve neither liberty nor safety." | DSA-KeyID: 0xDA153FAB - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD
Re: amanda-2.4.3b1 / dumporder warning
On Sat, Dec 08, 2001 at 10:55:27AM -0500, Jean-Louis Martineau wrote: > Try this patch. I guess I should try this patch with the too short dumporder-string again, right? Regards, Lipo -- "They that can give up essential liberty | Roland E. Lipovits to obtain a little temporary safety | Vienna, Austria deserve neither liberty nor safety." | DSA-KeyID: 0xDA153FAB - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD
Re: amanda-2.4.3b1 / dumporder warning
Hello Roland, Try this patch. Jean-Louis On Sat, Dec 08, 2001 at 12:03:29AM +, Roland E. Lipovits wrote: > Roland E. Lipovits <[EMAIL PROTECTED]> wrote: > >Int the logfile and the report, there are many lines saying > >| WARNING driver Unknown dumporder character ' > > > >The dumporder line in amanda.conf is: > >| dumporder "sssS" # specify the priority order of each dumper > > > >Any ideas why this warning is shown? > > Grmbl. May be this happens because I have 'inparallel 5' and so > 'dumporder' should be 5 chars long. I'll try that tomorrow. > > Regards, > Lipo > > -- > "They that can give up essential liberty | Roland E. Lipovits > to obtain a little temporary safety | Vienna, Austria > deserve neither liberty nor safety." | DSA-KeyID: 0xDA153FAB > - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD -- Jean-Louis Martineau email: [EMAIL PROTECTED] Departement IRO, Universite de Montreal C.P. 6128, Succ. CENTRE-VILLETel: (514) 343-6111 ext. 3529 Montreal, Canada, H3C 3J7Fax: (514) 343-5834 --- server-src/driver.c.1 Sat Dec 8 10:48:36 2001 +++ server-src/driver.c Sat Dec 8 10:52:29 2001 @@ -584,7 +584,7 @@ if(!accept) { char dumptype; char *dumporder = getconf_str(CNF_DUMPORDER); - if((strlen(dumporder)+1) <= (dumper-dmptable)) { + if(strlen(dumporder) <= (dumper-dmptable)) { if(dumper-dmptable < 3) dumptype = 's'; else
Re: amanda-2.4.3b1 / dumporder warning
Roland E. Lipovits <[EMAIL PROTECTED]> wrote: >Int the logfile and the report, there are many lines saying >| WARNING driver Unknown dumporder character ' > >The dumporder line in amanda.conf is: >| dumporder "sssS" # specify the priority order of each dumper > >Any ideas why this warning is shown? Grmbl. May be this happens because I have 'inparallel 5' and so 'dumporder' should be 5 chars long. I'll try that tomorrow. Regards, Lipo -- "They that can give up essential liberty | Roland E. Lipovits to obtain a little temporary safety | Vienna, Austria deserve neither liberty nor safety." | DSA-KeyID: 0xDA153FAB - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD
amanda-2.4.3b1 / dumporder warning
Hi, I've tried the new amanda-2.4.3b1, with a amanda.conf file which had the new keywords (dumporder & autoflush) in it, taken from example/amanda.conf. Int the logfile and the report, there are many lines saying | WARNING driver Unknown dumporder character ' The dumporder line in amanda.conf is: | dumporder "sssS" # specify the priority order of each dumper Any ideas why this warning is shown? Regards, Lipo -- "They that can give up essential liberty | Roland E. Lipovits to obtain a little temporary safety | Vienna, Austria deserve neither liberty nor safety." | DSA-KeyID: 0xDA153FAB - Benjamin Franklin, 1759 | RSA-KeyID: 0xBC39A5CD