Re: disklist problem

[Nathan Stratton Treadway]

On Fri, Aug 04, 2017 at 15:57:32 -0600, ghe wrote:
> STRANGE DUMP DETAILS:
>   /-- sbox.slsware.net / lev 1 STRANGE
>   sendbackup: start [sbox.slsware.net:/ level 1]
>   sendbackup: info BACKUP=/bin/tar
>   sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
>   sendbackup: info end
>   ? /bin/tar: ./boot: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./home: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./mnt: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./tmp: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./usr: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./var: directory is on a different filesystem; not dumped
>   | Total bytes written: 2938880 (2.9MiB, 6.6MiB/s)
>   sendbackup: size 2870
>   sendbackup: end
> 
> The directory may be on a different partition, but it's not on a
> different filesystem. They're all on /, and right at the top.

("partition" and "filesystem" mean the same thing in this context.  If
you look at the output of "df" or "mount", I think you will see that
each of these items refers to a mount point for a separate filesystem --
otherwise they would't need to be listed separately in the disklist
file)

> 
> Part of my disklist is:
> 
> ## sbox
> sbox.slsware.net  /   high-tar 40
> sbox.slsware.net  /boot   high-tar 40
> sbox.slsware.net  /home   high-tar 40
> sbox.slsware.net  /tmphigh-tar 40
> sbox.slsware.net  /usrhigh-tar 40
> sbox.slsware.net  /varhigh-tar 40
> 
> Notice that it doesn't ask for /mnt...
> 
> Then down at the bottom it says:
> 
> DUMP SUMMARY:
> HOSTNAMEDISKL ORIG-kB  OUT-kB ...
> -
> sbox.slsware.net/   128702870
> sbox.slsware.net/boot   1  10  10
> sbox.slsware.net/home   0 7691120 7691120
> sbox.slsware.net/tmp1 280 280
> sbox.slsware.net/usr0 3114740 3114740
> sbox.slsware.net/var0 1167590 1167590
> 
> It says some of the DLEs aren't dumped, then it dumps them. And I've
> restored from some of the 'not dumped' DLEs, using amrecover (so
> amanda's directory of the tape exists).

Note that the messages you quoted are in the message section for the 
"sbox.slsware.net /" DLE.  When Amanda calls GNU tar it passes the
--one-file-system option, so when the backup is run for the "/"
filesystem, tar prints those warning messages for each mount point it
runs across as it traverses the directory hierarchy under "/", warning
you that the filesystems mounted on those points are being skipped.

However, because you have listed certain other mount points explicitly
in your disklist file, Amanda will _also_ run tar (in a separate
invocation) for each of those mount points individually -- and thus each
of the DLEs show up in the DUMP SUMMARY section.

In other words, you should think of the message as "the current
invocation of GNU tar is not dumping this filesystem" rather than
"tonight's Amanda run is not dumping this filesystem".

(All of your mount points here are directly under "/", but say you had a
separate filesystem mounted on "/var/big_database".  In that case the message
section for */var* would be the one to include a warning message about
skipping the filesystem under "./big_database", etc.)

> 
> I've been backing up with Amanda for years, and didn't see this back
> then. (This has been happening for quite a while; I'm just wondering
> about it now.)

GNU tar was modified a few years ago to print that "not dumped" warning
message.  I don't remember the details off-hand, but I think it just
silently skipped mount points (assuming --one-file-system was given)
before that, which is why you didn't see those warning messages when you
started.  Let me know if you want additional details and I can dig up
links to more information.

Nathan


Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
Ray Ontko & Co.  -  Software consulting services  -   http://www.ontko.com/
 GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
 Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239

Re: disklist problem

[ghe]

On 08/05/2017 03:35 AM, Jon LaBadie wrote:


Gnutar writes on the standard error stream both error messages
and informational messages.  You are seeing the latter type
for your "/" DLE.

Typically, gnutar is called by amgtar.  Amgtar includes lists
of gnutar messages it can ignore and should not show to you.
But this can be easily tripped up by simple changes in the
message format, eg. an extra space or a case change.

Check the man page for amgtar to see the lists on messages
and for the way to make local changes to them.

Jon


Thank you very much.

From a combination of what you've told me and my guesses, it sounds 
like there's a buglet in amdump/gtar/amgtar/myDisklist that isn't really 
hurting anything, but worries people like me, and can be worked around 
with a little playing around with amgtar. From the man page, it looks 
like some attention to the dumptype param might be in order.


I'll do that RSN, but I'll quit worrying about it. The backups are 
working fine.


Thanks again.

--
Glenn English

Re: disklist problem

[Jon LaBadie]

On Fri, Aug 04, 2017 at 03:57:32PM -0600, ghe wrote:
> Debian Jessie, amanda 3.3.6
> 
> After a backup, amreport says things like:
> 
> STRANGE DUMP DETAILS:
>   /-- sbox.slsware.net / lev 1 STRANGE
>   sendbackup: start [sbox.slsware.net:/ level 1]
>   sendbackup: info BACKUP=/bin/tar
>   sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
>   sendbackup: info end
>   ? /bin/tar: ./boot: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./home: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./mnt: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./tmp: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./usr: directory is on a different filesystem; not dumped
>   ? /bin/tar: ./var: directory is on a different filesystem; not dumped
>   | Total bytes written: 2938880 (2.9MiB, 6.6MiB/s)
>   sendbackup: size 2870
>   sendbackup: end
> 
> The directory may be on a different partition, but it's not on a different
> filesystem. They're all on /, and right at the top. They're all RAID 1 too,
> if that makes any difference.
> 
> Part of my disklist is:
> 
> ## sbox
> sbox.slsware.net  /   high-tar 40
> sbox.slsware.net  /boot   high-tar 40
> sbox.slsware.net  /home   high-tar 40
> sbox.slsware.net  /tmphigh-tar 40
> sbox.slsware.net  /usrhigh-tar 40
> sbox.slsware.net  /varhigh-tar 40
> 
> Notice that it doesn't ask for /mnt...
> 
> Then down at the bottom it says:
> 
> DUMP SUMMARY:
> HOSTNAMEDISKL ORIG-kB  OUT-kB ...
> -
> sbox.slsware.net/   128702870
> sbox.slsware.net/boot   1  10  10
> sbox.slsware.net/home   0 7691120 7691120
> sbox.slsware.net/tmp1 280 280
> sbox.slsware.net/usr0 3114740 3114740
> sbox.slsware.net/var0 1167590 1167590
> 
> It says some of the DLEs aren't dumped, then it dumps them. And I've
> restored from some of the 'not dumped' DLEs, using amrecover (so amanda's
> directory of the tape exists).
> 
> I've been backing up with Amanda for years, and didn't see this back then.
> (This has been happening for quite a while; I'm just wondering about it
> now.)
> 
> I haven't been able to find the error. What's wrong (if anything)?
> 
Gnutar writes on the standard error stream both error messages
and informational messages.  You are seeing the latter type
for your "/" DLE.

Typically, gnutar is called by amgtar.  Amgtar includes lists
of gnutar messages it can ignore and should not show to you.
But this can be easily tripped up by simple changes in the
message format, eg. an extra space or a case change.

Check the man page for amgtar to see the lists on messages
and for the way to make local changes to them.

Jon
-- 
Jon H. LaBadie j...@jgcomp.com
 11226 South Shore Rd.  (703) 787-0688 (H)
 Reston, VA  20190  (703) 935-6720 (C)

Re: disklist

[John Aguilar]

True, but I read where if you edit it manually, the --m (modify) flag may
not work subsequently. so I was just wondering.

Thanks Paul for the help, I was thinking about just editing it directly,
but was hoping to maybe find a shortcut.

John

On Thu, Mar 24, 2016 at 2:32 PM, Paul Yeatman 
wrote:

> I don't know that it can be done with the amaddclient command but it may
> be faster to just edit the disklist file directly.  I believe that is all
> amaddclient is doing: adding entries to the disklist file.
>
> Paul
>
>
> --
> *From:* owner-amanda-us...@amanda.org  on
> behalf of John Aguilar 
> *Sent:* Thursday, March 24, 2016 11:07 AM
> *To:* amanda-users@amanda.org
> *Subject:* disklist
>
> I am currently setting up the latest release of amanda for centos 7 and I
> am wondering if it is possible, and if it is, how to specify it on command
> line.  I am trying to add a client to my amanda server with amaddclient
> command, but it seems I can only specify one path to directories I want to
> back up. i.e.* $amaddclient --config DailySet1 --client host. example.com
>  --diskdev /home --dumptype comp-user-tar*
>
> backing up my home directories is one path, but I would like to add
> others. i.e. /etc/, /var/www/htl..etc.. on the same disklist.
>
> Is there a way to do that without having needing to run amaddclient
> command for each --diskdev path to backup?
>
>
> Thanks,
> John
>



-- 
*John Aguilar,*
Sr. Computer Systems Administrator
Kno.e.sis Center(knoesis.wright.edu)
Wright State University
(937) 775-5255

Re: disklist and glob patterns

[Michael Stauffer]

For the sake of anyone following this thread in the future, some replies to
my last post:


On Tue, Mar 11, 2014 at 4:12 PM, Michael Stauffer mgsta...@gmail.comwrote:

 Thanks Jean-Lous.

 In my dumptype I have

   program GNUTAR

 but I don't know if this always means 'gtar' or if it's defined somewhere.
 I can't find a define for it.

 However, other 'include' expressions seem to be globbing correclty, even
 though the amgtar docs says gtar won't normal accept them. e.g. this
 expression is working:

  include ./[f-i]*

 So maybe GNUTAR is defined to use amgtar?


I haven't figured this out for sure, but based on the behavior of amdump
with the dumptype using 'program GNUTAR', yes, this means amdump is using
amgtar.


 Assuming I'm using amgtar, then it seems since it only manually globs
 expressions with a single forward slash, I should change my DLE to this,
 which includes the sub-dir in the DLE diskdevice:

 cfile.uphs.upenn.edu jet-grosspeople-h-z /jet/grosspeople {
gui-base
include ./[h-zH-Z]*
exclude ./Volumetric
}

 Does that seem right?


Yes, this works, i.e. when the DLE path itself references the sub-dir I'm
trying to break into smaller DLE's. This is consistent with the amgtar
documentation's explanation of processing only 'include' directives with a
single forward-slash.


 Regarding the related globbing issue:

 As an aside (or possibly related?) the case-sensitivity of globbing on my
 client is not behaving how I'd expect. e.g. 'echo [a-c]*' includes files
 that start with capital A-B, which I don't expect. Files starting with C
 are *not* listed. My shell option nocaseglob is off, and I've tried setting
 and unsetting it just to test. Nothing changes. I'll post about this last
 bit to another list too.


 It seems that with shift to unicode years ago, the sorting order doesn't
 follow ascii order by default anymore.

 If I add 'export LC_COLLATE=C' to my shell, then 'echo [a-c]*' behaves as
 expected. Assuming that amgtar uses shell globbing to do its manually
 globbing of 'include' expressions, and since gtar seems to use shell
 globbing on its own for 'exclude' expressions, I figure I should add
 'export LC_COLLATE=C' to my amandabackup profiles. It seems also that for
 'exclude' expressions, I could use  '[[:lower:]]' to indicate lower case,
 e.g. But then that wouldn't work for 'include' expressions b/c of amgtar's
 manual globbing.


This is working for me. i.e. globbing of ranges of characters within DLE's
follows older ascii-centric rules after setting LC_COLLATE=C.

-M

Re: disklist and glob patterns

[Jean-Louis Martineau]

Michael,

Look at the amgtar man page if you are using amgtar:

   Similarly, include expressions are supplied to GNU-tar's 
--files-from
   option. This option ordinarily does not accept any sort of 
wildcards,

   but amgtar manually applies glob pattern matching to include
   expressions with only one slash. The expressions must still 
begin with

   ./, so this effectively only allows expressions like ./[abc]* or
   ./*.txt.

Jean-Louis

On 03/11/2014 01:53 PM, Michael Stauffer wrote:

Amanda 3.3.4

Hi,

I'm confused about using glob patterns in disklist. Am I right that if 
I use dumptype include and exclude directives, I can use shell globbing?


This is part of my disklist:

cfile.uphs.upenn.edu http://cfile.uphs.upenn.edu jet-grosspeople-0-g 
/jet {

   gui-base
   #Get everything ^[0-9] and ^[a-zA-Z]
   include ./grosspeople
   exclude ./grosspeople/[h-zH-Z]*
   }
cfile.uphs.upenn.edu http://cfile.uphs.upenn.edu jet-grosspeople-h-z 
/jet {

   gui-base
   include ./grosspeople/[h-zH-Z]*
   exclude ./grosspeople/Volumetric
   }
cfile.uphs.upenn.edu http://cfile.uphs.upenn.edu 
jet-grosspeople-Volumetric /jet {

   gui-base
   include ./grosspeople/Volumetric
   }

My log shows me this warning:

STRANGE dumper cfile.uphs.upenn.edu http://cfile.uphs.upenn.edu 
jet-grosspeople-h-z 1 [sec 0.232 kb 10 kps 43.1 orig-kb 10]

  sendbackup: start [cfile.uphs.upenn.edu:jet-grosspeople-h-z level 1]
  sendbackup: info BACKUP=/bin/gtar
  sendbackup: info RECOVER_CMD=/bin/gtar -xpGf - ...
  sendbackup: info end
  ? /bin/gtar: ./grosspeople/[h-zH-Z]*: Warning: Cannot stat: No such 
file or directory
  ? /bin/gtar: ./grosspeople/[h-zH-Z]*: Warning: Cannot stat: No such 
file or directory

  | Total bytes written: 10240 (10KiB, 41MiB/s)
  sendbackup: size 10
  sendbackup: end

If I do a shell glob manually on the client, e.g. 'echo 
/jet/grosspeople/[h-zH-Z]*', there's no warning and it shows the list 
of files.


Interestingly, it does not complain about the same glob in the 
 jet-grosspeople-0-g DLE. And, the reported size from 'amstatus jet1' 
of the jet-grosspeople-0-g DLE looks to match the size I expect if the 
'exclude ./grosspeople/[h-zH-Z]*' directive was properly exectued.


Anyone know why this might be be responding differently to the same glob?

As an aside (or possibly related?) the case-sensitivity of globbing on 
my client is not behaving how I'd expect. e.g. 'echo [a-c]*' includes 
files that start with capital A-B, which I don't expect. Files 
starting with C are *not* listed. My shell option nocaseglob is off, 
and I've tried setting and unsetting it just to test. Nothing changes. 
I'll post about this last bit to another list too.


Thanks for any thoughts.

-M

Re: disklist and glob patterns

[Michael Stauffer]

Thanks Jean-Lous.

In my dumptype I have

  program GNUTAR

but I don't know if this always means 'gtar' or if it's defined somewhere.
I can't find a define for it.

However, other 'include' expressions seem to be globbing correclty, even
though the amgtar docs says gtar won't normal accept them. e.g. this
expression is working:

 include ./[f-i]*

So maybe GNUTAR is defined to use amgtar?

Assuming I'm using amgtar, then it seems since it only manually globs
expressions with a single forward slash, I should change my DLE to this,
which includes the sub-dir in the DLE diskdevice:

cfile.uphs.upenn.edu jet-grosspeople-h-z /jet/grosspeople {
   gui-base
   include ./[h-zH-Z]*
   exclude ./Volumetric
   }

Does that seem right?

Regarding the related globbing issue:

As an aside (or possibly related?) the case-sensitivity of globbing on my
 client is not behaving how I'd expect. e.g. 'echo [a-c]*' includes files
 that start with capital A-B, which I don't expect. Files starting with C
 are *not* listed. My shell option nocaseglob is off, and I've tried setting
 and unsetting it just to test. Nothing changes. I'll post about this last
 bit to another list too.


It seems that with shift to unicode years ago, the sorting order doesn't
follow ascii order by default anymore.

If I add 'export LC_COLLATE=C' to my shell, then 'echo [a-c]*' behaves as
expected. Assuming that amgtar uses shell globbing to do its manually
globbing of 'include' expressions, and since gtar seems to use shell
globbing on its own for 'exclude' expressions, I figure I should add
'export LC_COLLATE=C' to my amandabackup profiles. It seems also that for
'exclude' expressions, I could use  '[[:lower:]]' to indicate lower case,
e.g. But then that wouldn't work for 'include' expressions b/c of amgtar's
manual globbing.

Thoughts?

Thanks

-M


On Tue, Mar 11, 2014 at 2:04 PM, Jean-Louis Martineau
martin...@zmanda.comwrote:

  Michael,

 Look at the amgtar man page if you are using amgtar:

Similarly, include expressions are supplied to GNU-tar's
 --files-from
option. This option ordinarily does not accept any sort of
 wildcards,
but amgtar manually applies glob pattern matching to include
expressions with only one slash. The expressions must still begin
 with
./, so this effectively only allows expressions like ./[abc]* or
./*.txt.

 Jean-Louis


 On 03/11/2014 01:53 PM, Michael Stauffer wrote:

 Amanda 3.3.4

  Hi,

  I'm confused about using glob patterns in disklist. Am I right that if I
 use dumptype include and exclude directives, I can use shell globbing?

  This is part of my disklist:

  cfile.uphs.upenn.edu jet-grosspeople-0-g /jet {
gui-base
#Get everything ^[0-9] and ^[a-zA-Z]
include ./grosspeople
exclude ./grosspeople/[h-zH-Z]*
}
 cfile.uphs.upenn.edu jet-grosspeople-h-z /jet {
 gui-base
include ./grosspeople/[h-zH-Z]*
exclude ./grosspeople/Volumetric
}
 cfile.uphs.upenn.edu jet-grosspeople-Volumetric /jet {
gui-base
include ./grosspeople/Volumetric
}

  My log shows me this warning:

  STRANGE dumper cfile.uphs.upenn.edu jet-grosspeople-h-z 1 [sec 0.232 kb
 10 kps 43.1 orig-kb 10]
   sendbackup: start [cfile.uphs.upenn.edu:jet-grosspeople-h-z level 1]
   sendbackup: info BACKUP=/bin/gtar
   sendbackup: info RECOVER_CMD=/bin/gtar -xpGf - ...
   sendbackup: info end
   ? /bin/gtar: ./grosspeople/[h-zH-Z]*: Warning: Cannot stat: No such file
 or directory
   ? /bin/gtar: ./grosspeople/[h-zH-Z]*: Warning: Cannot stat: No such file
 or directory
   | Total bytes written: 10240 (10KiB, 41MiB/s)
   sendbackup: size 10
   sendbackup: end

  If I do a shell glob manually on the client, e.g. 'echo
 /jet/grosspeople/[h-zH-Z]*', there's no warning and it shows the list of
 files.

  Interestingly, it does not complain about the same glob in the
  jet-grosspeople-0-g DLE. And, the reported size from 'amstatus jet1' of
 the jet-grosspeople-0-g DLE looks to match the size I expect if the
 'exclude ./grosspeople/[h-zH-Z]*' directive was properly exectued.

  Anyone know why this might be be responding differently to the same glob?

  As an aside (or possibly related?) the case-sensitivity of globbing on
 my client is not behaving how I'd expect. e.g. 'echo [a-c]*' includes files
 that start with capital A-B, which I don't expect. Files starting with C
 are *not* listed. My shell option nocaseglob is off, and I've tried setting
 and unsetting it just to test. Nothing changes. I'll post about this last
 bit to another list too.

  Thanks for any thoughts.

  -M

Re: Disklist question

[Brian Cuttler]

Robert,

On Mon, Oct 06, 2008 at 10:16:29AM -0400, McGraw, Robert P wrote:
 I want to be able to use a different disklist for different runs. Other than
 use shell scripting to mv disklista to disklist is there a way to tell
 amanda what disklist to use during a run.

What needs to remain the same ? Similar tape pool ?
Soft link files between different amanda run and use
cron to choose which to run ?

What are you trying to achieve though ? What is the goal you
are shooting for ? You have proposed a solution but I don't know
what the problem statement was.

 Robert
 
 
 
 
 -- 
 Robert P. McGraw, Jr.
 Manager, Computer System   EMAIL: [EMAIL PROTECTED]
 Purdue University   ROOM: MATH-807
 Department of Mathematics  PHONE: (765) 494-6055
 150 N. University Street
 West Lafayette, IN 47907-2067   
 


---
   Brian R Cuttler [EMAIL PROTECTED]
   Computer Systems Support(v) 518 486-1697
   Wadsworth Center(f) 518 473-6384
   NYS Department of HealthHelp Desk 518 473-0773



IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure.  It
is intended only for the addressee.  If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments.  Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.

Re: Disklist question

[Jean-Louis Martineau]

man amanda.conf:
  diskfile  string
  Default: disklist. The file name for the disklist file 
holding client hosts, disks and other client dumping

  information.

You can use: amdump -odiskfile=disklista

Why do you want to do use different disklist file? You could have one 
disklist and specify the DLE on the command line.


Jean-Louis


McGraw, Robert P wrote:
I want to be able to use a different disklist for different runs. 
Other than use shell scripting to mv disklista to disklist is there a 
way to tell amanda what disklist to use during a run.


Robert




--
Robert P. McGraw, Jr.
Manager, Computer System   EMAIL: [EMAIL PROTECTED]
Purdue University   ROOM: MATH-807
Department of Mathematics  PHONE: (765) 494-6055
150 N. University Street 
West Lafayette, IN 47907-2067   

Re: disklist help

[Paul Bijnens]

khalid maqsudi wrote:
What should my disklist look like if I want to dump the entire c1t0d0s5 
slice (which is also my holding disk) to tape.   I have only 1 machine 
(master) and no clients connected to a powervault 128T tape device 
running solaris 8.  Any help would be appreciated. Thanks.  Also,  How 
can I verify if I have the client software on my master (or was that 
taken care of automatically when I compiled AMANDA on it as a master? )  


Running amcheck as user amanda on the server should pass without
errors.

The disklist entry could be as simple as:

my.master.com  c1t0d0s5  holding-disk

where holding-disk is a dumptype having the holdingdisk no
parameter (to avoid feeding the output to the input).  But by
bypassing the holdingdisk, you fall into the trouble of a non-optimal
backup with a streaming tape that does not stream anymore (and is
broken after a few weeks too probably).
It is better to configure the disk so that you can backup everything
except the holdingdisk directory.  Asume the slice is mounted
under /space, then:

mkdir /space/amandahold
mkdir /space/users
mkdir /space/projects

and then use gnutar to backup the /space/users and /space/projects
only (no need to bypass holdingdisk then).
Or you could use the gnutar exclude mechanism with a disklist
entry like:

my.master.com /space {
comp-user-tar
exclude append ./amandahold
}

--
Paul Bijnens, XplanationTel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax  +32 16 397.512
http://www.xplanation.com/  email:  [EMAIL PROTECTED]
***
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* init 0, kill -9 1, Alt-F4, Ctrl-Alt-Del, AltGr-NumLock, Stop-A, ... *
* ...  Are you sure?  ...   YES   ...   Phew ...   I'm out  *
***

RE: Disklist file

[Gavin Henry]

Two amanda configs needed and two cronjobs?

-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of Kaushal Shriyan
Sent: 23 August 2004 05:40
To: [EMAIL PROTECTED]
Subject: Disklist file


Hi !

 i have two sets of client
 one who owns laptop and others who own desktop
 now i wanted to take backup of Laptop
 at 13:00 hrs and desktop at 21:00 hrs
 is there any way out to have seperate disklist file
 i mean one for laptop and other for desktop


-- 
Regards,

Kaushal Shriyan

Technical Engineer
Red Hat India Pvt. Ltd.
Tel  : +91-22-22881326/27
Fax  : +91-22-22881318
Cell : +91-9820367783

Re: disklist and whitespaces

[Andrea Borgia]

Alexander Jolk wrote:

Then create a symlink to that directory with a simple name from
somewhere else and backup that symlink (not the directory containing it,
but the link itself).
So brilliantly simple and obvious that I didn't think of it 8-)
I still believe amanda should deal with such filenames properly, but 
this workaround is more than ok with me. Tested it today and it appears 
to be fine.

Thanks,
Andrea.
--
Homepage: http://andrea.borgia.bo.it /Amateur radio: IZ4FHT
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Re: Disklist

[Joshua Baker-LePain]

On Fri, 16 Jul 2004 at 4:15pm, Kaushal Shriyan wrote

 I wanted to know some more things qualifier used in the disklist file of
 amanda, At present i am using comp-root-tar to backup my 
 amanda client are there more qualifiers which can be used in the 
 disklist file

Please just email such questions to the amanda-users list.  This is off 
topic for amanda-hackers, and sending a copy straight to Paul is not 
really necessary.

If you're asking what I think you're asking, there are more dumptypes 
defined in the sample amanda.conf.  You can also define your own in there.

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Re: Disklist

[Paul Bijnens]

Kaushal Shriyan wrote:
Dear Paul Bijnens
I'm not married to Amanda :-)
While I'm waiting while some of my heavy program finish, I like to
respond to posts, where I believe I can help.
This little remark just to say that others may respond too.
(Amanda-hackers is appropriate for this post, I removed it.)
I wanted to know some more things qualifier used in the disklist file of
amanda, At present i am using comp-root-tar to backup my 
amanda client are there more qualifiers which can be used in the 
disklist file
From man amanda, section DISKLIST:
  hostname diskname [ diskdevice ] dumptype [ spindle [ interface ] ]
And dumptype itself is explained the DUMPTYPE SECTION.
(I'm not gonna copy/paste the man page here.)
To have a look at the different possibilities to use these,
have a look in the docs/disklist file where you find many examples.

--
Paul Bijnens, XplanationTel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax  +32 16 397.512
http://www.xplanation.com/  email:  [EMAIL PROTECTED]
***
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...*
* ...  Are you sure?  ...   YES   ...   Phew ...   I'm out  *
***

Re: disklist and whitespaces

[Michael D Schleif]

* Andrea Borgia [EMAIL PROTECTED] [2004:07:15:18:48:50+0200] scribed:
 Hello.
 
 On one host I would like to backup the Windows Documents and Settings 
 directory from Linux, but it appears Amanda is bitching about those 
 embedded whitespaces.
 
 I've tried both escaping them and quoting the whole path (as above), but 
 still no luck.
 
 Has anyone dealt with a similar problem before? Apart from simply 
 ignoring such paths, how should I add them to the disklist?

Assuming that you are backing up over Samba, what I do is share
Documents and Settings as something else (e.g., docs), which
alleviates this problem.

hth

-- 
Best Regards,

mds
mds resource
877.596.8237
-
Dare to fix things before they break . . .
-
Our capacity for understanding is inversely proportional to how much
we think we know.  The more I know, the more I know I don't know . . .
--


signature.asc
Description: Digital signature

Re: disklist and whitespaces

[Andrea Borgia]

Michael D Schleif wrote:
Assuming that you are backing up over Samba, what I do is share
Documents and Settings as something else (e.g., docs), which
alleviates this problem.
Nope, sorry I forgot that bit: the client is running Linux and I am 
trying to backup that directory that is located on the Windows partition 
on the same disk and mounted read-only.

Can anything be done in this situation? I mean, short of having XP 
zipping the directory and placing the result in a well-behaved path.

Andrea.
--
Homepage: http://andrea.borgia.bo.it /Amateur radio: IZ4FHT
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?

Re: disklist and whitespaces

[Alexander Jolk]

Andrea Borgia wrote:
 the client is running Linux and I am
 trying to backup that directory that is located on the Windows partition
 on the same disk and mounted read-only.

Then create a symlink to that directory with a simple name from
somewhere else and backup that symlink (not the directory containing it,
but the link itself).

Alex

-- 
Alexander Jolk / BUF Compagnie
tel +33-1 42 68 18 28 /  fax +33-1 42 68 18 29

Re: disklist with host that are turned off over the weekend

[Stefan G. Weichinger]

Hi, Jobst,

on Freitag, 30. Jänner 2004 at 02:18 you wrote to amanda-users:

JS I have machines that are turned off over the weekend
JS (friday late afternoon) but are mentioned in the disklist.

JS Is this OK to do?

Depends on what you want to do and how.

If you only dump on weekdays this won't be a problem.

If the DLEs are offline while amcheck or amdump you will get error
messages.

But the DLEs that are online will still get dumped.
-- 
best regards,
Stefan

Stefan G. Weichinger
mailto:[EMAIL PROTECTED]

Re: disklist and wildcard question

[Paul Bijnens]

Dan L. Ostrom wrote:

Running the command:

tar cvf /tmp/mytar *200300*

does just tar up the files requested. Since amdump is running 
the same tar command
(yes, tar is gtar here), shouldn't the disklist file entries work? Or am 
I misunderstanding
what the include/exclude statements are for or how they function?
The nice thing about open source software is that you can dive into the
code to find out.
And if you don't want to read the soure code, the exact command is
found in /tmp/amanda/runtar.TIMESTAMP.debug.
Amanda uses the --file-from directive of gnutar.  Each entry in that
file is a file or directory expanded *by amanda* from the pattern
that you give (hence the restriction of one level expansion only,
as the man page indicates).


--
Paul @ Home

Re: disklist file maintenance

[Brian Cuttler]

Bruno,

(No problem with list posting)

Yes, I see that as a valid potential way to go, if only it
was that simple the script you are suggesting could run in
advance of the amanda-server startup.

You know, actually its not a bad solution, I was going to
object that it wasn't a matter of home directories but of
directories on a non-home data disk on an amanda-client.

However the script could/would work just fine as the client
disks are all nfs mountable (automounter) and I can get the
list of top level directories.

I'd thought there was a more automated way but this should
work around the issue pretty well.

You have any experience with comp-tar usage on sgi and solaris
(vs discussions that I'd seen for linux systems) ?

thanks,

Brian

 (See brian, i´m posting your message on the list, ok?)
 
 Oh, Let´s see if I correctly understood your problem: you need to backup
 your user directories, like /home/brian,  /home/john, /home/melissa, (etc).
 But you can´t simply make a backup of your entire /home directory.
 So you, basically, would need a disklist file with the following entries:
 
 machine1/home/briancomp-root-tar
 machine1/home/jon   comp-root-tar
 machine1/home/melissa comp-root-tar
 (and so on for every user directory)
 
 However, you don´t want to manually edit the disklist file every time you
 create and delete a user.
 Is this right?
 
 Well, the first thing that occurred to me is to do a script that runs before
 amdump which would check what are the valid user directories of your machine
 and would write these names in the disklist file. (actually, it would write
 the correct disklist file entries, one for each directory).
 
 If someone know anything better, share with us.
 
 Hope it helps,
 Bruno.
 
 - Original Message - 
 From: Brian Cuttler
 To: Bruno Negrão
 Cc: Brian Cuttler
 Sent: Thursday, September 18, 2003 3:28 PM
 Subject: Re: Using amanda with just 1 tape - for testing - SOLVED
 
 
 
 Bruno,
 
 I realized you are writing about tapecycle (my latest guess was
 that a tapecycle of 0 is a problem) but as long as you are using
 tar rather than dump I've got a question.
 
 I've got a couple of large partitions that I'd like to move
 from dump to tar, but only specific partitions within the
 disklist and I don't know now to divide up the data ? Dividing
 at highest level on the partition by username might work ok
 (directories at that level named for the data owner) but I
 don't really want to maintain the disklist (look to see if
 it needs updating everytime I create/remove a user data area).
 
 Are you doing anything like this ? I'm in need of the feature
 for both Solaris/IRIX servers/clients.
 
 What version of amanda are you running ? Or rather, what is the
 min version that supports what I'm hoping to do ?
 
 thanks,
 
 Brian
 
 

Re: disklist file maintenance

[Bruno Negrão]

Yes, I see that as a valid potential way to go, if only it
was that simple the script you are suggesting could run in
advance of the amanda-server startup.
yes, in your crontab file you could add a line like this:
(say the script would be called dlmaintainer)

00 2 * * * (dlmaintainer  su amanda -c amdump DailySetX)

ps: the syntax above works for redhat linux

I'd thought there was a more automated way but this should
work around the issue pretty well.
If there is, i don´t know it...

You have any experience with comp-tar usage on sgi and solaris
(vs discussions that I'd seen for linux systems) ?
No I don´t. What I noticed in my redhat linux is that the amanda comp-tar backup truly 
uses the tar -cz option to make its backups.
I  proved it when i made a backup using amanda and read the tape contents without 
amanda, using the 3 commands bellow:

mt rewind
mt fsf NN
dd if=/dev/nst0 bs=32k skip=1 | tar -tvz

So, i believe that if your tar command works well on your system, amanda comp-tar 
would work fine also.

Bruno.

 (See brian, i´m posting your message on the list, ok?)

 Oh, Let´s see if I correctly understood your problem: you need to backup
 your user directories, like /home/brian,  /home/john, /home/melissa, (etc).
 But you can´t simply make a backup of your entire /home directory.
 So you, basically, would need a disklist file with the following entries:

 machine1/home/briancomp-root-tar
 machine1/home/jon   comp-root-tar
 machine1/home/melissa comp-root-tar
 (and so on for every user directory)

 However, you don´t want to manually edit the disklist file every time you
 create and delete a user.
 Is this right?

 Well, the first thing that occurred to me is to do a script that runs before
 amdump which would check what are the valid user directories of your machine
 and would write these names in the disklist file. (actually, it would write
 the correct disklist file entries, one for each directory).

 If someone know anything better, share with us.

 Hope it helps,
 Bruno.

 - Original Message - 
 From: Brian Cuttler
 To: Bruno Negrão
 Cc: Brian Cuttler
 Sent: Thursday, September 18, 2003 3:28 PM
 Subject: Re: Using amanda with just 1 tape - for testing - SOLVED



 Bruno,

 I realized you are writing about tapecycle (my latest guess was
 that a tapecycle of 0 is a problem) but as long as you are using
 tar rather than dump I've got a question.

 I've got a couple of large partitions that I'd like to move
 from dump to tar, but only specific partitions within the
 disklist and I don't know now to divide up the data ? Dividing
 at highest level on the partition by username might work ok
 (directories at that level named for the data owner) but I
 don't really want to maintain the disklist (look to see if
 it needs updating everytime I create/remove a user data area).

 Are you doing anything like this ? I'm in need of the feature
 for both Solaris/IRIX servers/clients.

 What version of amanda are you running ? Or rather, what is the
 min version that supports what I'm hoping to do ?

 thanks,

 Brian

RE: disklist file maintenance

[Dana Bourgeois]

Shouldn't you read the home directories out of /etc/passwd?  The fancy thing
to do would be to grep for /home (or equivalent) and then drop anything from
the list that doesn't exist.  


Dana Bourgeois


 -Original Message-
 From: [EMAIL PROTECTED] 
 [mailto:[EMAIL PROTECTED] On Behalf Of Brian Cuttler
 Sent: Thursday, September 18, 2003 12:37 PM
 To: Bruno Negrão
 Cc: Brian Cuttler; amanda users
 Subject: Re: disklist file maintenance
 
 
 
 Bruno,
 
 (No problem with list posting)
 
 Yes, I see that as a valid potential way to go, if only it
 was that simple the script you are suggesting could run in 
 advance of the amanda-server startup.
 
 You know, actually its not a bad solution, I was going to 
 object that it wasn't a matter of home directories but of 
 directories on a non-home data disk on an amanda-client.
 
 However the script could/would work just fine as the client 
 disks are all nfs mountable (automounter) and I can get the 
 list of top level directories.
 
 I'd thought there was a more automated way but this should
 work around the issue pretty well.
 
 You have any experience with comp-tar usage on sgi and 
 solaris (vs discussions that I'd seen for linux systems) ?
 
   thanks,
 
   Brian
 
  (See brian, i´m posting your message on the list, ok?)
  
  Oh, Let´s see if I correctly understood your problem: you need to 
  backup your user directories, like /home/brian,  /home/john, 
  /home/melissa, (etc). But you can´t simply make a backup of your 
  entire /home directory. So you, basically, would need a 
 disklist file 
  with the following entries:
  
  machine1/home/briancomp-root-tar
  machine1/home/jon   comp-root-tar
  machine1/home/melissa comp-root-tar
  (and so on for every user directory)
  
  However, you don´t want to manually edit the disklist file 
 every time 
  you create and delete a user. Is this right?
  
  Well, the first thing that occurred to me is to do a script 
 that runs 
  before amdump which would check what are the valid user 
 directories of 
  your machine and would write these names in the disklist file. 
  (actually, it would write the correct disklist file 
 entries, one for 
  each directory).
  
  If someone know anything better, share with us.
  
  Hope it helps,
  Bruno.
  
  - Original Message -
  From: Brian Cuttler
  To: Bruno Negrão
  Cc: Brian Cuttler
  Sent: Thursday, September 18, 2003 3:28 PM
  Subject: Re: Using amanda with just 1 tape - for testing - SOLVED
  
  
  
  Bruno,
  
  I realized you are writing about tapecycle (my latest guess 
 was that a 
  tapecycle of 0 is a problem) but as long as you are using 
 tar rather 
  than dump I've got a question.
  
  I've got a couple of large partitions that I'd like to move 
 from dump 
  to tar, but only specific partitions within the disklist 
 and I don't 
  know now to divide up the data ? Dividing at highest level on the 
  partition by username might work ok (directories at that 
 level named 
  for the data owner) but I don't really want to maintain the 
 disklist 
  (look to see if it needs updating everytime I create/remove a user 
  data area).
  
  Are you doing anything like this ? I'm in need of the 
 feature for both 
  Solaris/IRIX servers/clients.
  
  What version of amanda are you running ? Or rather, what is the min 
  version that supports what I'm hoping to do ?
  
  thanks,
  
  Brian
  
  
 
 

Re: disklist file maintenance

[Paul Bijnens]

Bruno Negrão wrote:
Yes, I see that as a valid potential way to go, if only it
was that simple the script you are suggesting could run in
advance of the amanda-server startup.
yes, in your crontab file you could add a line like this:
(say the script would be called dlmaintainer)
00 2 * * * (dlmaintainer  su amanda -c amdump DailySetX)
(the parentesis are not needed, actually, and I would put
amdump in the crontab of amanda itself, instead of root and
not needing to do a su -- but that's only details.)
But the problem is that the disks that you want to add
to disklist with the yet-te-written program dlmaintainer
are on the clients.  So you're pushing the real problem inside
this program.  How can a program on the server get to each
client and get the uptodate disklist for that client back
to the server.  Not easily accomplished...
What I use in my disklist is a setup like:

#first this very big directory gets his own dle
bigserver  /bigdisk/verybig/subdir  /bigdisk  {
comp-user-tar
include ./verybig/subdir
}  1
# all files/dirs A-M, except bigdir
bigserver  /bigdisk/files_A-M  /bigdisk {
comp-user-tar
include ./[a-mA-M]
exclude ./bigdir
}  1
# all files/dirs N-Z
bigserver  /bigdisk/files_N-Z  /bigdisk {
comp-user-tar
include ./[n-zN-Z]
}  1
# all the rest, include .xxx files etc
bigserver  /bigdisk/files_rest /bigdisk {
comp-user-tar
exclude append ./[a-mA-M]
exclude append ./[n-zN-Z]
}  1
The trick here is to divide large chunks, and keep a rest
chunk that includes everything else.  Now even clients
that create files/directories on places where they shouldn't
are still included in the backup.
You need 2.4.3 at least for this to work, and, because
of a bug in 2.4.4p1, you need 2.4.4p1-20030716 to get the
first entry (without wildcards) to work.


--
Paul Bijnens @ Home

Re: disklist + subdirs

[Paul Bijnens]

Hinrich Aue wrote:

Hello,

I have a problem with my disklist, I entered my / partition, and now I only
want to backup
the /users/mentor/avtstud2/ directory with all it's subdirs.
Is that possible? when yes does anyone know how?
I tried:

mentor1 / {
   comp-user-tar
   include ./users/ ./users/mentor ./users/mentor/avtstud2
   }
but this soesn't work, although amcheck doesn't report any problems.
You're hit by a little bug that was solved just after the release of
2.4.4p1:
Use the snapshot version with this bug fixed:

http://www.iro.umontreal.ca/~martinea/amanda/amanda-2.4.4p1-20030716.tar.gz

From Changelog :

2003-07-02  Jean-Louis Martineau [EMAIL PROTECTED]
patch by Paul Bijnens [EMAIL PROTECTED]

* client-src/client_util.c (add_include): Count include with 2 /,
  Fix descriptor leak.

2003-06-27  Jean-Louis Martineau [EMAIL PROTECTED]

* Amanda 2.4.4p1 released.
* configure.in: AM_INIT_AUTOMAKE(amanda, 2.4.4p1).


But nevertheless, why don't you just use as DLE:

   mentor1 /users/mentor/avtstud2 comp-user-tar

--
Paul Bijnens, XplanationTel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax  +32 16 397.512
http://www.xplanation.com/  email:  [EMAIL PROTECTED]
***
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...*
* ...  Are you sure?  ...   YES   ...   Phew ...   I'm out  *
***

Re: disklist question

[Paul Bijnens]

Michael Packer wrote:
I'm trying to break up a large disk into smaller sections when it does a
backup.. I tried:
localhost /sda5/other sda5 { 
   exclude ./files
   exclude ./snap
   exclude ./web
   always-full 
} 1
You are using amanda 2.4.3 or above, I presume.

Is your base dumptype always-full using gnutar?  Only gnutar
can use the exclude parameter.
Your base dumptype is at the end of the list - strange. I've
never tried it like that; it may work. Verify with:
   amadmin YourConf disklist localhost /sda5/other
to verify that all parameters are indeed correct.
If specifying more than one exclude you want to add the append
keyword, otherwise it will overwrite the previous exclude.
And, not really an error, but it's best to use the real host name
instead of localhost.  This will bite you sooner or later.
Don't tell us we didn't warn you.
Like this:

host.dom.ain  /mount/point {
allways-full
program GNUTAR
exclude append ./files
exclude append ./snap
exclude append ./web
} 1
--
Paul Bijnens, XplanationTel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUMFax  +32 16 397.512
http://www.xplanation.com/  email:  [EMAIL PROTECTED]
***
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...*
* ...  Are you sure?  ...   YES   ...   Phew ...   I'm out  *
***

Re: disklist question

[Gene Heskett]

On Thursday 19 June 2003 08:44, Michael Packer wrote:
I'm trying to break up a large disk into smaller sections when it
 does a backup.. I tried:

localhost /sda5/other sda5 {
   exclude ./files
   exclude ./snap
   exclude ./web
   always-full
} 1

but that doesn't seem to be working  how can I tell it to exclude
 those three directories on that /dev/sda5

thanks!

Don't use localhost, use the FQDN of the machine.

You can only pass one argument that way, and its the last one thats 
effective.  Put them into a single file, and name the file.  The 
syntax changes slightly to tell it its a file, so check the manpages.

-- 
Cheers, Gene
AMD [EMAIL PROTECTED] 320M
[EMAIL PROTECTED]  512M
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.

Re: Disklist question

[Joshua Baker-LePain]

On Mon, 2 Jun 2003 at 10:42am, Jason Edgecombe wrote

 I have a question about the disklist format.
 
 I am using tar to dump my filesystems. I would like to dump / (without 
 /home) and have /home be on a separate DLE. How do I do that? Must I 
 specify each directory in / as a separate DLE?

It depends on your partition layout.  Amanda runs tar with the 
--one-file-system option, so backing up / will, by default, grab only 
what's in the same partition with /.  If home is in that partition, 
you'll need to exclude it, and then add a DLE for it.

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Re: Disklist question

[Jon LaBadie]

On Mon, Jun 02, 2003 at 10:42:23AM -0400, Jason Edgecombe wrote:
 I have a question about the disklist format.
 
 I am using tar to dump my filesystems. I would like to dump / (without 
 /home) and have /home be on a separate DLE. How do I do that? Must I 
 specify each directory in / as a separate DLE?

Depends on your disk configuration.  Gnutar, as run by amanda, will not
cross from one file system to another.  When it reaches a directory that
is a mount point for another partition (another common term for file system)
it will not enter that directory.

If /home is part of the root file system you will only need the / DLE.
If /home is on a different file system (disk partition) it will need a
separate DLE.  Note, on systems that automount home directories, it
is possible that /home should not be backed up at all.  Ignore the
comment if your environment does not automount home dirs - Solaris is
often set up this way.

Note also that not every directory in / is a separate file system.
for example /etc is not a separate file system.  But on my desktop
computer /usr is a separate file system and so is /usr/local.  Each
need their own DLE's.  Yet my laptop, with the same OS and version
is configured with only one file system, /.  So /usr and /usr/local
do not need separate DLE's on the laptop.

-- 
Jon H. LaBadie  [EMAIL PROTECTED]
 JG Computing
 4455 Province Line Road(609) 252-0159
 Princeton, NJ  08540-4322  (609) 683-7220 (fax)

Re: Disklist question

[Gene Heskett]

On Monday 02 June 2003 10:42, Jason Edgecombe wrote:
I have a question about the disklist format.

I am using tar to dump my filesystems. I would like to dump /
 (without /home) and have /home be on a separate DLE. How do I do
 that? Must I specify each directory in / as a separate DLE?

I am, but you can also specify the ./home as an exclude, then back it 
up in a seperate DLE.

Sincerely,
Jason Edgecombe

-- 
Cheers, Gene
AMD [EMAIL PROTECTED] 320M
[EMAIL PROTECTED]  512M
99.26% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attornies please note, additions to this message
by Gene Heskett are:
Copyright 2003 by Maurice Eugene Heskett, all rights reserved.

Re: Disklist format

[Alexander JOLK]

 pantera /home/depot-af /home/depot {
 hard-disk-tar
 include ./[Aa-Ff]*
 } 1

I'd be pretty much surprised if this include statement didn't mean
`include all files starting with either A, any character between a and
F, or f'.  That's probably not what you meant to say, and moreover,
[a-F] is the empty set.  So there you are, you don't have anything
included in your DLE. :-)

What you meant to say was probably something like ./[A-Fa-f]*.

Alex

-- 
Alexander Jolk / BUF Compagnie
tel +33-1 42 68 18 28 /  fax +33-1 42 68 18 29

Re: Disklist format

[Paul Bijnens]

Robert Hazbun wrote:

Anyway, my problem is that I am doing just this, but am running into the following 
error:

 pantera/home/depot-wz lev 0 FAILED [missing result for /home/depot-wz in
pantera response]
...
Upon looking in the amanda source to attempt to find out what's wrong, I found out 
that the planner did not receive a size estimate for each entry from the client. Why, I 
don't know, and hence the question I'm asking.

I suppose that depot is very large.  If /tmp/amanda/sendsize*debug does 
contain
the correct numbers, see how long it took and make sure the etimeout 
parameter
is large enough.

Below are the corresponding entries in the 'disklist' file for my box. The boxes name 
is pantera ( I'm a car nut ).

pantera /home/depot-af /home/depot {
   hard-disk-tar
   include ./[Aa-Ff]*
   } 1
oh-oh, you probably mean [A-Fa-f] instead of [Aa-Ff]...

Re: Disklist - partial directory backup - advice please

[Jean-Louis Martineau]

Hello John,

2.4.3b3 is buggy with relative exclude list. Try the latest snapshot
from http://www.iro.umontreal.ca/~martinea/amanda .

Jean-Louis

On Mon, Aug 19, 2002 at 08:47:52PM -0400, John Ouellette wrote:
 
 Ok, now that's interesting...  I'll agree that the fact that you're using
 2.4.3b3 and aren't having problems indicates that you're right.  
 However...  The comments in the amanda.conf file say:
 
 Note that the `full pathname' of a file within its
 filesystem starts with `./', because of the way amanda runs
 gnutar: `tar -C $mountpoint -cf - --lots-of-options .' (note
 the final dot!)  Thus, if you're backing up `/usr' with a
 diskfile entry like ``host /usr gnutar-root', but you don't
 want to backup /usr/tmp, your exclude list should contain
 the pattern `./tmp', as this is relative to the `/usr' above.
 Please refer to the man-page of gnutar for more information.
 If a relative pathname is specified as the exclude list,
 it is searched from within the directory that is
 going to be backed up.
 
 The man page says:
 
  For exclude list, If the file name is relative, the
 disk name being backed up is prepended.  So if this
 is entered:
 
exclude list .amanda.excludes
 
 the actual file use would be  /var/.amanda.excludes
 for  a  backup of /var, /usr/local/.amanda.excludes
 for a backup of /usr/local, and so on.
 
 These suggest that the exclude pattern, whether as a exclude pattern or a 
 file containing a list of patterns, is supposed to have *relative* 
 pathnames, not absolute pathnames.  Yes, your list is relative, but your 
 path for the exclude list file itself is absolute 
 (/usr/local/etc/amanda/exclude.gtar) which is not quite what the docs 
 recommend
 
 I'll try some absolute paths later: if they work, it suggests a 
 documentation bug, rather than a coding one... 
 
 J.
 
 On Mon, 19 Aug 2002, Gene Heskett wrote:
 
  On Monday 19 August 2002 20:09, John Ouellette wrote:
  We're going a bit far afield from what Kevin had originally posted
   but oh well.
  
  Are you using version 2.4.2, Gene?
  
  No, 2.4.3b3-20020805 right now.  And I'm not having any known 
  problems, none.  And other than some broken scripting in the 
  amstatus command, I've had zilch troubles with amanda in the last 4 
  or more months, thru a goodly number of 2.4.3b* versions.
  
Kevin and I are using v2.4.3b3
   and have found the same problem with exclude lists.  What I have
   found (again, with v2.4.3b3) was that the exclude keyword in a
   dumptype definition *causes the size estimates from tar to fail*.
The debug info from Amanda shows that tar is never even called. 
   From one of my tests (sendsize.*.debug):
  
  ++
  sendsize: debug 1 pid 27066 ruid 499 euid 499 start time Mon Aug
   19 14:31:57 2002
  /home/amanda/libexec/sendsize: version 2.4.3b3
  sendsize: calculating for amname '/', dirname '/'
  sendsize: getting size via gnutar for / level 0
  ++
  
  Any one of my copious numbers of sendsize.*.debug files is exactly 
  like the rest, containing only a copy of the exclude file as 
  follows:
  -
  ./usr/dumps/*
  ./var/pacct
  -
  So I again submit that the exclude logic isn't broken, but that the 
  file itself simply isn't being found due to a lack of a full 
  pathlist in the dumptype specification, as is shown that I am using 
  below.
  
  (Note that there were three other partitions called with the same
  dumptype: the size estimates weren't even attempted by amanda --
   the above is the entire contents of the debug file.)  This with a
   dumptype definition of:
  
  
  This is yours
  
  define dumptype TEST {
  global
  comment test dumptype
  program GNUTAR
  compress none
  index
  exclude ./amanda ---whats the current pwd when this 
  executes?
  record no
  }
  
  (The idea of the exclude was the same as Kevin's: to exclude the
   holding disk areas from those disks.)  The result is the same if
   I use:
  
  exclude ./amanda*
  exclude *amanda*
  exclude list .amanda.exclude
  exclude list ./.amanda.exclude
  
  etc., in the dumptype definition.  The exclude keyword seemed to
   work in v2.4.2p2, but I decided to upgrade to 2.4.3b3 to get
   other features working (I'm not sure if it was worth it now...).
  
  Thats odd. I also note that you still aren't giving it a full path 
  to the exclude file.  Its possible that the fact that it can't find 
  it might be enough of an error to cause the tar call to fail.
  
  I've taken a brief look into the code for how exclude lists are
   handled, but there was such a big change from v2.4.2 to v2.4.3,
   I'm not sure if I'll be able to figure out what broke the exclude
   keyword...
  
  Note that in the amandad debug file, the exclude list seems to be
   parsed OK, so it really must be in the call to sendsize that the
   error is occurring...
  
  J.
  
  On Mon, 19 Aug 2002, Gene Heskett wrote:
 

Re: Disklist - partial directory backup - advice please

[John Ouellette]

Hi Kevin,

Are you using Amanda 2.4.3b3?  I think there is a bug in the 'exclude'
lists in the definition of dumptypes (including the inline exclude 
lists like you're using) for that version.  The bug causes the dumps to 
fail, at least it does in my case.

How to work around your problem without using exclude lists, I'm not sure, 
although you might try 'include' lists -- I'm not sure if the same bug 
affects these.

J.

-- 
++
John Ouellette | Ph: 212-313-7919 
Department of Astrophysics | Fax: 212-769-5007 
American Museum of Natural History | e-mail: [EMAIL PROTECTED]
Central Park West at 79th St.  | http://research.amnh.org/astrophysics
New York, NY  10024-5192   |
++

RE: Disklist - partial directory backup - advice please

[Kevin Passey]

Thanks John

That is what I am using.

I only what to omit the /dumps folders.

I think I will have to create them in a separate partition.

Thanks for that.

Regards

Kevin



-Original Message-
From: John Ouellette [mailto:[EMAIL PROTECTED]]
Sent: 19 August 2002 16:11
To: [EMAIL PROTECTED]
Subject: Re: Disklist - partial directory backup - advice please



Hi Kevin,

Are you using Amanda 2.4.3b3?  I think there is a bug in the 'exclude'
lists in the definition of dumptypes (including the inline exclude 
lists like you're using) for that version.  The bug causes the dumps to 
fail, at least it does in my case.

How to work around your problem without using exclude lists, I'm not sure, 
although you might try 'include' lists -- I'm not sure if the same bug 
affects these.

J.

-- 
++
John Ouellette | Ph: 212-313-7919 
Department of Astrophysics | Fax: 212-769-5007 
American Museum of Natural History | e-mail: [EMAIL PROTECTED]
Central Park West at 79th St.  | http://research.amnh.org/astrophysics
New York, NY  10024-5192   |
++

Re: Disklist - partial directory backup - advice please

[Kirk Strauser]

At 2002-08-19T16:00:21Z, Kevin Passey [EMAIL PROTECTED] writes:

 I only what to omit the /dumps folders.
 
 I think I will have to create them in a separate partition.

Try this:

1) Create a new dumptype with an exclude list option.  For example:

 define dumptype compressed-tar-with-excludes {
 compressed-tar
 comment Non-root partitions without compression, plus an exclude list
 exclude list .amanda.excludes
 }

2) Edit the file specified above.  For example:

 root@kanga:/usr/share/media# cat .amanda.excludes 
 ./music/all

   That will cause Amanda to skip the files/directories listed in the
   exclude file.
-- 
Kirk Strauser
The Strauser Group - http://www.strausergroup.com/

Re: Disklist - partial directory backup - advice please

[Gene Heskett]

On Monday 19 August 2002 12:00, Kevin Passey wrote:
Thanks John

That is what I am using.

I only what to omit the /dumps folders.

Did you specify them as ./dumps (without the quotes of course)
The missing leading dot in your line above says its your exclude 
file thats broken.   Tar's exludes are relative to the currently 
being processed directory, hence the leading ./

I think I will have to create them in a separate partition.

Thanks for that.

Regards

Kevin



-Original Message-
From: John Ouellette [mailto:[EMAIL PROTECTED]]
Sent: 19 August 2002 16:11
To: [EMAIL PROTECTED]
Subject: Re: Disklist - partial directory backup - advice please



Hi Kevin,

Are you using Amanda 2.4.3b3?  I think there is a bug in the
 'exclude' lists in the definition of dumptypes (including the
 inline exclude lists like you're using) for that version.  The
 bug causes the dumps to fail, at least it does in my case.

How to work around your problem without using exclude lists, I'm
 not sure, although you might try 'include' lists -- I'm not sure
 if the same bug affects these.

J.

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.12% setiathome rank, not too shabby for a WV hillbilly

Re: Disklist - partial directory backup - advice please

[John Ouellette]

I've played around with the exclude lists for a while (as defined in the 
dumptypes) and have had no luck in getting any configs with exclude lists 
to work, whether I define the paths with the leading ./ or no

Kevin was using an exclude list defined in the disklist, but I assume it 
is the same code that parses this and passes it on to 'tar', so the same 
bug might exist.

And, according to the docs and examples, you do need the quotes... (or get 
a parse error if you exclude them).

I've started to trudge through the code to see if the bug is obvious, but 
haven't managed to get far.

J.

On Mon, 19 Aug 2002, Gene Heskett wrote:

 On Monday 19 August 2002 12:00, Kevin Passey wrote:
 Thanks John
 
 That is what I am using.
 
 I only what to omit the /dumps folders.
 
 Did you specify them as ./dumps (without the quotes of course)
 The missing leading dot in your line above says its your exclude 
 file thats broken.   Tar's exludes are relative to the currently 
 being processed directory, hence the leading ./
 
 I think I will have to create them in a separate partition.
 
 Thanks for that.
 
 Regards
 
 Kevin
 
 
 
 -Original Message-
 From: John Ouellette [mailto:[EMAIL PROTECTED]]
 Sent: 19 August 2002 16:11
 To: [EMAIL PROTECTED]
 Subject: Re: Disklist - partial directory backup - advice please
 
 
 
 Hi Kevin,
 
 Are you using Amanda 2.4.3b3?  I think there is a bug in the
  'exclude' lists in the definition of dumptypes (including the
  inline exclude lists like you're using) for that version.  The
  bug causes the dumps to fail, at least it does in my case.
 
 How to work around your problem without using exclude lists, I'm
  not sure, although you might try 'include' lists -- I'm not sure
  if the same bug affects these.
 
 J.
 
 

-- 
++
John Ouellette | Ph: 212-313-7919 
Department of Astrophysics | Fax: 212-769-5007 
American Museum of Natural History | e-mail: [EMAIL PROTECTED]
Central Park West at 79th St.  | http://research.amnh.org/astrophysics
New York, NY  10024-5192   |
++

Re: Disklist - partial directory backup - advice please

[Gene Heskett]

On Monday 19 August 2002 16:12, John Ouellette wrote:
I've played around with the exclude lists for a while (as defined
 in the dumptypes) and have had no luck in getting any configs
 with exclude lists to work, whether I define the paths with the
 leading ./ or no

Kevin was using an exclude list defined in the disklist, but I
 assume it is the same code that parses this and passes it on to
 'tar', so the same bug might exist.

And, according to the docs and examples, you do need the quotes...
 (or get a parse error if you exclude them).

I'm not here, but I'm using an exclude file too, here it is:
-
./usr/dumps/*
./var/pacct
-

Also, I recall the filename didn't give the full path in your earler 
example, which should look like this in your amanda.conf:
--
define dumptype root-tar {
global
program GNUTAR
comment root partitions dumped with tar
compress none
index yes
exclude list /usr/local/etc/amanda/exclude.gtar
priority low
}

define dumptype user-tar {
root-tar
comment user partitions dumped with tar
exclude list /usr/local/etc/amanda/exclude.gtar
compress server best
priority medium
}
-etc, etc taking note that each dumptype can have 
its own unique name-it-whatever-file.

I've started to trudge through the code to see if the bug is
 obvious, but haven't managed to get far.

J.

On Mon, 19 Aug 2002, Gene Heskett wrote:
 On Monday 19 August 2002 12:00, Kevin Passey wrote:
 Thanks John
 
 That is what I am using.
 
 I only what to omit the /dumps folders.

 Did you specify them as ./dumps (without the quotes of course)
 The missing leading dot in your line above says its your exclude
 file thats broken.   Tar's exludes are relative to the currently
 being processed directory, hence the leading ./

 I think I will have to create them in a separate partition.
 
 Thanks for that.
 
 Regards
 
 Kevin
 
 
 
 -Original Message-

 From: John Ouellette [mailto:[EMAIL PROTECTED]]

[...]

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.12% setiathome rank, not too shabby for a WV hillbilly

Re: Disklist - partial directory backup - advice please

[Jean-Louis Martineau]

Hello Kevin,

Try the latest snapshot of 2.4.3b3 from
http://www.iro.umontreal.ca/~martinea/amanda
it fixe a few problem with the exclude.
Don't forget to upgrade the server and the client.

I'm not sure it works if you use the device name in the disklist,
you should try with the directory name.

dilmom /usr {
  high-tar
  exclude ./dumps
  } 1

Jean-Louis
-- 
Jean-Louis Martineau email: [EMAIL PROTECTED] 
Departement IRO, Universite de Montreal
C.P. 6128, Succ. CENTRE-VILLETel: (514) 343-6111 ext. 3529
Montreal, Canada, H3C 3J7Fax: (514) 343-5834

Re: Disklist - partial directory backup - advice please

[John Ouellette]

We're going a bit far afield from what Kevin had originally posted but oh 
well.

Are you using version 2.4.2, Gene?  Kevin and I are using v2.4.3b3 and
have found the same problem with exclude lists.  What I have found (again,
with v2.4.3b3) was that the exclude keyword in a dumptype definition
*causes the size estimates from tar to fail*.  The debug info from Amanda
shows that tar is never even called.  From one of my tests 
(sendsize.*.debug):

++
sendsize: debug 1 pid 27066 ruid 499 euid 499 start time Mon Aug 19 
14:31:57 2002
/home/amanda/libexec/sendsize: version 2.4.3b3
sendsize: calculating for amname '/', dirname '/'
sendsize: getting size via gnutar for / level 0
++

(Note that there were three other partitions called with the same 
dumptype: the size estimates weren't even attempted by amanda -- the 
above is the entire contents of the debug file.)  This with a dumptype 
definition of:

define dumptype TEST {
global
comment test dumptype
program GNUTAR
compress none
index
exclude ./amanda
record no
}

(The idea of the exclude was the same as Kevin's: to exclude the holding 
disk areas from those disks.)  The result is the same if I use:

exclude ./amanda*
exclude *amanda*
exclude list .amanda.exclude
exclude list ./.amanda.exclude

etc., in the dumptype definition.  The exclude keyword seemed to work in 
v2.4.2p2, but I decided to upgrade to 2.4.3b3 to get other features 
working (I'm not sure if it was worth it now...).

I've taken a brief look into the code for how exclude lists are handled, 
but there was such a big change from v2.4.2 to v2.4.3, I'm not sure if 
I'll be able to figure out what broke the exclude keyword...

Note that in the amandad debug file, the exclude list seems to be parsed 
OK, so it really must be in the call to sendsize that the error is 
occurring...

J.


On Mon, 19 Aug 2002, Gene Heskett wrote:

 On Monday 19 August 2002 16:12, John Ouellette wrote:
 I've played around with the exclude lists for a while (as defined
  in the dumptypes) and have had no luck in getting any configs
  with exclude lists to work, whether I define the paths with the
  leading ./ or no
 
 Kevin was using an exclude list defined in the disklist, but I
  assume it is the same code that parses this and passes it on to
  'tar', so the same bug might exist.
 
 And, according to the docs and examples, you do need the quotes...
  (or get a parse error if you exclude them).
 
 I'm not here, but I'm using an exclude file too, here it is:
 -
 ./usr/dumps/*
 ./var/pacct
 -
 
 Also, I recall the filename didn't give the full path in your earler 
 example, which should look like this in your amanda.conf:
 --
 define dumptype root-tar {
 global
 program GNUTAR
 comment root partitions dumped with tar
 compress none
 index yes
 exclude list /usr/local/etc/amanda/exclude.gtar
 priority low
 }
 
 define dumptype user-tar {
 root-tar
 comment user partitions dumped with tar
 exclude list /usr/local/etc/amanda/exclude.gtar
 compress server best
 priority medium
 }
 -etc, etc taking note that each dumptype can have 
 its own unique name-it-whatever-file.
 
 I've started to trudge through the code to see if the bug is
  obvious, but haven't managed to get far.
 
 J.
 
 On Mon, 19 Aug 2002, Gene Heskett wrote:
  On Monday 19 August 2002 12:00, Kevin Passey wrote:
  Thanks John
  
  That is what I am using.
  
  I only what to omit the /dumps folders.
 
  Did you specify them as ./dumps (without the quotes of course)
  The missing leading dot in your line above says its your exclude
  file thats broken.   Tar's exludes are relative to the currently
  being processed directory, hence the leading ./
 
  I think I will have to create them in a separate partition.
  
  Thanks for that.
  
  Regards
  
  Kevin
  
  
  
  -Original Message-
 
  From: John Ouellette [mailto:[EMAIL PROTECTED]]
 
 [...]
 
 

-- 
++
John Ouellette | Ph: 212-313-7919 
Department of Astrophysics | Fax: 212-769-5007 
American Museum of Natural History | e-mail: [EMAIL PROTECTED]
Central Park West at 79th St.  | http://research.amnh.org/astrophysics
New York, NY  10024-5192   |
++

Re: Disklist - partial directory backup - advice please

[Gene Heskett]

On Monday 19 August 2002 20:09, John Ouellette wrote:
We're going a bit far afield from what Kevin had originally posted
 but oh well.

Are you using version 2.4.2, Gene?

No, 2.4.3b3-20020805 right now.  And I'm not having any known 
problems, none.  And other than some broken scripting in the 
amstatus command, I've had zilch troubles with amanda in the last 4 
or more months, thru a goodly number of 2.4.3b* versions.

  Kevin and I are using v2.4.3b3
 and have found the same problem with exclude lists.  What I have
 found (again, with v2.4.3b3) was that the exclude keyword in a
 dumptype definition *causes the size estimates from tar to fail*.
  The debug info from Amanda shows that tar is never even called. 
 From one of my tests (sendsize.*.debug):

++
sendsize: debug 1 pid 27066 ruid 499 euid 499 start time Mon Aug
 19 14:31:57 2002
/home/amanda/libexec/sendsize: version 2.4.3b3
sendsize: calculating for amname '/', dirname '/'
sendsize: getting size via gnutar for / level 0
++

Any one of my copious numbers of sendsize.*.debug files is exactly 
like the rest, containing only a copy of the exclude file as 
follows:
-
./usr/dumps/*
./var/pacct
-
So I again submit that the exclude logic isn't broken, but that the 
file itself simply isn't being found due to a lack of a full 
pathlist in the dumptype specification, as is shown that I am using 
below.

(Note that there were three other partitions called with the same
dumptype: the size estimates weren't even attempted by amanda --
 the above is the entire contents of the debug file.)  This with a
 dumptype definition of:


This is yours

define dumptype TEST {
global
comment test dumptype
program GNUTAR
compress none
index
exclude ./amanda ---whats the current pwd when this 
executes?
record no
}

(The idea of the exclude was the same as Kevin's: to exclude the
 holding disk areas from those disks.)  The result is the same if
 I use:

exclude ./amanda*
exclude *amanda*
exclude list .amanda.exclude
exclude list ./.amanda.exclude

etc., in the dumptype definition.  The exclude keyword seemed to
 work in v2.4.2p2, but I decided to upgrade to 2.4.3b3 to get
 other features working (I'm not sure if it was worth it now...).

Thats odd. I also note that you still aren't giving it a full path 
to the exclude file.  Its possible that the fact that it can't find 
it might be enough of an error to cause the tar call to fail.

I've taken a brief look into the code for how exclude lists are
 handled, but there was such a big change from v2.4.2 to v2.4.3,
 I'm not sure if I'll be able to figure out what broke the exclude
 keyword...

Note that in the amandad debug file, the exclude list seems to be
 parsed OK, so it really must be in the call to sendsize that the
 error is occurring...

J.

On Mon, 19 Aug 2002, Gene Heskett wrote:
 On Monday 19 August 2002 16:12, John Ouellette wrote:
 I've played around with the exclude lists for a while (as
  defined in the dumptypes) and have had no luck in getting any
  configs with exclude lists to work, whether I define the paths
  with the leading ./ or no
 
 Kevin was using an exclude list defined in the disklist, but I
  assume it is the same code that parses this and passes it on
  to 'tar', so the same bug might exist.
 
 And, according to the docs and examples, you do need the
  quotes... (or get a parse error if you exclude them).

 I'm not here, but I'm using an exclude file too, here it is:
 -
 ./usr/dumps/*
 ./var/pacct
 -

 Also, I recall the filename didn't give the full path in your
 earler example, which should look like this in your amanda.conf:
 --
 define dumptype root-tar {
 global
 program GNUTAR
 comment root partitions dumped with tar
 compress none
 index yes
 exclude list /usr/local/etc/amanda/exclude.gtar
^
 priority low
 }

 define dumptype user-tar {
 root-tar
 comment user partitions dumped with tar
 exclude list /usr/local/etc/amanda/exclude.gtar
Again
 compress server best
 priority medium
 }
 -etc, etc taking note that each dumptype can
 have its own unique name-it-whatever-file.

 I've started to trudge through the code to see if the bug is
  obvious, but haven't managed to get far.
 
 J.
 
 On Mon, 19 Aug 2002, Gene Heskett wrote:
  On Monday 19 August 2002 12:00, Kevin Passey wrote:
  Thanks John
  
  That is what I am using.
  
  I only what to omit the /dumps folders.
 
  Did you specify them as ./dumps (without the quotes of
  course) The missing leading dot in your line above says its
  your exclude file thats broken.   Tar's exludes are relative
  to the currently being processed directory, hence the leading
  ./
 
  I think I will have 

Re: Disklist - partial directory backup - advice please

[John Ouellette]

Ok, now that's interesting...  I'll agree that the fact that you're using
2.4.3b3 and aren't having problems indicates that you're right.  
However...  The comments in the amanda.conf file say:

Note that the `full pathname' of a file within its
filesystem starts with `./', because of the way amanda runs
gnutar: `tar -C $mountpoint -cf - --lots-of-options .' (note
the final dot!)  Thus, if you're backing up `/usr' with a
diskfile entry like ``host /usr gnutar-root', but you don't
want to backup /usr/tmp, your exclude list should contain
the pattern `./tmp', as this is relative to the `/usr' above.
Please refer to the man-page of gnutar for more information.
If a relative pathname is specified as the exclude list,
it is searched from within the directory that is
going to be backed up.

The man page says:

 For exclude list, If the file name is relative, the
disk name being backed up is prepended.  So if this
is entered:

   exclude list .amanda.excludes

the actual file use would be  /var/.amanda.excludes
for  a  backup of /var, /usr/local/.amanda.excludes
for a backup of /usr/local, and so on.

These suggest that the exclude pattern, whether as a exclude pattern or a 
file containing a list of patterns, is supposed to have *relative* 
pathnames, not absolute pathnames.  Yes, your list is relative, but your 
path for the exclude list file itself is absolute 
(/usr/local/etc/amanda/exclude.gtar) which is not quite what the docs 
recommend

I'll try some absolute paths later: if they work, it suggests a 
documentation bug, rather than a coding one... 

J.

On Mon, 19 Aug 2002, Gene Heskett wrote:

 On Monday 19 August 2002 20:09, John Ouellette wrote:
 We're going a bit far afield from what Kevin had originally posted
  but oh well.
 
 Are you using version 2.4.2, Gene?
 
 No, 2.4.3b3-20020805 right now.  And I'm not having any known 
 problems, none.  And other than some broken scripting in the 
 amstatus command, I've had zilch troubles with amanda in the last 4 
 or more months, thru a goodly number of 2.4.3b* versions.
 
   Kevin and I are using v2.4.3b3
  and have found the same problem with exclude lists.  What I have
  found (again, with v2.4.3b3) was that the exclude keyword in a
  dumptype definition *causes the size estimates from tar to fail*.
   The debug info from Amanda shows that tar is never even called. 
  From one of my tests (sendsize.*.debug):
 
 ++
 sendsize: debug 1 pid 27066 ruid 499 euid 499 start time Mon Aug
  19 14:31:57 2002
 /home/amanda/libexec/sendsize: version 2.4.3b3
 sendsize: calculating for amname '/', dirname '/'
 sendsize: getting size via gnutar for / level 0
 ++
 
 Any one of my copious numbers of sendsize.*.debug files is exactly 
 like the rest, containing only a copy of the exclude file as 
 follows:
 -
 ./usr/dumps/*
 ./var/pacct
 -
 So I again submit that the exclude logic isn't broken, but that the 
 file itself simply isn't being found due to a lack of a full 
 pathlist in the dumptype specification, as is shown that I am using 
 below.
 
 (Note that there were three other partitions called with the same
 dumptype: the size estimates weren't even attempted by amanda --
  the above is the entire contents of the debug file.)  This with a
  dumptype definition of:
 
 
 This is yours
 
 define dumptype TEST {
 global
 comment test dumptype
 program GNUTAR
 compress none
 index
 exclude ./amanda ---whats the current pwd when this 
 executes?
 record no
 }
 
 (The idea of the exclude was the same as Kevin's: to exclude the
  holding disk areas from those disks.)  The result is the same if
  I use:
 
 exclude ./amanda*
 exclude *amanda*
 exclude list .amanda.exclude
 exclude list ./.amanda.exclude
 
 etc., in the dumptype definition.  The exclude keyword seemed to
  work in v2.4.2p2, but I decided to upgrade to 2.4.3b3 to get
  other features working (I'm not sure if it was worth it now...).
 
 Thats odd. I also note that you still aren't giving it a full path 
 to the exclude file.  Its possible that the fact that it can't find 
 it might be enough of an error to cause the tar call to fail.
 
 I've taken a brief look into the code for how exclude lists are
  handled, but there was such a big change from v2.4.2 to v2.4.3,
  I'm not sure if I'll be able to figure out what broke the exclude
  keyword...
 
 Note that in the amandad debug file, the exclude list seems to be
  parsed OK, so it really must be in the call to sendsize that the
  error is occurring...
 
 J.
 
 On Mon, 19 Aug 2002, Gene Heskett wrote:
  On Monday 19 August 2002 16:12, John Ouellette wrote:
  I've played around with the exclude lists for a while (as
   defined in the dumptypes) and have had no luck in getting any
   configs with exclude lists to work, whether I define the paths
   with the leading ./ or no
  
  Kevin was using an exclude list defined in the 

Re: Disklist - partial directory backup - advice please

[Gene Heskett]

On Monday 19 August 2002 20:47, John Ouellette wrote:
Ok, now that's interesting...  I'll agree that the fact that
 you're using 2.4.3b3 and aren't having problems indicates that
 you're right. However...  The comments in the amanda.conf file
 say:

Note that the `full pathname' of a file within its
filesystem starts with `./', because of the way amanda runs
gnutar: `tar -C $mountpoint -cf - --lots-of-options .' (note
the final dot!)  Thus, if you're backing up `/usr' with a
diskfile entry like ``host /usr gnutar-root', but you don't
want to backup /usr/tmp, your exclude list should contain
the pattern `./tmp', as this is relative to the `/usr' above.
Please refer to the man-page of gnutar for more information.
If a relative pathname is specified as the exclude list,
it is searched from within the directory that is
going to be backed up.

Which refers to the CONTENTS of the exclude file.

The man page says:

 For exclude list, If the file name is relative, the
disk name being backed up is prepended.  So if this
is entered:

   exclude list .amanda.excludes

Ohmygawd, who wrote that??  Never mind, I don't wanna know that 
badly cause it would cause my faith in the real authors to slip a 
notch.

The meaning somehow got lost in the translation to english at least 
as I understand it and I've been speaking it for 67 of my nearly 68 
years!  In the first place (IIRC) amdump itself actually cd's to 
each directory in a tree, backing up each and every subdir in that 
subdir using tar, with tar itself skipping the subdirs that match 
in the ./here format from the listing in the exclude file.

So the CONTENTS of the exclude file are then stated in a format 
relative TO the current directory.

No wonder its confusing the troops here, gaahhck!

the actual file use would be  /var/.amanda.excludes
for  a  backup of /var, /usr/local/.amanda.excludes
for a backup of /usr/local, and so on.

I think we are getting the fact that I'm argueing to the point of 
boreing this whole list for a full pathlist arguement leading to 
the exclude file, with the exclude files actual contents.  Apples 
vs Oranges, or was it kiwi fruit...  Whatever.

These suggest that the exclude pattern, whether as a exclude
 pattern or a file containing a list of patterns, is supposed to
 have *relative* pathnames, not absolute pathnames.  Yes, your
 list is relative, but your path for the exclude list file itself
 is absolute
(/usr/local/etc/amanda/exclude.gtar) which is not quite what the
 docs recommend

Again, the pathlist TO the exclude file is absolute.  BUT,
The contents of the exclude file are relative to ./

The docs don't say in plain language that the exclude files location 
is relative, anyplace in them.  They assume, and rightly so IMO, 
that the location is an absolute, just as the location of any other 
file in the system is barring the installation of links by the 
thousands all over the system pointing back to the real file.

I'll try some absolute paths later: if they work, it suggests a
documentation bug, rather than a coding one...

See above, that particular paragraph would be better off if excised 
from the docs rather than to continue its confusion facter.  I'm 
not normally one to beat on the doc writers as its generally a 
rather thankless job anyway,  but I could make an exception this 
one time.

[... way past time for a snippage party here folks]

-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.12% setiathome rank, not too shabby for a WV hillbilly

Re: disklist and sub directories.

[Jon LaBadie]

On Mon, Jul 22, 2002 at 07:06:07PM -0500, Craig Hancock [EMAIL PROTECTED] 
wrote:
 Hello all after playing with amanda all week I have finally gotten into
 a situation were my entry in disklist file is to big. From what I've read
 amanda isn't smart enough to pick were it left off if it ran out of tape

smart enough sounds so judgemental :(   mods always welcome.

 it will attempt to start from the begining again. Because of this I was thinking 
 of making of these disklist.
 
 Origianally I have a disklist of /home/data but unfortantely is is 92G but tape
 can't write a complete 92 gigs. So I was thinking of spliting this in 3's
 
 1) /home/data/directory 
 2) /home/data/directory2
 3) /home/data
 
 What I would like is to do is backup 1) and 2) seperately, but
 once we get to 3 I don't want thost 2 directories to be backed up 
 from disk 3 
 
 Now is this possible.

done all the time.

Have to use tar, not dump to do directories.
Set up a uniq dumptype for /home/data that differs only in the
exclude list file which would exclude directory and directory2
as well as all the usual suspects ... err directories.

-- 
Jon H. LaBadie  [EMAIL PROTECTED]
 JG Computing
 4455 Province Line Road(609) 252-0159
 Princeton, NJ  08540-4322  (609) 683-7220 (fax)

Re: disklist and sub directories.

[Gene Heskett]

On Monday 22 July 2002 20:06, [EMAIL PROTECTED] 
wrote:
Hello all after playing with amanda all week I have finally gotten
 into a situation were my entry in disklist file is to big. From
 what I've read amanda isn't smart enough to pick were it left off
 if it ran out of tape it will attempt to start from the begining
 again. Because of this I was thinking of making of these
 disklist.

Origianally I have a disklist of /home/data but unfortantely is is
 92G but tape can't write a complete 92 gigs. So I was thinking of
 spliting this in 3's

1) /home/data/directory
2) /home/data/directory2
3) /home/data

Nope, you'll have to continue with /home/directory3, home/directory4 
etc.  Unless you want to write up a filelist of excludes that lists 
the ones already done and pass that to the backup spec for 
/home/data.  But then the remains must fit on a tape too...  Do 
they?

But lets take this as an opportunity to clear up something else that 
the average newbie *might* think is carved in stone and maybe add 
this to the FAQ even:

Please note that one is NOT constrained by the choice of backup 
specifications currently included in the amanda.conf file.  Those 
are the most common candidates, but there isn't a thing to prevent 
you from building your own backup spec by suitable cutpaste or 
imaginative editing to make up a whole new specification.  You 
could write a spec for each entry in your disklist if there is a 
visible advantage to doing so.  Making a special, for this 
situation, exclude file is quite within the realm of customising 
how amanda works.  I've done it here.  Just give that exclude file 
a unique name, and use that unique name in your spec as the exclude 
file.  Its that easy.

What I would like is to do is backup 1) and 2) seperately, but
once we get to 3 I don't want thost 2 directories to be backed up
from disk 3

Now is this possible.

I'm going to assume you forget the question mark above.

see above. :-)


-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.07% setiathome rank, not too shabby for a WV hillbilly

Re: disklist and sub directories.

[Gene Heskett]

On Monday 22 July 2002 23:15, Jon LaBadie wrote:
On Mon, Jul 22, 2002 at 07:06:07PM -0500, Craig Hancock 
[EMAIL PROTECTED] wrote:
 Hello all after playing with amanda all week I have finally
 gotten into a situation were my entry in disklist file is to
 big. From what I've read amanda isn't smart enough to pick were
 it left off if it ran out of tape

smart enough sounds so judgemental :(   mods always welcome.

 it will attempt to start from the begining again. Because of
 this I was thinking of making of these disklist.

 Origianally I have a disklist of /home/data but unfortantely is
 is 92G but tape can't write a complete 92 gigs. So I was
 thinking of spliting this in 3's

 1) /home/data/directory
 2) /home/data/directory2
 3) /home/data

 What I would like is to do is backup 1) and 2) seperately, but
 once we get to 3 I don't want thost 2 directories to be backed
 up from disk 3

 Now is this possible.

done all the time.

Have to use tar, not dump to do directories.
Set up a uniq dumptype for /home/data that differs only in the
exclude list file which would exclude directory and directory2
as well as all the usual suspects ... err directories.

As usual, you said it in fewer words, Jon.  And I forgot to 
emphasize that one would have to use tar, but then, from his 
disklist, *I* assumed (without even thinking about it) that the  
choice of tar had already been made.  But I should have mentioned 
it anyway, thanks.
-- 
Cheers, Gene
AMD K6-III@500mhz 320M
Athlon1600XP@1400mhz  512M
99.07% setiathome rank, not too shabby for a WV hillbilly

Re: disklist and hw compression

[Marc W. Mengel]

When you are using hardware compression on the tape drive, the
backup image cached on disk is of course not yet compressed.
If it thinks the dump won't fit on the tape, did you pick
a different tapetype for the compressed device?

On Tue, 2 Apr 2002, Matthew Boeckman wrote:

 I have two (hopefully) quick questions.

 I'm running an ultra5, and trying to level0 a 60+gig filesystem. Using
 software compression took over 22 hours (i killed it at that point) so I
   decided to use hardware, specifying /dev/rmt/0cn as the tape device in
 amanda.conf. When I tried to run amdump, it failed saying that dumps
 too large for disk, which is true if you don't count the hardware
 compression I was trying to use. The tapes are AIT2 50GB tapes. Is there
 a way to force amanda to do this? I assume perhaps amadmin csd force
 diskname ? I wanted to know for sure before kicking off another
 20+hour backup cycle. Or is there something that I have to configure in
 amanda to let it know i'm using hardware compression and to ignore tape
 warnings?

 The second and sort-of corollary to that is what is the exact syntax of
 exclude files referenced from disklist? I can't seem to find an exact
 answer in the documentation/FAQOMATIC. The examples/disklist file has
 the following:
 slowsrv /usr {
user-tar
exclude list .exclude
compress server fast
 } 2 le0 # no line break before spindle and interface
   Does that mean that .exclude is in /usr ? Is it exclude absolute
 pathnames or any pattern match? In other words if I want to exclude
 /usr/bob, and in the exclude I have /bob, will that exclude any match
 for bob, or only subdirs of the mount being backed up? Likewise, what
 are the compress server fast lines about?

 Thanks!



-- 
-
Marc Mengel [EMAIL PROTECTED]

Re: disklist and hw compression

[Joshua Baker-LePain]

On Tue, 2 Apr 2002 at 7:59am, Matthew Boeckman wrote

 20+hour backup cycle. Or is there something that I have to configure in 
 amanda to let it know i'm using hardware compression and to ignore tape 
 warnings?

To use hardware compression, you fib to amanda about your tapelength.  In 
your tapetype, put a reasonable guess, based upon your data, as to how 
much you can get on tape.  If it's all code and text files, 90-100GB 
wouldn't be far off.  If it's all jpgs, well, 50GB may be about right.  
You may need to adjust if you keep hitting EOT.

 slowsrv /usr {
user-tar
exclude list .exclude
compress server fast
 } 2 le0 # no line break before spindle and interface
   Does that mean that .exclude is in /usr ? Is it exclude absolute 

I believe so.

 pathnames or any pattern match? In other words if I want to exclude 
 /usr/bob, and in the exclude I have /bob, will that exclude any match 
 for bob, or only subdirs of the mount being backed up? Likewise, what 

./bob would exclude /usr/bob.

 are the compress server fast lines about?
 
That's software (gzip) compression being done on the amanda server (rather 
than the client).  If you turn hardware compression on, be sure to turn 
software off.

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Re: disklist and hw compression

[Eric Trager]

On Tue, 2 Apr 2002, Matthew Boeckman wrote:

 I'm running an ultra5, and trying to level0 a 60+gig filesystem. Using
 software compression took over 22 hours (i killed it at that point) so I
   decided to use hardware, specifying /dev/rmt/0cn as the tape device in
 amanda.conf. When I tried to run amdump, it failed saying that dumps
 too large for disk, which is true if you don't count the hardware
 compression I was trying to use. The tapes are AIT2 50GB tapes. Is there
 a way to force amanda to do this? I assume perhaps amadmin csd force
 diskname ? I wanted to know for sure before kicking off another
 20+hour backup cycle. Or is there something that I have to configure in
 amanda to let it know i'm using hardware compression and to ignore tape
 warnings?

Make sure that you configure the tape length in amanda.conf to specify
what you think it is that you'll be able to get on a tape counting
hardware compression.

When I used (note past tense) hardware compression with our systems, I was
using 40 GB DLT tapes. With regular, unzipped data I was getting close to
70 GB on a tape, so I set the tape length to 65000 Mb. It worked fine with
several partitions and combinations, anywhere from 45 to 65 GB in size.
Unfortunately, one particular partition, only 45-50 GB in size, was loaded
with small directories containing nothing but gzips. I was not able to get
that on a tape until I used GNUTAR to break it up.

I stopped using hardware compression because the general attitude among
the experts here was that amanda is better off doing her own packing. I
haven't really noticed a significant difference, but, as always, YMMV,
etc.

- -  -   --
Eric Trager

Re: disklist and hw compression

[John R. Jackson]

The second and sort-of corollary to that is what is the exact syntax of 
exclude files referenced from disklist?  ...

Andrew Hall wrote a nice description of exclusion patterns.  It is
part of the 2.4.3 docs directory, so you could grab a recent beta from
www.amanda.org.

Matthew Boeckman

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: Disklist Piority

[Johannes Niess]

[EMAIL PROTECTED] writes:

 Is there are way of forcing amanda to backup the directories in the 
 disklist in a certain order or at least tell it to back up a thew 
 directories first?

David,

There is a priority parameter for dumptypes. But it does not work for
your purpose.

 priority string
  Default: medium.  When there is no  tape  to  write
  to,  Amanda will do incremental backups in priority
  order to the holding disk.   The  priority  may  be
  high(2),  medium(1),  low(0)  or  a  number of your
  choice.

Having the backup sequence fixed is counterproductive in normal
environments: Fast, high backup levels get stalled until the slow
level 0 of that big disk is done. Total backup time goes skyrocketing
because you loose the parallesism between clients.

Why do you want a fixed sequence? Would this help:

 starttime int
  Default: none.  Backups will not start until  after
  this  time  of day.  The value should be hh*100+mm,
  e.g. 6:30PM (18:30) would be entered as 1830.

You could delay the unimportant directories.

Johannes Nieß

Re: Disklist ?

[bhlewis]

Hi,

[EMAIL PROTECTED] wrote:

 I'm going to deploy AMANDA as a backup system.
 However, I'm in doubt some parts in disklist file
 What should it be in the file if
 
 - I want backup only index.html in every user's home dir
   /home/*/htdocs/index.html
 - I want backup only all file have .pl extension
  /htdocs/*.pl
 - I want backup every file whose name led by phplib
  /usr/local/apache/phplib*
 - I want backup only shadow file in /etc
  /etc/shadow

Amanda really doesn't work that way.  It is intended to back up whole 
filesystems rather than selected files througout the filesystem.  It is
possible to exclude files but I think you'd have a tough time writing
exclusion rules that do what you want (exlude all but certain filenames).

Instead, you might consider writing a short script that went out and found 
the files you want and copied them to one (perhaps very small) filesystem.
You could then start your script sometime before the Amanda run is scheduled
and point Amanda at your special filesystem.

Another option is to use a GNUTAR wrapper script that pretends to be GNU 
tar to Amanda but really does something of your own devising.  Perhaps it
could run GNU tar with a list of filenames generated by find. John Jackson
has an example GNUTAR wrapper script here:

ftp://gandalf.cc.purdue.edu/pub/amanda/gtar-wrapper.README
ftp://gandalf.cc.purdue.edu/pub/amanda/gtar-wrapper.ksh

I hope this helps,

-Ben

-- 
Benjamin LewisThank goodness modern convenience is a 
Database Analyst/Programmer  thing of the remote future.
Purdue University Computing Center  -- Pogo, by Walt Kelly
[EMAIL PROTECTED] 

Re: disklist config

[Joshua Baker-LePain]

On Wed, 5 Sep 2001 at 11:33pm, [EMAIL PROTECTED] wrote

 My dumptype has no exclusions at all. How come
 the original disklist with:
 hostname / dumptype

 does not pickup the other file systems which are mounted as /usr and /var?

'dump' always works on a per-filesystem basis, and amanda passes the
--one-filesystem (or somesuch) flag to tar.  This is the way backups have
always been run.

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University

Re: disklist format

[John R. Jackson]

Trying to get this running on MacOSX.

Disks are labeled as /dev/rdisk0,/dev/rdis1, etc.

How does one write the entry for the 9th partition on /dev/rdisk0?  ...

What does df -k say for that 9th partition?

Why aren't you using mount points instead of /dev names?

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: Disklist and network units

[Olivier Nicole]

but i get an error reading disklist. I have seen in the disklist file example

Interesting, what error?

2) When i tried to backup a directory of a mounted linux server unit i don't
get errors but it doesn't backup anything. Any idea?

What is a directory of a mounted linux server unit ?

Olivier

Re: Disklist and network units

[John R. Jackson]

1) How should i specify a device for backup?  ...

By far the easiest way is to list the mount point.  For instance /var.
There is no reason to convert that to a disk name.  Amanda will do all
of that for you, if needed.

... i get an error reading disklist.  ...

Please post what you tried to do (hopefully, amcheck) and the exact
error message.

2) When i tried to backup a directory of a mounted linux server unit i don't
get errors but it doesn't backup anything. Any idea?

I don't understand.  Are you trying to back up an NFS mounted
directory?  Why not install Amanda on the NFS server and do it that way?
In particular, using GNU tar across an NFS mount is going to have problems
unless you set the NFS mount options properly (allow root access).

Ramon Guiu

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: Disklist formatting questions:

[John R. Jackson]

   the error messages complain about not being able to open /dev/root.

This and your other item are two symptoms of the same problem.

   Does amanda have the ability to interpret the LABEL= portion
   of fstab then dig the information out of ext2fs?  ...

Yes.  You need to upgrade to 2.4.2p2 and also check out the Amanda
patches page for one more item beyond that.

--jason

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]

Re: disklist

[John R. Jackson]

what is the content of this "disklist" and where should I put it?

The disklist file tells Amanda what clients and disks to back up.
It goes in the same directory as amanda.conf.  "man amanda" describes
the syntax, and there is an example file with the sources.

John R. Jackson, Technical Software Specialist, [EMAIL PROTECTED]