Re: [Bacula-users] backup size prediction?

2006-07-07 Thread Ludovic Strappazon




Hello Kern,

Is it correctly attached now ?

Ludovic.

Kern Sibbald wrote:

  Hello Ludovic,

Could you send the script as an attachment.  As it came, it was line 
wrapped ...

Regards,

Kern

On Thursday 06 July 2006 20:22, Ludovic Strappazon wrote:
  
  
Hello Arno,

This is the script :

#! /bin/sh
# Author : Ludovic Strappazon. [EMAIL PROTECTED]
# Copyright : Kern Sibbald
# Any comment, advice or enhancement are welcome :-)

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
MYSQL="/usr/bin/mysql -u bacula --password=X"
TMP=/tmp
BACULA=/usr/local/bacula

PROGNAME=`basename $0`
PROGPATH=`echo $0 | sed -e 's,[\\/][^\\/][^\\/]*$,,'`
STATUS=""

. $PROGPATH/utils.sh

print_usage() {
echo "Usage: $PROGNAME -P pool -M media-type -w warning
threshold -c critical threshold [-S]"
}

print_help() {
echo ""
print_usage
echo ""
echo "This plugin checks the space available in the pool against
the space required for the next scheduled backups"
echo "Example : $PROGNAME -P default -M LTO -w 20 -c 10 will
check the default pool, return OK if (available space)  1,20*(required
space), WARNING if 1,20*(required space)  (available space) 
1,10*(required space), and CRITICAL else."
echo ""
echo "With the -S option, it will check the pool named Scratch and
return WARNING instead of CRITICAL if the Scratch pool can save the
situation."
echo "Example : $PROGNAME -P default -M LTO -w 20 -c 10 -S will
check the default pool, return OK if (available space)  1,20*(required
space), WARNING if 1,20*(required space)  (available space) 
1,10*(required space) or if (available space in default and Scratch) 
1,10*(required space)  (available space in default), and CRITICAL else."
echo ""
echo "The evaluation of the space required is done by adding the
biggest backups of the same level than the scheduled jobs"
echo "The available space is evaluated by the number of out of
retention tapes and the average VolBytes of these Full tapes"
echo ""
echo "The Information Status are : \"Required, Available, Volume
Errors\" and \"Will use Scratch pool\" if necessary."
echo ""
echo "I think this plugin should be used in passive mode, and ran by
a RunAfterJob"
exit 3
}

NB_ARGS=$#
SCRATCH=0
while getopts :P:M:w:c:hS OPTION
do
  case $OPTION in
P) POOL="$OPTARG"
   ;;
M) MEDIA_TYPE="$OPTARG"
   ;;
S) SCRATCH=1
   ;;
w) WARNING="$OPTARG"
   ;;
c) CRITICAL="$OPTARG"
   ;;
h) print_help
   exit 3
   ;;
*) print_usage
   exit 3
   ;;
  esac
done
shift $(($OPTIND - 1))

if [ "$NB_ARGS" -ne 8 -a "$NB_ARGS" -ne 9 ]; then
print_revision $PROGNAME 13/06/2006
print_usage
exit 3
fi

LAST_CHECK=`ps -ef | grep check_ba[Cc]ula_pools.sh | awk {'print $5'} |
uniq | wc -l`
if [ "$LAST_CHECK" -gt 1 ]; then
echo "The last check was not complete, you should increase the
check_period."
exit 3
fi

NB_VOLUMES=`$MYSQL  EOF
USE bacula
SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
and Pool.Name="$POOL" AND Inchanger = "1";
EOF`

NB_VOLUMES=`echo $NB_VOLUMES | cut -f 2 -d ' '`

echo "st
1
q" | $BACULA/etc/bconsole | sed -n /Scheduled/,/Running/p | grep Backup

| tr -s [:blank:] | tr '[:blank:]' '@'  ${TMP}/Scheduled.txt

CAPA_REQUIRED=0
for LINE in `cat ${TMP}/Scheduled.txt`
do
  SCHEDULED_JOB=`echo $LINE | awk -F@ '{print $6}'`
  LEVEL=`echo $LINE | awk -F@ '{print $1}' | cut -c 1`

MAX_VOLUME_JOB_FOR_LEVEL=`$MYSQL  EOF
USE bacula
SELECT MAX(JobBytes) from Job, Pool where Level="$LEVEL" AND
Job.Name="$SCHEDULED_JOB" AND Job.PoolId=Pool.PoolId AND Pool.Name="$POOL";
EOF
`
MAX_VOLUME_JOB_FOR_LEVEL=`echo $MAX_VOLUME_JOB_FOR_LEVEL | cut -f 2 -d ' '
`

CAPA_REQUIRED=$[CAPA_REQUIRED+MAX_VOLUME_JOB_FOR_LEVEL]
done

rm ${TMP}/Scheduled.txt


if [ $NB_VOLUMES -gt 0 ]
then

  NB_VOLUMES_OUT_OF_RETENTION=`$MYSQL  EOF
USE bacula
SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
and Pool.Name="$POOL" AND LastWritten  "-00-00 00:00:00" AND
UNIX_TIMESTAMP()-UNIX_TIMESTAMP(LastWritten)Media.VolRetention AND
Inchanger = "1";
EOF
`
  NB_VOLUMES_OUT_OF_RETENTION=`echo $NB_VOLUMES_OUT_OF_RETENTION | cut
-f 2 -d ' '`

NB_VOLUMES_ERROR=`$MYSQL  EOF
USE bacula
SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
and Pool.Name="$POOL" AND VolStatus="Error" AND Inchanger = "1";
EOF
`
NB_VOLUMES_ERROR=`echo $NB_VOLUMES_ERROR | cut -f 2 -d ' '`

AVERAGE_CAPA_VOLUME=`$MYSQL  EOF
USE bacula
SELECT SUM(VolBytes)/COUNT(MediaId) FROM Media where VolStatus="Full"
AND MediaType="$MEDIA_TYPE";
EOF
`
AVERAGE_CAPA_VOLUME=`echo $AVERAGE_CAPA_VOLUME | cut -f 2 -d ' ' | cut
-f 1 -d '.'`

CAPA_VOLUMES_APPEND=`$MYSQL  EOF
USE bacula
SELECT SUM("$AVERAGE_CAPA_VOLUME"-VolBytes) from Media, Pool where
Media.PoolId=Pool.PoolId and Pool.Name="$POOL" AND (VolStatus = "Append"
OR VolStatus = "Recycle" OR VolStatus = "Purge") AND Inchanger = "1" AND

Re: [Bacula-users] backup size prediction?

2006-07-07 Thread Kern Sibbald
On Friday 07 July 2006 08:17, Ludovic Strappazon wrote:
 Hello Kern,

 Is it correctly attached now ?

Yes, thanks.  I put it in the 1.39 source in examples/database ...

Thanks.


 Ludovic.

 Kern Sibbald wrote:
  Hello Ludovic,
 
  Could you send the script as an attachment.  As it came, it was line
  wrapped ...
 
  Regards,
 
  Kern
 
  On Thursday 06 July 2006 20:22, Ludovic Strappazon wrote:
  Hello Arno,
 
  This is the script :
 
  #! /bin/sh
  # Author : Ludovic Strappazon. [EMAIL PROTECTED]
  # Copyright : Kern Sibbald
  # Any comment, advice or enhancement are welcome :-)
 
  PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
  MYSQL=/usr/bin/mysql -u bacula --password=X
  TMP=/tmp
  BACULA=/usr/local/bacula
 
  PROGNAME=`basename $0`
  PROGPATH=`echo $0 | sed -e 's,[\\/][^\\/][^\\/]*$,,'`
  STATUS=
 
  . $PROGPATH/utils.sh
 
  print_usage() {
  echo Usage: $PROGNAME -P pool -M media-type -w warning
  threshold -c critical threshold [-S]
  }
 
  print_help() {
  echo 
  print_usage
  echo 
  echo This plugin checks the space available in the pool against
  the space required for the next scheduled backups
  echo Example : $PROGNAME -P default -M LTO -w 20 -c 10 will
  check the default pool, return OK if (available space)  1,20*(required
  space), WARNING if 1,20*(required space)  (available space) 
  1,10*(required space), and CRITICAL else.
  echo 
  echo With the -S option, it will check the pool named Scratch and
  return WARNING instead of CRITICAL if the Scratch pool can save the
  situation.
  echo Example : $PROGNAME -P default -M LTO -w 20 -c 10 -S will
  check the default pool, return OK if (available space)  1,20*(required
  space), WARNING if 1,20*(required space)  (available space) 
  1,10*(required space) or if (available space in default and Scratch) 
  1,10*(required space)  (available space in default), and CRITICAL
  else. echo 
  echo The evaluation of the space required is done by adding the
  biggest backups of the same level than the scheduled jobs
  echo The available space is evaluated by the number of out of
  retention tapes and the average VolBytes of these Full tapes
  echo 
  echo The Information Status are : \Required, Available, Volume
  Errors\ and \Will use Scratch pool\ if necessary.
  echo 
  echo I think this plugin should be used in passive mode, and ran by
  a RunAfterJob
  exit 3
  }
 
  NB_ARGS=$#
  SCRATCH=0
  while getopts :P:M:w:c:hS OPTION
  do
case $OPTION in
  P) POOL=$OPTARG
 ;;
  M) MEDIA_TYPE=$OPTARG
 ;;
  S) SCRATCH=1
 ;;
  w) WARNING=$OPTARG
 ;;
  c) CRITICAL=$OPTARG
 ;;
  h) print_help
 exit 3
 ;;
  *) print_usage
 exit 3
 ;;
esac
  done
  shift $(($OPTIND - 1))
 
  if [ $NB_ARGS -ne 8 -a $NB_ARGS -ne 9 ]; then
  print_revision $PROGNAME 13/06/2006
  print_usage
  exit 3
  fi
 
  LAST_CHECK=`ps -ef | grep check_ba[Cc]ula_pools.sh | awk {'print $5'} |
  uniq | wc -l`
  if [ $LAST_CHECK -gt 1 ]; then
  echo The last check was not complete, you should increase the
  check_period.
  exit 3
  fi
 
  NB_VOLUMES=`$MYSQL  EOF
  USE bacula
  SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
  and Pool.Name=$POOL AND Inchanger = 1;
  EOF`
 
  NB_VOLUMES=`echo $NB_VOLUMES | cut -f 2 -d ' '`
 
  echo st
  1
  q | $BACULA/etc/bconsole | sed -n /Scheduled/,/Running/p | grep Backup
 
  | tr -s [:blank:] | tr '[:blank:]' '@'  ${TMP}/Scheduled.txt
 
  CAPA_REQUIRED=0
  for LINE in `cat ${TMP}/Scheduled.txt`
  do
SCHEDULED_JOB=`echo $LINE | awk -F@ '{print $6}'`
LEVEL=`echo $LINE | awk -F@ '{print $1}' | cut -c 1`
 
  MAX_VOLUME_JOB_FOR_LEVEL=`$MYSQL  EOF
  USE bacula
  SELECT MAX(JobBytes) from Job, Pool where Level=$LEVEL AND
  Job.Name=$SCHEDULED_JOB AND Job.PoolId=Pool.PoolId AND
  Pool.Name=$POOL; EOF
  `
  MAX_VOLUME_JOB_FOR_LEVEL=`echo $MAX_VOLUME_JOB_FOR_LEVEL | cut -f 2 -d '
  ' `
 
  CAPA_REQUIRED=$[CAPA_REQUIRED+MAX_VOLUME_JOB_FOR_LEVEL]
  done
 
  rm ${TMP}/Scheduled.txt
 
 
  if [ $NB_VOLUMES -gt 0 ]
  then
 
NB_VOLUMES_OUT_OF_RETENTION=`$MYSQL  EOF
  USE bacula
  SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
  and Pool.Name=$POOL AND LastWritten  -00-00 00:00:00 AND
  UNIX_TIMESTAMP()-UNIX_TIMESTAMP(LastWritten)Media.VolRetention AND
  Inchanger = 1;
  EOF
  `
NB_VOLUMES_OUT_OF_RETENTION=`echo $NB_VOLUMES_OUT_OF_RETENTION | cut
  -f 2 -d ' '`
 
  NB_VOLUMES_ERROR=`$MYSQL  EOF
  USE bacula
  SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
  and Pool.Name=$POOL AND VolStatus=Error AND Inchanger = 1;
  EOF
  `
  NB_VOLUMES_ERROR=`echo $NB_VOLUMES_ERROR | cut -f 2 -d ' '`
 
  AVERAGE_CAPA_VOLUME=`$MYSQL  EOF
  USE bacula
  SELECT SUM(VolBytes)/COUNT(MediaId) FROM Media where VolStatus=Full
  AND MediaType=$MEDIA_TYPE;
  EOF
  `
  

Re: [Bacula-users] backup size prediction?

2006-07-07 Thread Ludovic Strappazon




We also use this one to summarily check (with nagios) the backups.
If it can be of any use to someone else...

Ludovic. 

Kern Sibbald a crit:

  On Friday 07 July 2006 08:17, Ludovic Strappazon wrote:
  
  
Hello Kern,

Is it correctly attached now ?

  
  
Yes, thanks.  I put it in the 1.39 source in examples/database ...

Thanks.

  
  
Ludovic.

Kern Sibbald wrote:


  Hello Ludovic,

Could you send the script as an attachment.  As it came, it was line
wrapped ...

Regards,

Kern

On Thursday 06 July 2006 20:22, Ludovic Strappazon wrote:
  
  
Hello Arno,

This is the script :

#! /bin/sh
# Author : Ludovic Strappazon. [EMAIL PROTECTED]
# Copyright : Kern Sibbald
# Any comment, advice or enhancement are welcome :-)

PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
MYSQL="/usr/bin/mysql -u bacula --password=X"
TMP=/tmp
BACULA=/usr/local/bacula

PROGNAME=`basename $0`
PROGPATH=`echo $0 | sed -e 's,[\\/][^\\/][^\\/]*$,,'`
STATUS=""

. $PROGPATH/utils.sh

print_usage() {
echo "Usage: $PROGNAME -P pool -M media-type -w warning
threshold -c critical threshold [-S]"
}

print_help() {
echo ""
print_usage
echo ""
echo "This plugin checks the space available in the pool against
the space required for the next scheduled backups"
echo "Example : $PROGNAME -P default -M LTO -w 20 -c 10 will
check the default pool, return OK if (available space)  1,20*(required
space), WARNING if 1,20*(required space)  (available space) 
1,10*(required space), and CRITICAL else."
echo ""
echo "With the -S option, it will check the pool named Scratch and
return WARNING instead of CRITICAL if the Scratch pool can save the
situation."
echo "Example : $PROGNAME -P default -M LTO -w 20 -c 10 -S will
check the default pool, return OK if (available space)  1,20*(required
space), WARNING if 1,20*(required space)  (available space) 
1,10*(required space) or if (available space in default and Scratch) 
1,10*(required space)  (available space in default), and CRITICAL
else." echo ""
echo "The evaluation of the space required is done by adding the
biggest backups of the same level than the scheduled jobs"
echo "The available space is evaluated by the number of out of
retention tapes and the average VolBytes of these Full tapes"
echo ""
echo "The Information Status are : \"Required, Available, Volume
Errors\" and \"Will use Scratch pool\" if necessary."
echo ""
echo "I think this plugin should be used in passive mode, and ran by
a RunAfterJob"
exit 3
}

NB_ARGS=$#
SCRATCH=0
while getopts :P:M:w:c:hS OPTION
do
  case $OPTION in
P) POOL="$OPTARG"
   ;;
M) MEDIA_TYPE="$OPTARG"
   ;;
S) SCRATCH=1
   ;;
w) WARNING="$OPTARG"
   ;;
c) CRITICAL="$OPTARG"
   ;;
h) print_help
   exit 3
   ;;
*) print_usage
   exit 3
   ;;
  esac
done
shift $(($OPTIND - 1))

if [ "$NB_ARGS" -ne 8 -a "$NB_ARGS" -ne 9 ]; then
print_revision $PROGNAME 13/06/2006
print_usage
exit 3
fi

LAST_CHECK=`ps -ef | grep check_ba[Cc]ula_pools.sh | awk {'print $5'} |
uniq | wc -l`
if [ "$LAST_CHECK" -gt 1 ]; then
echo "The last check was not complete, you should increase the
check_period."
exit 3
fi

NB_VOLUMES=`$MYSQL  EOF
USE bacula
SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
and Pool.Name="$POOL" AND Inchanger = "1";
EOF`

NB_VOLUMES=`echo $NB_VOLUMES | cut -f 2 -d ' '`

echo "st
1
q" | $BACULA/etc/bconsole | sed -n /Scheduled/,/Running/p | grep Backup

| tr -s [:blank:] | tr '[:blank:]' '@'  ${TMP}/Scheduled.txt

CAPA_REQUIRED=0
for LINE in `cat ${TMP}/Scheduled.txt`
do
  SCHEDULED_JOB=`echo $LINE | awk -F@ '{print $6}'`
  LEVEL=`echo $LINE | awk -F@ '{print $1}' | cut -c 1`

MAX_VOLUME_JOB_FOR_LEVEL=`$MYSQL  EOF
USE bacula
SELECT MAX(JobBytes) from Job, Pool where Level="$LEVEL" AND
Job.Name="$SCHEDULED_JOB" AND Job.PoolId=Pool.PoolId AND
Pool.Name="$POOL"; EOF
`
MAX_VOLUME_JOB_FOR_LEVEL=`echo $MAX_VOLUME_JOB_FOR_LEVEL | cut -f 2 -d '
' `

CAPA_REQUIRED=$[CAPA_REQUIRED+MAX_VOLUME_JOB_FOR_LEVEL]
done

rm ${TMP}/Scheduled.txt


if [ $NB_VOLUMES -gt 0 ]
then

  NB_VOLUMES_OUT_OF_RETENTION=`$MYSQL  EOF
USE bacula
SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
and Pool.Name="$POOL" AND LastWritten  "-00-00 00:00:00" AND
UNIX_TIMESTAMP()-UNIX_TIMESTAMP(LastWritten)Media.VolRetention AND
Inchanger = "1";
EOF
`
  NB_VOLUMES_OUT_OF_RETENTION=`echo $NB_VOLUMES_OUT_OF_RETENTION | cut
-f 2 -d ' '`

NB_VOLUMES_ERROR=`$MYSQL  EOF
USE bacula
SELECT COUNT(MediaId) from Media, Pool where Media.PoolId=Pool.PoolId
and Pool.Name="$POOL" AND VolStatus="Error" AND Inchanger = "1";
EOF
`
NB_VOLUMES_ERROR=`echo $NB_VOLUMES_ERROR | cut -f 2 -d ' '`

AVERAGE_CAPA_VOLUME=`$MYSQL  EOF
USE bacula
SELECT SUM(VolBytes)/COUNT(MediaId) FROM Media where VolStatus="Full"
AND MediaType="$MEDIA_TYPE";
EOF
`

Re: [Bacula-users] backup size prediction?

2006-07-06 Thread Ludovic Strappazon

Hi Mark,

I've written a nagios shell script to check my pools in the way you 
describe.
I also remember that Arno Lehmann did something similar in perl (it is 
distibuted with the bacula sources)


Regards,
Ludovic Strappazon.

[EMAIL PROTECTED] a écrit :
Is there any way to have bacula predict how many volumes will be needed for the 
next night's backup? 


My goal is to make sure that the autochanger is always loaded with enough tapes
to handle the next night's backup, without changing tapes every day.

I'm interested in having bacula notify me in advance that it guesses that the
following night's backup will require N volumes. I don't want to run
estimate for each client (which is a time-consuming and resource intensive
action), and then sum the results. It would be great if bacula made use of
information that's in the database to get the average of the sizes of the last
X backups of the same level that's scheduled for the next backup (ie., full,
incremental, differential) for each client, and then added a user-defined
fudge-factor (maybe 10%) to guess at the size of the upcoming backup for all
scheduled jobs.

For example, most of our non-full backups are under 100GB for all clients
combined, so our autochanger can run for a long time without manually removing
full tapes and putting in new ones. However, full backups can be anywhere from
40GB to over 2TB (depending on the client). To reduce contention, full backups
are set up with a staggered schedule--each client does a full on a different
night. It would be nice to get some warning from bacula that the next night's
backup will probably take 6 volumes, so that I can make sure the enough empty
(or over-writeable) tapes are loaded.


Any suggestions about scripting such an early warning system? I'm thinking of 
something like:


--- bad pseudo-code ---
foreach $client sqlquery(Get list of clients scheduled to backup 
tonight)
{
$backup_level=sqlquery(Get level of next backup for $client);

$size=average(sqlquery size in GB of all backups for
$client of $backup_level);

$predictedsize=$total + $size;
}

$predictedsize=$predictedsize * $fudgefactor;

foreach $volsize (sqlquery(Select VolumeSize in GB
where VolumeStatus='Full'))
{
$volsizetotal=$volsize + $volsizetotal;
		$volcount++; 
	}


$average_GB_to_fill_volume=$volsizetotal/$volcount;

$volumes_needed=$predictedsize / $average_GB_to_fill_volume;

roundup($volumes_needed);

print There will be $volumes_needed needed for tonight's backup;
 EO bad pseudo-code ---



Thanks,

Mark

Mark Bergman  [EMAIL PROTECTED]
System Administrator
Section of Biomedical Image Analysis 215-662-7310
Department of Radiology,   University of Pennsylvania

http://pgpkeys.pca.dfn.de:11371/pks/lookup?search=mark.bergman%40.uphs.upenn.edu



The information contained in this e-mail message is intended only for the 
personal and confidential use of the recipient(s) named above. If the reader of 
this message is not the intended recipient or an agent responsible for 
delivering it to the intended recipient, you are hereby notified that you have 
received this document in error and that any review, dissemination, 
distribution, or copying of this message is strictly prohibited. If you have 
received this communication in error, please notify us immediately by e-mail, 
and delete the original message.

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users

  




check_bacula_pools.sh
Description: application/shellscript
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] backup size prediction?

2006-07-06 Thread Arno Lehmann
Hello,

On 7/6/2006 5:10 PM, Ludovic Strappazon wrote:
 Hi Mark,
 
 I've written a nagios shell script to check my pools in the way you 
 describe.
 I also remember that Arno Lehmann did something similar in perl (it is 
 distibuted with the bacula sources)

Erm, no, not really.
I considered adding such a function to the baculareport scrip I created, 
but never actually found it necessary. But I really like to see what 
you've got :-)

 Regards,
 Ludovic Strappazon.
 
 [EMAIL PROTECTED] a écrit :
 
 Is there any way to have bacula predict how many volumes will be 
 needed for the next night's backup?
 My goal is to make sure that the autochanger is always loaded with 
 enough tapes
 to handle the next night's backup, without changing tapes every day.

 I'm interested in having bacula notify me in advance that it guesses 
 that the
 following night's backup will require N volumes. I don't want to run
 estimate for each client (which is a time-consuming and resource 
 intensive
 action), and then sum the results. It would be great if bacula made 
 use of
 information that's in the database to get the average of the sizes of 
 the last
 X backups of the same level that's scheduled for the next backup 
 (ie., full,
 incremental, differential) for each client, and then added a user-defined
 fudge-factor (maybe 10%) to guess at the size of the upcoming backup 
 for all
 scheduled jobs.

I would even try to estimate the expected growth (does *anybody* have 
backup sets that become smaller over time?) from the existing backups. 
Simply assume a linear growth and add the observed error to your result. 
Or something. I would need to brush up my mathematical / statistical 
knowledge a bit :-)

 For example, most of our non-full backups are under 100GB for all clients
 combined, so our autochanger can run for a long time without manually 
 removing
 full tapes and putting in new ones. However, full backups can be 
 anywhere from
 40GB to over 2TB (depending on the client). To reduce contention, full 
 backups
 are set up with a staggered schedule--each client does a full on a 
 different
 night. It would be nice to get some warning from bacula that the next 
 night's
 backup will probably take 6 volumes, so that I can make sure the 
 enough empty
 (or over-writeable) tapes are loaded.


 Any suggestions about scripting such an early warning system? I'm 
 thinking of something like:

 --- bad pseudo-code ---
 foreach $client sqlquery(Get list of clients scheduled to backup 
 tonight)
 {
 $backup_level=sqlquery(Get level of next backup for $client);

One problem might be to get the pool the next backup goes to.

 $size=average(sqlquery size in GB of all backups for
 $client of $backup_level);
 
 $predictedsize=$total + $size;
 }

 $predictedsize=$predictedsize * $fudgefactor;

This would need to be calculated per pool, IMO.

 foreach $volsize (sqlquery(Select VolumeSize in GB
 where VolumeStatus='Full'))
 {
 $volsizetotal=$volsize + $volsizetotal;
 $volcount++; }

 $average_GB_to_fill_volume=$volsizetotal/$volcount;

I do have some function to estimate volume capacity in baculareport.pl 
which is a little more complicated, but still gives reasonable results :-)

 $volumes_needed=$predictedsize / $average_GB_to_fill_volume;

 roundup($volumes_needed);

 print There will be $volumes_needed needed for tonight's backup;
  EO bad pseudo-code 
 ---


Apart from my obsession with with putting backups of different levels to 
different pools and storage devices, I fully agree - such a script would 
be a nice addition.

Arno


 Thanks,

 Mark
 
 Mark Bergman  [EMAIL PROTECTED]
 System Administrator
 Section of Biomedical Image Analysis 215-662-7310
 Department of Radiology,   University of Pennsylvania

 http://pgpkeys.pca.dfn.de:11371/pks/lookup?search=mark.bergman%40.uphs.upenn.edu
  




 The information contained in this e-mail message is intended only for 
 the personal and confidential use of the recipient(s) named above. If 
 the reader of this message is not the intended recipient or an agent 
 responsible for delivering it to the intended recipient, you are 
 hereby notified that you have received this document in error and that 
 any review, dissemination, distribution, or copying of this message is 
 strictly prohibited. If you have received this communication in error, 
 please notify us immediately by e-mail, and delete the original message.

 Using Tomcat but need to do more? Need to support web services, security?
 Get stuff done quickly with pre-integrated technology to make your job 
 easier
 Download IBM WebSphere Application Server v.1.0.1 based on Apache 
 Geronimo
 http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
 

Re: [Bacula-users] backup size prediction?

2006-07-06 Thread Arno Lehmann
Ludovic,

On 7/6/2006 8:22 PM, Ludovic Strappazon wrote:
   Hello Arno,
 
 This is the script :

thanks a lot!

This looks like it's meant for use as a Nagios check, is this assumption 
correct?

Arno

-- 
IT-Service Lehmann[EMAIL PROTECTED]
Arno Lehmann  http://www.its-lehmann.de

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users


Re: [Bacula-users] backup size prediction?

2006-07-06 Thread Ludovic Strappazon




Arno Lehmann wrote:

  Ludovic,

On 7/6/2006 8:22 PM, Ludovic Strappazon wrote:
  
  
  Hello Arno,

This is the script :

  
  
thanks a lot!

This looks like it's meant for use as a Nagios check, is this assumption 
correct?

Arno

  

Yes, we use it with Nagios.

Ludovic.


Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users