Re: [Bacula-users] Deadweight volumes, how to discard?
Hello there,
I often use mysql directly on the bacula catalog to check for old
volumes that haven't been used for a while for whatever reason. It's
quite simple if you take a look at the "Media" table structure:
# Select all volumes where LastWritten is older than Jan 1st, 2021:
echo 'select VolumeName, LastWritten, VolBytes from Media where LastWritten <
"2021-01-01";' | mysql -pXXX bacula
VolumeName LastWritten VolBytes
client0298-0206 2020-12-31 10:37:13 45526845926
You can then pipe that through awk or some other tool to create a bconsole
"script":
for v in $(echo 'select VolumeName, LastWritten, VolBytes from Media where
LastWritten < "2021-01-01";' | mysql -s -p bacula | awk '{print
$1}' ) ; do echo purge volume="$v"; done
purge volume=client0298-0206
and then pipe that directly to bacula / bconsole if you're feeling adventurous
like so:
( for v in $(echo 'select VolumeName, LastWritten, VolBytes from Media where
LastWritten < "2021-01-01";' | mysql -s -p bacula | awk '{print
$1}' ) ; do echo purge volume="$v"; done ) | bconsole
That's most likely not the most elegant scripting you've ever seen,
but it works and will let you review the consequences of your actions
before actually committing them.
You can also use the sql statement above to restrict the search to
volumes of a certain name pattern ("AND VolumeName LIKE 'OFFLINE%'" for
instance)
or that are in a certain pool and so on.
All the best,
Uwe
--
Uwe Schürkamp | email:
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deadweight volumes, how to discard?
On 8/5/21 3:36 PM, Robert Earl wrote:
> Hi folks,
> I have a large number of abandoned volumes with no data, no associated
> file, and haven't been written in years. I believe they are the obstacle
> to automatic labeling of new volumes; my Maximum Volumes for this pool
> is 120.
>
> Alarmingly, I just found a "Full" volume whose mtime is April 9, 2021
> but Baculum's "Last Written" record is Feb. 28, 2019.
>
> I also have 10 old volumes that are outdated and I can purge/recycle.
>
> I am looking at batch-processing a large number of volumes, though, so
> this is unwieldy through the GUI, and I'm dense on the niceties of
> bconsole. What would be the procedure for a set of mass operations like
> this from a script/command line?
Robert,
You might try something like the following example. This configuration
sets all purged volumes to be automatically moved into the Scratch pool,
then the Job "Clean Expired Volumes" periodically invokes the
clean_volumes script in order to scan the Scratch pool, delete the
volumes from the catalog, and delete the physical files from disk.
Pool {
Name = Scratch
Pool Type = Backup
}
Pool {
Name = Full-Disk
...
RecyclePool = Scratch
}
Pool {
Name = Diff-Disk
...
RecyclePool = Scratch
}
[and so on]
Job {
Name = "Clean Expired Volumes"
Type = Admin
Enabled = Yes
Pool = Scratch
Storage = asgard-file
FileSet = Dummy
Client = asgard
Level = Full
RunBeforeJob = "/etc/bacula/clean_volumes -v"
Messages = Daemon
Priority = 20
Rerun Failed Levels = yes
Allow Duplicate Jobs = no
Cancel Queued Duplicates = yes
Schedule = "Volume Cleanup"
}
#!/usr/bin/perl
use strict;
use Getopt::Long;
use IPC::Open2;
use IO::Handle;
my $bconsole = '/usr/sbin/bconsole';
my (%opts, @purged, $pid);
GetOptions(\%opts,
'verbose|v',
'test');
my ($IN, $OUT) = (IO::Handle->new(), IO::Handle->new());
$pid = open2($OUT, $IN, $bconsole) || die "Unable to open bconsole";
if (scalar (@purged = check_volumes()))
{
printf("Bacula reports the following purged volumes:\n\t%s\n",
join("\n\t", @purged)) if ($opts{verbose});
my $deleted = delete_volumes(@purged);
print "$deleted volumes deleted.\n" if ($opts{verbose});
}
elsif ($opts{verbose})
{
print "No purged volumes found to delete.\n";
}
print $IN "exit\n";
waitpid($pid, 0);
exit (0);
sub check_volumes
{
my $dividers = 0;
my (@purged, @row);
print $IN "list volumes pool=Scratch\n";
for (;;)
{
my $resp = <$OUT>;
last if ($resp =~ /No results to list./);
$dividers++ if ($resp =~ /^[\+\-]+$/);
last if ($dividers == 3);
@row = split(/\s+/, $resp);
push (@purged, $row[3]) if ($row[5] eq 'Purged');
}
return (@purged);
}
sub delete_volumes
{
my $volume_dir = '/spool/bacula/';
my $count = 0;
foreach my $vol (@_)
{
my $l;
my $file = $volume_dir.$vol;
print "Deleting volume $vol from catalog ... " if ($opts{verbose});
print $IN "delete volume=$vol yes\n";
$l = <$OUT>;
$l = <$OUT>;
print "Done.\nDeleting volume $file from disk ... " if
($opts{verbose});
if (-f $file)
{
$count++;
unlink ($file);
}
print "Done.\n" if ($opts{verbose});
}
return ($count);
}
--
Phil Stracchino
Babylon Communications
ph...@caerllewys.net
p...@co.ordinate.org
Landline: +1.603.293.8485
Mobile: +1.603.998.6958
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Deadweight volumes, how to discard?
Hello Robert
You can create a file with list of volumes that you want to delete and use
some script something like this below:
Example of file /tmp/volumes.txt
volume01
volume02
volume03
#!/bin/bash
bconsole=$(which bconsole)
for volname in $(cat /tmp/volumes.txt); do
echo $volname;
echo "delete volume=${volname} pool=PoolName yes" | ${bconsole} ;
rm -f /path/to/storage/${volname};
done
Best regards
*Wanderlei Hüttel*
Em qui., 5 de ago. de 2021 às 16:38, Robert Earl
escreveu:
> Hi folks,
> I have a large number of abandoned volumes with no data, no associated
> file, and haven't been written in years. I believe they are the obstacle to
> automatic labeling of new volumes; my Maximum Volumes for this pool is 120.
>
> Alarmingly, I just found a "Full" volume whose mtime is April 9, 2021 but
> Baculum's "Last Written" record is Feb. 28, 2019.
>
> I also have 10 old volumes that are outdated and I can purge/recycle.
>
> I am looking at batch-processing a large number of volumes, though, so
> this is unwieldy through the GUI, and I'm dense on the niceties of
> bconsole. What would be the procedure for a set of mass operations like
> this from a script/command line?
>
> Thanks
> Robert
> ___
> Bacula-users mailing list
> Bacula-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[Bacula-users] Deadweight volumes, how to discard?
Hi folks, I have a large number of abandoned volumes with no data, no associated file, and haven't been written in years. I believe they are the obstacle to automatic labeling of new volumes; my Maximum Volumes for this pool is 120. Alarmingly, I just found a "Full" volume whose mtime is April 9, 2021 but Baculum's "Last Written" record is Feb. 28, 2019. I also have 10 old volumes that are outdated and I can purge/recycle. I am looking at batch-processing a large number of volumes, though, so this is unwieldy through the GUI, and I'm dense on the niceties of bconsole. What would be the procedure for a set of mass operations like this from a script/command line? Thanks Robert ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
