[BackupPC-users] Cron routine for checking logs

Here is a routine that I put in my crontab to daily check the logs daily for
all the various types of error messages, many of which are not caught
on the web interface summary.

Basically, the shell script looks through the last N minutes (defaults
to 1500 which is just more than a day) of the following log files:

   1. $TopDir/pc/*/XferLog.* [any pc-tree XferLog's modified (i.e., written)  
within the last N minutes]
   2. $TopDir/pc/*/Log.* [any host-specific log entries added within the last N 
minutes]
   3. $TopDir/log/LOG*  [any BackupPC server log entries added within the last 
N minutes]

The Log file entries are listed by timestamp.

The more verbose XferLog file entries are sorted and tabulated by
error type, giving the count of each error entry (since some errors
may be repeated multiple times)

An optional input parameter allows you to specify how many *minutes*
to look back with the default being 1500

Note that you need to run the routine as a user with privileges to
access above log files as well as to execute the BackupPC_zcat script.

If you find any errors that my script doesn't catch or conversely any
non-errors that erroneously reported, please let me know and I will
adjust the filtering rules.

Enjoy!
Jeff

----------------------------------------------------------------------------------------------------------

#!/bin/bash
#========================================================================
#
# BackupPC_filterErrors.sh
#                       
#
# DESCRIPTION:
#
#   Filter for errors through:
#     1. $TopDir/pc/*/Log\.[0-9][0-9][0-9][0-9][0-9][0-9]*
#          [any XferLog files modified (i.e. written) within the last N minutes]
#     2. $TopDir/pc/*/XferLog\.*
#          [any pc host-specific log entries added within the last N minutes]
#     3. $TopDir/log/LOG*
#          [any BackupPC server log entries added within the last N minutes]
#
#    The pc host and BackupPC server Log file entries are listed by timestamp.
#    The more verbose XferLog file entries are sorted and tabulated by
#    error type, giving the count of each error entry (since some errors
#    may be repeated many times)
#
# USAGE:
#   BackupPC_filterErrors.sh [<minutes>]
#
#       where <minutes> is the optional number of minutes to look back
#       default = 1500 minutes
#
#   Note: need to run as user with privileges to access above log files
#         as well as to execute the BackupPC_zcat script
#
# AUTHOR
#   Jeff Kosowsky
#
# COPYRIGHT
#   Copyright (C) 2022  Jeff Kosowsky
#
#   This program is free software; you can redistribute it and/or modify
#   it under the terms of the GNU General Public License as published by
#   the Free Software Foundation; either version 2 of the License, or
#   (at your option) any later version.
#
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#
#   You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
#
#========================================================================
#
# Version 0.5, released January 2022
#
# CHANGELOG:
#
#========================================================================


MIN=$1
[ -z "$MIN" ] && MIN=1500

TOPDIR=/var/lib/backuppc
BACKUPPC_ZCAT=/usr/share/backuppc/bin/BackupPC_zcat

#Test that you have permissions to run BackupPC_zcat
echo | $BACKUPPC_ZCAT
EXIT=$?
[ $EXIT -ne 0 ] && exit $EXIT

function filterdate () { #Filter last $1 minutes of log entries
    awk -v start=$(($(date +%s) - $1*60)) '{cmd = "date -d "$1"T"$2" +%s"; cmd 
| getline mydt; close(cmd); if(mydt > start){print}}'
    }

function catlog () { #Use cat or BackupPC_zcat as appropriate
    if file $1 | grep -q compressed; then
        $BACKUPPC_ZCAT $1
    else
        cat $1
    fi
}

function printoutput
    if [ -n "$OUTPUT" ]; then
        echo "##### $1 [$(date -d @$(stat -c %Y $1) +'%Y-%m-%d %H:%M:%S')]"
        echo "$OUTPUT" | sed "s/^\s*/\t/"
        echo
    fi


cd $TOPDIR/pc
for file in $( find * -maxdepth 1 -mmin -$MIN \( -name "XferLOG\.*" -o -name 
"LOG\.[0-9][0-9][0-9][0-9][0-9][0-9]*" \) | sort); do
    if [[ $file == *XferLOG* ]]; then
        OUTPUT=$(catlog $file | \
                     grep -i "\(^\| 
\)\(errors\?\|warning\|vanished\|ignored\|failed\|discarded\|denied\|unrecognized\|aborted\|aborting\)\(:\|
 \|$\)" | \
                     grep -v "^\s\+\(same\|create\|pool\)\s\+" | \
                     grep -v " 0 errors" | \
                     sort | uniq -c)
    else
        OUTPUT=$( catlog $file | \
                      filterdate $MIN | \
                      grep -i "\b\(error\|fail\|cancel\|abort\)" | \
                      grep -v "0 errors")
    fi

    printoutput $file
done

cd $TOPDIR/log
for file in $( find * -maxdepth 0 -mmin -$MIN -name "LOG*" | sort -r); do
    OUTPUT=$( catlog $file | \
                  filterdate $MIN | \
                  grep -i "\b\(error\|fail\|cancel\|abort\|can't\)")

    printoutput $file
done


_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/