[htdig-dev] silly new rundig.sh

Ted Stresen-Reuter Mon, 05 Jul 2004 01:05:49 -0700

Hi,

I was in need of a rundig.sh script that was a little more flexible and easy to use than what's in the contrib directory. I wanted one that would allow me to override default values with command line parameters and that would fail gracefully when there was a problem. I spent about two or three days working on a shell script that would meet these needs and produced the attached.

Toward the end of the exercise, while consulting rundig in the /installdir directory for help on how to do things like iterate through the parameters, I realized that rundig more or less does what I need already, and with more elegance (using 'shift' when -c is encountered rather than trying to use sed, like I did...) so I felt rather humbled, but decided to send along my script should someone find it useful.

Pointers are always appreciated, but keep in mind that I'm still marveling over installdir/rundig...

Ted Stresen-Reuter

======================
#! /bin/sh

# This is the directory where all the htdig stuff lives
BASEDIR="/Library/htdig"

# Report destination - who to send the report to
REPORT_DEST="[EMAIL PROTECTED]"
export REPORT_DEST

# This is the name (with complete path) of the conf file to use
CONF="$BASEDIR/conf/htdig.conf"

###### You shouldn't need to modify anything below this line
debug=0

stats= opts= alt= main=
for arg
do
    case "$arg" in
    -a)  alt="$arg" ;;
    -s)  stats="$arg" ;;
    -i)  main="$arg" ;;
    *)   opts="$opts $arg" ;; # e.g. -v or -c config
    esac
done

# if a config file exists, use it, otherwise, use the default
configfile=`echo $opts | sed 's/.*\-c *\([^ ][^ ]*\).*/\1/g'`

if test -z "$configfile"; then
    CONF=$CONF
else
    CONF=$configfile
fi

if test -z "$REPORT_DEST"; then

echo "No destination has been set for the report. Please set DESTINATION to a valid email address." exit 1

else

    if test -d $BASEDIR; then

        if test -r $CONF; then

# Get the db dir new_db_dir=`awk '/^[^#a-zA-Z]*database_dir/ { print $NF }' < $CONF`

            if [ "$new_db_dir" != "" ]; then

                DBDIR=$new_db_dir

                if test -d $DBDIR && test $debug -lt 1; then

                    # This is the name of a temporary report file
                    REPORT=$DBDIR/htdig.report

                    # This is the subject line of the report
                    SUBJECT="htdig report"

# This is a little intro to the report echo "This report produced by the script $0 running on `hostname` with the following parameters: $main $alt $stats $opts" > $REPORT

                    ##### Dig phase
                    STARTTIME=`date`
                    echo "Start time: $STARTTIME"
                    echo "rundig: Start time: $STARTTIME" >> $REPORT

                    # by default we use -a?
                    $BASEDIR/bin/htdig $main $alt $stats $opts

                    TIME=`date`
                    echo "Done Digging: $TIME"
                    echo "htdig: Done Digging: $TIME" >> $REPORT

                    ##### Purge Phase
                    # (clean out broken links, etc.)
                    $BASEDIR/bin/htpurge $alt $opts >> $REPORT

                    TIME=`date`
                    echo "Done Purging: $TIME"
                    echo "htpurge: Done Purging: $TIME" >> $REPORT

# Move 'em into place. cp $DBDIR/db.docs.index.work $DBDIR/db.docs.index cp $DBDIR/db.docdb.work $DBDIR/db.docdb cp $DBDIR/db.excerpts.work $DBDIR/db.excerpts cp $DBDIR/db.words.db.work $DBDIR/db.words.db test -f $DBDIR/db.words.db.work_weakcmpr && cp $DBDIR/db.words.db.work_weakcmpr $DBDIR/db.words.db_weakcmpr

                    ##### Fuzzy Phase
                    $BASEDIR/bin/htfuzzy $opts endings >> $REPORT
                    $BASEDIR/bin/htfuzzy $opts synonyms >> $REPORT

                    TIME=`date`
                    echo "Done Fuzzying: $TIME"
                    echo "htfuzzy: Done Fuzzying: $TIME" >> $REPORT

##### Cleanup Phase # To get additional statistics, uncomment the following line $BASEDIR/bin/htstat $opts >>$REPORT

                    END=`date`
                    echo "End time: $END"
                    echo "rundig: End time: $END" >> $REPORT
                    echo

                    # Grab the important statistics from the report file
                    # All lines begin with htdig: or htmerge:
                    fgrep "htdig:" $REPORT
                    echo
                    fgrep "htpurge:" $REPORT
                    echo
                    fgrep "htfuzzy:" $REPORT
                    echo
                    fgrep "rundig:" $REPORT
                    echo

                    echo "Total lines in $REPORT: `wc -l $REPORT`"

# Send out the report ... mail -s "$SUBJECT - $STARTTIME" $REPORT_DEST < $REPORT

                    # ... and clean up
                    rm $REPORT
                    exit 0

                else

                    if test $debug -eq 1 -o $debug -gt 1; then

                        echo $CONF
                        exit 0

                    else

echo "No database_dir has been specified in $CONF" exit 1

fi

                fi
            else

                echo "$DBDIR is NOT a directory"
                exit 1

fi

        else

if test -z "$configfile"; then echo "$CONF is NOT readable." else echo "$CONF is NOT readable. You need to specify the complete path to the config file when passing it as a parameter to this script." fi exit 1

fi

    else

        echo "$BASEDIR is NOT a directory"
        exit 1

fi

fi

------------------------------------------------------- This SF.Net email sponsored by Black Hat Briefings & Training. Attend Black Hat Briefings & Training, Las Vegas July 24-29 - digital self defense, top technical experts, no vendor pitches, unmatched networking opportunities. Visit www.blackhat.com _______________________________________________ ht://Dig Developer mailing list: [EMAIL PROTECTED] List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-dev

[htdig-dev] silly new rundig.sh

Reply via email to