Hello Thorsten and Tibor, > > On Fri, 02 Mar 2012, Thorsten S wrote: >> however, why are individual bad input records not simply skipped *and >> logged as such for later inspection* and the queue continues otherwise >> unaffected? a single bad edit shouldn't stall everything afterwards, >> unless there is a convincing reason that I am not aware of > > Briefly put, upload jobs are usually run sequentially, since in general > a future upload job may depend on the result of a previous upload job. > So, if an upload job ends in an error, manual inspection and resolution > may be required. Historically, in this case, the upload queue was > stopped and an SMS alert was sent to the admin support person, who then > logged in and inspected the problem and unblocked the queue. This `easy > technical solution' works, but requires human intervention, which is > definitely sub-optimal.
After several years of checking that, in my installations, I never found an error that required, so to say, a general stop-panic-and-see situation, I wrote a crude yes-I-know script that I run every 20 minutes and leaves a couple of logs for later inspection. I can leave the office for whatever reason with confidence that the systems will keep running. You may have to adjust some Invenio paths, but it may help you to decide to roll your own. Hope it helps, Ferran
# -*- coding: utf-8 -*- # Time-stamp: <2010.07.19 08:59:38 error2ack.sh [email protected]> # This scripts acknowledges bibtasks errors and resumes the scheduler what=error2ack cd ~/tmp filerotate() { if [ -s $filename ]; then test -s $filename.8 && mv $filename.8 $filename.9 test -s $filename.7 && mv $filename.7 $filename.8 test -s $filename.6 && mv $filename.6 $filename.7 test -s $filename.5 && mv $filename.5 $filename.6 test -s $filename.4 && mv $filename.4 $filename.5 test -s $filename.3 && mv $filename.3 $filename.4 test -s $filename.2 && mv $filename.2 $filename.3 test -s $filename.1 && mv $filename.1 $filename.2 test -s $filename.0 && mv $filename.0 $filename.1 mv $filename $filename.0 fi } for filename in $what.log $what.sql; do filerotate done echo "select id, status, runtime, proc, arguments from schTASK where status in ('ERROR', 'DONE WITH ERRORS');" | \ ~/invenio/bin/dbexec >$what.log if [ -s $what.log ]; then awk 'NR > 1 { q = sprintf("%c",39); \ printf("update schTASK set status=%s%s%s where id=%s;\n", \ q, "ACK ERROR", q, $1) }' $what.log >$what.sql cat $what.sql | ~/invenio/bin/dbexec ~/invenio/bin/bibsched restart fi
