Dear Jan,
Here are error messages after your patch was used:
bibsched_task_216.log:
2013-11-16 17:00:21 --> Task #216 started.
2013-11-16 17:00:21 --> harvesting from selected repositories
2013-11-16 17:00:21 --> running in postmode p-r-a-t-u
2013-11-16 17:05:21 --> an error occurred while harvesting from source
cdsweb for the dates chosen:
Harvesting interrupted (after 10 attempts) at 2013-11-16 17:05:21 --> :
set=cerncds%3ACONF&verb=ListRecords&from=2013-07-01&metadataPrefix=marcxml&u
ntil=2013-07-05
2013-11-16 17:05:21 --> An error occurred. Task is configured to stop
2013-11-16 17:05:21 --> Task #216 finished. [DONE WITH ERRORS]
Invenio.err:
* 2013-11-16 17:05:26 -> StandardError: BibTask with ERRORS:
#216 oaiharvest -> DONE WITH ERRORS (bibsched.py:1239:check_errors)
** User details
No client information available
** Traceback details
Traceback (most recent call last):
File "/usr/lib64/python2.6/site-packages/invenio/bibsched.py", line 1302,
in watch_loop
calculate_rows()
File "/usr/lib64/python2.6/site-packages/invenio/bibsched.py", line 1246,
in calculate_rows
check_errors()
File "/usr/lib64/python2.6/site-packages/invenio/bibsched.py", line 1239,
in check_errors
raise StandardError(msg)
StandardError: BibTask with ERRORS:
#216 oaiharvest -> DONE WITH ERRORS
** Stack frame details
Frame check_errors in /usr/lib64/python2.6/site-packages/invenio/bibsched.py
at line 1239
----------------------------------------------------------------------------
---
1236 msg = 'BibTask with ERRORS:\n%s' %
"\n".join(msg_errors)
1237 err_types = set(e[2] for e in errors if e[2])
1238 if 'ERROR' in err_types or 'DONE WITH ERRORS' in
err_types:
----> 1239 raise StandardError(msg)
1240 else:
1241 raise RecoverableError(msg)
1242
----------------------------------------------------------------------------
---
err_types = "set(['DONE WITH ERRORS'])"
errors = "((216L, 'oaiharvest', 'DONE WITH ERRORS'),)"
msg_errors = "[' #216 oaiharvest -> DONE WITH ERRORS']"
sql = '"SELECT count(id) FROM schTASK WHERE
status=\'ERROR\' OR status=\'DONE WITH ERRORS\' OR STATUS=\'CERROR\'"'
msg = "'BibTask with ERRORS:\\n #216 oaiharvest
-> DONE WITH ERRORS'"
row = "(216L, 'oaiharvest', 'DONE WITH ERRORS')"
Frame calculate_rows in
/usr/lib64/python2.6/site-packages/invenio/bibsched.py at line 1249
----------------------------------------------------------------------------
---
1246 check_errors()
1247 except RecoverableError, msg:
1248 register_emergency('Light emergency from %s:
BibTask failed: %s' % (CFG_SITE_URL, msg))
----> 1249 run_sql("UPDATE schTASK SET status='ERRORS
REPORTED' WHERE status='CERROR'")
1250
1251 max_bibupload_priority, min_bibupload_priority =
run_sql(
1252 """SELECT MAX(priority), MIN(priority)
----------------------------------------------------------------------------
---
self = '<invenio.bibsched.BibSched object at
0x3706bd0>'
check_errors = '<function check_errors at 0x34772a8>'
Frame watch_loop in /usr/lib64/python2.6/site-packages/invenio/bibsched.py
at line 1322
----------------------------------------------------------------------------
---
1319 else:
1320 time.sleep(CFG_BIBSCHED_REFRESHTIME)
1321 except Exception, err:
----> 1322 register_exception(alert_admin=True)
1323 try:
1324 register_emergency('Emergency from %s: BibSched
halted: %s' % (CFG_SITE_URL, err))
1325 except NotImplementedError:
----------------------------------------------------------------------------
---
check_errors = '<function check_errors at 0x34772a8>'
calculate_rows = '<function calculate_rows at 0x37158c0>'
task = "(216L, 'oaiharvest', datetime.datetime(2013,
11, 16, 17, 0, 4), 'RUNNING', 0, 'lt-tv.jinr.ru', None)"
self = '<invenio.bibsched.BibSched object at
0x3706bd0>'
err = "StandardError('BibTask with ERRORS:\\n
#216 oaiharvest -> DONE WITH ERRORS',)"
From: Jan Åge Lavik [mailto:[email protected]]
Sent: Friday, November 15, 2013 4:38 PM
To: Tibor Simko
Cc: Genis Musulmanbekov; project-invenio-general
Subject: Re: Harvesting error
Dear Genis,
I think you are getting this error due to an exception while processing the
original exception due to a bad variable name.
This patch on my public should fix that, and then the "real" error will
reveal itself - which is notifying you about connection troubles:
http://invenio-software.org/repo/personal/invenio-jlavik/log/?h=oaiharvest-e
xception-handling
Tibor, it is on latest master, so should merge cleanly.
Cheers,
Jan
On Fri, Nov 15, 2013 at 12:49 PM, Tibor Simko <[email protected]> wrote:
On Fri, 15 Nov 2013, [email protected] wrote:
> Recommended patch does not work (see below)
Yes, I was mostly addressing Alessio and SamK to quick-check the error
reporting improvements in the concerned patch (and in related follow-up
patches) in the master branch, to see whether you could get nicer
tracebacks after possible back-porting of some of them to maint-1.1.
The referenced patch was not meant to be applied on your maint-1.1-based
instance...
Best regards
--
Tibor Simko
