> 3) the failure callback that yum gives to urlgrabber /always/
> re-raises the exception, preventing any possible retries that that
> level
>
> I found #3 the most surprising since yum goes to the trouble of
> passing the retry option to urlgrabber
Yes, was surprised too when I had discovered this, but the idea
is that it's often better to try next mirror instead of retrying
(think of timing out on a dead mirror and retrying that 20 times).
> 503 errors are usually meant to be retried. Possibly there are other
> http statuses that should be retried as well.
I agree.
> I've attached a overly simple patch to urlgrabber that does solve my
> immediate issue, but I'm not sure if it's the right fix overall. I'd
> be happy to work on a better fix, but I thought I would solicit
> feedback/advice first.
If you return from the Yum failure callback instead, the retry
logic will work. Could you try this instead?
diff --git a/output.py b/output.py
index b3f2c75..4a55357 100755
--- a/output.py
+++ b/output.py
@@ -465,6 +465,9 @@ class YumOutput:
:raises: *errobj*.exception
"""
self.logger.error('%s: %s', errobj.url, errobj.exception)
+ if errorobj.exception.code == 503:
+ errorobj.exception.errno = -1
+ return
self.logger.error(_('Trying other mirror.'))
raise errobj.exception
Unfortunately, urlgrabber caches errno, we can't change it
in callback.. Have to patch it too:
diff --git a/urlgrabber/grabber.py b/urlgrabber/grabber.py
index 3127b48..c46f96f 100644
--- a/urlgrabber/grabber.py
+++ b/urlgrabber/grabber.py
@@ -1011,7 +1011,6 @@ class URLGrabber(object):
# beware of infinite loops :)
tries = tries + 1
exception = None
- retrycode = None
callback = None
if DEBUG: DEBUG.info('attempt %i/%s: %s',
tries, opts.retry, args[0])
@@ -1022,7 +1021,6 @@ class URLGrabber(object):
except URLGrabError, e:
exception = e
callback = opts.failure_callback
- retrycode = e.errno
except KeyboardInterrupt, e:
exception = e
callback = opts.interrupt_callback
@@ -1040,6 +1038,7 @@ class URLGrabber(object):
if DEBUG: DEBUG.info('retries exceeded, re-raising')
raise
+ retrycode = getattr(exception, 'errno', None)
if (retrycode is not None) and (retrycode not in opts.retrycodes):
if DEBUG: DEBUG.info('retrycode (%i) not in list %s,
re-raising',
retrycode, opts.retrycodes)
@@ -2176,7 +2175,7 @@ def parallel_wait(meter = 'text'):
try: _run_callback(opts.failure_callback, opts)
except URLGrabError, ug_err:
retry = 0 # no retries
- if opts.tries < retry and ug_err.args[0] in opts.retrycodes:
+ if opts.tries < retry and ug_err.errno in opts.retrycodes:
start(opts, opts.tries + 1) # simple retry
continue
_______________________________________________
Yum-devel mailing list
Yum-devel@lists.baseurl.org
http://lists.baseurl.org/mailman/listinfo/yum-devel
