jenkins-bot has submitted this change and it was merged.

Change subject: repository: make http verification more robust
......................................................................


repository: make http verification more robust

And add comments in the code to clarify why each improvement is actually
needed.

Change-Id: I34410d306c4caff5b7c05c30fd70d5c8db92be81
Signed-off-by: Loic Dachary <l...@dachary.org>
---
M FLOSSbot/repository.py
1 file changed, 17 insertions(+), 2 deletions(-)

Approvals:
  Dachary: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/FLOSSbot/repository.py b/FLOSSbot/repository.py
index 3a3948e..5d4d9d0 100644
--- a/FLOSSbot/repository.py
+++ b/FLOSSbot/repository.py
@@ -397,9 +397,24 @@
 
     def verify_http(self, url):
         try:
-            r = requests.head(url, allow_redirects=True)
+            #
+            # although head() would be more light weight, some
+            # servers do not respond to it. For instance
+            # https://src.openvz.org/projects/OVZ/ returned 405
+            #
+            # The user agent is required for some servers. For
+            # instance http://marabunta.laotracara.com/descargas/
+            # returns 406 if no User-Agent header is set.
+            #
+            r = requests.get(url,
+                             headers={'User-Agent': 'FLOSSbot'},
+                             verify=False)
+            log.debug("GET " + url + " status " + str(r.status_code))
+            if r.status_code != requests.codes.ok:
+                log.debug("GET " + url + " " + r.text)
             return r.status_code == requests.codes.ok
-        except:
+        except Exception as e:
+            log.debug("GET failed with " + str(e))
             return False
 
     def verify_protocol(self, url, protocol, credentials):

-- 
To view, visit https://gerrit.wikimedia.org/r/311843
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I34410d306c4caff5b7c05c30fd70d5c8db92be81
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/bots/FLOSSbot
Gerrit-Branch: master
Gerrit-Owner: Dachary <l...@dachary.org>
Gerrit-Reviewer: Dachary <l...@dachary.org>
Gerrit-Reviewer: jenkins-bot <>

_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

Reply via email to