Hey Gerald.
I noticed your continual effort to change http:// links to https://.
I've written a simple script that can do that in automatic way.
What do you think about using it?
Martin
#!/usr/bin/env python3
import os
import re
import requests
urls = set()
def check_url(url):
try:
r = requests.get(url, timeout = 5)
print(r)
return True
except requests.exceptions.ConnectionError as e:
print(e)
return False
for root, dirs, files in os.walk('.'):
for f in files:
if f.endswith('.html'):
full = os.path.join(root, f)
for l in open(full).readlines():
m = re.search(r'"(http:\/\/[^"]*)"', l)
if m:
urls.add(m.group(1))
urls = list(sorted(list(urls)))
to_replace = []
for i, url in enumerate(urls):
https = url.replace('http://', 'https://')
print('%d/%d: %s' % (i, len(urls), https))
if (check_url(https)):
to_replace.append(url)
print()
print('Can use HTTPS for %d of links:' % (len(to_replace)))
for url in to_replace:
print(url)
0/329: https://CobolForGCC.sourceforge.net/
HTTPSConnectionPool(host='cobolforgcc.sourceforge.net', port=443): Max retries
exceeded with url: / (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection
refused'))
1/329: https://annwm.lbl.gov/bench/
HTTPSConnectionPool(host='annwm.lbl.gov', port=443): Max retries exceeded with
url: /bench/ (Caused by ConnectTimeoutError(, 'Connection to annwm.lbl.gov timed out. (connect
timeout=5)'))
2/329: https://archive.adaic.com/standards/83lrm/html/ada_lrm.html
HTTPSConnectionPool(host='archive.adaic.com', port=443): Max retries exceeded
with url: /standards/83lrm/html/ada_lrm.html (Caused by
ConnectTimeoutError(, 'Connection to archive.adaic.com timed out. (connect
timeout=5)'))
3/329: https://archive.adaic.com/standards/83rat/html/Welcome.html
HTTPSConnectionPool(host='archive.adaic.com', port=443): Max retries exceeded
with url: /standards/83rat/html/Welcome.html (Caused by
ConnectTimeoutError(, 'Connection to archive.adaic.com timed out. (connect
timeout=5)'))
4/329: https://awards.acm.org/about/2015-technical-awards
5/329: https://blackfin.uclinux.org/gf/
HTTPSConnectionPool(host='blackfin.uclinux.org', port=443): Max retries
exceeded with url: /gf/ (Caused by SSLError(SSLCertVerificationError("hostname
'blackfin.uclinux.org' doesn't match either of '*.azurewebsites.net',
'*.scm.azurewebsites.net', '*.azure-mobile.net', '*.scm.azure-mobile.net',
'*.sso.azurewebsites.net'")))
6/329: https://buildbot.net/
7/329: https://c-faq.com/
HTTPSConnectionPool(host='c-faq.com', port=443): Max retries exceeded with url:
/ (Caused by SSLError(SSLCertVerificationError("hostname 'c-faq.com' doesn't
match either of '*.eskimo.com', 'eskimo.com'")))
8/329: https://chasm-interop.sourceforge.net/
HTTPSConnectionPool(host='chasm-interop.sourceforge.net', port=443): Max
retries exceeded with url: / (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection
refused'))
9/329: https://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.39.1922
10/329: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.37.7180
11/329: https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.2235
12/329: https://cobolforgcc.sourceforge.net/cobol_toc.html
HTTPSConnectionPool(host='cobolforgcc.sourceforge.net', port=443): Max retries
exceeded with url: /cobol_toc.html (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection
refused'))
13/329: https://compilerconnection.com
14/329: https://deuce.doc.wustl.edu/Download.html
15/329: https://developer.axis.com/
HTTPSConnectionPool(host='developer.axis.com', port=443): Max retries exceeded
with url: / (Caused by SSLError(SSLCertVerificationError("hostname
'developer.axis.com' doesn't match either of 'www.axis.com', 'axis.com',
'beta.www.axis.com'")))
16/329: https://developer.classpath.org/doc/
HTTPSConnectionPool(host='developer.classpath.org', port=443): Max retries
exceeded with url: /doc/ (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection
refused'))
17/329: https://developer.classpath.org/mediation/ClasspathGraphicsImagesText
HTTPSConnectionPool(host='developer.classpath.org', port=443): Max retries
exceeded with url: /mediation/ClasspathGraphicsImagesText (Caused by
NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection
refused'))
18/329: https://dwarfstd.org/
HTTPSConnectionPool(host='dwarfstd.org', port=443): Max retries exceeded with
url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate
(_ssl.c:1123)')))
19/329: https://exactcode.com/opensource/openbench/
20/329: https://fastjar.sourceforge.net/
HTTPSConnectionPool(host='fastjar.sourceforge.net', port=443): Max retries
