Public bug reported: Simplestreams do not catch errors created for being behind a proxy.
In MAAS, we were expecting / validating for json to be returned from simplestreams, but we kept seeing this failure: [...] line 325, in load_content return json.loads(content) File "/usr/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) To debug, I had to do this: --- simplestreams/util.py 2016-11-28 14:54:35 +0000 +++ simplestreams/util.py 2018-08-31 14:32:54 +0000 @@ -320,6 +320,7 @@ def load_content(content): + print(content) if isinstance(content, bytes): content = content.decode('utf-8') return json.loads(content) And this resulted in: 2018-08-31 14:13:36 stdout: [info] body 2018-08-31 14:13:36 stdout: [info] :lang(fa) { direction: rtl; font-size: 100%; font-family: Tahoma, Roya, sans-serif; float: right; } 2018-08-31 14:13:36 stdout: [info] :lang(he) { direction: rtl; } 2018-08-31 14:13:36 stdout: [info] --></style> 2018-08-31 14:13:36 stdout: [info] </head><body id=ERR_CONNECT_FAIL> 2018-08-31 14:13:36 stdout: [info] <div id="titles"> 2018-08-31 14:13:36 stdout: [info] <h1>ERROR</h1> 2018-08-31 14:13:36 stdout: [info] <h2>The requested URL could not be retrieved</h2> 2018-08-31 14:13:36 stdout: [info] </div> 2018-08-31 14:13:36 stdout: [info] <hr> 2018-08-31 14:13:36 stdout: [info] 2018-08-31 14:13:36 stdout: [info] <div id="content"> 2018-08-31 14:13:36 stdout: [info] <p>The following error was encountered while trying to retrieve the URL: <a href="http://[2001:67c:1562:800a::6]:5240/MAAS/images-stream/streams/v1/index.json">http://[2001:67c:1562:800a::6]:5240/MAAS/images-stream/streams /v1/index.json</a></p> 2018-08-31 14:13:36 stdout: [info] 2018-08-31 14:13:36 stdout: [info] <blockquote id="error"> 2018-08-31 14:13:36 stdout: [info] <p><b>Connection to 2001:67c:1562:800a::6 failed.</b></p> 2018-08-31 14:13:36 stdout: [info] </blockquote> 2018-08-31 14:13:36 stdout: [info] 2018-08-31 14:13:36 stdout: [info] <p id="sysmsg">The system returned: <i>(110) Connection timed out</i></p> 2018-08-31 14:13:36 stdout: [info] 2018-08-31 14:13:36 stdout: [info] <p>The remote host or network may be down. Please try the request again.</p> 2018-08-31 14:13:36 stdout: [info] 2018-08-31 14:13:36 stdout: [info] <p>Your cache administrator is <a href="mailto:webmaster?subject=CacheErrorInfo%20-%20ERR_CONNECT_FAIL&body=CacheHost%3A%20localhost%0D%0AErrPage%3A%20ERR_CONNECT_FAIL%0D%0AErr%3A%20(110)%20Connection%20timed%20out%0D% 0ATimeStamp%3A%20Fri,%2031%20Aug%202018%2014%3A13%3A36%20GMT%0D%0A%0D%0AClientIP%3A%2010.246.88.6%0D%0AServerIP%3A%202001%3A67c%3A1562%3A800a%3A%3A6%0D%0A%0D%0AHTTP%20Request%3A%0D%0AGET%20%2FMAAS%2Fimages-stream%2Fstreams%2Fv1%2Findex.json%20HTTP%2F1.1%0AU ser-Agent%3A%20python-simplestreams%2F0.1%0D%0AAccept-Encoding%3A%20gzip,%20deflate%0D%0AAccept%3A%20*%2F*%0D%0AConnection%3A%20keep-alive%0D%0AHost%3A%20%5B2001%3A67c%3A1562%3A800a%3A%3A6%5D%3A5240%0D%0A%0D%0A%0D%0A">webmaster</a>.</p> 2018-08-31 14:13:36 stdout: [info] 2018-08-31 14:13:36 stdout: [info] <br> 2018-08-31 14:13:36 stdout: [info] </div> 2018-08-31 14:13:36 stdout: [info] 2018-08-31 14:13:36 stdout: [info] <hr> 2018-08-31 14:13:36 stdout: [info] <div id="footer"> 2018-08-31 14:13:36 stdout: [info] <p>Generated Fri, 31 Aug 2018 14:13:36 GMT by localhost (squid/3.1.19)</p> 2018-08-31 14:13:36 stdout: [info] <!-- ERR_CONNECT_FAIL --> 2018-08-31 14:13:36 stdout: [info] </div> 2018-08-31 14:13:36 stdout: [info] </body></html> 2018-08-31 14:13:36 stdout: [info] Which shows that simplestreams was failing to access the mirror because of a proxy issue, but it is not catching such issues. ** Affects: simplestreams Importance: Undecided Status: New ** Affects: simplestreams (Ubuntu) Importance: Undecided Status: New ** Also affects: simplestreams Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1790161 Title: simplestreams do not raise errors for failures behind a proxy To manage notifications about this bug go to: https://bugs.launchpad.net/simplestreams/+bug/1790161/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs