Re: [yocto] Missing certificates
On 2015-07-24 12:02, Gary Thomas wrote: I was trying to run a simple fetch from python using url = 'https://raw.github.com/Itseez/opencv/master/samples/c/fruits.jpg' filedata = urllib2.urlopen(url).read() This failed: Traceback (most recent call last): File ./edge.py, line 36, in module filedata = urllib2.urlopen(url).read() File /usr/lib/python2.7/urllib2.py, line 154, in urlopen return opener.open(url, data, timeout) File /usr/lib/python2.7/urllib2.py, line 431, in open response = self._open(req, data) File /usr/lib/python2.7/urllib2.py, line 449, in _open '_open', req) File /usr/lib/python2.7/urllib2.py, line 409, in _call_chain result = func(*args) File /usr/lib/python2.7/urllib2.py, line 1240, in https_open context=self._context) File /usr/lib/python2.7/urllib2.py, line 1197, in do_open raise URLError(err) urllib2.URLError: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) I can see that it was looking for some certificates in /usr/lib/ssl/certs but that directory is missing. Anyone know what I might be missing (or have misconfigured)? Thanks I've found a discussion about this problem on the OpenEmbedded development list: http://lists.openembedded.org/pipermail/openembedded-devel/2015-July/102160.html So the problem that this has uncovered is twofold: 1) Python (and OpenSSL) are not using the certificates that are installed by the ca-certificates package OpenSSL expects the certificates in /usr/lib/ssl/certs and ca-certificates uses /etc/ssl/certs 2) The certificates from ca-certificates are not immediately usable by OpenSSL since they are not hashed. This is done by the 'c_rehash' program but has been explicitly disabled by a patch. Further exploration implies that this was disabled because not all targets will have c_rehash available and since the hashing is expected to be done on the target when the certificates are loaded/updated. Finally, c_rehash, may or may not exist in the OpenSSL packages, depending on whether or not perl is available on the target (it's a perl script) How best to solve this? As is, python htts:// support is broken in OE-core, so I think an off-the-shelf solution is warranted. Perhaps the PACKAGECONFIG for openssl should default to supporting perl on the target, and hence the c_rehash utility would be available? Certainly the choice of where the certificates live, etc, should be standardized. Maybe the c_rehash can be run at package build time for ca-certificates? This would make things work, at least for the real CA certificates. Ideas? -- Gary Thomas | Consulting for the MLB Associates |Embedded world -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Missing certificates
On Fri, Jul 24, 2015 at 12:49 PM, Gary Thomas g...@mlbassoc.com wrote: On 2015-07-24 13:30, Aníbal Limón wrote: Hi Gary, What version of python do you use?. Since 2.7.9 cert checking is enabled by default causing this kind of errors. [1] [1] https://www.python.org/dev/peps/pep-0476/ Kind regards, alimon I'm using the stock python 2.7.9 from Poky/Yocto master:901be2cb69892595443ed41ab4be285932db15eb Is there an answer for this that's a bit less intrusive? Perhaps there could be a DISTRO or even IMAGE feature to enable/disable this checking? The pep you referenced mostly talks about why this was changed and how to disable it - manually within the python code itself. What I don't see is where/how/what to change/import to actually let the full certificate checking happen. I think the better bet is to fix it so it actually finds the certs from ca-certificates rather than bypassing certificate checking, personally, but I can see how that would be a useful workaround. :) -- Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Missing certificates
On 2015-07-24 14:09, Christopher Larson wrote: On Fri, Jul 24, 2015 at 12:49 PM, Gary Thomas g...@mlbassoc.com mailto:g...@mlbassoc.com wrote: On 2015-07-24 13:30, Aníbal Limón wrote: Hi Gary, What version of python do you use?. Since 2.7.9 cert checking is enabled by default causing this kind of errors. [1] [1] https://www.python.org/dev/peps/pep-0476/ Kind regards, alimon I'm using the stock python 2.7.9 from Poky/Yocto master:901be2cb69892595443ed41ab4be285932db15eb Is there an answer for this that's a bit less intrusive? Perhaps there could be a DISTRO or even IMAGE feature to enable/disable this checking? The pep you referenced mostly talks about why this was changed and how to disable it - manually within the python code itself. What I don't see is where/how/what to change/import to actually let the full certificate checking happen. I think the better bet is to fix it so it actually finds the certs from ca-certificates rather than bypassing certificate checking, personally, but I can see how that would be a useful workaround. :) I tried this same code on my Ubuntu 15.04 desktop and it looks like they've disabled it in the main python http[s] code - there were no certificates examined during the transaction as far as I could tell (strace is my friend) I'll see if I can figure out how to stitch this together with our [Poky/Yocto/OE-core] setup for OpenSSL and ca-certificates. -- Gary Thomas | Consulting for the MLB Associates |Embedded world -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Missing certificates
On 24/07/15 14:49, Gary Thomas wrote: On 2015-07-24 13:30, Aníbal Limón wrote: Hi Gary, What version of python do you use?. Since 2.7.9 cert checking is enabled by default causing this kind of errors. [1] [1] https://www.python.org/dev/peps/pep-0476/ Kind regards, alimon I'm using the stock python 2.7.9 from Poky/Yocto master:901be2cb69892595443ed41ab4be285932db15eb Is there an answer for this that's a bit less intrusive? Perhaps there could be a DISTRO or even IMAGE feature to enable/disable this checking? I don't think that Python guys include a configuration flags to disable this behavior because it's the default now due to security issues. The pep you referenced mostly talks about why this was changed and how to disable it - manually within the python code itself. What I don't see is where/how/what to change/import to actually let the full certificate checking happen. You can use this code for disable per urlopen call or globally [1]. Regards, alimon [1] https://www.python.org/dev/peps/pep-0476/#opting-out On 24/07/15 13:02, Gary Thomas wrote: I was trying to run a simple fetch from python using url = 'https://raw.github.com/Itseez/opencv/master/samples/c/fruits.jpg' filedata = urllib2.urlopen(url).read() This failed: Traceback (most recent call last): File ./edge.py, line 36, in module filedata = urllib2.urlopen(url).read() File /usr/lib/python2.7/urllib2.py, line 154, in urlopen return opener.open(url, data, timeout) File /usr/lib/python2.7/urllib2.py, line 431, in open response = self._open(req, data) File /usr/lib/python2.7/urllib2.py, line 449, in _open '_open', req) File /usr/lib/python2.7/urllib2.py, line 409, in _call_chain result = func(*args) File /usr/lib/python2.7/urllib2.py, line 1240, in https_open context=self._context) File /usr/lib/python2.7/urllib2.py, line 1197, in do_open raise URLError(err) urllib2.URLError: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) I can see that it was looking for some certificates in /usr/lib/ssl/certs but that directory is missing. Anyone know what I might be missing (or have misconfigured)? Thanks -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Missing certificates
Hi Gary, What version of python do you use?. Since 2.7.9 cert checking is enabled by default causing this kind of errors. [1] [1] https://www.python.org/dev/peps/pep-0476/ Kind regards, alimon On 24/07/15 13:02, Gary Thomas wrote: I was trying to run a simple fetch from python using url = 'https://raw.github.com/Itseez/opencv/master/samples/c/fruits.jpg' filedata = urllib2.urlopen(url).read() This failed: Traceback (most recent call last): File ./edge.py, line 36, in module filedata = urllib2.urlopen(url).read() File /usr/lib/python2.7/urllib2.py, line 154, in urlopen return opener.open(url, data, timeout) File /usr/lib/python2.7/urllib2.py, line 431, in open response = self._open(req, data) File /usr/lib/python2.7/urllib2.py, line 449, in _open '_open', req) File /usr/lib/python2.7/urllib2.py, line 409, in _call_chain result = func(*args) File /usr/lib/python2.7/urllib2.py, line 1240, in https_open context=self._context) File /usr/lib/python2.7/urllib2.py, line 1197, in do_open raise URLError(err) urllib2.URLError: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) I can see that it was looking for some certificates in /usr/lib/ssl/certs but that directory is missing. Anyone know what I might be missing (or have misconfigured)? Thanks -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Missing certificates
On Fri, Jul 24, 2015 at 11:02 AM, Gary Thomas g...@mlbassoc.com wrote: I was trying to run a simple fetch from python using url = ' https://raw.github.com/Itseez/opencv/master/samples/c/fruits.jpg' filedata = urllib2.urlopen(url).read() This failed: Traceback (most recent call last): File ./edge.py, line 36, in module filedata = urllib2.urlopen(url).read() File /usr/lib/python2.7/urllib2.py, line 154, in urlopen return opener.open(url, data, timeout) File /usr/lib/python2.7/urllib2.py, line 431, in open response = self._open(req, data) File /usr/lib/python2.7/urllib2.py, line 449, in _open '_open', req) File /usr/lib/python2.7/urllib2.py, line 409, in _call_chain result = func(*args) File /usr/lib/python2.7/urllib2.py, line 1240, in https_open context=self._context) File /usr/lib/python2.7/urllib2.py, line 1197, in do_open raise URLError(err) urllib2.URLError: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) I can see that it was looking for some certificates in /usr/lib/ssl/certs but that directory is missing. Anyone know what I might be missing (or have misconfigured)? At least in the past, we’ve been highly inconsistent in certificate store configuration in various recipes, some pointing at a dir, some a file, and the paths vary. I don’t know if anyone ever fixed that, however — this is from memory. -- Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Missing certificates
On 2015-07-24 13:30, Aníbal Limón wrote: Hi Gary, What version of python do you use?. Since 2.7.9 cert checking is enabled by default causing this kind of errors. [1] [1] https://www.python.org/dev/peps/pep-0476/ Kind regards, alimon I'm using the stock python 2.7.9 from Poky/Yocto master:901be2cb69892595443ed41ab4be285932db15eb Is there an answer for this that's a bit less intrusive? Perhaps there could be a DISTRO or even IMAGE feature to enable/disable this checking? The pep you referenced mostly talks about why this was changed and how to disable it - manually within the python code itself. What I don't see is where/how/what to change/import to actually let the full certificate checking happen. On 24/07/15 13:02, Gary Thomas wrote: I was trying to run a simple fetch from python using url = 'https://raw.github.com/Itseez/opencv/master/samples/c/fruits.jpg' filedata = urllib2.urlopen(url).read() This failed: Traceback (most recent call last): File ./edge.py, line 36, in module filedata = urllib2.urlopen(url).read() File /usr/lib/python2.7/urllib2.py, line 154, in urlopen return opener.open(url, data, timeout) File /usr/lib/python2.7/urllib2.py, line 431, in open response = self._open(req, data) File /usr/lib/python2.7/urllib2.py, line 449, in _open '_open', req) File /usr/lib/python2.7/urllib2.py, line 409, in _call_chain result = func(*args) File /usr/lib/python2.7/urllib2.py, line 1240, in https_open context=self._context) File /usr/lib/python2.7/urllib2.py, line 1197, in do_open raise URLError(err) urllib2.URLError: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581) I can see that it was looking for some certificates in /usr/lib/ssl/certs but that directory is missing. Anyone know what I might be missing (or have misconfigured)? Thanks -- Gary Thomas | Consulting for the MLB Associates |Embedded world -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
