Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-22 Thread Thomas Goirand
On 09/21/2014 11:30 PM, Ian Cordasco wrote: Hi Thomas, Several people, many of whom are core contributors to other projects, have asked that this discussion not be continued in this venue. Discussion of the decisions of the core-developers of requests are not appropriate for this list. All

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-21 Thread Thomas Goirand
Hi Ian and Donald, I've read the full thread, and couldn't help to reply to it still, even though I previously thought I shouldn't, as what I care is OpenStack, not really requests, and more largely, the topic of the wrong reasons why upstream are embedding foreign library code copies. I

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-21 Thread Thomas Goirand
On 09/18/2014 05:22 AM, Dean Troyer wrote: On Wed, Sep 17, 2014 at 3:53 PM, Robert Collins robe...@robertcollins.net mailto:robe...@robertcollins.net wrote: On 18 September 2014 08:01, Dean Troyer dtro...@gmail.com mailto:dtro...@gmail.com wrote: Interestingly enough, the

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-21 Thread Ian Cordasco
Hi Thomas, Several people, many of whom are core contributors to other projects, have asked that this discussion not be continued in this venue. Discussion of the decisions of the core-developers of requests are not appropriate for this list. All three of us have email addresses that you can

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-21 Thread Jay Pipes
On 09/21/2014 11:30 AM, Ian Cordasco wrote: Hi Thomas, Several people, many of whom are core contributors to other projects, have asked that this discussion not be continued in this venue. Discussion of the decisions of the core-developers of requests are not appropriate for this list. All

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Jeremy Stanley
On 2014-09-18 14:35:10 + (+), Ian Cordasco wrote: Except that even OpenStack doesn’t pin requests because of how extraordinarily stable our API is. [...] FWIW, I nearly capped it a few weeks ago with https://review.openstack.org/117848 but since the affected projects were able to rush

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Ian Cordasco
On 9/19/14, 9:01 AM, Jeremy Stanley fu...@yuggoth.org wrote: On 2014-09-18 14:35:10 + (+), Ian Cordasco wrote: Except that even OpenStack doesn’t pin requests because of how extraordinarily stable our API is. [...] FWIW, I nearly capped it a few weeks ago with

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Brant Knudson
I don't think anyone would be complaining if glanceclient didn't have the need to reach into and monkeypatch requests's connection pool manager[1]. Is there a way to tell requests to build the https connections differently without monkeypatching urllib3.poolmanager? glanceclient's monkeypatching

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Donald Stufft
On Sep 19, 2014, at 11:54 AM, Brant Knudson b...@acm.org wrote: I don't think anyone would be complaining if glanceclient didn't have the need to reach into and monkeypatch requests's connection pool manager[1]. Is there a way to tell requests to build the https connections differently

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Mark Washenberger
On Fri, Sep 19, 2014 at 8:59 AM, Donald Stufft don...@stufft.io wrote: On Sep 19, 2014, at 11:54 AM, Brant Knudson b...@acm.org wrote: I don't think anyone would be complaining if glanceclient didn't have the need to reach into and monkeypatch requests's connection pool manager[1]. Is

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Donald Stufft
On Sep 19, 2014, at 12:42 PM, Mark Washenberger mark.washenber...@markwash.net wrote: On Fri, Sep 19, 2014 at 8:59 AM, Donald Stufft don...@stufft.io mailto:don...@stufft.io wrote: On Sep 19, 2014, at 11:54 AM, Brant Knudson b...@acm.org mailto:b...@acm.org wrote: I don't

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Chmouel Boudjnah
On Fri, Sep 19, 2014 at 6:58 PM, Donald Stufft don...@stufft.io wrote: So you can remove all that code and just let requests/urllib3 handle it on 3.2+, 2.7.9+ and for anything less than that either use conditional dependencies to have glance client depend on pyOpenSSL, ndg-httpsclient, and

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Donald Stufft
On Sep 19, 2014, at 2:26 PM, Chmouel Boudjnah chmo...@enovance.com wrote: On Fri, Sep 19, 2014 at 6:58 PM, Donald Stufft don...@stufft.io mailto:don...@stufft.io wrote: So you can remove all that code and just let requests/urllib3 handle it on 3.2+, 2.7.9+ and for anything less than

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Clint Byrum
Excerpts from Ian Cordasco's message of 2014-09-18 10:33:04 -0700: On 9/18/14, 11:29 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 07:30:27 -0700: On Sep 18, 2014, at 10:18 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-19 Thread Mark Washenberger
On Fri, Sep 19, 2014 at 11:26 AM, Chmouel Boudjnah chmo...@enovance.com wrote: On Fri, Sep 19, 2014 at 6:58 PM, Donald Stufft don...@stufft.io wrote: So you can remove all that code and just let requests/urllib3 handle it on 3.2+, 2.7.9+ and for anything less than that either use conditional

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Chmouel Boudjnah
Ian Cordasco ian.corda...@rackspace.com writes: urllib3 do that automatically. I haven’t started to investigate exactly why they do this. Likewise, glance client has custom certificate verification in glanceclient.common.https. Why? I’m not exactly certain this probably come from pre-requests

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Flavio Percoco
On 09/18/2014 04:43 AM, Donald Stufft wrote: On Sep 17, 2014, at 10:24 PM, Thomas Goirand z...@debian.org mailto:z...@debian.org wrote: On 09/18/2014 08:22 AM, Morgan Fainberg wrote: I think that all of the conversation to this point has been valuable, the general consensus is vendoring a

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Alan Pevec
2014-09-17 17:15 GMT+02:00 Thomas Goirand z...@debian.org: File bla/tests/test_ssl.py, line 19, in module from requests.packages.urllib3 import poolmanager ImportError: No module named packages.urllib3 This is in tests only, in runtime code there is conditional import of vendorized

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Thomas Goirand
On 09/18/2014 04:01 PM, Flavio Percoco wrote: After having gone through the whole thread and read all the concerns, problems and reasonings, I think we should stick to requests as-is for now and deal with this particular issue. Regardless of the vendorized urllib3 package, I believe requests

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Donald Stufft
On Sep 18, 2014, at 7:43 AM, Thomas Goirand z...@debian.org wrote: On 09/18/2014 04:01 PM, Flavio Percoco wrote: After having gone through the whole thread and read all the concerns, problems and reasonings, I think we should stick to requests as-is for now and deal with this particular

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Thomas Goirand
On 09/18/2014 10:43 AM, Donald Stufft wrote: Obviously we can work with the requests team to figure out the best approach. There's only a single approach that works: have the requests upstream authors to stop embedding foreign code, and use the dependency instead. There are legitimate

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Donald Stufft
On Sep 18, 2014, at 7:54 AM, Thomas Goirand z...@debian.org wrote: Linux distributions are not the end be all of distribution models and they don’t get to dictate to upstream. Well, distributions is where the final user is, and where software gets consumed. Our priority should be the

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Chmouel Boudjnah
On Thu, Sep 18, 2014 at 1:58 PM, Donald Stufft don...@stufft.io wrote: Distributions are not the only place that people get their software from, unless you think that the ~3 million downloads requests has received on PyPI in the last 30 days are distributions downloading requests to package

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Donald Stufft
On Sep 18, 2014, at 9:00 AM, Chmouel Boudjnah chmo...@enovance.com wrote: On Thu, Sep 18, 2014 at 1:58 PM, Donald Stufft don...@stufft.io mailto:don...@stufft.io wrote: Distributions are not the only place that people get their software from, unless you think that the ~3 million

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Clint Byrum
Excerpts from Donald Stufft's message of 2014-09-18 04:58:06 -0700: On Sep 18, 2014, at 7:54 AM, Thomas Goirand z...@debian.org wrote: Linux distributions are not the end be all of distribution models and they don’t get to dictate to upstream. Well, distributions is where the

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Ian Cordasco
On 9/17/14, 9:24 PM, Thomas Goirand z...@debian.org wrote: On 09/18/2014 08:22 AM, Morgan Fainberg wrote: I think that all of the conversation to this point has been valuable, the general consensus is vendoring a library is not as desirable as using it strictly as a dependency. It would be

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Ian Cordasco
On 9/18/14, 2:27 AM, Chmouel Boudjnah chmo...@enovance.com wrote: Ian Cordasco ian.corda...@rackspace.com writes: urllib3 do that automatically. I haven’t started to investigate exactly why they do this. Likewise, glance client has custom certificate verification in

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Donald Stufft
On Sep 18, 2014, at 10:18 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 04:58:06 -0700: On Sep 18, 2014, at 7:54 AM, Thomas Goirand z...@debian.org wrote: Linux distributions are not the end be all of distribution models and they don’t

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Ian Cordasco
On 9/18/14, 9:18 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 04:58:06 -0700: On Sep 18, 2014, at 7:54 AM, Thomas Goirand z...@debian.org wrote: Linux distributions are not the end be all of distribution models and they don’t get to

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Clint Byrum
Excerpts from Donald Stufft's message of 2014-09-18 07:30:27 -0700: On Sep 18, 2014, at 10:18 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 04:58:06 -0700: On Sep 18, 2014, at 7:54 AM, Thomas Goirand z...@debian.org wrote: Linux

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Donald Stufft
On Sep 18, 2014, at 12:29 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 07:30:27 -0700: On Sep 18, 2014, at 10:18 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 04:58:06 -0700: On Sep 18, 2014,

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Clint Byrum
Excerpts from Ian Cordasco's message of 2014-09-18 07:35:10 -0700: On 9/18/14, 9:18 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 04:58:06 -0700: On Sep 18, 2014, at 7:54 AM, Thomas Goirand z...@debian.org wrote: Linux

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-18 Thread Ian Cordasco
On 9/18/14, 11:29 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 07:30:27 -0700: On Sep 18, 2014, at 10:18 AM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Donald Stufft's message of 2014-09-18 04:58:06 -0700: On Sep 18, 2014, at

[openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Thomas Goirand
Hi, I'm horrified by what I just found. I have just found out this in glanceclient: File bla/tests/test_ssl.py, line 19, in module from requests.packages.urllib3 import poolmanager ImportError: No module named packages.urllib3 Please *DO NOT* do this. Instead, please use urllib3 from ...

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Donald Stufft
I don't know the specific situation but it's appropriate to do this if you're using requests and wish to interact with the urllib3 that requests is using. On Sep 17, 2014, at 11:15 AM, Thomas Goirand z...@debian.org wrote: Hi, I'm horrified by what I just found. I have just found out this

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Donald Stufft
Looking at the code on my phone it looks completely correct to use the vendored copy here and it wouldn't actually work otherwise. On Sep 17, 2014, at 11:17 AM, Donald Stufft don...@stufft.io wrote: I don't know the specific situation but it's appropriate to do this if you're using

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Clint Byrum
This is where Debian's one urllib3 to rule them all model fails in a modern fast paced world. Debian is arguably doing the right thing by pushing everyone to use one API, and one library, so that when that one library is found to be vulnerable to security problems, one update covers everyone.

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Davanum Srinivas
I was trying request-ifying oslo.vmware and ran into this as well: https://review.openstack.org/#/c/121956/ And we don't seem to have urllib3 in global-requirements either. Should we do that first? -- dims On Wed, Sep 17, 2014 at 1:05 PM, Clint Byrum cl...@fewbar.com wrote: This is where

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Clint Byrum
Excerpts from Davanum Srinivas's message of 2014-09-17 10:15:29 -0700: I was trying request-ifying oslo.vmware and ran into this as well: https://review.openstack.org/#/c/121956/ And we don't seem to have urllib3 in global-requirements either. Should we do that first? Honestly, after

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Mike Bayer
On Sep 17, 2014, at 2:46 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Davanum Srinivas's message of 2014-09-17 10:15:29 -0700: I was trying request-ifying oslo.vmware and ran into this as well: https://review.openstack.org/#/c/121956/ And we don't seem to have urllib3 in

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Ian Cordasco
On 9/17/14, 1:46 PM, Clint Byrum cl...@fewbar.com wrote: Excerpts from Davanum Srinivas's message of 2014-09-17 10:15:29 -0700: I was trying request-ifying oslo.vmware and ran into this as well: https://review.openstack.org/#/c/121956/ And we don't seem to have urllib3 in global-requirements

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Dean Troyer
Interestingly enough, the distros are doing exactly what they don't want us to do, ie, rebuilding things to use 'their' tested version of dependencies rather than the included one... On Wed, Sep 17, 2014 at 2:42 PM, Ian Cordasco ian.corda...@rackspace.com wrote: That aside, I’ve been mulling

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Mike Bayer
On Sep 17, 2014, at 3:42 PM, Ian Cordasco ian.corda...@rackspace.com wrote: Circling back to the issue of vendoring though: it’s a conscious decision to do this, and in the last two years there have been 2 CVEs reported for requests. There have been none for urllib3 and none for chardet.

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Ian Cordasco
On 9/17/14, 3:11 PM, Mike Bayer mba...@redhat.com wrote: On Sep 17, 2014, at 3:42 PM, Ian Cordasco ian.corda...@rackspace.com wrote: Circling back to the issue of vendoring though: it’s a conscious decision to do this, and in the last two years there have been 2 CVEs reported for requests.

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Robert Collins
On 18 September 2014 08:01, Dean Troyer dtro...@gmail.com wrote: Interestingly enough, the distros are doing exactly what they don't want us to do, ie, rebuilding things to use 'their' tested version of dependencies rather than the included one... Indeed - but the distros are solving for two

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Dean Troyer
On Wed, Sep 17, 2014 at 3:53 PM, Robert Collins robe...@robertcollins.net wrote: On 18 September 2014 08:01, Dean Troyer dtro...@gmail.com wrote: Interestingly enough, the distros are doing exactly what they don't want us to do, ie, rebuilding things to use 'their' tested version of

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Mike Bayer
On Sep 17, 2014, at 4:31 PM, Ian Cordasco ian.corda...@rackspace.com wrote: Project X pins a version of requests. Alice doesn’t know anything about requests and does pip install X. Until Alice takes a more active role in the development of Project X and looks into requests, she will never

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Morgan Fainberg
@lists.openstack.org Subject:  Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3) On 9/17/14, 5:39 PM, Mike Bayer wrote: On Sep 17, 2014, at 4:31 PM, Ian Cordasco wrote: Project X pins a version of requests. Alice

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Thomas Goirand
On 09/18/2014 08:22 AM, Morgan Fainberg wrote: I think that all of the conversation to this point has been valuable, the general consensus is vendoring a library is not as desirable as using it strictly as a dependency. It would be nice in a perfect world if vendoring wasn’t and issue, but in

Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)

2014-09-17 Thread Donald Stufft
On Sep 17, 2014, at 10:24 PM, Thomas Goirand z...@debian.org wrote: On 09/18/2014 08:22 AM, Morgan Fainberg wrote: I think that all of the conversation to this point has been valuable, the general consensus is vendoring a library is not as desirable as using it strictly as a dependency. It