Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-30 Thread Julien Danjou
On Mon, Sep 29 2014, Joshua Harlow wrote:

 Do we know that the users (keystone, neutron...) aren't vulnerable?

 From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure 
 seems
 like we would likely still have issues if custom implementations are being
 used/created. Perhaps we should just use the defusedxml libraries until proven
 otherwise (better to be safe than sorry).

According to LP#1100282¹, Keystone and Neutron are supposed to not be
vulnerable with different fixes than Nova.

Since all the solutions are different, I'm not sure it covers the
problem in its entirety in all cases.

I see 2 options:
1. Put effort to move all projects to defusedxml
2. Since XML API are going to be deprecated (at least in Nova), move
   xmlutils to Nova and be done with it.

Solution 1 requires a lot more effort, and I wonder if it's worth it.


¹  https://bugs.launchpad.net/bugs/1100282

-- 
Julien Danjou
// Free Software hacker
// http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-30 Thread Davanum Srinivas
Julien,

I believe all the lessons learned from defusedxml (see the release
dates) have been folded back into the different libraries. For example
plain old etree.fromstring() even without any special options is ok
with the specially crafted xml bombs that you can find as test cases
in defusedxml repo. There is some more information here as well
(http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this
point, unless we see a new attack vector other than the ones that
caused folks to whip up defusedxml, we should be good. So Option #2 is
definitely the way to go

thanks,
dims

On Tue, Sep 30, 2014 at 3:45 AM, Julien Danjou jul...@danjou.info wrote:
 On Mon, Sep 29 2014, Joshua Harlow wrote:

 Do we know that the users (keystone, neutron...) aren't vulnerable?

 From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure 
 seems
 like we would likely still have issues if custom implementations are being
 used/created. Perhaps we should just use the defusedxml libraries until 
 proven
 otherwise (better to be safe than sorry).

 According to LP#1100282¹, Keystone and Neutron are supposed to not be
 vulnerable with different fixes than Nova.

 Since all the solutions are different, I'm not sure it covers the
 problem in its entirety in all cases.

 I see 2 options:
 1. Put effort to move all projects to defusedxml
 2. Since XML API are going to be deprecated (at least in Nova), move
xmlutils to Nova and be done with it.

 Solution 1 requires a lot more effort, and I wonder if it's worth it.


 ¹  https://bugs.launchpad.net/bugs/1100282

 --
 Julien Danjou
 // Free Software hacker
 // http://julien.danjou.info

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Davanum Srinivas :: https://twitter.com/dims

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-30 Thread Daniel P. Berrange
On Tue, Sep 30, 2014 at 09:28:22AM +0930, Christopher Yeoh wrote:
 On Mon, 29 Sep 2014 18:03:20 +0200
 Julien Danjou jul...@danjou.info wrote:
  
  It seems that Python fixed that issue with 2 modules released on PyPI:
  
https://pypi.python.org/pypi/defusedxml
https://pypi.python.org/pypi/defusedexpat
  
  I'm no XML expert, and I've only a shallow understanding of the issue,
  but I wonder if we should put some efforts to drop xmlutils and our
  custom XML fixes to used instead these 2 modules.
 
 Nova XML API support is marked as deprecated in Juno. So hopefully
 we'll be able to just drop XML and associated helper modules within a
 couple of cycles.

Even if Nova doesn't have an XML API, internals of Nova still need to
deal with XML. eg the libvirt driver needs to parse  format XML docs.
At least these docs are coming from a trusted source, not the untrusted
end user, but we'll always have a need for python XML modules of some
kind.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-30 Thread Julien Danjou
On Tue, Sep 30 2014, Davanum Srinivas wrote:

 I believe all the lessons learned from defusedxml (see the release
 dates) have been folded back into the different libraries. For example
 plain old etree.fromstring() even without any special options is ok
 with the specially crafted xml bombs that you can find as test cases
 in defusedxml repo. There is some more information here as well
 (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this
 point, unless we see a new attack vector other than the ones that
 caused folks to whip up defusedxml, we should be good. So Option #2 is
 definitely the way to go

Thanks for this information dims! I'll start working on that ASAP.

-- 
Julien Danjou
-- Free Software hacker
-- http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-30 Thread Doug Hellmann
I agree, it sounds like option 2 is safe.

Julien, I updated your commit message on 
https://review.openstack.org/#/c/125021/ to point to this thread.

Write-it-down-ly,
Doug

On Sep 30, 2014, at 7:17 AM, Davanum Srinivas dava...@gmail.com wrote:

 Julien,
 
 I believe all the lessons learned from defusedxml (see the release
 dates) have been folded back into the different libraries. For example
 plain old etree.fromstring() even without any special options is ok
 with the specially crafted xml bombs that you can find as test cases
 in defusedxml repo. There is some more information here as well
 (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this
 point, unless we see a new attack vector other than the ones that
 caused folks to whip up defusedxml, we should be good. So Option #2 is
 definitely the way to go
 
 thanks,
 dims
 
 On Tue, Sep 30, 2014 at 3:45 AM, Julien Danjou jul...@danjou.info wrote:
 On Mon, Sep 29 2014, Joshua Harlow wrote:
 
 Do we know that the users (keystone, neutron...) aren't vulnerable?
 
 From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure 
 seems
 like we would likely still have issues if custom implementations are being
 used/created. Perhaps we should just use the defusedxml libraries until 
 proven
 otherwise (better to be safe than sorry).
 
 According to LP#1100282¹, Keystone and Neutron are supposed to not be
 vulnerable with different fixes than Nova.
 
 Since all the solutions are different, I'm not sure it covers the
 problem in its entirety in all cases.
 
 I see 2 options:
 1. Put effort to move all projects to defusedxml
 2. Since XML API are going to be deprecated (at least in Nova), move
   xmlutils to Nova and be done with it.
 
 Solution 1 requires a lot more effort, and I wonder if it's worth it.
 
 
 ¹  https://bugs.launchpad.net/bugs/1100282
 
 --
 Julien Danjou
 // Free Software hacker
 // http://julien.danjou.info
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
 -- 
 Davanum Srinivas :: https://twitter.com/dims
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-30 Thread Ben Nemec
This was also needed for Python 2.6, right?  Do we have confirmation
that we can drop that for Kilo?

-Ben

On 09/30/2014 08:28 AM, Doug Hellmann wrote:
 I agree, it sounds like option 2 is safe.
 
 Julien, I updated your commit message on 
 https://review.openstack.org/#/c/125021/ to point to this thread.
 
 Write-it-down-ly,
 Doug
 
 On Sep 30, 2014, at 7:17 AM, Davanum Srinivas dava...@gmail.com wrote:
 
 Julien,

 I believe all the lessons learned from defusedxml (see the release
 dates) have been folded back into the different libraries. For example
 plain old etree.fromstring() even without any special options is ok
 with the specially crafted xml bombs that you can find as test cases
 in defusedxml repo. There is some more information here as well
 (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this
 point, unless we see a new attack vector other than the ones that
 caused folks to whip up defusedxml, we should be good. So Option #2 is
 definitely the way to go

 thanks,
 dims

 On Tue, Sep 30, 2014 at 3:45 AM, Julien Danjou jul...@danjou.info wrote:
 On Mon, Sep 29 2014, Joshua Harlow wrote:

 Do we know that the users (keystone, neutron...) aren't vulnerable?

 From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure 
 seems
 like we would likely still have issues if custom implementations are being
 used/created. Perhaps we should just use the defusedxml libraries until 
 proven
 otherwise (better to be safe than sorry).

 According to LP#1100282¹, Keystone and Neutron are supposed to not be
 vulnerable with different fixes than Nova.

 Since all the solutions are different, I'm not sure it covers the
 problem in its entirety in all cases.

 I see 2 options:
 1. Put effort to move all projects to defusedxml
 2. Since XML API are going to be deprecated (at least in Nova), move
   xmlutils to Nova and be done with it.

 Solution 1 requires a lot more effort, and I wonder if it's worth it.


 ¹  https://bugs.launchpad.net/bugs/1100282

 --
 Julien Danjou
 // Free Software hacker
 // http://julien.danjou.info

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 -- 
 Davanum Srinivas :: https://twitter.com/dims

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-30 Thread Doug Hellmann
Yes, I think we are still on track to drop 2.6 support for the servers in Kilo.

This wasn’t used in the client libraries, right?

On Sep 30, 2014, at 10:25 AM, Ben Nemec openst...@nemebean.com wrote:

 This was also needed for Python 2.6, right?  Do we have confirmation
 that we can drop that for Kilo?
 
 -Ben
 
 On 09/30/2014 08:28 AM, Doug Hellmann wrote:
 I agree, it sounds like option 2 is safe.
 
 Julien, I updated your commit message on 
 https://review.openstack.org/#/c/125021/ to point to this thread.
 
 Write-it-down-ly,
 Doug
 
 On Sep 30, 2014, at 7:17 AM, Davanum Srinivas dava...@gmail.com wrote:
 
 Julien,
 
 I believe all the lessons learned from defusedxml (see the release
 dates) have been folded back into the different libraries. For example
 plain old etree.fromstring() even without any special options is ok
 with the specially crafted xml bombs that you can find as test cases
 in defusedxml repo. There is some more information here as well
 (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this
 point, unless we see a new attack vector other than the ones that
 caused folks to whip up defusedxml, we should be good. So Option #2 is
 definitely the way to go
 
 thanks,
 dims
 
 On Tue, Sep 30, 2014 at 3:45 AM, Julien Danjou jul...@danjou.info wrote:
 On Mon, Sep 29 2014, Joshua Harlow wrote:
 
 Do we know that the users (keystone, neutron...) aren't vulnerable?
 
 From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure 
 seems
 like we would likely still have issues if custom implementations are being
 used/created. Perhaps we should just use the defusedxml libraries until 
 proven
 otherwise (better to be safe than sorry).
 
 According to LP#1100282¹, Keystone and Neutron are supposed to not be
 vulnerable with different fixes than Nova.
 
 Since all the solutions are different, I'm not sure it covers the
 problem in its entirety in all cases.
 
 I see 2 options:
 1. Put effort to move all projects to defusedxml
 2. Since XML API are going to be deprecated (at least in Nova), move
  xmlutils to Nova and be done with it.
 
 Solution 1 requires a lot more effort, and I wonder if it's worth it.
 
 
 ¹  https://bugs.launchpad.net/bugs/1100282
 
 --
 Julien Danjou
 // Free Software hacker
 // http://julien.danjou.info
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 
 -- 
 Davanum Srinivas :: https://twitter.com/dims
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-30 Thread Julien Danjou
On Tue, Sep 30 2014, Doug Hellmann wrote:

 Yes, I think we are still on track to drop 2.6 support for the servers in 
 Kilo.

 This wasn’t used in the client libraries, right?

After a quick grep of the code I've around, it doesn't look being used
by anything else than Nova itself.

-- 
Julien Danjou
-- Free Software hacker
-- http://julien.danjou.info


signature.asc
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-29 Thread Doug Hellmann

On Sep 29, 2014, at 12:03 PM, Julien Danjou jul...@danjou.info wrote:

 Hi,
 
 I was looking at xmlutils today, and I took a look at the history of
 this file that seems to come from a CVE almost 2 years ago.
 
 What is surprising is that, unless I missed something, the only user of
 that lib is Nova. Other projects such as Keystone or Neutron implemented
 things in a different way.
 
 It seems that Python fixed that issue with 2 modules released on PyPI:
 
  https://pypi.python.org/pypi/defusedxml
  https://pypi.python.org/pypi/defusedexpat
 
 I'm no XML expert, and I've only a shallow understanding of the issue,
 but I wonder if we should put some efforts to drop xmlutils and our
 custom XML fixes to used instead these 2 modules.
 
 Hint appreciated.

I thought those fixes were also eventually rolled into language releases, and 
we had planned to stop worrying about using xmlutils after we drop python 2.6 
support for master. Am I mistaken about those being rolled into the release?

The defused* packages may have been created/released at the same time as, or 
after, the module in the incubator. If we do need to continue carrying support 
for the fix I agree that moving to the 3rd party libraries would make sense.

Doug


 
 -- 
 Julien Danjou
 /* Free Software hacker
   http://julien.danjou.info */
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-29 Thread Joshua Harlow
Do we know that the users (keystone, neutron...) aren't vulnerable?

From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure 
seems like we would likely still have issues if custom implementations are 
being used/created. Perhaps we should just use the defusedxml libraries until 
proven otherwise (better to be safe than sorry).

On Sep 29, 2014, at 9:03 AM, Julien Danjou jul...@danjou.info wrote:

 Hi,
 
 I was looking at xmlutils today, and I took a look at the history of
 this file that seems to come from a CVE almost 2 years ago.
 
 What is surprising is that, unless I missed something, the only user of
 that lib is Nova. Other projects such as Keystone or Neutron implemented
 things in a different way.
 
 It seems that Python fixed that issue with 2 modules released on PyPI:
 
  https://pypi.python.org/pypi/defusedxml
  https://pypi.python.org/pypi/defusedexpat
 
 I'm no XML expert, and I've only a shallow understanding of the issue,
 but I wonder if we should put some efforts to drop xmlutils and our
 custom XML fixes to used instead these 2 modules.
 
 Hint appreciated.
 
 -- 
 Julien Danjou
 /* Free Software hacker
   http://julien.danjou.info */
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [oslo] Fate of xmlutils

2014-09-29 Thread Christopher Yeoh
On Mon, 29 Sep 2014 18:03:20 +0200
Julien Danjou jul...@danjou.info wrote:
 
 It seems that Python fixed that issue with 2 modules released on PyPI:
 
   https://pypi.python.org/pypi/defusedxml
   https://pypi.python.org/pypi/defusedexpat
 
 I'm no XML expert, and I've only a shallow understanding of the issue,
 but I wonder if we should put some efforts to drop xmlutils and our
 custom XML fixes to used instead these 2 modules.

Nova XML API support is marked as deprecated in Juno. So hopefully
we'll be able to just drop XML and associated helper modules within a
couple of cycles.

Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev