Re: [openstack-dev] [oslo] Fate of xmlutils
On Mon, Sep 29 2014, Joshua Harlow wrote: Do we know that the users (keystone, neutron...) aren't vulnerable? From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure seems like we would likely still have issues if custom implementations are being used/created. Perhaps we should just use the defusedxml libraries until proven otherwise (better to be safe than sorry). According to LP#1100282¹, Keystone and Neutron are supposed to not be vulnerable with different fixes than Nova. Since all the solutions are different, I'm not sure it covers the problem in its entirety in all cases. I see 2 options: 1. Put effort to move all projects to defusedxml 2. Since XML API are going to be deprecated (at least in Nova), move xmlutils to Nova and be done with it. Solution 1 requires a lot more effort, and I wonder if it's worth it. ¹ https://bugs.launchpad.net/bugs/1100282 -- Julien Danjou // Free Software hacker // http://julien.danjou.info signature.asc Description: PGP signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
Julien, I believe all the lessons learned from defusedxml (see the release dates) have been folded back into the different libraries. For example plain old etree.fromstring() even without any special options is ok with the specially crafted xml bombs that you can find as test cases in defusedxml repo. There is some more information here as well (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this point, unless we see a new attack vector other than the ones that caused folks to whip up defusedxml, we should be good. So Option #2 is definitely the way to go thanks, dims On Tue, Sep 30, 2014 at 3:45 AM, Julien Danjou jul...@danjou.info wrote: On Mon, Sep 29 2014, Joshua Harlow wrote: Do we know that the users (keystone, neutron...) aren't vulnerable? From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure seems like we would likely still have issues if custom implementations are being used/created. Perhaps we should just use the defusedxml libraries until proven otherwise (better to be safe than sorry). According to LP#1100282¹, Keystone and Neutron are supposed to not be vulnerable with different fixes than Nova. Since all the solutions are different, I'm not sure it covers the problem in its entirety in all cases. I see 2 options: 1. Put effort to move all projects to defusedxml 2. Since XML API are going to be deprecated (at least in Nova), move xmlutils to Nova and be done with it. Solution 1 requires a lot more effort, and I wonder if it's worth it. ¹ https://bugs.launchpad.net/bugs/1100282 -- Julien Danjou // Free Software hacker // http://julien.danjou.info ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: https://twitter.com/dims ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
On Tue, Sep 30, 2014 at 09:28:22AM +0930, Christopher Yeoh wrote: On Mon, 29 Sep 2014 18:03:20 +0200 Julien Danjou jul...@danjou.info wrote: It seems that Python fixed that issue with 2 modules released on PyPI: https://pypi.python.org/pypi/defusedxml https://pypi.python.org/pypi/defusedexpat I'm no XML expert, and I've only a shallow understanding of the issue, but I wonder if we should put some efforts to drop xmlutils and our custom XML fixes to used instead these 2 modules. Nova XML API support is marked as deprecated in Juno. So hopefully we'll be able to just drop XML and associated helper modules within a couple of cycles. Even if Nova doesn't have an XML API, internals of Nova still need to deal with XML. eg the libvirt driver needs to parse format XML docs. At least these docs are coming from a trusted source, not the untrusted end user, but we'll always have a need for python XML modules of some kind. Regards, Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
On Tue, Sep 30 2014, Davanum Srinivas wrote: I believe all the lessons learned from defusedxml (see the release dates) have been folded back into the different libraries. For example plain old etree.fromstring() even without any special options is ok with the specially crafted xml bombs that you can find as test cases in defusedxml repo. There is some more information here as well (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this point, unless we see a new attack vector other than the ones that caused folks to whip up defusedxml, we should be good. So Option #2 is definitely the way to go Thanks for this information dims! I'll start working on that ASAP. -- Julien Danjou -- Free Software hacker -- http://julien.danjou.info signature.asc Description: PGP signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
I agree, it sounds like option 2 is safe. Julien, I updated your commit message on https://review.openstack.org/#/c/125021/ to point to this thread. Write-it-down-ly, Doug On Sep 30, 2014, at 7:17 AM, Davanum Srinivas dava...@gmail.com wrote: Julien, I believe all the lessons learned from defusedxml (see the release dates) have been folded back into the different libraries. For example plain old etree.fromstring() even without any special options is ok with the specially crafted xml bombs that you can find as test cases in defusedxml repo. There is some more information here as well (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this point, unless we see a new attack vector other than the ones that caused folks to whip up defusedxml, we should be good. So Option #2 is definitely the way to go thanks, dims On Tue, Sep 30, 2014 at 3:45 AM, Julien Danjou jul...@danjou.info wrote: On Mon, Sep 29 2014, Joshua Harlow wrote: Do we know that the users (keystone, neutron...) aren't vulnerable? From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure seems like we would likely still have issues if custom implementations are being used/created. Perhaps we should just use the defusedxml libraries until proven otherwise (better to be safe than sorry). According to LP#1100282¹, Keystone and Neutron are supposed to not be vulnerable with different fixes than Nova. Since all the solutions are different, I'm not sure it covers the problem in its entirety in all cases. I see 2 options: 1. Put effort to move all projects to defusedxml 2. Since XML API are going to be deprecated (at least in Nova), move xmlutils to Nova and be done with it. Solution 1 requires a lot more effort, and I wonder if it's worth it. ¹ https://bugs.launchpad.net/bugs/1100282 -- Julien Danjou // Free Software hacker // http://julien.danjou.info ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: https://twitter.com/dims ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
This was also needed for Python 2.6, right? Do we have confirmation that we can drop that for Kilo? -Ben On 09/30/2014 08:28 AM, Doug Hellmann wrote: I agree, it sounds like option 2 is safe. Julien, I updated your commit message on https://review.openstack.org/#/c/125021/ to point to this thread. Write-it-down-ly, Doug On Sep 30, 2014, at 7:17 AM, Davanum Srinivas dava...@gmail.com wrote: Julien, I believe all the lessons learned from defusedxml (see the release dates) have been folded back into the different libraries. For example plain old etree.fromstring() even without any special options is ok with the specially crafted xml bombs that you can find as test cases in defusedxml repo. There is some more information here as well (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this point, unless we see a new attack vector other than the ones that caused folks to whip up defusedxml, we should be good. So Option #2 is definitely the way to go thanks, dims On Tue, Sep 30, 2014 at 3:45 AM, Julien Danjou jul...@danjou.info wrote: On Mon, Sep 29 2014, Joshua Harlow wrote: Do we know that the users (keystone, neutron...) aren't vulnerable? From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure seems like we would likely still have issues if custom implementations are being used/created. Perhaps we should just use the defusedxml libraries until proven otherwise (better to be safe than sorry). According to LP#1100282¹, Keystone and Neutron are supposed to not be vulnerable with different fixes than Nova. Since all the solutions are different, I'm not sure it covers the problem in its entirety in all cases. I see 2 options: 1. Put effort to move all projects to defusedxml 2. Since XML API are going to be deprecated (at least in Nova), move xmlutils to Nova and be done with it. Solution 1 requires a lot more effort, and I wonder if it's worth it. ¹ https://bugs.launchpad.net/bugs/1100282 -- Julien Danjou // Free Software hacker // http://julien.danjou.info ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: https://twitter.com/dims ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
Yes, I think we are still on track to drop 2.6 support for the servers in Kilo. This wasn’t used in the client libraries, right? On Sep 30, 2014, at 10:25 AM, Ben Nemec openst...@nemebean.com wrote: This was also needed for Python 2.6, right? Do we have confirmation that we can drop that for Kilo? -Ben On 09/30/2014 08:28 AM, Doug Hellmann wrote: I agree, it sounds like option 2 is safe. Julien, I updated your commit message on https://review.openstack.org/#/c/125021/ to point to this thread. Write-it-down-ly, Doug On Sep 30, 2014, at 7:17 AM, Davanum Srinivas dava...@gmail.com wrote: Julien, I believe all the lessons learned from defusedxml (see the release dates) have been folded back into the different libraries. For example plain old etree.fromstring() even without any special options is ok with the specially crafted xml bombs that you can find as test cases in defusedxml repo. There is some more information here as well (http://lxml.de/FAQ.html#is-lxml-vulnerable-to-xml-bombs). So at this point, unless we see a new attack vector other than the ones that caused folks to whip up defusedxml, we should be good. So Option #2 is definitely the way to go thanks, dims On Tue, Sep 30, 2014 at 3:45 AM, Julien Danjou jul...@danjou.info wrote: On Mon, Sep 29 2014, Joshua Harlow wrote: Do we know that the users (keystone, neutron...) aren't vulnerable? From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure seems like we would likely still have issues if custom implementations are being used/created. Perhaps we should just use the defusedxml libraries until proven otherwise (better to be safe than sorry). According to LP#1100282¹, Keystone and Neutron are supposed to not be vulnerable with different fixes than Nova. Since all the solutions are different, I'm not sure it covers the problem in its entirety in all cases. I see 2 options: 1. Put effort to move all projects to defusedxml 2. Since XML API are going to be deprecated (at least in Nova), move xmlutils to Nova and be done with it. Solution 1 requires a lot more effort, and I wonder if it's worth it. ¹ https://bugs.launchpad.net/bugs/1100282 -- Julien Danjou // Free Software hacker // http://julien.danjou.info ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Davanum Srinivas :: https://twitter.com/dims ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
On Tue, Sep 30 2014, Doug Hellmann wrote: Yes, I think we are still on track to drop 2.6 support for the servers in Kilo. This wasn’t used in the client libraries, right? After a quick grep of the code I've around, it doesn't look being used by anything else than Nova itself. -- Julien Danjou -- Free Software hacker -- http://julien.danjou.info signature.asc Description: PGP signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [oslo] Fate of xmlutils
Hi, I was looking at xmlutils today, and I took a look at the history of this file that seems to come from a CVE almost 2 years ago. What is surprising is that, unless I missed something, the only user of that lib is Nova. Other projects such as Keystone or Neutron implemented things in a different way. It seems that Python fixed that issue with 2 modules released on PyPI: https://pypi.python.org/pypi/defusedxml https://pypi.python.org/pypi/defusedexpat I'm no XML expert, and I've only a shallow understanding of the issue, but I wonder if we should put some efforts to drop xmlutils and our custom XML fixes to used instead these 2 modules. Hint appreciated. -- Julien Danjou /* Free Software hacker http://julien.danjou.info */ signature.asc Description: PGP signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
On Sep 29, 2014, at 12:03 PM, Julien Danjou jul...@danjou.info wrote: Hi, I was looking at xmlutils today, and I took a look at the history of this file that seems to come from a CVE almost 2 years ago. What is surprising is that, unless I missed something, the only user of that lib is Nova. Other projects such as Keystone or Neutron implemented things in a different way. It seems that Python fixed that issue with 2 modules released on PyPI: https://pypi.python.org/pypi/defusedxml https://pypi.python.org/pypi/defusedexpat I'm no XML expert, and I've only a shallow understanding of the issue, but I wonder if we should put some efforts to drop xmlutils and our custom XML fixes to used instead these 2 modules. Hint appreciated. I thought those fixes were also eventually rolled into language releases, and we had planned to stop worrying about using xmlutils after we drop python 2.6 support for master. Am I mistaken about those being rolled into the release? The defused* packages may have been created/released at the same time as, or after, the module in the incubator. If we do need to continue carrying support for the fix I agree that moving to the 3rd party libraries would make sense. Doug -- Julien Danjou /* Free Software hacker http://julien.danjou.info */ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
Do we know that the users (keystone, neutron...) aren't vulnerable? From https://pypi.python.org/pypi/defusedxml#python-xml-libraries it sure seems like we would likely still have issues if custom implementations are being used/created. Perhaps we should just use the defusedxml libraries until proven otherwise (better to be safe than sorry). On Sep 29, 2014, at 9:03 AM, Julien Danjou jul...@danjou.info wrote: Hi, I was looking at xmlutils today, and I took a look at the history of this file that seems to come from a CVE almost 2 years ago. What is surprising is that, unless I missed something, the only user of that lib is Nova. Other projects such as Keystone or Neutron implemented things in a different way. It seems that Python fixed that issue with 2 modules released on PyPI: https://pypi.python.org/pypi/defusedxml https://pypi.python.org/pypi/defusedexpat I'm no XML expert, and I've only a shallow understanding of the issue, but I wonder if we should put some efforts to drop xmlutils and our custom XML fixes to used instead these 2 modules. Hint appreciated. -- Julien Danjou /* Free Software hacker http://julien.danjou.info */ ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [oslo] Fate of xmlutils
On Mon, 29 Sep 2014 18:03:20 +0200 Julien Danjou jul...@danjou.info wrote: It seems that Python fixed that issue with 2 modules released on PyPI: https://pypi.python.org/pypi/defusedxml https://pypi.python.org/pypi/defusedexpat I'm no XML expert, and I've only a shallow understanding of the issue, but I wonder if we should put some efforts to drop xmlutils and our custom XML fixes to used instead these 2 modules. Nova XML API support is marked as deprecated in Juno. So hopefully we'll be able to just drop XML and associated helper modules within a couple of cycles. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
