Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
Jim Fulton wrote: On Jul 18, 2006, at 2:55 PM, David Pratt wrote: Hi Jim. I was noticing a 0.4.0-zope in distutils I don't know what you mean by this. that looks patched with NotImplementedErrors for the offending code in docutils.parsers.rst.directives.misc. Can you when this will land in the Zope3 trunk? Hi Jim. Yes, I mean docutils, sorry. If you mean patching the docutils, then as far as I'm concerned, it will never land in the Zope 3 trunk. The right solution to this problem is to write applications that use docutils correctly, not to patch docutils. You are probably right but just the same I'd rather see the patched version for z3 also since I am certain this will become less obvious over time if it is left the way it is. Alternatively, perhaps a text file for these security issues could be included in the distribution so it is not forgotten with any recommendations for a programmer to avoid known security issues. Regards, David ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
David Pratt wrote: You are probably right but just the same I'd rather see the patched version for z3 also since I am certain this will become less obvious over time if it is left the way it is. Instead of maintaining a fork of docutils, Zope 3 should (and may already, I haven't been keeping up with this issue) include tests to make sure we're using docutils appropriately. Best of both worlds: we have continued assurance we don't regress, and we don't have to maintain a fork/patches. -- Benji York Senior Software Engineer Zope Corporation ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
Benji York wrote: David Pratt wrote: You are probably right but just the same I'd rather see the patched version for z3 also since I am certain this will become less obvious over time if it is left the way it is. Instead of maintaining a fork of docutils, Zope 3 should (and may already, I haven't been keeping up with this issue) include tests to make sure we're using docutils appropriately. Best of both worlds: we have continued assurance we don't regress, and we don't have to maintain a fork/patches. Hi Benji. Fair enough. What about the idea of maintaining a text file in the distribution specific to possible security issues. Is this worth considering for historical purposes so they do not get lost over time or implicitly understood by only a handful of people. Many thanks. Regards, David ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
David Pratt wrote: What about the idea of maintaining a text file in the distribution specific to possible security issues. Is this worth considering for historical purposes so they do not get lost over time or implicitly understood by only a handful of people. Exactly. Any package that needs security-related things verified should have a test (doctest in a text file) describing the problem and verifying that it has been fixed. I don't think we want a single file to hold them though, tests (including these) should normally live near the package that they test. -- Benji York Senior Software Engineer Zope Corporation ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
Benji York wrote: David Pratt wrote: What about the idea of maintaining a text file in the distribution specific to possible security issues. Is this worth considering for historical purposes so they do not get lost over time or implicitly understood by only a handful of people. Exactly. Any package that needs security-related things verified should have a test (doctest in a text file) describing the problem and verifying that it has been fixed. I don't think we want a single file to hold them though, tests (including these) should normally live near the package that they test. Ok this all makes perfect sense. The doctest is the right place for this for sure. Just took me a while to see that everthing was already there to deal with this as consistently as all other parts of zope3. It's all good :-) Regards, David ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
On Jul 19, 2006, at 8:47 AM, Benji York wrote: David Pratt wrote: What about the idea of maintaining a text file in the distribution specific to possible security issues. Is this worth considering for historical purposes so they do not get lost over time or implicitly understood by only a handful of people. Exactly. Any package that needs security-related things verified should have a test (doctest in a text file) describing the problem and verifying that it has been fixed. Of course, that, by itself, doesn't solve the problem. docutils may introduce a new feature in the furture that shouldn't be exposed through the web. Whenever we integrate a new version, we need to review it to make sure there aren't new security issues. This is especially true of anything that is exposed TTW. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
On Jul 18, 2006, at 2:55 PM, David Pratt wrote: Hi Jim. I was noticing a 0.4.0-zope in distutils I don't know what you mean by this. that looks patched with NotImplementedErrors for the offending code in docutils.parsers.rst.directives.misc. Can you when this will land in the Zope3 trunk? If you mean patching the docutils, then as far as I'm concerned, it will never land in the Zope 3 trunk. The right solution to this problem is to write applications that use docutils correctly, not to patch docutils. I can understand why this solution was used for Zope 2, at least in the short run. I don't think it's a good long-ter solution. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
Jim Fulton wrote: Recently, a serious security flaw was found in Zope 2 due to it's improper support for allowing reStructuredText to be edited through-the-web. reStructuredText has directives that allow inclusion of any file a Zope process could read and inclusion of data obtained from fetching arbitrary URLs. In a trusted environment, these directives have legitimate uses. The feature of including files and URL results should not be enabled for text entered from untrusted sources, which applies to most through-the-web interactions. Hi Jim. In the case of a wiki, it is the nature of a wiki that folks are able to edit through the web. Wouldn't data validation and any necessary alterations to the directives some sense as opposed to removing it from the zope3 mix? The recent hotfix: http://www.zope.org/Products/Zope/Hotfix-2006-07-05/Hotfix-2006-07-05 addresses the problem for Zope 2. It is safe to allow reStructuredText through the web with care. The inclusion of files or URL results can be disabled, but the programmer must explicitly disable the feature. It is not disabled by default. It is also critical that a developer who exposes through-the-web reStructuredText have tests to verify that the file/url inclusion feature has been disabled. Zope 3 itself, as released, doesn't have this problem because it doesn't allow reST entry through the web. There are third-party applications, however, including 2 packages in the Zope 3 subversion tree that do have this problem. I strongly urge you to avoid using any Zope package that allows through-the-web input of reStructuredText unless you can verify that file/url has been properly disabled. The zwiki and bugtracker packages do not currently disable file/url inclusion and should not be used in situations in which users who are not highly trusted have access to these applications. Can you be explicit about the process of disabling file/url inclusion for zope3 (if this is the critical point you are making ). The use of restructured text is valuable in zope and obviously it is important to understand security measures that would allow its continued use. If this can be done, why remove the products from the repository tree? Would it not be better to apply the necessary fixes? Many thanks. Regards, David ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Security alert: use of Through-the-Web reStructuredText
On Jul 8, 2006, at 11:49 AM, David Pratt wrote: Jim Fulton wrote: Recently, a serious security flaw was found in Zope 2 due to it's improper support for allowing reStructuredText to be edited through-the-web. reStructuredText has directives that allow inclusion of any file a Zope process could read and inclusion of data obtained from fetching arbitrary URLs. In a trusted environment, these directives have legitimate uses. The feature of including files and URL results should not be enabled for text entered from untrusted sources, which applies to most through-the- web interactions. Hi Jim. In the case of a wiki, it is the nature of a wiki that folks are able to edit through the web. But a wiki can be edited in other formats that restructured text. (Personally, I think wikis should use tools like Epoz or Kupu to allow direct HTML editing, but that's a different matter. Wouldn't data validation and any necessary alterations to the directives some sense as opposed to removing it from the zope3 mix? Sure, if someone is willing to do it and take responsibility. Note that I'm not removing these from the release, because they've never been in the release. I didn't even remove them from the repository, I just removed them from the Zope 3 tree. I'm convinced that TTW reST can be safe with suitable attention to detail. So far though, that hasn't happened. No one has come forward yet and said I'll maintain this and be responsible for making sure we're secure wrt reST. The recent hotfix: http://www.zope.org/Products/Zope/Hotfix-2006-07-05/ Hotfix-2006-07-05 addresses the problem for Zope 2. Perhaps. We don't know for sure. We don't have tests. We don't know if it can be defeated using a reload product. It is also a very crude fix. It prevents people from creating add-ons that make legitimate use of file-inclusion or the raw directive. It was a great fix in an emergency -- and this was a serious emergency, but I don't want to use such a fix in Zope 3. It is safe to allow reStructuredText through the web with care. The inclusion of files or URL results can be disabled, but the programmer must explicitly disable the feature. It is not disabled by default. It is also critical that a developer who exposes through-the-web reStructuredText have tests to verify that the file/url inclusion feature has been disabled. Zope 3 itself, as released, doesn't have this problem because it doesn't allow reST entry through the web. There are third-party applications, however, including 2 packages in the Zope 3 subversion tree that do have this problem. I strongly urge you to avoid using any Zope package that allows through-the-web input of reStructuredText unless you can verify that file/url has been properly disabled. The zwiki and bugtracker packages do not currently disable file/ url inclusion and should not be used in situations in which users who are not highly trusted have access to these applications. Can you be explicit about the process of disabling file/url inclusion for zope3 (if this is the critical point you are making ). The use of restructured text is valuable in zope and obviously it is important to understand security measures that would allow its continued use. The reStructuredText documentation gives instructions for disabling it. But something this risk needs people to be responsible. I'm not seeing that. I expect someone to come forward eventually. Part of being responsible is writing reasonably extensive tests. If this can be done, why remove the products from the repository tree? Would it not be better to apply the necessary fixes? Many thanks. Because their presence in the Zope 3 tree put people at serious risk. If someone wants to work on them, great, and they can release them as add-on packages. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
