[Zope-dev] Re: New test runner work
t test that fails due to this check is the culprit. I see now though that this could be easily modeled by having a 'global' or 'base' layer in your test suite, and mandate its use by all tests in your application. Or the check could go in a more specific layer if appropriate. >> These sorts of policies are important > >> for us as we run our tests in an automated environment (we can't >> commit to >> our trunk. Instead, we send a request to a daemon which runs the test >> suite >> and commits on our behalf if the tests all pass). > > Hm, seems rather restrictive... We like it ;) Our RCS (Bazaar) allows us to trivially merge branches into other branches, so we can avoid fallout from any delays in landing stuff to the trunk. And it is guarenteed that any changes landing in the trunk run, and more importantly, run with the current versions of all the dependant libraries and tools. So for example, if we add some sanity checks to SQLObject stopping certain dangerous operations, nobody can accidently commit code that breaks under the new version. It means that every day the trunk is rolled out to a staging server automatically and actually runs, and production rollouts can be done with confidence by simply picking an arbitrary revision on the trunk, tagging it and pushing it out. > I guess making the new test runner class-based would more easily allow this > sort of customization. > >> Our full test suite currently takes 45 minutes to run and it is >> becoming an >> issue. > > Hm, I can see why you'd like to parallelize things. Of course, this > only helps > you if you have enough hardware to benefit from the parallelization. I think the parallelization might take some work (perhaps not the test runner, but making our existing test suite work with it ;) ). I think there are lower hanging fruit to reach for first ;) -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ signature.asc Description: OpenPGP digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: New test runner work
rhaps this would be best done by allowing people to write wrappers around the one-true-testrunner? This seems to be the simplest way of allowing customization of the test runner: def pretest(...): [...] def posttest(...): [...] if __name__ == '__main__': zope.test.testrunner.main(pretest=pretest,posttest=posttest) Other policy could also be configured - e.g. 'Run these tests or tests using this resource first. If any failures, don't bother running any more'. Or 'Stop running tests after 1 failure'. These sorts of policies are important for us as we run our tests in an automated environment (we can't commit to our trunk. Instead, we send a request to a daemon which runs the test suite and commits on our behalf if the tests all pass). Our full test suite currently takes 45 minutes to run and it is becoming an issue. We need to speed them up, determine slow tests in need of pruning or optimization, short circuit test runs and reduce test suite maintenance. So I should be able to get time to help (although I need to look closer at the SchoolTool and py.test runners to see if they are closer to what we need). -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ signature.asc Description: OpenPGP digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Extending Zope's ZConfig Schema in a Product?
Sidnei da Silva wrote: > Right, seems that's my only choice for the time being. > We currently do this in our 'start Z3' script before any of the Zope imports are done: # Disgusting hack to use our extended config file schema rather than the # Z3 one. TODO: Add command line options or other to Z3 to enable # overriding this -- StuartBishop 20050406 from zdaemon.zdoptions import ZDOptions ZDOptions.schemafile = the_path_to_our_schema_dot_xml You can then write a schema.xml that inherits from the Z3 one: We are trying to centralize all our knobs and buttons into one place - ZCML isn't suitable for tweaking an application being rolled out to multiple servers (dev boxes, staging servers, production server instances) because there is too much boilerplate and syntax around the knob you need to tweak, and way too complex for a drunk sysadmin to handle. eg. it is much easier to have 'smtp on' in launchpad.conf than it is to configure the correct IMailer directives and whatnot with he correct magic names so that the application sends email via SMTP rather than sendmail. So far, ZConfig has been way overkill for this but allows us to keep our config in the same config as the port numbers and logfile destinations (without having to generate it from some other source, which we will probably do in the future to solve some production rollout issues). -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ signature.asc Description: OpenPGP digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] PageTemplates Unicode to trunk (?)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I'd like to get feedback and/or approval about a branch before I merge it to the trunk. svn+ssh://svn.zope.org/repos/main/Zope/branches/stub-unicode-zpt Differences from trunk are: - File upload box grows a charset field to specify the character set of the file being uploaded - pt_upload converts an uploaded file to Unicode if the charset has been passed. - "assert type(text) is type('')" removed from write() if management_page_charset is set: - pt_setTitle sets the title to a Unicode string if it isn't a valid ASCII string - pt_edit sets the contents to a Unicode string if it isn't a valid ASCII string - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAxZEOAfqZj7rGN0oRAifPAJ4wjUEw97vtLFSKEW1SDOhHY5ZRtwCeKi2w 62m9rP7LbttL634EOL7FWZM= =Jihw -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [patch] More secure cookie crumbler?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/04/2004, at 1:40 PM, Shane Hathaway wrote: On 04/12/04 09:04, Chris Withers wrote: For me, that's worth patching for, it's up to you if you want to include it in an offical CookieCrumbler release or not ;-) BTW, I wouldn't mind if you or Stuart took over maintainership of CookieCrumbler after the next release. Then you'd be able to take it any direction you want. I don't believe its model can support well the things you're asking it to do, but I'll happily give you the chance to prove me wrong. :-) I wouldn't be taking it in any direction - I like the product the way it is (Now I have checked in my code :-) ). Give me a poke if you don't feel like applying bug fixes though. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAgLQXAfqZj7rGN0oRAtRqAJ92YgRN62v9S/NLKrihGZguNnQF9gCgmyg4 EGTx2vtEJ7EKLlFjMEBWe8s= =uEZG -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [patch] More secure cookie crumbler?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/04/2004, at 4:28 PM, Kapil Thangavelu wrote: fwiw, Simon Eisenmann checked in a SessionStorage product into the collective which does much the same. released under the zpl http://cvs.sourceforge.net/viewcvs.py/collective/SessionCrumbler/ Looks pretty similar. It looks like it was written against the CMF's Cookie Crumbler rather than the standalone - I don't know if these two implementations have diverged much or not. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAgLKtAfqZj7rGN0oRApVJAJ93KDOQZ2qV9v8gDv5adu5ITDu8rgCfa/0R ZRfH3ojpD4qQjx/XQ3B3wrM= =AtNX -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [patch] More secure cookie crumbler?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/04/2004, at 3:34 PM, Tres Seaver wrote: I haven't looked at the code. Do you have actual experience using core sessions over ZEO? I pondered that recently for a client, and fell back to using a hacked together version of Anthony Baxter's SQLSession product, instead. No experience unfortunately, although I do use sessions stored in the ZODB rather than temporary storage. I would consider it a bug if it didn't work :-) Performance may be an issue depending on how you use your SESSIONs of course which I suspect would be the major issue with doing this. SessionStorage would work either as a separate product, or as a knob for the CookieCrumbler, I think. If ZPL is an adequate license, why don't you check it in there? I've checked a version into Products/CookieCrumber and added a note to the CHANGES.txt - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAgK/jAfqZj7rGN0oRAuvLAJ0dc8+B6NuTnIbUQWOV2OqCQYCfJQCeNHJm t6lrGtXwIkmhRr+O9VPrHuQ= =f7dh -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [patch] More secure cookie crumbler?
On 12/04/2004, at 10:39 PM, Shane Hathaway wrote: On Mon, 12 Apr 2004, Chris Withers wrote: I think the attached patch (against CookieCrumbler 1.1) makes CookieCrumbler a little more secure. Your patch won't work with multiple ZEO app servers. It appears to store the tokens in a module global. Do not apply it. I've attached some similar code we are using. Instead of patching CookieCrumbler, it extends it and is drop-in compatible. We are stuffing the auth credentials into the SESSION, so it will work with ZEO if your SESSION machinery copes (either using server affinity or your session storage is mounted by the ZEO clients from a central storage). I was going to pack this up and release it as a product under BSD or MIT licence, but I either forgot or came up with a technical reason not to. Either way, I'm having memory issues :-) SessionCookieCrumbler.py Description: Binary data Does this look worth releasing as a separate product? PS: To make cookie auth properly secure, you really need to be working over SSL only I agree--SSL is required. Let's not give people a false sense of security by changing CookieCrumbler. Unfortunately it causes performance to blow. We compromise by having the auth form on the SSL server, but the rest of the application on raw HTTP. This at least reduces the window that a replay attack can be used. It would be possible to tie the auth credential down to a particular IP address, but that is entering the world of diminishing returns and incompatibilities (think ISP's with farms of proxies - is this still a problem nowadays?). -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ PGP.sig Description: This is a digitally signed message part ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Fwd: [ZPT] Making PageTemplate's edit pages Unicode aware
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I havn't received much feedback on the ZPT mailing list, so I thought I'd bring it over here to a wider audience (thread is at http://mail.zope.org/pipermail/zpt/2004-March/005218.html ). Begin forwarded message: From: Stuart Bishop <[EMAIL PROTECTED]> Date: 29 March 2004 6:13:06 PM To: Dieter Maurer <[EMAIL PROTECTED]> Cc: [EMAIL PROTECTED] Subject: Re: [ZPT] Makeing PageTemplate's edit pages Unicode aware On 27/03/2004, at 9:57 PM, Dieter Maurer wrote: Stuart Bishop wrote at 2004-3-25 12:27 +1100: Currently, if you enter non-ascii text into the title or contents fields on a PageTemplate's edit page, the data ends up stored as an encoded string (using management_page_charset, if it is set. Unknown encoding if it is not). This should be easy to fix using the foo:charset:ustring notation to have Zope convert the encoded strings to Unicode. However, the file upload feature is more problematic. Should the file upload try converting the file to Unicode from UTF-8 and raise an exception if this is not possible? I personally feel this is preferable to ending up with arbitrarily enncoded document source, with no idea of the character set used. I do not think that Zope should convert when it does not know the encoding. I am unaware that a missing "management_page_charset" can be interpreted as "UTF-8". If this were the case, converstion to unicode might be correct. By the way: the HTML specification says that uploaded files should come with a "content-type" declaration. In this case, the charset specified there (if any) should be used to determine the encoding. Yes - A missing management_page_charset should probably be interpreted as either US-ASCII or ISO-8859-1. US-ASCII is probably more correct, but I would guess that most browsers will be configured to use ISO-8859-1 as their default (and this might be specified in the HTML spec?) I guess using the charset type the browser tells us for file uploads means we can blame the browser. I don't know how this could be reliable (since text files themselves don't encode their character set unless they happen to be UTF-16 or have a BOM). I am wondering if having a file upload function is incompatible with a Unicode aware page templates product. If management_page_charset is not set, it is unknown what charset is being used. The only way of knowing the character set of data that has been submitted is to know the character set of the form that it was submitted from. All other mechanisms do not work due to incompatibilities in how the browsers work. Currently, if you create a page template that contains non-ASCII characters, any tal:content or tal:replace expressions that return Unicode will now raise a Unicode error. This can be demonstrated simply: My 2¢ My 2¢ These are the things I think need to be fixed in Zope's Page Templates implementation to make them Unicode aware. There may be more (?): - It should be possible for the actual page template source to be stored as a Unicode string. Currently, there is an assert ensuring it is a traditional string. - The title property should be a Unicode string. - PageTemplateFile should grow an optional charset parameter, defaulting to US-ASCII. - PageTemplate.write(text) should raise an exception if text is not either a Unicode string or an ASCII string. - The ZopePageTemplate edit page should use Zope's :charset:ustring notation so Unicode strings get passed to its handler. - The file upload widget needs to either be removed, or grow a charset box. I don't think either of these solutions are ideal :-( Note that when I say 'Unicode string', we can still store ASCII text using a traditional string to save space. My application is currently using a ZopePageTemplate subclass that has been modified to use Unicode strings for the document source and title, and it seems to be functioning just fine. Does anyone know if that "assert type(text) == type('')" in PageTemplate.write is there for a reason? - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAdfPfAfqZj7rGN0oRAkBuAJ0WLSC3V2eL+zNzkQqBqjJ2bl5degCfe2SB DlT7NTsieQlDhVgEnHYaXp8= =6XPE -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] OFS.ObjectManager raising string exceptions?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In 2.7, ObjectManager raises 'Bad Request' most of the time, but in one case actually raises zExceptions.BadRequest. Did this get missed when all the string exceptions were getting replaced, or is it deliberately like this for backwards compatibility? - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAToYoAfqZj7rGN0oRAk20AJwI82ItBf4WqhCDwhQGVRFT/U8tDQCcCpjd X6XXkgrNEEGGWCiVKbBON5k= =MPo8 -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: 2.7rc1 - Unauthorized: You are not allowed to access '' in this context
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 21/01/2004, at 12:54 PM, Stuart Bishop wrote: I think I've tracked down a minimal example, the trigger being my use of __allow_access_to_unprotected_subobjects__ = None. I'm thinking this recent change is incompatible if a parent object tightens security in this way or uses security.setDefaultAccess('deny'). The work around is to explicity grant access to the name '' as I've done in the attached example. Should policy.validate(name='') be changed to cope with this situation, or shall I update CHANGES.txt and ClassSecurityInfo.setDefaultAccess attempting to explain the situation and the fix? I've opened this up as an issue on the collector: http://collector.zope.org/Zope/1202 I've so far seen one other report on the main Zope mailing list. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAEb/5AfqZj7rGN0oRAt/IAKCMG8wXYX3niyJ24phTqHOIm21qyQCdEdSN /SxsrihaAm7Yn6pog+exdN0= =wjUk -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: 2.7rc1 - Unauthorized: You are not allowed to access '' in this context
On 21/01/2004, at 2:34 AM, Tres Seaver wrote: Zope 2.6.3 added a new security check for untrusted code, to ensure that the "bindings" created (in particular, 'context' and 'container') weren't set up if the user didn't have access to the bound objects. You can either: - On the template's "Bindings" tab, unbind the 'context' name (assuming that your template does not use either 'context' or 'here') - Give the template a proxy role of 'Manager'. Don't suppose you can be more specific on 'has access'. According to my security tab, my container has both View and Access Contents Information granted to Authenticated. Somewhere, I'm losing authorization where in 2.7b3 I wasn't. I think I've tracked down a minimal example, the trigger being my use of __allow_access_to_unprotected_subobjects__ = None. I'm thinking this recent change is incompatible if a parent object tightens security in this way or uses security.setDefaultAccess('deny'). The work around is to explicity grant access to the name '' as I've done in the attached example. Should policy.validate(name='') be changed to cope with this situation, or shall I update CHANGES.txt and ClassSecurityInfo.setDefaultAccess attempting to explain the situation and the fix? __init__.py Description: Binary data -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ PGP.sig Description: This is a digitally signed message part ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] 2.7rc1 - Unauthorized: You are not allowed to access '' in this context
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 In Shared.DC.Scripts.Bindings._getContext(self), there seems to be a new security check: getSecurityManager().validate(parent, container, '', self) This is now giving me the following traceback: Traceback (innermost last): Module ZPublisher.Publish, line 100, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 40, in call_object Module Products.CGPublisher.storage.Storage, line 911, in editPane Module Shared.DC.Scripts.Bindings, line 261, in __call__ Module Shared.DC.Scripts.Bindings, line 292, in _bindAndExec Module Products.PageTemplates.PageTemplateFile, line 106, in _exec Module Products.PageTemplates.PageTemplate, line 90, in pt_render - Module Products.PageTemplates.PageTemplateFile, line 74, in pt_getContext Module Shared.DC.Scripts.Bindings, line 224, in _getContext Module AccessControl.ImplPython, line 398, in validate Module AccessControl.ImplPython, line 263, in validate Unauthorized: You are not allowed to access '' in this context editPaneHelper is just a PageTemplateFile. Storage.editPane (Python - not Python Script) is calling it like: return self.editPaneHelper(**options) Can anyone give me a hint on tracking this down? I have so far been unable to write a minimal example that fails (they all work), so I'm unsure if this is a Zope problem or my problem. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFADPECAfqZj7rGN0oRAgUYAJsG+0Bx51I2ZBufS476v7mbOIjT7QCgmBD/ NfboK3y2rSJiT/eR55/ECP8= =ctqN -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Security audit introduced problem in PageTemplates/Expression.py
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 17/01/2004, at 10:34 AM, Jim Fulton wrote: I I'm pretty sure that I can redo the way we protect dictionaries and lists so that we can provide backward compatability. If I can do this, I will, because backward compatability *is* important, especially for bug-fix releases. This is done and checked into the Zope 2.7 branch (Zope-2_7-branch). Stuart, can you try this out and make sure that your application works as it did before? All appears to be working as before. If this is definitely deprecated, I'll note that in AccessControl.py. I don't have a problem with deprecating this feature if it makes the Zope code saner - I was only using it because it was there and did what I wanted. I don't particularly like the idea of this mechanism working for getattr access but not for getitem access. I've always tended to stick with using getitem over getattr, partly as a holdover from when it was incredibly painful to mix getattr overrides with ExtensionClass, and partly because you are less likely to recursively shoot yourself in the foot. Indeed - an argument could be made for deprecating getattr in favor of getitem, as the latter could make use of Unicode keys if Zope's traversal mechanisms were updated to cope. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAC2d1AfqZj7rGN0oRAjPWAJ0VHsN8Rptk21xf90EyXTk5abgWiACeKZXM l6yznxwTidlY2vooA9b+o0s= =xCpW -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Security audit introduced problem in PageTemplates/Expression.py
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 16/01/2004, at 9:23 AM, Jim Fulton wrote: Dieter Maurer wrote: Jim Fulton wrote at 2004-1-15 10:03 -0500: ... Right. The name attribute was intended for attribute-based access. IMO, it makes no sense to consider key values when doing security checks. I had thought we had more, but most of that logic is now in a ComputedAttribute __ac_local_roles__ and in our __getitem__ hooks. What we currently have is a BTree storing key -> value. Valid keys are defined by a schema. If that schema changes, we do not want to lose the information until we are sure it has been archived, but we no longer want it available. The sorts of things we *were* doing was allowing access to certain attributes if the currently authenticated user had required permissions on a related object. eg. A bag-of-metadata has an associated Publisher object, and the Publisher has Editors (implemented as a local role assigned in the Publisher). Access to bits of metadata would be calculated based on the bag's schema, which determined if each bit was private, public or shared. Shared was available to people with certain permissions on the bag, or to people with certain permissions on the associated Publisher (the Editors). Some metadata has calculated privacy settings (eg. EmailAddress is shared or public depending on the value of PrivateEmailAddress) We now have the situation that this is possible if bag-of-metadata is accessed via getattr (stored as attributes on the object), but not via getitem (stored anywhere else). I can also imagine BTrees keyed to userid (eg storing settings), where people can only access their branch or branches of people in their workgroup. BTW, telling me that an algorithm has changed doesn't constitute a use case. :) I know that algorithm has changed. I assert that we don't need the feature that the change broke. I am open to evidence to the contrary. Its probably a feature we don't *need*, but some of us happen to be using it, and have been using this documented feature since Zope 2.5. I can probably work around it (although it means the next milestone release next week will be on b3 instead of b4), but I'd assumed that if one person who is using the beta triggered this issue, there will be plenty more who may be upset when they try porting their apps to 2.7.0 release. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAB5kEAfqZj7rGN0oRAgGZAJ9gQe9xVX9pg/XdQKXpPVOruoD+/gCdG6vn V1SPuM5ZOpsmy+hpI94JGc4= =q473 -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Security audit introduced problem in PageTemplates/Expression.py
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/01/2004, at 4:19 PM, Stuart Bishop wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The 'security audit work for the 2.7 branch' commit on 8th Jan made the following change in PageTemplates/Expression.py: As well as in other locations such as ZopeGuards.py. I've opened http://collector.zope.org/Zope/1182 with some example code. Anyone know if None is being passed as the name in some locations? I don't think it would be helpful for me to go around reversing code changed by a security audit without some background. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFABgNqAfqZj7rGN0oRApeyAJ0Y4BzVbQfOdq2rpaH/m1e9cip/RACfUqzq i1nr0FrFG544SCKh7dReZVk= =4TUc -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Security audit introduced problem in PageTemplates/Expression.py
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The 'security audit work for the 2.7 branch' commit on 8th Jan made the following change in PageTemplates/Expression.py: *** *** 312,318 # Skip directly to item access o = object[name] # Check access to the item. ! if not validate(object, object, name, o): raise Unauthorized, name object = o continue - --- 307,313 # Skip directly to item access o = object[name] # Check access to the item. ! if not validate(object, object, None, o): raise Unauthorized, name object = o continue *** *** 367,373 raise else: # Check access to the item. ! if not validate(object, object, name, o): raise Unauthorized, name object = o - --- 362,368 raise else: # Check access to the item. ! if not validate(object, object, None, o): raise Unauthorized, name object = o This has the side effect of not passing the name attribute to my security assertion methods registered via ClassSecurityInfo.setDefaultAccess: class Foo(blah, blah, blah): security = ClassSecurityInfo() def _checkAccess(self, name, value): if name.startswith('CG'): return 1 return 0 security.setDefaultAccess(_checkAccess) def __getitem__(self, key): ''' Access via dictionary interface, with security provided via _checkAccess ''' return 'example' Reversing the changes to Expression.py seems to break lots of things (including SiteErrorLog), so I'm sure this is much more involved. Can anyone shed light onto what is going on? - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQFAA4AFAfqZj7rGN0oRArWMAJ96sb9wKkx9qqstiB+78cZ1LrtW8ACggNX8 +uCQkzQGvbgIzW8Sb4C9kAE= =7xyW -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks "refresh"
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/12/2003, at 11:20 AM, Chris McDonough wrote: On Fri, 2003-12-12 at 23:43, Stuart Bishop wrote: I've just clarified the issue in the collector. We have since found that manual product refresh is working, and it is just the automatic product refresh that gives this error. I don't mean to sound glib (really), but refresh has always caused more problems for me than it has solved, so when I see bug reports for refresh I'm always reminded of "Doctor, doctor, it hurts when I do this..." With ZEO, Zope typically restarts in under a second. I just do that and don't bother with refresh. Is there any commanding reason to use refresh other than startup/teardown time? Firstly, I've *never* had problems with refresh (as a user of) until Zope 2.7 (I only use it on my own products, so I must naturally code in a refresh friendly manner). I'm used to developing with autorefresh switched on for my products and it just *works*. If I make a change and don't see the expected behavior, I switch to the product refresh page to see where the syntax error is that caused the refresh to fail. I've occasionally also used manual refresh to reload products on production servers without nuking sessions, but apart from that it is startup/teardown time. This is a problem though because doc/INSTALL.txt tells us to: ./configure make instance Until following the steps in doc/INSTALL.txt fires up a single ZEO server bound to the loopback address and a single ZEO client (with authentication on), the vast majority of developer and production installations will be ZEO-less. This also makes ZEO-less installations the best tested and most stable environment, or at least people will assume so. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/29SpAfqZj7rGN0oRAjAtAJ9GZh1iPTlUS3pYh4EDfEvrz8ysDQCfY8V5 O2w5LfPybUiEelaTvQep6Qw= =NXRf -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] problem with TAL statements across multiple lines in zope 2.7b3
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/12/2003, at 3:34 AM, Gerry Kirk wrote: i didn't have this problem with zope 2.6.2. I have a TAL statement as follows: tal:define = "url python:test(item_typeinfo.getId() in use_folder_contents, item.absolute_url()+'/folder_contents', item.absolute_url()+'/'+action);" this statement works if it is all on one line, but i get a syntax error if it is spread across multiple lines. is this a bug, or a change in TAL design? Works for me. You aren't accidently splitting inside of a string are you? - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/2puKAfqZj7rGN0oRAs9IAJ0SudIu20xmevZrSIHTPu2Aug9cDwCgiRFw 7x6vS4ticldQQ6H+Q4dL8xU= =hBXD -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks "refresh"
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/12/2003, at 5:05 AM, Dieter Maurer wrote: Then, it is highly likely that the problems in Zope 2.7 has a different cause. I probably will start working with Zope 2.7 in 2 weeks. And as refresh is an essential functionality for me, I will make it working (should it not work). I've just clarified the issue in the collector. We have since found that manual product refresh is working, and it is just the automatic product refresh that gives this error. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/2pj2AfqZj7rGN0oRAqltAJwM0oaQN2zV+mnWuRNU2BpxJ7824ACdGRXq PN6CjW0fqAjAiwHQpOj4uKY= =aG8y -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks "refresh"
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/12/2003, at 5:50 AM, Dieter Maurer wrote: Stuart Bishop wrote at 2003-12-10 11:24 +1100: On 08/12/2003, at 12:15 AM, Dieter Maurer wrote: Playing with Zope Head (as of 2003-12-04) revealed problems with refresh. This is not just a 2.8 issue - the behavior is in the 2.7 betas (at least up to beta 2 - havn't tested autorefresh with beta 3 yet) as well: http://collector.zope.org/Zope/1010 Manually refreshing works fine, but automatic refreshes cause the ValueErrors. Then, try my patch... The patch is incompatible with the 2.7 branch - self._reader and ConnectionObjectReader don't seem to exist in 2.7 at all, so there is nothing to shuffle around. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/2OoZAfqZj7rGN0oRAnWhAJoDezgxOKwswrhZEvAWeYrXyLNZ2ACcCmqG TVUNPb7Ts0pndeNvEWgtHP0= =1/Bz -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Head (2.8) breaks "refresh"
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/12/2003, at 12:15 AM, Dieter Maurer wrote: Playing with Zope Head (as of 2003-12-04) revealed problems with refresh. This is not just a 2.8 issue - the behavior is in the 2.7 betas (at least up to beta 2 - havn't tested autorefresh with beta 3 yet) as well: http://collector.zope.org/Zope/1010 Manually refreshing works fine, but automatic refreshes cause the ValueErrors. - -- Stuart Bishop <[EMAIL PROTECTED]> http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/1mfNAfqZj7rGN0oRAj9PAJ4+EUiHT9uI48ne5fRbO7+Tfa2fSQCdETcl ZF0IU3quFDv6QGlnzRdxSz4= =HIIJ -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Unicode id's and utf8 url encoding
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 12/11/2003, at 1:41 AM, Toby Dickenson wrote: This is going to break every use equivalent to getattr(some_object, id). Its not obvious to me how this can be cleanly resolved in zope 2. Your right - this is the killer :-( I had naively assumed setattr(ob, u'\N{TRADEMARK SIGN}', val) would work. It will be quite possible to make ObjectManager.__getitem__ handle Unicode ids, but hacking ExtensionClass to accept Unicode ids is beyond my ken and I'm unsure if this is even possible. - -- Stuart Bishop <[EMAIL PROTECTED]> â http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/sgz8AfqZj7rGN0oRAmCSAJ9zZK9T7+dR22BuF+0PRo/hG2jEXwCgnHIa LhkA45RB4HP/GCujHiHONQc= =obky -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Unicode id's and utf8 url encoding
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi. I think the last thread on this issue was: http://aspn.activestate.com/ASPN/Mail/Message/zope-Dev/1843637 From my reading of RFC2396 and RFC2277, the character set encoding for URLs is UTF8. This is confirmed by http://www.w3.org/International/O-URL-code.html . Does this settle the issue on how to handle non-ascii strings in URLs? If so, is their anything stopping us allowing Unicode ids in Zope? This would involve patching OFS.ObjectManager.checkValidId to accept strings not in [0-9a-zA-Z\$\-_\.\+!\*'(),], and writing replacement urllib.quote and urllib.unquote methods for use by HTTPRequest.py. If this is deemed a sane plan, I'd like to try getting this into Zope 2.7. It would also be worth fixing Python's urllib.quote and urllib.unquote methods, but that is an issue for a seperate mailing list... - -- Stuart Bishop <[EMAIL PROTECTED]> â http://www.stuartbishop.net/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (Darwin) iD8DBQE/sFP1AfqZj7rGN0oRAhGGAJ9dGMyhxFhG6GFr877qNx5+GP/S3wCghcOy T6lXZz60sR1VJLWjK76BmVk= =rQ0I -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Z2.7 & ValueError: Cache values may only be in onecache
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Monday, August 4, 2003, at 03:05 PM, robert wrote: I have similar tracebacks sometimes when I try to reload a product. Restarting Zope allways helps. Robert Yup. It is the product reloads that are triggering it. I'll stick an issue in the Collector. http://collector.zope.org/Zope/1010 - -- Stuart Bishop <[EMAIL PROTECTED]> http://shangri-la.dropbear.id.au/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.2 (Darwin) iD8DBQE/L3Lph8iUz1x5geARAl3hAJ4mSxSwwEnsWza80cGFZrTlRgbQiQCfYenK 29V3OcXBMX72u2l+fA6T20U= =wQXg -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Z2.7 & ValueError: Cache values may only be in one cache
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I'm getting tracebacks like the following under Zope 2.7b1, but not in Zope 2.6b5. Can somebody please shed some light on what this error is trying to tell me, so I can work out if this is a Zope 2.7 bug or my own fault? Traceback (innermost last): Module ZPublisher.Publish, line 102, in publish Module Zope.App.startup, line 220, in commit Module ZODB.Transaction, line 233, in commit Module ZODB.Transaction, line 348, in _commit_objects Module ZODB.Connection, line 427, in commit - __traceback_info__: (('BTrees.IOBTree', 'IOBucket'), '\x00\x00\x00\x00\x00\x00\x00G', '') ValueError: Cache values may only be in one cache. The problem is intermittent. - -- Stuart Bishop <[EMAIL PROTECTED]> http://shangri-la.dropbear.id.au/ -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.2 (Darwin) iD8DBQE/Lc/Jh8iUz1x5geARAnZVAJ9HPijFIBoYgwCwjrIGFOp3VOppwgCfXhvv OZC/ySC+jywzZvrQgYjeYbY= =j1/n -END PGP SIGNATURE- ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.python.org/mailman/listinfo/zope-announce http://mail.python.org/mailman/listinfo/zope )
Re: [Zope-dev] How (in)secure is Zope?
On Thursday, March 13, 2003, at 11:54 AM, Christian Tismer wrote: Dear Zope community, please excuse my ignorance, but I am asked from time to time how secure or insecure Zope actually is, and I always have to say that I actually don't know. From a sysadmin's point of view, it is roughly equivalent to Apache with CGI or PHP. The major differences are: - Zope's authentication & authorization systems are implicit in everything you write. It is harder to write insecure code than in PHP or CGI. - Anyone with ability to create dynamic content (dtml, python, zpt) can DOS your server. - You usually need to run Apache in front of Zope, which adds an additional attack point. -- Stuart Bishop <[EMAIL PROTECTED]> http://shangri-la.dropbear.id.au/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] interaction with local FS
On Tuesday, January 28, 2003, at 01:24 AM, Mario Bianchi wrote: This thread probably belongs in the main zope mailing list. LocalFS will let you map a local directory into Zope. AdaptableStorage probably will too, but it will be harder to setup at the moment. No way to access 'someuser''s files with the uid of 'someuser'? To do this, you would need to run Zope (or any other multi user application for that matter, such as Apache) as a superuser. Which you really don't want to do. The only other possible way of doing this would be to start a new process for every request which changes its euid (using a SUID root wrapper), authenticates itself, and does what you want to the FileSystem. People generally gave up on this approach for web systems about a decade ago due to it being a performance nightmare, which is why nothing does this out of the box. A better approach can come from becoming more familiar with your OS's group membership features. Under many Unix variants, for example, you can set up a group 'zopedocs', and add your users as members of this group. Also add the user the Zope server runs as to this group. Set up the local directories so that they are group writable, and group ownership propagates to newly created files: mkdir ~/ZopeDocs chgrp zopedocs ~/ZopeDocs chmod g+rwxs ~/ZopeDocs Make sure files that get created in this folder are group readable (at a minimum) if created from the shell either through chmod or setting the umask. Set the Zope server's umask to what you want (probably 002 - group writable, world readable) so when a file is created via Zope, it is modifiable by members of the ZopeDocs group. If you need to make sure users can't modify or access each others documents, you need to do the same thing but with a different unique group for each user (and the Zope user a member of all of them). These steps can of course be scripted and even added to the account creation procedures to make things quicker or automatic, but you need to write the script as it embodies your own security policy which is probably different to everyone elses. -- Stuart Bishop <[EMAIL PROTECTED]> http://shangri-la.dropbear.id.au/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] restrictedTraverse
On Sunday, May 12, 2002, at 01:27 AM, Florent Guillaume wrote: > With an object path /A/B/C where C has a local role allowing a user to > view C but where B disallows acquisition of the View permission, the > publisher correctly allows the user to see C. > > However restrictedTraverse('/A/B/C') fails ("You are not allowed to > access B in this context"). This is because restrictedTraverse checks > the security (using "validate") at *every* step, and obviously the > user is not allowed to see B. Is there a reason for this ? Why not > simply validate only at the last step ? Note that it doesn't check for the View permission though - it checks for the 'Access contents information' permission. If this fails, it fails because the site manager has explicitly said that a group of users is not allowed to access any objects below this point. If restrictedTraverse did not behave live this, a site manager would need to check the permissions on every single object in a subtree if they needed to restrict access to it, as well as every single object created in that subtree in the future. The current behaviour means that someone with 'Manage permissions' cannot make an object world viewable if it is in an area that the site manager has designated as restricted. It means that an object can be created in a restricted area and it stays restricted, even if it defaults to world viewable (such as an object that defines __allow_access_to_unprotected_subobjects__=1, or objects dynamically created from external sources like the file system or an RDBMS). > I have the need to programatically access object protected in such a > way. The workaround I'm going to use in my code for now is to call > unrestrictedTraverse and validate() by hand the resulting object. But > I'm concerned that there may be a more profound security reason I'm > missing. You may end up exposing an object that was never meant to be exposed to the user, as previously it was relying on permissions set in a parent container? You may be better off by granting the 'Access contents information' on B or allowing it to be aquired. -- Stuart Bishop <[EMAIL PROTECTED]> http://shangri-la.dropbear.id.au/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )