Re: [Zope-dev] Security validation issue
Herman Geldenhuys wrote: I've written a Zope product that exposes a MenuItem. I add a menuItem in a Zope folder, and I have no difficulty accessing and editing it via the ZMI. I've written an xml-rpc-like protocol for Zope, that basically validates the security manually. What do you mean by manually? This code works for any other default Zope type, but not mine. Did I perhaps forgot a permission or something? Did you do security declarations for that method? I can access this fine via the ZMI, but when I validate it this way, python just starts cursing at me. Why are you doing you own validation? ;-) cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Security validation issue
Hi Herman Geldenhuys wrote: I've written a Zope product that exposes a MenuItem. I add a menuItem in a Zope folder, and I have no difficulty accessing and editing it via the ZMI. I've written an xml-rpc-like protocol for Zope, that basically validates the security manually. What do you mean by manually? By manually I mean that I have to do the validation myself. I have written a new protocol that plugs into the Zope application server. It's called OZE and I am about to release the source on sourceforge. Its an RPC-like protocol. But in a nutshell, I must do the security validation myself, because I bypass a few usual-Zope elements in the framework. I will gladly answer any other questions, but will this satisfy for now? H - Original Message - From: Chris Withers [EMAIL PROTECTED] To: Herman Geldenhuys [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Friday, January 30, 2004 10:48 AM Subject: Re: [Zope-dev] Security validation issue Herman Geldenhuys wrote: I've written a Zope product that exposes a MenuItem. I add a menuItem in a Zope folder, and I have no difficulty accessing and editing it via the ZMI. I've written an xml-rpc-like protocol for Zope, that basically validates the security manually. What do you mean by manually? This code works for any other default Zope type, but not mine. Did I perhaps forgot a permission or something? Did you do security declarations for that method? I can access this fine via the ZMI, but when I validate it this way, python just starts cursing at me. Why are you doing you own validation? ;-) cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] ZCatalog fast as admin, dog slow as anonymous and other users
Zopistas, My ZCatalog is fast as admin, but dog slow as anonymous and other users. Anyone had this same experience? Details: marjors: Zope 2.6.2 CMF 1.3 Plone 1.0.5 about 50,000 cataloged objects (dual xenon server, plenty of ram, RAID) User folder has 15k users in it, but admin is one of those users. i've factored it way down and a simple search with just 1 Event loaded, like: return context.portal_catalog(Type='Event') is instantaneous for the 'admin' user, but takes about 10 seconds to return 1 SINGLE object. (only 1 Event loaded) imagine the time it takes to search and display 1300 events (which is what's normally in there). ouch. i've rebuilt the indexes, and still no dice. Any help would be greatly appreciated. All my best, Jason Spisak ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog fast as admin, dog slow as anonymous and other users
Jason Spisak wrote: Zopistas, My ZCatalog is fast as admin, but dog slow as anonymous and other users. Anyone had this same experience? Details: This is certainly to do with queries which run against FieldIndexes only for anonymous users. This is something the CMF does (e.g. expires and effective dates only applied for anonymous). FieldIndexes are really bad for DateTimes. Delete your date-related FieldIndexes and replace them with DateIndexes. seb ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog fast as admin, dog slow as anonymous and other users
Seb, thanks! i'll give that a shot. i also thought of adding daterange indexes for the calendar portion or things. does that also garner any speed improvement? Seb Bacon wrote: Jason Spisak wrote: Zopistas, My ZCatalog is fast as admin, but dog slow as anonymous and other users. Anyone had this same experience? Details: This is certainly to do with queries which run against FieldIndexes only for anonymous users. This is something the CMF does (e.g. expires and effective dates only applied for anonymous). FieldIndexes are really bad for DateTimes. Delete your date-related FieldIndexes and replace them with DateIndexes. seb ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog fast as admin, dog slow as anonymous and other users
Seb, I've replaced the following date indexes that werer FieldIndexes with DateIndexes Date effective expires modified i'll let you know what kind of speed improvement I get. Jason Spisak wrote: Zopistas, My ZCatalog is fast as admin, but dog slow as anonymous and other users. Anyone had this same experience? Details: This is certainly to do with queries which run against FieldIndexes only for anonymous users. This is something the CMF does (e.g. expires and effective dates only applied for anonymous). FieldIndexes are really bad for DateTimes. Delete your date-related FieldIndexes and replace them with DateIndexes. seb Jason Spisak ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZCatalog fast as admin, dog slow as anonymous and other users
Jason Spisak wrote:
Zopistas,
My ZCatalog is fast as admin, but dog slow as anonymous and other users.
Anyone had this same experience? Details:
marjors:
Zope 2.6.2
CMF 1.3
Plone 1.0.5
about 50,000 cataloged objects (dual xenon server, plenty of ram, RAID)
User folder has 15k users in it, but admin is one of those users.
i've factored it way down and a simple search with just 1 Event loaded,
like:
return context.portal_catalog(Type='Event')
is instantaneous for the 'admin' user, but takes about 10 seconds to
return 1 SINGLE object. (only 1 Event loaded) imagine the time it
takes to search and display 1300 events (which is what's normally in
there). ouch. i've rebuilt the indexes, and still no dice.
Any help would be greatly appreciated.
This symptom probably has to do with the login in the CMF catalog which
filters results based on the effective - expiration dates, for anybody
without the View inactive content permission. I would guess that your
portal_catalog is *not* using a DateRangeIndex to filter such content,
but is still using the individual 'effective' and 'expires' indexes.
To fix this:
- Add a DateRangeIndex, 'effectiveRange', to your portal_catalog;
set its start attribute to 'effective' and its stop attribute to
'expires'.
- Patch CMFCore/CatalogTool.py using the attached patch file (made
against the released 1.3 version).
Tres.
--
===
Tres Seaver[EMAIL PROTECTED]
Zope Corporation Zope Dealers http://www.zope.com
Index: CMFCore/CatalogTool.py
===
RCS file: /cvs-repository/CMF/CMFCore/CatalogTool.py,v
retrieving revision 1.30.4.6
diff -c -r1.30.4.6 CatalogTool.py
*** CMFCore/CatalogTool.py 1 Aug 2002 19:07:55 - 1.30.4.6
--- CMFCore/CatalogTool.py 30 Jan 2004 18:16:37 -
***
*** 12,18
##
Basic portal catalog.
! $Id$
import os
--- 12,18
##
Basic portal catalog.
! $Id: CatalogTool.py,v 1.30.4.6 2002/08/01 19:07:55 tseaver Exp $
import os
***
*** 202,215
if not _checkPermission(
CMFCorePermissions.AccessInactivePortalContent, self ):
base = aq_base( self )
! now = DateTime()
! if hasattr( base, 'addIndex' ): # Zope 2.4 and above
! kw[ 'effective' ] = { 'query' : now, 'range' : 'max' }
! kw[ 'expires' ] = { 'query' : now, 'range' : 'min' }
! else: # Zope 2.3
! kw[ 'effective' ] = kw[ 'expires' ] = now
! kw[ 'effective_usage'] = 'range:max'
! kw[ 'expires_usage' ] = 'range:min'
return apply(ZCatalog.searchResults, (self, REQUEST), kw)
--- 202,208
if not _checkPermission(
CMFCorePermissions.AccessInactivePortalContent, self ):
base = aq_base( self )
! kw[ 'effectiveRange' ] = DateTime()
return apply(ZCatalog.searchResults, (self, REQUEST), kw)
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: ZCatalog fast as admin, dog slow as anonymous and other users
Tres,
You are right. I've also got that permission turned off to take
advantage of that part of the CMF workflow, which compounds the problem
since it's site wide. I'm building the individual DateIndexs now and
when that's done, I'll check the performance (for my own edification if
nothing else 8-). Then I'll add that DateRange index and patch the CMF
and check again.
Do you guys want metrics? If so, preferred tool/output method?
(although my guess is, you already know how spiffy this makes things 8-)
Tres Seaver wrote:
Jason Spisak wrote:
Zopistas,
My ZCatalog is fast as admin, but dog slow as anonymous and other
users. Anyone had this same experience? Details:
marjors:
Zope 2.6.2
CMF 1.3
Plone 1.0.5
about 50,000 cataloged objects (dual xenon server, plenty of ram, RAID)
User folder has 15k users in it, but admin is one of those users.
i've factored it way down and a simple search with just 1 Event
loaded, like:
return context.portal_catalog(Type='Event')
is instantaneous for the 'admin' user, but takes about 10 seconds to
return 1 SINGLE object. (only 1 Event loaded) imagine the time it
takes to search and display 1300 events (which is what's normally in
there). ouch. i've rebuilt the indexes, and still no dice.
Any help would be greatly appreciated.
This symptom probably has to do with the login in the CMF catalog which
filters results based on the effective - expiration dates, for anybody
without the View inactive content permission. I would guess that your
portal_catalog is *not* using a DateRangeIndex to filter such content,
but is still using the individual 'effective' and 'expires' indexes.
To fix this:
- Add a DateRangeIndex, 'effectiveRange', to your portal_catalog;
set its start attribute to 'effective' and its stop attribute to
'expires'.
- Patch CMFCore/CatalogTool.py using the attached patch file (made
against the released 1.3 version).
Tres.
Index: CMFCore/CatalogTool.py
===
RCS file: /cvs-repository/CMF/CMFCore/CatalogTool.py,v
retrieving revision 1.30.4.6
diff -c -r1.30.4.6 CatalogTool.py
*** CMFCore/CatalogTool.py 1 Aug 2002 19:07:55 - 1.30.4.6
--- CMFCore/CatalogTool.py 30 Jan 2004 18:16:37 -
***
*** 12,18
##
Basic portal catalog.
! $Id$
import os
--- 12,18
##
Basic portal catalog.
! $Id: CatalogTool.py,v 1.30.4.6 2002/08/01 19:07:55 tseaver Exp $
import os
***
*** 202,215
if not _checkPermission(
CMFCorePermissions.AccessInactivePortalContent, self ):
base = aq_base( self )
! now = DateTime()
! if hasattr( base, 'addIndex' ): # Zope 2.4 and above
! kw[ 'effective' ] = { 'query' : now, 'range' : 'max' }
! kw[ 'expires' ] = { 'query' : now, 'range' : 'min' }
! else: # Zope 2.3
! kw[ 'effective' ] = kw[ 'expires' ] = now
! kw[ 'effective_usage'] = 'range:max'
! kw[ 'expires_usage' ] = 'range:min'
return apply(ZCatalog.searchResults, (self, REQUEST), kw)
--- 202,208
if not _checkPermission(
CMFCorePermissions.AccessInactivePortalContent, self ):
base = aq_base( self )
! kw[ 'effectiveRange' ] = DateTime()
return apply(ZCatalog.searchResults, (self, REQUEST), kw)
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZCatalog fast as admin, dog slow as anonymous and other users
Jason Spisak wrote: You are right. I've also got that permission turned off to take advantage of that part of the CMF workflow, which compounds the problem since it's site wide. I'm building the individual DateIndexs now and when that's done, I'll check the performance (for my own edification if nothing else 8-). Then I'll add that DateRange index and patch the CMF and check again. Do you guys want metrics? If so, preferred tool/output method? (although my guess is, you already know how spiffy this makes things 8-) While *I* have a gut feel for what the result should be (using DateIndexes will ease most of your pain; using a DateRange index will blow your socks off), it would be good to confirm that I'm right (or not :) in public. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZCatalog fast as admin, dog slow as anonymous and other users
Consider me awaiting my socks to be blown off. 8-) Tres Seaver wrote: Jason Spisak wrote: You are right. I've also got that permission turned off to take advantage of that part of the CMF workflow, which compounds the problem since it's site wide. I'm building the individual DateIndexs now and when that's done, I'll check the performance (for my own edification if nothing else 8-). Then I'll add that DateRange index and patch the CMF and check again. Do you guys want metrics? If so, preferred tool/output method? (although my guess is, you already know how spiffy this makes things 8-) While *I* have a gut feel for what the result should be (using DateIndexes will ease most of your pain; using a DateRange index will blow your socks off), it would be good to confirm that I'm right (or not :) in public. Tres. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZCatalog fast as admin, dog slow as anonymous and other users
Tres and Seb, Got rid of all the date FieldIndexes and am reindexing the new DateIndexes one at a time from the indexes tab, but I've hit a snag. when I go to reindex 'expires' I get this error immediately: Site error This site encountered an error trying to fulfill your request. The errors were: Error Details Error Type OverflowError Error Value integer multiplication any thoughts on how to get past this? Jason Spisak wrote: You are right. I've also got that permission turned off to take advantage of that part of the CMF workflow, which compounds the problem since it's site wide. I'm building the individual DateIndexs now and when that's done, I'll check the performance (for my own edification if nothing else 8-). Then I'll add that DateRange index and patch the CMF and check again. Do you guys want metrics? If so, preferred tool/output method? (although my guess is, you already know how spiffy this makes things 8-) While *I* have a gut feel for what the result should be (using DateIndexes will ease most of your pain; using a DateRange index will blow your socks off), it would be good to confirm that I'm right (or not :) in public. Tres. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZCatalog fast as admin, dog slow as anonymous and other users
I didn't see this come through from the list, so I'm doing a quick resend. Tres and Seb, Got rid of all the date FieldIndexes and am reindexing the new DateIndexes one at a time from the indexes tab, but I've hit a snag. when I go to reindex 'expires' I get this error immediately: Site error This site encountered an error trying to fulfill your request. The errors were: Error Details Error Type OverflowError Error Value integer multiplication any thoughts on how to get past this? Jason Spisak wrote: You are right. I've also got that permission turned off to take advantage of that part of the CMF workflow, which compounds the problem since it's site wide. I'm building the individual DateIndexs now and when that's done, I'll check the performance (for my own edification if nothing else 8-). Then I'll add that DateRange index and patch the CMF and check again. Do you guys want metrics? If so, preferred tool/output method? (although my guess is, you already know how spiffy this makes things 8-) While *I* have a gut feel for what the result should be (using DateIndexes will ease most of your pain; using a DateRange index will blow your socks off), it would be good to confirm that I'm right (or not :) in public. Tres. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZCatalog fast as admin, dog slow as anonymous and other users
Jason Spisak wrote: Tres and Seb, Got rid of all the date FieldIndexes and am reindexing the new DateIndexes one at a time from the indexes tab, but I've hit a snag. when I go to reindex 'expires' I get this error immediately: Site error This site encountered an error trying to fulfill your request. The errors were: Error Details Error Type OverflowError Error Value integer multiplication Update lib/python/Products/PluginIndexesn/DateIndex/DateIndex.py to the current head of the 2.6 branch, e.g. from: As a quick workaround, you could patch CMFDefault/DublinCore.py:: 8 --- *** CMFDefault/DublinCore.py14 Nov 2002 06:48:20 - 1.19.4.3 --- CMFDefault/DublinCore.py30 Jan 2004 22:22:53 - *** *** 253,259 date = getattr( self, 'creation_date', None ) return date is None and self.__FLOOR_DATE or date ! __CEILING_DATE = DateTime( , 0 ) # never expires security.declarePublic( 'expires' ) def expires( self ): --- 253,259 date = getattr( self, 'creation_date', None ) return date is None and self.__FLOOR_DATE or date ! __CEILING_DATE = DateTime( 2037, 0 ) # never expires security.declarePublic( 'expires' ) def expires( self ): 8 --- The better fix would involve not returning the silly _CEILING_DATE at all, and then telling the index whether to treat 'None' as low or high. Tres. -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZCatalog fast as admin, dog slow as anonymous and other users
Thanks. I'll update the index, since it will eventually be in proper zope anyway, it'll just be a bit ahead it's time 8-) Just to throw another wrench in, when updating the 'start' and 'end' indexes I get this error now that they are DateIndexes: Site error This site encountered an error trying to fulfill your request. The errors were: Error Details Error Type IndexError Error Value string index out of range So I'm guessing something that parses for these dates is blank and it doesn't like that too much. Technically you should never have a blank start or end date, I guess. Tres Seaver wrote: Jason Spisak wrote: Tres and Seb, Got rid of all the date FieldIndexes and am reindexing the new DateIndexes one at a time from the indexes tab, but I've hit a snag. when I go to reindex 'expires' I get this error immediately: Site error This site encountered an error trying to fulfill your request. The errors were: Error Details Error Type OverflowError Error Value integer multiplication Update lib/python/Products/PluginIndexesn/DateIndex/DateIndex.py to the current head of the 2.6 branch, e.g. from: As a quick workaround, you could patch CMFDefault/DublinCore.py:: 8 --- *** CMFDefault/DublinCore.py14 Nov 2002 06:48:20 - 1.19.4.3 --- CMFDefault/DublinCore.py30 Jan 2004 22:22:53 - *** *** 253,259 date = getattr( self, 'creation_date', None ) return date is None and self.__FLOOR_DATE or date ! __CEILING_DATE = DateTime( , 0 ) # never expires security.declarePublic( 'expires' ) def expires( self ): --- 253,259 date = getattr( self, 'creation_date', None ) return date is None and self.__FLOOR_DATE or date ! __CEILING_DATE = DateTime( 2037, 0 ) # never expires security.declarePublic( 'expires' ) def expires( self ): 8 --- The better fix would involve not returning the silly _CEILING_DATE at all, and then telling the index whether to treat 'None' as low or high. Tres. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
