[Zope-dev] ZCatalog Indexes tab crawls...
Hi, Has anyone noticed that the ZCatalog Indexes tab crawls if you have loads of objects indexed. My guess is that some types of index take way too long to figure out how many objects are indexed. Anyone know which index types those could be? BTW, would anyone object if I removed that object count, since it's not often very useful... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] ZCatalog Indexes tab crawl reason confirmed
Chris Withers wrote: My guess is that some types of index take way too long to figure out how many objects are indexed. This was confirmed by commenting out: dtml-var numObjects missing=n/a ...in catalogIndexes.dtml BTW, would anyone object if I removed that object count, since it's not often very useful... so... would anyone mind? cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog Indexes tab crawl reason confirmed
so... would anyone mind? Well, I've often been interested to note the numbers. It gave me a feeling for which indexes are heavily used. Sure, I could figure this out without looking at this page, but the (lack of) speed hasn't bugged me .. -- Jean Jordaan http://www.upfrontsystems.co.za ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Access Rule
Wycinite cytrusyHello, Anybody knows how to restore acces to folder with bad written site access rule ? Versioning works fine for me, but is any simplier/alternative solution ? regards Hubert Muller ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog Indexes tab crawl reason confirmed
--On Donnerstag, 17. Juli 2003 12:26 Uhr +0200 Jean Jordaan [EMAIL PROTECTED] wrote: so... would anyone mind? Well, I've often been interested to note the numbers. It gave me a feeling for which indexes are heavily used. Sure, I could figure this out without looking at this page, but the (lack of) speed hasn't bugged me . The problem is caused by calling len() on the indexes btrees. Instead a counter implemented btree.Length should be used in the future. -aj ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog indexes tab - which Index Types are guilty?
--On Donnerstag, 17. Juli 2003 12:29 Uhr +0100 Chris Withers [EMAIL PROTECTED] wrote: The problem is caused by calling len() on the indexes btrees. Instead a counter implemented btree.Length should be used in the future. Which Index types are currently guilty of this? I think all except ZCTextIndex. How about re-naming the column to Number of Documents Indexed and making sure this is actually what the indexes return. ?? I have a feeling that not all index types actually return the number of objects indexed. Can anyone confirm this? TextIndex in an older version returned the number of indexed words but this is fixed at least since 2.6. -aj ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Access Rule
You should be able to put export SUPPRESS_ACCESSRULE=1 into your start script, to disable existing access rules. Stefan --On Donnerstag, 17. Juli 2003 12:46 +0200 Hubert Muller [EMAIL PROTECTED] wrote: Anybody knows how to restore acces to folder with bad written site access rule ? Versioning works fine for me, but is any simplier/alternative solution ? -- The time has come to start talking about whether the emperor is as well dressed as we are supposed to think he is. /Pete McBreen/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] New zope.org rollout and transition plan
Hi all - At long last, we are preparing to 'go live' with a new and improved zope.org. The new site will put an architecture in place that will make it easier to maintain and improve the site. We're planning to make the new site live as www.zope.org at the end of July. The current site will still be accessible as old.zope.org. We plan to keep the old site available for a reasonable amount of time to ensure that important content that may have been missed by the migration can be hand-moved, etc. We'll give plenty of notice before decommissioning the old instance. Q: What does this mean for zope.org members? Generally, all membership info and most content will be transitioned to the new site automatically. Due to the amount of content and the 'organic growth' of the current zope.org over time however, there may be some bits of content that are either missed or get broken in some subtle way during transition. If you maintain product releases, articles or other content on zope.org, it would be a good idea to check on it after the transition. If you find things missing, you should be able to find it on the old.zope.org site and add it manually to the new site. If you find something broken and you don't know why, let us know so we can fix it. Q: Will there be some kind of 'public beta' period? Not per se. Experience has shown that it is hard to drum up the critical mass of usage required to really uncover non-obvious issues using a separate 'beta' site. In addition, zope.org presents a challenge in that you really need a short and choreographed 'switchover' window due to the community- contribution nature of the site. Members generally don't want to contribute their content twice (once on a beta site and once on the 'real' site), and we also need to go live quickly after a final content migration to minimize the inconvenience for people who might contribute content in the window between final migration and the launch of the new site. Q: Will the transition be quick, painless and invisible? Ha ha - silly human. :) We know from experience during the last zope.org transition that there will a period of hand-fixing after the new site goes live. Leaving the old site accessible during the transition will help with any pain we experience. I'll send another note when we finalize the dates for the final content migration and go-live date. I want to keep the window between these down to a day if possible to minimize disruptions. Brian Lloyd[EMAIL PROTECTED] V.P. Engineering 540.361.1716 Zope Corporation http://www.zope.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Untrusted developers
Howdy, I work with a Zope deployment at a University. Each school has a folder within the Zope deploy where they have complete control. We allow each student, staff, and faculty member to have their own personal folder. One of the problems with this is that users can write a script which loops indefinitely. When a script gets stuck in a loop it bogs down the ZEO client running it until the system kills that python process. Usually this is because someone is developing something new, when it doesn't work they make a change and try it again. Eventually all of the ZEO Clients are hung and everything is slow (and Zope looks bad to the bosses because this didn't happen with apache.) What I would like to see is a timeout associated with code objects (Python Scripts, Page Templates) that is set to some small value like 10 seconds by default. If the script does not complete within the timeout Zope would raise an exception. The user could bump up the timeout if they are writing something time intensive on purpose, but they wouldn't kill the whole web server (and important web pages) during development. Has anything like this been considered previously? Is it something that would ever make it into a zope release if I was to work on a patch? Thank you, Brian Brinegar Engineering Computer Network Purdue University ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Untrusted developers
This would be a fairly difficult thing to do internal to Zope given the numerous ways people can write infinite loops. You might want to take a look at the Autolance product at http://www.zope.org/Members/mcdonc/Products/AutoLance. It wasn't written for this scenario but might work anyway. On Thu, 2003-07-17 at 12:10, Brian Brinegar wrote: Howdy, I work with a Zope deployment at a University. Each school has a folder within the Zope deploy where they have complete control. We allow each student, staff, and faculty member to have their own personal folder. One of the problems with this is that users can write a script which loops indefinitely. When a script gets stuck in a loop it bogs down the ZEO client running it until the system kills that python process. Usually this is because someone is developing something new, when it doesn't work they make a change and try it again. Eventually all of the ZEO Clients are hung and everything is slow (and Zope looks bad to the bosses because this didn't happen with apache.) What I would like to see is a timeout associated with code objects (Python Scripts, Page Templates) that is set to some small value like 10 seconds by default. If the script does not complete within the timeout Zope would raise an exception. The user could bump up the timeout if they are writing something time intensive on purpose, but they wouldn't kill the whole web server (and important web pages) during development. Has anything like this been considered previously? Is it something that would ever make it into a zope release if I was to work on a patch? Thank you, Brian Brinegar Engineering Computer Network Purdue University ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Untrusted developers
On Thu, Jul 17, 2003 at 11:10:45AM -0500, Brian Brinegar wrote: One of the problems with this is that users can write a script which loops indefinitely. When a script gets stuck in a loop it bogs down the ZEO client running it until the system kills that python process. We're talking about the process on the Zope, not ZEO server, right? Does the process continue even though the request for it has terminated? I think that I recall that it does. (I don't understand why it works that way.) Is this true for both FastCGI and regular (proxied) HTTP requests? Just being able to kill processes when their requests have been terminated would improve the situation dramatically. It would also allow termination policies to be implemented in the front-end server (Apache). This would not be as nice as the suggestion you made, but we could whip up a simple solution quickly. Can you programmatically determine when a process is associated with a terminated request or is it a fuzzy exercise (like watching top for awhile)? Do you use /Control_Panel/DebugInfo? When you find it, is it sufficient and safe to just kill it? --kyler ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Untrusted developers
Just being able to kill processes when their requests have been terminated would improve the situation dramatically. It would also allow termination policies to be implemented in the front-end server (Apache). This would not be as nice as the suggestion you made, but we could whip up a simple solution quickly. Can you programmatically determine when a process is associated with a terminated request or is it a fuzzy exercise (like watching top for awhile)? Do you use /Control_Panel/DebugInfo? When you find it, is it sufficient and safe to just kill it? --kyler I'm not sure if there is a way to know if the request has been terminated, but if you could killing the thread should work. I think Zope creates new threads until the maximum set in the start script is reached. If one was killed I would assume that Zope would just start up another one the next time it is needed until the maximum is again reached. This would greatly reduce the problem I agree. -Brian ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog Indexes tab crawls...
Chris Withers wrote at 2003-7-17 11:12 +0100: Has anyone noticed that the ZCatalog Indexes tab crawls if you have loads of objects indexed. My guess is that some types of index take way too long to figure out how many objects are indexed. Anyone know which index types those could be? The one that provide the correct number of indexed objects (rather than just the number of indexed terms). Because the same object can be indexed under several terms, determining the number of indexed objects requires to build the union of all the index values. This almost surely has quadratic (worst case) runtime characteristics. BTW, would anyone object if I removed that object count, since it's not often very useful... You probably should replace it with the size of the index (i.e. the number of index terms). Formerly, the index overview displayed this information but under a buggy # objects title. Someone fixed this for most indexes, they now show the number of objects but at a high price. I suggest to change the title to # index terms and revert for the indexes to the old behaviour. Others pointed out, that also the size determination for an index may be expensive. However, it is at most linear in the number (rather than quadratic) and all recently created indexes now use BTrees.Length to maintain their size (which gives constant time). Having a feeling how large an index is is valuable information. Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog Indexes tab crawls...
Actually I regard the current behavior as a feature. Using a stopwatch and a slide-rule I can estimate to within 100 objects, how many values are indexed in a catalog by measuring the time it takes to draw the indexes page. Please do not remove this most valued feature! -Casey On Thursday 17 July 2003 04:35 pm, Dieter Maurer wrote: Chris Withers wrote at 2003-7-17 11:12 +0100: Has anyone noticed that the ZCatalog Indexes tab crawls if you have loads of objects indexed. My guess is that some types of index take way too long to figure out how many objects are indexed. Anyone know which index types those could be? The one that provide the correct number of indexed objects (rather than just the number of indexed terms). Because the same object can be indexed under several terms, determining the number of indexed objects requires to build the union of all the index values. This almost surely has quadratic (worst case) runtime characteristics. BTW, would anyone object if I removed that object count, since it's not often very useful... You probably should replace it with the size of the index (i.e. the number of index terms). Formerly, the index overview displayed this information but under a buggy # objects title. Someone fixed this for most indexes, they now show the number of objects but at a high price. I suggest to change the title to # index terms and revert for the indexes to the old behaviour. Others pointed out, that also the size determination for an index may be expensive. However, it is at most linear in the number (rather than quadratic) and all recently created indexes now use BTrees.Length to maintain their size (which gives constant time). Having a feeling how large an index is is valuable information. Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog Indexes tab crawls...
--On Donnerstag, 17. Juli 2003 18:22 Uhr -0400 Casey Duncan [EMAIL PROTECTED] wrote: Actually I regard the current behavior as a feature. Using a stopwatch and a slide-rule I can estimate to within 100 objects, how many values are indexed in a catalog by measuring the time it takes to draw the indexes page. Please do not remove this most valued feature! I agree but the current implementation sux. Switching to a counter based solution would solve the problem. The only problem I see is to keep the code fully backward compatible. -aj ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZCatalog Indexes tab crawls...
Andreas Jung wrote I agree but the current implementation sux. Switching to a counter based solution would solve the problem. The only problem I see is to keep the code fully backward compatible. if there's no counter present: create one, do a count of the docs, initialise the counter display counter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
