Re: [Zope-dev] Zope and Curl
There was some brief chatting about it a while ago on the list, do a search. seb On Sat, 2001-12-08 at 21:00, Dirk Datzert wrote: Hi, has any zoper played with the new internet technology curl from http://www.curl.com ? Regards, Dirk ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Is it possible to get REQUEST from dtml tag class.
Hi,
There is a bug in dtml-tree, the bug should be in file:
Zope-2.5.0b2-src/lib/python/TreeDisplay.py
we need to know BASEPATH1 around line 626 which is:
--
if exp:
ptreeData['tree-item-expanded']=1
output('A NAME=%s HREF=%s?%stree-c=%s#%s'
'IMG SRC=%s/p_/mi ALT=- BORDER=0/A' %
(id, root_url, param, s, id, script))
else:
output('A NAME=%s HREF=%s?%stree-e=%s#%s'
'IMG SRC=%s/p_/pl ALT=+ BORDER=0/A' %
(id, root_url, param, s, id, script))
output('/TD\n')
--
sorry I don't know how to fix this,
thank for your great work for zope.
Yuan-Chen Cheng
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] yes, segv11 and Broken pipes
Wasn't this the problem where asyncore.py is not catching the operating system's EINTR. I used to get these all the time, and was able to stop it using a modified asyncore which loops on an OS select() call, restarting the call if it catches EINTR from the OS ... I can send a modified asyncore.py for anyone who wants to give it a try?? Ziniti Jens Quade wrote: Dirk Datzert [EMAIL PROTECTED] writes: Its a Linux 2.2.19. What does IIRC means ? If I remember correctly. I could fix the problem (or a similar one) last summer by changing the Linux kernel. http://mailman.beehive.de/pipermail/zope/2001-June/000590.html http://mailman.beehive.de/pipermail/zope/2001-November/000923.html ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] 100k+ objects, or...Improving Performance of BTreeFolder...
I'm not sure if this is taken into consideration in your work so far/future plans... but just in case you were unaware, it is not necessary for you to persistently store objects in the ZODB that you intend to index in a ZCatalog. All that is required is that the object to be cataloged is accessible via a URL path. ZSQL methods can be set up to be URL-traversable, and to wrap a class around the returned row. To load the items into the catalog, you can use a PythonScript or similar to loop over a multi-row query, passing the objects directly to the catalog along with a path that matches the one they'll be retrievable from. This approach would eliminate the need for BTreeFolder altogether, although of course it requires access to the RDBMS for retrievals. This should reduce the number of writes and allow for bigger subtransactions in a given quantity of memory. At 07:36 PM 12/9/01 -0800, [EMAIL PROTECTED] wrote: Interesting FYI for those looking to support lots of cataloged objects in ZODB and Zope (Chris W., et al)... I'm working on a project to put ~350k Cataloged objects (customer database) in a single BTreeFolder-derived container; these objects are 'proxy' objects which each expose a single record in a relational dataset, and allow about 8 fields to be indexed (2 of which, TextIndexes). ... - Also, I want to make it clear that if I had a data access API that needed more than simple information about my datasets (i.e. I was trying to do reporting on patterns, like CRM-ish types of applications), I would likely wrap a function around indexes done in the RDB, not in Catalog. My requires no reporting functionality, and thus really needs no indexes, other than for finding a record for customer service purposes and account validation purposes. The reason, however, that I chose ZCatalog was for full text indexing that I could control/hack/customize easily. My slightly uninformed belief now is that for big datasets or enterprise applications (whatever that means), I would use a hybrid set of (faster) indexes using the RDB's indexes where appropriate (heavily queried fields), and ZCatalog for TextIndexes (convenient). I'm sure inevitable improvements to ZCatalog (there seems to be community interest in such) will help here. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope collector is broken
Okay it seems fixed, but I wanted to mention a similar user-experience
problem in CMF and workflows in general:
Suppose a user clicks publish in the action box, enters his comment
and then clicks the Publish button. And the re-clicks on the Publish
button (either because he double-clicked, or because he retried because
the process seemed slow). Then the first click will correctly change the
object state, and the second click will be unable to publish because
from the public state there is (obviously) no further publish
transition. So the user will get a message like the one below.
Because I saw that these double-clicks are somewhat frequent in our
case, I put a try: except: in content_status_modify.
Florent
Christopher N. Deckard [EMAIL PROTECTED] wrote:
I tried multiple times. I tried again this time, but deselected the
security related checkbox. It went in. The issue _is_ security
related, though not something that makes zope insecure.
Thanks,
-Chris
Ken Manheimer wrote:
I'm sorry, i'm unable to reprovoke the problem! Is there anything
unusual about how you were submitting? Did you try more than once, to
see if the problem is consistent?
On Fri, 07 Dec 2001 11:43:52 -0500, Chris Deckard wrote:
I tried submitting a new issue to the Zope collector at 11:42AM
EST. Here's the traceback and error message:
Zope Error
Zope has encountered an error while publishing this resource.
Error Type: WorkflowException Error Value: No workflow
provides the request action.
[...]
WorkflowException: (see above)
--
Ken
[EMAIL PROTECTED]
--
Christopher N. Deckard | Lead Web Systems Developer
[EMAIL PROTECTED]|Engineering Computer Network
http://www.ecn.purdue.edu/| Purdue University
zlib.decompress('x\234K\316Kq((-J)M\325KM)\005\000)\005w') ---
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
--
Florent Guillaume, Nuxeo (Paris, France)
+33 1 40 33 79 10 http://nuxeo.com mailto:[EMAIL PROTECTED]
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope collector is broken
Florent Guillaume wrote: Okay it seems fixed, but I wanted to mention a similar user-experience problem in CMF and workflows in general: Suppose a user clicks publish in the action box, enters his comment and then clicks the Publish button. And the re-clicks on the Publish button (either because he double-clicked, or because he retried because the process seemed slow). Then the first click will correctly change the object state, and the second click will be unable to publish because from the public state there is (obviously) no further publish transition. So the user will get a message like the one below. Because I saw that these double-clicks are somewhat frequent in our case, I put a try: except: in content_status_modify. For one of my applications (not CMF based), I use unique form ids to catch if a form is submitted twice. I also use a bit of javascript to ignore a second click on the submit button, which reduces the load on the form-ids manager. I'd post the javascript... but I can't find it just now. It is just a three-liner to set a variable when the button is first clicked, and only allow the submit action to run if the variable is unset. Actually, now that CST is in the Zope core, I should rewrite the form-id manager to take advantage of that. -- Steve Alexander ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] 100k+ objects, or...Improving Performance ofBTreeFolder...
Phillip J. Eby wrote: ZCatalog. All that is required is that the object to be cataloged is accessible via a URL path. Even that's not true... all you need to rpovide is an object and an identifier. I used to use integer ids for the identifier before ZCatalog decided to ban them :-( cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] 100k+ objects, or...Improving Performance of BTreeFolder...
Excellent analysis, many thanks Sean! This is much-needed info for people whom are attempting to scale. - Original Message - From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Sunday, December 09, 2001 10:36 PM Subject: [Zope-dev] 100k+ objects, or...Improving Performance of BTreeFolder... Interesting FYI for those looking to support lots of cataloged objects in ZODB and Zope (Chris W., et al)... I'm working on a project to put ~350k Cataloged objects (customer database) in a single BTreeFolder-derived container; these objects are 'proxy' objects which each expose a single record in a relational dataset, and allow about 8 fields to be indexed (2 of which, TextIndexes). Some informal stress tests using 100k+ _Cataloged_ objects in a BTreeFolder in Zope 2.3.3 on my PIII/500/256mb laptop are proving to be successful, but not without some stubborn investigation and a few caveats. BTreeFolder, using ObjectManager APIs, frankly, just won't scale for bulk-adds of objects to folders. I was adding CatalogAware objects to my folder (including index_object()). After waiting for bulk-add processes to finish after running for 2 days, I killed Zope and started trying to optimize, figuring that the problem was related to Catalog and my own RDB access code, and got nowhere (well, I tuned my app, bu this didn't solve my problem), then went to #zope, got a few ideas, and ended up with the conclusion that my problem was not Catalog-related, but related to BTreeFolder; I initially thought it was a problem with the C-Based generic BTree implementation scaling well past 10k objects, but felt I couldn't point the finger at that before some more basic stuff was ruled out. The easiest thing to do in this case, was to figure out what was heavily accessing the BTree via its dictionary-like interface, and the thought occurred to me that there might be multiple has_key checks, security stuff, and the like called by ObjectManager._setObject(), and I was right. I figured a switch to use the simple BasicBTreeFolder._setOb() for my stress tests might reveal an increase in speed, and... ...it works, acceptably, no less, on my slow laptop for 100,000 objects. It took ~50 minutes to do this on meager hardware with a 4200 RPM ide disk, and I figure a bulk add process like this on fast, new hardware (i.e. something with upwards of 22k pystones and lots of RAM) with a dedicated server for my RDB, would likely take 1/5th this time, or about 10 minutes (by increasing both MySQL performance, and Zope performance); combine this with ZEO and have a dedicated node do this, and I think this is a small amount of proof of Zope's ability to scale to many objects. (See my caveats at the bottom of this message, though). After days of frustration, I'm actually impressed by what I found: My data-access APIs are very computationally expensive, since they establish a MySQLdb cursor object for each call and execute a query; these data access methods used in bulk adding 100k objects after using _setOb() during Cataloging via index_object() (the transaction done all in memory for now, but likely moved to subtransactions soon to support up to 4x that data). So far, the moral of the story: use _setOb(), not _setObject() for this many objects! I haven't seen any material documenting anything like this for BTreeFolder, so I figured I would share with zope-dev what I found in the hopes that developers creating products with BTreeFolder and/or future implementations of BTreeFolder might take this into account, in docs, if nothing else. Caveats: - I'm using FileStorage and an old version of Zope (2.3.3). I can't say how this will perform with Python 2.1/Zope 2.[4/5]. I imagine that one would want to pack the storage between full rebuilds or have very, very fast storage hardware. - Catalog searches without any limiting queries to indexes will simply be too slow for practical use with this many objects, so they need to be forbidden with a permission to prevent accidental over-utilization of system resources or DOS-style attacks. Otherwise, Catalog searches on my slow hard drive seem acceptable. - I'm not too concerned with BTreeFolder __getattr__() performance penalties, though I modified BTreeFolder.__getattr__ just in case to remove the 'if tree and tree.has_key(name)', replacing with try/except; I'm not sure if this helps/hinders, because my stress-test code uses _getOb() instead. - objectIds() doesn't work; or, more accurately, at first glance, dtml-var _.len(objectIds()) doesn't work; I haven't tested anything else. I would like to find out why this is, and fix it. I suppose that there is something done in ObjectManager that BTreeFolder's simple _setOb() doesn't do. If anyone wants to help me figure out the obvious here, I'd appreciate it. ;) - I don't think un-indexed access of records is likely to be very practical with this many, esp. if things like objectIds()
Re: [Zope-dev] 100k+ objects, or...Improving Performance of BTreeFolder...
At 04:08 PM 12/10/01 +, Tony McDonald wrote: On 10/12/01 2:54 pm, Phillip J. Eby [EMAIL PROTECTED] wrote: I'm not sure if this is taken into consideration in your work so far/future plans... but just in case you were unaware, it is not necessary for you to persistently store objects in the ZODB that you intend to index in a ZCatalog. All that is required is that the object to be cataloged is accessible via a URL path. ZSQL methods can be set up to be URL-traversable, and to wrap a class around the returned row. To load the items into the catalog, you can use a PythonScript or similar to loop over a multi-row query, passing the objects directly to the catalog along with a path that matches the one they'll be retrievable from. This approach would eliminate the need for BTreeFolder altogether, although of course it requires access to the RDBMS for retrievals. This should reduce the number of writes and allow for bigger subtransactions in a given quantity of memory. Gad! - are you saying you don't need to store a 1Mb .doc file into the ZODB, but can still index the thing, store the index information in the Zcatalog (presumably a lot smaller than 1Mb) and have the actual file accessible from a file system URL? If so, that's really neat! Yep. By URL path, though, I meant a *Zope* path. However it would be straightforward to create a Zope object that represents a filesystem path and does traversal/retrieval, assuming that one of the 'FS'-products out there doesn't already do this for you. Chris Withers has pointed out that technically you don't even need the path string to be valid, it just has to be unique. However, the standard tools and the method for getting the real object referred to by the catalog record do expect it to be a valid path IIRC. I personally find it most convenient, therefore, to use a real Zope path. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] 100k+ objects, or...Improving Performance ofBTreeFolder...
Phillip J. Eby wrote: Gad! - are you saying you don't need to store a 1Mb .doc file into the ZODB, but can still index the thing, store the index information in the Zcatalog (presumably a lot smaller than 1Mb) and have the actual file accessible from a file system URL? If so, that's really neat! Yep. By URL path, though, I meant a *Zope* path. However it would be straightforward to create a Zope object that represents a filesystem path and does traversal/retrieval, assuming that one of the 'FS'-products out there doesn't already do this for you. LocalFS would cover Tone's needs... Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMCacheManager and gzip
Ok, that was my though, so I have uninstalled IE6, just incase the install was broken, and I will do some more testing tonight. I really need to put this to one side in favour of another caching problem I have been asked to look at though. I will probably pick it up again in a day or two. Adrian... -- Adrian Hungate All views expressed in this email are those of the whole world, however some people don't realise this yet. - Original Message - From: Martijn Jacobs [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Monday, December 10, 2001 2:42 PM Subject: Re: [Zope-dev] RAMCacheManager and gzip There is zlib support in Python, and it is used in Medusa already. ZServer does not call it, but the compression stuff is there. There is one good reason not to use zlib though, Micro$oft Internet Distorter does not support content-encoding: gzip and for content-encoding: compress it expects data to be compressed using the Unix compress tool... (I think someone reported this already... oh well) Hmm, how does mod_gzip compress it then? In every request I get from IE includes the accept-encoding: gzip string, and apache is sending it gzip-compressed back. Do you mean that the mod_gzip is compressing it different then zlib? mart(ijn) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Python Conference News
Python 10 Conference News Tim Berners-Lee, Director of the World Wide Web Consortium, and Andrew Koenig, Principal Research Staff Member at ATT Shannon Laboratory, will give the keynote addresses. Tim Berners-Lee will speak on Webizing Python and Andrew Koenig will discuss Notes from a Polygot Outsider. The entire program of the Refereed Paper Track is complete. You can view the titles and authors of all the presentations within each session and check out nominations for the Best Paper Award on the Python 10 Web site. The Tutorials Day program is also complete. Silver Sponsors of Python 10 are: Hostway Corporation: http://www.hostway.com and New Riders Publishing: http://www.newriders.com Registration for Python 10 is open. Save with early bird registration rates when you register online before January 7, 2002. To register and for conference details, visit: http://www.python10.org The Tenth International Python Conference February 4th through February 7th The Hilton Alexandria Mark Center Alexandria, Virginia ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RAMCacheManager and gzip
Ok, I stand corrected (and with egg on my face for abusing zlib). The moral of the story is that zlib.compress(data) is NOT the same as gzip data browser (Which I guess I should have known). Thanks for the pointer Jan! I think I have figured out what I did wrong, but I now have compression working for IE If anyone wants the patch to play with, drop me a line (It's a file to add to the PatchKit folder. PatchKit auto-loads all patches it finds - just restart Zope, or refresh PatchKit). Adrian... -- Adrian Hungate All views expressed in this email are those of the whole world, however some people don't realise this yet. - Original Message - From: Adrian Hungate [EMAIL PROTECTED] To: Martijn Jacobs [EMAIL PROTECTED]; [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Monday, December 10, 2001 6:13 PM Subject: Re: [Zope-dev] RAMCacheManager and gzip Ok, that was my though, so I have uninstalled IE6, just incase the install was broken, and I will do some more testing tonight. I really need to put this to one side in favour of another caching problem I have been asked to look at though. I will probably pick it up again in a day or two. Adrian... -- Adrian Hungate All views expressed in this email are those of the whole world, however some people don't realise this yet. - Original Message - From: Martijn Jacobs [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Monday, December 10, 2001 2:42 PM Subject: Re: [Zope-dev] RAMCacheManager and gzip There is zlib support in Python, and it is used in Medusa already. ZServer does not call it, but the compression stuff is there. There is one good reason not to use zlib though, Micro$oft Internet Distorter does not support content-encoding: gzip and for content-encoding: compress it expects data to be compressed using the Unix compress tool... (I think someone reported this already... oh well) Hmm, how does mod_gzip compress it then? In every request I get from IE includes the accept-encoding: gzip string, and apache is sending it gzip-compressed back. Do you mean that the mod_gzip is compressing it different then zlib? mart(ijn) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: My thoughts on the development process
I've taken a while to respond on this, because I wanted to talk it over with folks here, to think about the specifics of what different ideas would mean, etc. In summary, your last paragraph says: So let's trade in some risks to the Zope core development (rash action and messed up stuff happening once every while), in exchange for a lot more active contributors. We agree, and it's my job to get us there. Here's the next level of specificity: 1) Lighter weight process. Brian and I have talked about it, and we agree that we should investigate ways to lower the bar on ceremony, *without* abandoning organization. However, we both feel that, as you noted, the ultimate issues lie elsewhere. Thus... 2) Improve the tools. There's merit to the argument that the state of the fishbowl isn't very discoverable nor usable. Folks on zope-web know that we're proposing work that we'll do. Also, we are open to work that others will do. 3) Delegate leadership. This is, IMO, the most important thing. We need more people in the core, leading important areas and making decisions. For ZC, this really means that we have to trust the trustworthy and accept the paragraph of yours I quoted above. 4) Find the leadership. We can't sit passively by and say, We opened CVS and nobody's done anything. Somebody has to work specifically and repeatedly to bring people in, make sure they know what to do, and perhaps even prod them occassionally to get working. This takes publicity, promotion, cajoling, communications, and all kinds of other things to get the momentum going. I've signed up for the last part. In general, I'm sure everyone is pretty talked out on this subject and ready to see some action and results to go with all the voluminous emails. Me too. So let this general state of the problem discussion rest for a while, and let's work on some of the good ideas already discussed. --Paul Martijn Faassen wrote: Hi there, I've read parts of the open letter threads just now. There's a lot of talk about how if only we have better tools the whole process will go better and Zope will get more contributors. That's a typical hacker response, and I do this myself as well. Throwing more technology at a problem doesn't always make a problem go away. And though technological solutions to social problems are nice if you can have them, and we should look for them, they don't always work. I'm not convinced more technology will make the dead fish problem go away. I think the contributing process is in fact too heavyweight. It should be easier for people to get in drastic changes to Zope. The only way for people to take more responsibility if they can actually have it. Only a few people will take it, but that's more than what is possible now, with possibly the single exception of my taking responsibility for ParsedXML. And until recently I was still in the position of doubting whether I really had it formally, not just de-facto. I kept asking for approval and guidelines from the official maintainers, but they were too busy (no blame to them), so I went on anyway and did a release eventually. I dread having to go through the fishbowl to add in my 'node path' implementation to ParsedXML. I've done the design work, I've implemented most of it, and I feel I'd have mostly wasted time writing a fishbowl proposal. I hadn't even explored the problem enough to be able to do that. I needed to prototype it to understand it. I've discussed some issues with people locally and and on the Zope-XML mailing list. And I'll probably release a version in a few days. Perhaps adding Formulator to the Zope core would be nice eventually. But going through the fishbowl bureaucracy would take forever. I only have so much time to spend on it, and I'd rather spend time improving the product itself. And now look at how the Zope core is actually being developed. Sure, there's lots of stuff in the fishbowl about what the Zope future should be like. Plenty of stuff, though some stuff is rather hard to find. But I have a lot of praise for what the Zope Corp people have accomplished it it; it's a lot better than having no such thing at all, even if it's only used as a notification service in part. The main thinking about the directions of Zope is not done in the fishbowl or on the lists, it's in the minds of the talented people at Zope Corp and in the brainstorm sessions they hold together. That's the natural way people work. I work that way too. Such a process can occur on mailing lists as well, but it's very hard to break into it. I've tried several times. I'll keep trying as I'm convinced it's possible, but it takes a lot of persistence. Time will tell. On the Zope-XML list I just post regular updates about my thinking to encourage discussion, and sometimes that works. So what am I trying to get at with this mail? One thing is that the process is too heavy-weight right
EINTR ... was Re: [Zope-dev] browser closing connection
Also, for the record we usually get a bunch of these quite often: 2001-11-04T09:04:33 ERROR(200) ZServer uncaptured python exception, closing channel zhttp_channel connected XXX.XXX.XXX.XXX:2181 at fb4edc channel#: 2286 requests:4 (socket.error:(32, 'Broken pipe') [/usr/local/zope/dist/Zope-2.4.1/ZServer/medusa/asynchat.py|initiate_send|21 4] [/usr/local/zope/dist/Zope-2.4.1/ZServer/medusa/http_server.py|send|414] [/usr/local/zope/sw/Python2.1.1/lib/python2.1/asyncore.py|send|330]) We were seeing the same error (asyncore.py|send|330, etc) on solaris. For what its worth, I tracked this down in the sources and confirmed that in Zope 2.3, we shipped a modified asyncore.py with Medusa that handled EINTR, but in Zope 2.4 we used stock Python's asyncore which does NOT handle EINTR being returned from select(). IMHO, the distributed Python 2.1 asyncore behavior is incorrect. I've attached a diff of a portion of the differences (manually edited to take out other patches). I suspect this patch never got integrated due to ugliness of while 1 Also, the what should this be comment relates to NT's error numbers. Visual C++ has an errno.h that lists EINTR as 4 -- And winsock.h defines WSAEINTR as 10004 (ie add 10,000 to the errno). SO that number should be 10004, not 0 for correctness on Windows. --- /usr/local/python-2.1.1/lib/python2.1/asyncore.py Fri Nov 9 16:28:15 2001 +++ asyncore.py Sun Oct 1 11:58:56 2000 @@ -59,8 +39,10 @@ ECONNRESET = 10054 ENOTCONN= 10057 ESHUTDOWN = 10058 +EINTR = 0 # what should this be? else: -from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET, ENOTCONN, ESHUTDOWN +from errno import EALREADY, EINPROGRESS, EWOULDBLOCK, ECONNRESET +from errno import ENOTCONN, ESHUTDOWN, EINTR try: socket_map @@ -83,7 +65,13 @@ r.append (fd) if obj.writable(): w.append (fd) -r,w,e = select.select (r,w,e, timeout) + +while 1: +try: r,w,e = select.select (r,w,e, timeout) +except select.error, v: +if v[0] != EINTR: raise +else: break + if DEBUG: print r,w,e
[Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris
Hello.
We are facing zope restarts on the solaris 5.6 platform with zope
2.4.3 and python 2.1.1. I put together a script based some
information on an old posting to the apache mailing list. The
following shell/perl script allows one to get a core file from a dying
zope child process and also allow the zope to restart without any side
effects.
The script
#!/bin/sh
PATH=$PATH:/usr/local/bin
export PATH
cd /tmp
for PID in `ps -u zfs -f -o pid,comm,args | fgrep z2.py | cut -d' ' -f1`
do
export PID
truss -f -l -t\!all -S SIGSEGV,SIGILL -p $PID 21 \
| perl -pe 'system(gcore $ENV{'PID'} sleep 5 kill -9 $ENV{'PID'}),
exit($ENV{'PID'}) if /(SIGSEGV|SIGILL)/;'
done
Step 1: modify script to match your environment.
Step 2: execute script
Step 3: wait for core file to be dumped in /tmp.
Step 4: analyze with gdb where $PID is the pid of the dumped process
#bash gdb /path/to/bin/python /tmp/core.$PID
#0 0xef5b9810 in _lwp_sema_wait ()
(gdb) where
#0 0xef5b9810 in _lwp_sema_wait ()
#1 0xef647ea0 in _park ()
#2 0xef647b84 in _swtch ()
#3 0xef6468a4 in cond_wait ()
#4 0xef6467c8 in _ti_pthread_cond_wait ()
#5 0x50220 in PyThread_acquire_lock (lock=0xd9d878, waitflag=1)
at Python/thread_pthread.h:313
#6 0x51f18 in lock_PyThread_acquire_lock (self=0xda39b8, args=0x0)
at ./Modules/threadmodule.c:67
#7 0x35db4 in fast_cfunction (func=0xda39b8, pp_stack=0xed40f828,
na=0)
at Python/ceval.c:2994
#8 0x33ca0 in eval_code2 (co=0x267848, globals=0x51ec4, locals=0x0,
args=0x0,
argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
at Python/ceval.c:1951
:
:
It seems that we are facing trouble due to the thread library on
solaris (unless the truss command has introduced a side-effect).
Anyone else facing similiar troubles? or maybe I should post
this to a python mailing list.
- joe
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
