Re: [Zope] zope2.2.0 and what is high load
On Fri, 11 Aug 2000, Chris McDonough wrote: i went to each Zeo client control_panel, and see that there's a mzximum of four connections open at any given time. comments to the above. i saw only four threads because i started that particular box with the default. on the box with -t25, i see 7 open connections at any given time. sorry, my mistake. That's about it for your problem. Maybe the profiling stuff can give more info, I'm not sure. i'll try and read more about that profiling. but not knowing what it does really scares me :) It shows that four threads have database connections. I'm curious as to why you're only seeing four threads being served by database connections too. as mentioned above, my bad. right after i sent the prev mail, i went to the other box with -t25, and it does have 7 open connections. You may want to ramp your number of threads down to 5 (-t5). 25 doesn't do much for you because only 7 threads can be using database connections at any given time. The high number of threads you're servicing might actually be slowing things down as they contend for available database connections. will do. and hope to see that it does something. Why are you frontending this with apache? how are you servicing two ZEO clients via a single apache? Does it do some sort of round-robining? Or does each box have its own apache? each box has its own apache. i'm round robining (straight dns round robin). ZEO is also a concern here because i'm using it blindly, unfortunately. but it works great if i got less than 150 simultaneous connections. and this symptom i saw prevuiously, even without ZEO. Is there any reason you're not just using straight ZServer? I'm wondering if apache processes contending to talk to available Zope python threads might be slowing stuff down? we need to serve some othe things from apache, that can't be done thru ZServer alone (a little php stuff) It doesn't seem as if bumping your number of database connections will do us any good here, because it doesn't seem you're using all 7. I'm not sure *why* you're not using all 7, because you've got at least 15 processes waiting around just on your top screen for DB connections. i did some grepping and saw the magical number 7 in ZODB/DB.py. is that the number to turn up if i wanted to try increasing the db pool connection? i do have postgresql with ZPygreSQL. will that have anything to do with what i'm facing? Maybe manage_debug isn't telling you the whole story, or maybe you caught it at a low-load period. I wonder if the profiling stuff would help here. Maybe someone else can jump in here, this is all I can think of at the moment. thanks for sparing some time helping me out. i love zope, and open source in general, and i can't bear having to switch to using something i don't really like. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] zope2.2.0 and what is high load
On Fri, 11 Aug 2000, Chris McDonough wrote: i went to each Zeo client control_panel, and see that there's a mzximum of four connections open at any given time. comments to the above. i saw only four threads because i started that particular box with the default. on the box with -t25, i see 7 open connections at any given time. sorry, my mistake. Oh, ok... that makes more sense. You may want to ramp your number of threads down to 5 (-t5). 25 doesn't do much for you because only 7 threads can be using database connections at any given time. The high number of threads you're servicing might actually be slowing things down as they contend for available database connections. will do. and hope to see that it does something. Based on your findings that all 7 DB connections are being used, I'd revise the number of threads to 8. It doesn't seem as if bumping your number of database connections will do us any good here, because it doesn't seem you're using all 7. I'm not sure *why* you're not using all 7, because you've got at least 15 processes waiting around just on your top screen for DB connections. i did some grepping and saw the magical number 7 in ZODB/DB.py. is that the number to turn up if i wanted to try increasing the db pool connection? i do have postgresql with ZPygreSQL. will that have anything to do with what i'm facing? Yes, it's the pool_size parameter. You can increment this value *carefully* if you see that you have more threads hanging around than DB connections, like you're seeing now. I have no idea what this will do to your relational stuff. The "right thing to do" is to test this outside of production. I don't want to tell you to go ahead and muck around on a production server. But if it were *me*, I might be tempted to: - First, lower the number of threads via -t8 and see what effect that has on speed. - If that doesn't solve anything, bump up both the NUMBER_OF_THREADS and the pool_size up to 10 or 12, then do some checking, and see if you still have threads waiting for a DB. Continue doing this in increments of 2 or 4, making notes as you go. Note that if it breaks your site, I'll deny I ever gave you this advice. :-) Proceed at your own risk. i love zope, and open source in general, and i can't bear having to switch to using something i don't really like. Geez, I hope not. This is an eminently solveable problem. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] zope2.2.0 and what is high load
Based on your findings that all 7 DB connections are being used, I'd revise the number of threads to 8. will give it a go this monday. Yes, it's the pool_size parameter. You can increment this value *carefully* if you see that you have more threads hanging around than DB connections, like you're seeing now. I have no idea what this will do to your relational stuff. ---that really scares me. sailing the uncharted see, and me a non swimmer! The "right thing to do" is to test this outside of production. I don't want to tell you to go ahead and muck around on a production server. But if it were *me*, I might be tempted to: --like you, i'm tempted too. i'll do backup first before anything else. furthermore, i can't get the amount of traffic to really test. - First, lower the number of threads via -t8 and see what effect that has on speed. - If that doesn't solve anything, bump up both the NUMBER_OF_THREADS and the pool_size up to 10 or 12, then do some checking, and see if you still have threads waiting for a DB. Continue doing this in increments of 2 or 4, making notes as you go. Note that if it breaks your site, I'll deny I ever gave you this advice. :-) Proceed at your own risk. ---thanks for the advice that's not given ;) , i'll report progress monday! Geez, I hope not. This is an eminently solveable problem. --i hope to solve this, and prove the greatness of OSS! ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] zope2.2.0 and what is high load
It sounds like your load may not be very small. Your -t25 setting to threads does not help too much, because the number of database connections is hard-limited to 7 in the ZODB source. It might help to visit the debug section of the control_panel to get an understanding of how many threads are actually being used at any given moment. If you find that all 7 threads are being continually used simultaneously, and your box can handle more threads (e.g. it's not CPU- or RAM-bound already), we can probably figure out how to bump you up to more database connection threads per client by changing the source. This should really be a knob somewhere. There is also a profiling facility inside Control_Panel/debug that does some sort of automated performance compilation for you, though I've never had a reason to use it. When it's running, it also purportedly slows Zope down quite a bit. Or, of course, you could always throw in another ZEO client if you're getting hit too hard to handle the load. I suggest that you: 1) Monitor your Zope clients' thread usages by using their debug facilities. 2) Monitor your Zope clients' general CPU and memory usages via vmstat or top or what-have you. Then report back your findings and we'll see if we can spot a bottleneck. -Original Message- From: bak @ kedai [mailto:[EMAIL PROTECTED]] Sent: Thursday, August 10, 2000 11:02 AM To: [EMAIL PROTECTED] Subject: [Zope] zope2.2.0 and what is high load oo zopistas i have a question. being a small time site maintainer, i don't quite know how to classify the situation i'm having now. the site is front ended with apache, with mod_pcgi2. zope 2.2.0 (running with -t 25) and rh 6.2 how can i know that the load i'm having is a big load? i've been getting more than 200 simultaneos connections(ps ax|grep http|wc -l) and that affect the response time. if the connection is less than 150, then zope's response time is great. my Q: what do you BIG SITE admin have? do you encounter slow down? i need some sort of verification that more than 200 simultaneous connect is really a big load. i hope not to have to resort to other costly and proprietery solutions just because of my ignorance. i know table design does affect loading time, and that will be tackled. cache is an option that i've looked at but have to disregard for now since i can't get ie to behave. i know of the how to and will be looking at it. i've scaled the site (was once more than 400 connections at one time) with ZEO to two servers. i now hope to have insights from yu BIG SITE admins looking forward to your input, help, and pointers. i'll try and document every thing and put a howto. thanks ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
