On 26.06.2007, at 12:52, Sean Reifschneider wrote:
On Mon, Jun 25, 2007 at 11:44:07PM +0200, Laura Creighton wrote:Jodok Batlogg of Lovely Systems has already proposed donating a machineThe problem is that nobody has the 8-ish hours of time required to migratethe services to a new machine.The quick fix would be to engage XS4ALL to upgrade the RAM in that box, leaving the box otherwise untouched. The system has only 1GB of RAM in it. It's got a 2.8GHz Xeon CPU in it, so I would expect it can take at least4GB of RAM, if not 8 or 16GB.Thomas: If the PSF threw a grand or two at XS4ALL, could we get the memoryin ximinez upgraded? Preferably to 4 or 8GB of RAM?
very good idea. memory is cheap. that should solve the issues without a lot of work.
regarding the cheeseshop-software:it seems hard to find qualified persons that can improve the pypi software (for money). me and parts of my team will be at europython and have some time for sprinting on code improvements (baking static files,...) probably we can organize a cheeseshop sprint there?
jodok
Apache at 20MB each consumes all RAM. 20MB each for Apache processes is not unusual. Our web-site, which is fairly light-weight apache +mod_pythonFrom what I've seen, the problem really boils down to 50 instances ofuses around 20MB per Apache process resident, for example.The Apache processes consuming memory cause other problems. As memory is consumed, disc caching is reduced to nearly nothing, causing more data to have to be pulled from disc, causing fewer requests able to be processedper second. For example, right now I'm looking at the system and it'sspending half it's time waiting for the disc. This is after a restart ofApache. I think it's a little premature to call 20 to 40MB Apache processes"leaking". As I said, our web site is running 20MB Apache processes and it's not doing that much. One of our clients runs a fairly high traffic web site that is fairly dynamic, using mod_perl, and their Apache processes stabilize around 30MB each. Their problem is that they want to handle a concurrency of 200 or more, which adds up to 6GB right there, not countingthe database. Once system load starts going up above 1, it is probably just going to result in a death spiral because memory accesses are now taking milliseconds instead of tens of nanoseconds.I've taken the fairly drastic measure of pushing down the requests that Apache will handle before restarting from 1,000 to 10. Whether the problem is that something is leaking memory in the Apache processes, or that wejust don't have enough RAM for the Apache processes to stabilize atwhatever their natural limit is once they have everything they need loaded,this should help.We still have 60 concurrent clients allowed, so we're less vulnerable to running out of hooks due to slow clients or spiders with high concurrency.So far, things are looking good. I don't have much real-world time with it running this way, but I'm optimistic that this will help prevent the Apacherestart process from being required much if any. However, I'd really like to see this box get 4 to 8GB of RAM. This isabsolutely a place where the PSF can throw money at a problem to make it goaway.The donation of another machine to host pypi is very generous, but it's not something that we really have the manpower to be able to take advantage ofit seems.Anyway, the current config seems to be working well based on the baseline Itook and some other changes I played around with. By now the baseline would have been consuming 20MB per Apache instance. Thomas: Can you contact someone at XS4ALL about getting us a quote for upgrading the RAM in ximinez to 4 or 8GB? This will require the least amount of time from the community.Sorry I haven't been able to look at this in more depth sooner. Busy busy.Sean -- "I was on IRC once and got mistaken for Dan Bernstein. I still have nightmares." -- Donnie Barnes Sean Reifschneider, Member of Technical Staff <[EMAIL PROTECTED]>tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability_______________________________________________ Catalog-SIG mailing list [email protected] http://mail.python.org/mailman/listinfo/catalog-sig
-- "Flat is better than nested." -- The Zen of Python, by Tim Peters Jodok Batlogg, Lovely Systems Schmelzhütterstraße 26a, 6850 Dornbirn, Austria phone: +43 5572 908060, fax: +43 5572 908060-77
smime.p7s
Description: S/MIME cryptographic signature
_______________________________________________ Catalog-SIG mailing list [email protected] http://mail.python.org/mailman/listinfo/catalog-sig
