On 2013-01-16 15:41:00 +0000, Donald Stufft said:
On Wednesday, January 16, 2013 at 4:43 AM, Jannis Leidel wrote:
The mirror script was hanging on d, too. Since I've had to fix the
mirror every other month now I wonder if the pep381client is up to the
task. Does anyone have recommendations for fixing the situation? Is
there an easy way to monitor the client and kill it if it runs longer
than x?
Should be able to use the timeout command.
timeout -k HARD_LIMIT SOFT_LIMIT /path/pep381client/pep381run -q /var/pypi
I used to run this with "timeout 5m pep381run" but it seems that the
client really didn't like getting a SIGTERM.
Looking at the code there seems to be something special going on
regarding handling KeyboardInterrupt so I switched to using "timeout -2
5m …".
Still, I saw a weird problem and I'm suspicious that the client really
doesn't always manage to write a consistent status file when being
terminated.
For some reason I'm reinitializing my whole mirror after only a day.
I think I'm happy running Tarek's branch on the f mirror for now, but
I'm not super sure about that …
My proposal would be to sit down at PyCon to dig a bit more into the
code to make it more robust. My feeling is that the current client
tries way to hard on a very low-level API (httplib) for a lot of
mechanics. I'd be happy to refactor and provide tests, I think.
Also, a script to determine internal consistency and consistency
compared to PyPI would be nice. Then again, the rsync idea might not be
that far off regarding the amount of work and the problems we're
dealing with …
Christian
_______________________________________________
Catalog-SIG mailing list
Catalog-SIG@python.org
http://mail.python.org/mailman/listinfo/catalog-sig
