On 10.05.2012 17:42, Julian Gilbey wrote:
On Mon, Feb 13, 2012 at 01:23:49PM +0000, Adam D. Barratt wrote:
On 06.02.2012 20:58, Adam D. Barratt wrote:
>migration/testing.pl needs to be updated to make it react better
when
>called repeatedly, or to avoid the need to do so. Right now it's
>sufficiently resource hungry to make a 16-core machine unhappy.
>
>An initial idea might be to statically generate package pages,
>although
>this would need to be benchmarked for feasibility. testing.pl
could
>then simply become a redirect to /package/blah or similar.
Stupid question: is it running under mod_perl?
The server in question doesn't appear to have libapache2-mod-perl2
installed, so I'm going to say "no". ;-)
If so, surely there's
some way to cache the database between invocations so it doesn't have
to be regenerated on each call?
The per-package pages were already cached and it's that cache that
we're currently using, via mod_rewrite, to serve the pages that are
linked from services such as the PTS. There's no support for any of the
other querystring parameters right now, because we'd end up with
n-combinatorial versions of each file around.
In theory, yes, it could probably be rewritten to store the interesting
data somewhere and cross-reference that, rather than re-reading the
packages files all the time. As I mentioned in my mail opening this
report though, whatever is available needs to be sufficiently sane not
to make a machine of the size of ftp-master/release.d.o fall over when
comment spammers spider several thousand of the pages in a short time
and thus make DSA frown a good deal.
Regards,
Adam
--
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]
Archive:
http://lists.debian.org/[email protected]
