Hi!
On Tue, Jul 02, 2002 at 10:17:37AM +0200, Per Einar Ellefsen wrote:
Thomas has said he wanted to do one in Austria (that's the country, right?).
Yes, that's what's behind .at
Rather small (~8 million people), between German and Italy, Switzerland and Hungaria, Czech & Slovak Republic...
That makes it one for Europe. I'm sure we can get more, for example I'm currently looking at sunsite.dk which seem to have hosted a mirror for the XEmacs site and others (they have the apache.org website), so that'd make for another one in Europe. Other continents might appear to us in the future.
If we have *total control*, is everyone ok with proceeding with this?
I'm planning to write two scripts
1) build-mirror:
This script will be run on each mirror via a cronjob. Frequency should be definitly not more than weekly, better dayly.
*) fetches the docs from cvs *) builds them with DocSet *) runs the search indexer (using SWISH)
Sites running this script will need DocSet, cvs, SWISH-E installed. This should be a requirement for all "official" mirrors
i actually have these crontabs already:
this one forces a complete rebuild, which is quite resource intensive because of the pdfs (needed when templates change)
/home/perlwww/bin/site_build_force: ----------------------------------- #!/bin/sh
umask 002
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/bin:/usr/X11R6/bin:$HOME/bin; export PATH
PERL5LIB=$HOME/lib/perl5/5.00503:$HOME/lib/perl5/site_perl/5.005:$HOME/lib/perl5/site_perl:$HOME/lib/perl5; export PERL5LIB
cd /home/perlwww/apache.org/modperl-docs cvs up -dP >/dev/null 2>&1 bin/build -df bin/makeindex
this one is similar but does only updates (suitable on doc updates)
/home/perlwww/bin/site_build_update: ------------------------------------- #!/bin/sh
umask 002
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/bin:/usr/X11R6/bin:$HOME/bin; export PATH
PERL5LIB=$HOME/lib/perl5/5.00503:$HOME/lib/perl5/site_perl/5.005:$HOME/lib/perl5/site_perl:$HOME/lib/perl5; export PERL5LIB
cd /home/perlwww/apache.org/modperl-docs cvs up -dP >/dev/null 2>&1 bin/build -d bin/makeindex
crontab -l # every monday rebuild all, including pdf 30 05 * * 1 /home/perlwww/bin/site_build_force # update all (only changes) every 6 hours 15 */6 * * * /home/perlwww/bin/site_build_update
2) test-mirrors
This script will be run on perl.apache.org via a cronjob. Should be in sync with the building there, maybe even in the same script
*) request /index.html from each mirror (LWP) *) check the timestamp *) if last-modified is recent enough (say, 1 week) add to mirrors
you mean the opposite, no? when not recent enough, remove from the mirrors list.
*) add each mirror to a (new) Mirrors-DocSet
What do you think?
there is a problem with any automatic modification, since they require auth to commit changes. currently the online site is checked out as an anonymous user: pserver:[EMAIL PROTECTED]:/home/cvspublic
another reason for not doing *any* changes on the apache.org site is to avoid colissions with automatic updates. if nobody changes things there, we can be sure that 'cvs up' will never render the documents broken.
I'd rather have this script maintain a black-list which we will manually apply to our list every so often.
what if the mirror updates the first page but fails to do the same for other pages?
do we require that the search works locally? and if so should we verify that the indices are up to date?
__________________________________________________________________ Stas Bekman JAm_pH ------> Just Another mod_perl Hacker http://stason.org/ mod_perl Guide ---> http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
