After a bit of further investigation, the machine runs out of ram. It has 512Mb (including virtual) available to it. Is there not a faster and less processor/memory intensive way of doing this as this makes the server with all our live sites so slow to respond it is almost unusable - the perl script hogs 98% of the processor and all the available ram.
Cheers Mike. -----Original Message----- From: Eero af Heurlin [mailto:[EMAIL PROTECTED] Sent: 23 April 2004 14:33 To: [EMAIL PROTECTED] Subject: Re: [midgard-user] Staging / Live Server Synchronisation Problem -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Michael Ross (PCT North West) wrote: | | I have another problem however. I really could do with resynchronising the | staging and live servers using repligard with the -a option to ensure that | the live server is up to date and that no previously unapproved articles | exist. Here in lies the problem. The xml file that repligard generates if | 156Mb in size (it has data from 16 local websites). The remove_unapproved | script crashes before it manages to parse such a huge xml file. Normal use | of the remove_unapproved script (without repligard -a) is fine. Any ideas on | how this can be fixed? | Crashes ? Are you sure, it will propably take several hours to parse such a file even on a fast machine, especially if it's got plenty of blobs (those are a real performance killer for the perl expat module), on one customer where we have a custom filter (which uses the same parsing logic) processing a 40MB xml file takes approx 5 hours on a 1ghz P3 (256mb memory). If the script, ie Perl itself or the Expat module, actually crashes then we need someone who knows more about them. /rambo -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.2-nr2 (Windows 2000) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFAiRsOi7iLtQbxbW8RAvIlAKCxhfTRF+KhvvwGR6+kjFFWDgJrxACeIbHM waq2kjDfQ+EAse7V/ULKkt4= =Q3MO -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
