Bob wrote: > >Everything was going along fine, with daily updates being added to the >archive. I then wanted to refresh the entire archive and ran bin/arch >--wipe list1 > >it failed. > >Archive articles go from 0000 to 66990 > >list1.mbox is: 526313754 Aug 21 20:19 list1.mbox > >The server NetBSD 3.1, has 2GB memory > >bin/arch first went through every article > >Week-of-Mon-20070702 >#65136 <[EMAIL PROTECTED]> >figuring article archives >[...] > >then after many > >Updating HTML for article 4584 >Updating HTML for article 4585 >Updating HTML for article 4586 >Updating index files for archive [Week-of-Mon-20050321] > Date > Subject > Author > Thread >Computing threaded index >Updating HTML for article 4587 >Updating HTML for article 4588 >Updating HTML for article 4589 >Updating HTML for article 4590 >Updating HTML for article 4591 >Updating HTML for article 4592 >Updating HTML for article 4593 > >[...] > >Updating HTML for article 9353 >Updating index files for archive [Week-of-Mon-20041122] >Traceback (most recent call last): > File "bin/arch", line 200, in ? > main() > File "bin/arch", line 190, in main > archiver.close() > File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 322, >in close self.update_dirty_archives() > File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 538, >in update_dirty_archives self.update_archive(i) > File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperArch.py", line 1121, >in update_archive self.__super_update_archive(archive) > File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 439, >in update_archive self.__set_parameters(archive) > File "/usr/pkg/lib/mailman/Mailman/Archiver/pipermail.py", line 354, >in __set_parameters firstdate = self.database.firstdate(archive) > File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line >209, in firstdate self.__openIndices(archive) > File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line >240, in __openIndices self.__closeIndices() > File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line >265, in __closeIndices index.close() > File "/usr/pkg/lib/mailman/Mailman/Archiver/HyperDatabase.py", line >187, in close fp.write(marshal.dumps(self.dict)) >MemoryError >
This problem is mentioned in the description of the -e option of bin/arch --help. >I can successfully run about the last 10,000 > >bin/arch -s 50000 -e 66990 list1 > >But that is about 1/7th of the archive >Can I produce a complete archive in chunks of 10,000? How? bin/arch --wipe -e 99999 list1 bin/arch -s 10000 -e 19999 list1 bin/arch -s 20000 -e 29999 list1 ... bin/arch -s 60000 list1 Note that --wipe is specified only on the first command. Also, it is a good idea to stop Mailman during this process. The reason is that if a new message arrives and is archived say between the second and third commands, it will be assigned message number 20000 in the new archive, but it will be message number 66991 in the .mbox. This will guarantee that if you ever rebuild the archive again, all message numbers from 20000 on will change, thus invalidating any saved archive URLs. Finally, it is a good idea to first check the .mbox file with bin/cleanarch to make sure there are no unescaped From_ lines in message bodies. -- Mark Sapiro <[EMAIL PROTECTED]> The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan ------------------------------------------------------ Mailman-Users mailing list [email protected] http://mail.python.org/mailman/listinfo/mailman-users Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/ Unsubscribe: http://mail.python.org/mailman/options/mailman-users/archive%40jab.org Security Policy: http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.027.htp
