On 2012-01-06, Neck Acm wrote: > Hi all, I am new to sphinx, I am trying to convert massive plain text > files to well-organized html, not program documentation, just some > plain text record.
> The source text files is about 116 MB ... > eventually eats all my memory ( 1.5 GB ), returning MemoryError, abort > the build process This is gigantic. How many lines are this? > I've tried to build with less files( 9.8 M ), sucessfully create > beautiful html, In the docutils-users list http://docutils.sourceforge.net/docs/user/mailing-lists.html#docutils-users is a recent thread with exactly this problem: even with input files of about 4 Mb, compilation took half an hour. Investigation showed that the Docutils parser does not scale well - Docutils is simply not built for massive input files. As Sphinx uses Docutils for the document conversion, the problem should be the same here. > Is there any way to reduce memory usage in building * No easy way. You might try to fix some issues in the Docutils parser/writer but the developers currently have no ressources to deal with this. * The recommended way is to split the document into separate documents. Sphinx provides good support for inter-document links. > and the file size of output html ? * does the html size scale linear with the input file size? * you might consider converting to e.g. epub, which is basically zipped HTML. Günter -- You received this message because you are subscribed to the Google Groups "sphinx-dev" group. To post to this group, send email to sphinx-dev@googlegroups.com. To unsubscribe from this group, send email to sphinx-dev+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sphinx-dev?hl=en.
