Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lenya Wiki" for change notification.
The following page has been changed by DominiqueDeMunck: http://wiki.apache.org/lenya/Large_number_of_documents_HOWTO ------------------------------------------------------------------------------ - * WORKING ON THIS RIGHT NOW * + * It is possible to include '''30 000+''' documents in a Lenya publication on even a modest server with 512MB of RAM. - It is possible to include 30 000+ documents in a Lenya publication. + * Yet some constructs make the default Lenya publication slow down a lot with that number of documents. - Yet some problems make Lenya slow down a lot with that number of - documents. I used following steps in my Lenya 1.2.4 installation to speed Lenya + * I used following steps in my Lenya 1.2.4 installation to speed up Lenya - enormously: + enormously: - 1) Java XML processing supports XML documents of 10's of thousands of nodes, - yet therefore the default heap space needs to be enlarged. - - In Linux/tomcat (jetty probably also) - set the environment variable : + 1. Java XML processing supports XML documents of 10's of thousands of nodes, yet [[BR]] therefore the '''default heap space''' needs to be enlarged. + + * In Linux/tomcat (jetty probably also) set the environment variable : [[BR]]JAVA_OPTS="-Xmx256m" + * In Windows/tomcat, there is a GUI tool called "tomcat5w" where you can [[BR]] set this option for the tomcat service (Maximum Memory Pool option) . - JAVA_OPTS="-Xmx256m" + 1. The main slow-down problem is that the whole sitetree is passed through 4 + navigation pipes (breadcrumb, menu, tabs and search). This is not really necessary, because generally, one + only wants to see the root folders, the current document, its parents and its direct descendants. + This is why navigation in "Site" mode goes fast: it uses the 'SiteTreeFragmentGenerator'. + A similar solution can be applied for the left navigation menu in authoring mode. + I created a Cocoon generator ('SelectiveSitetreeGenerator') which only generates the minimal tree to navigate + from the currently selected document (see Attachement SelectiveSitetreeGenerator.java at the end ). + 1. - - In Windows/tomcat, there is a GUI tool called "tomcat5w" where you can set - this option for the tomcat service (Maximum Memory Pool option) . + * You need to include the SelectiveSitetreeGenerator.java file into the right Lenya src folder: - 2) The main slow-down problem is that the whole sitetree is passed through a - number of navigation pipes. This is not really necessary, because generally, one only - wants to see the current document, its parents and its direct descendants. - This is why navigation in "Site" mode goes fast: it uses the SiteTreeFragmentGenerator. - A similar solution can be applied for the left navigation menu in authoring mode. - I created a Cocoon generator which only generates the minimal tree to navigate from the - currently selected document (see Attachement). - You need to include that file into the lenya folder: + {{{src/java/org/apache/lenya/cms/cocoon/generation/}}} + * Now we declare our generator in {{{src/webapp/sitemap.xmap}}}. Add to the generators list in front: + {{{ - src/java/org/apache/lenya/cms/cocoon/generation/ - - And then we will make that generator our menu generator. - - in src/webapp/sitemap.xmap add to the generators list: - - <map:generator name="sitetree-selective" label="content,data" logger="sitemap.generator.sitetree-selective" pool-grow="2" pool-max="16" pool-min="2" src="org.apache.lenya.cms.cocoon.generation.SelectiveSitetreeGenerator"/> + <map:generator name="sitetree-selective" label="content,data" logger="sitemap.generator.sitetree-selective" pool-grow="2" pool-max="16" pool-min="2" src="org.apache.lenya.cms.cocoon.generation.SelectiveSitetreeGenerator"/> + }}} + 1. + * Now we can reference it in our navigation xmap ({{{src/webapp/lenya/navigation.xmap}}}) - - and then we use it in our navigation: - - src/webapp/lenya/navigation.xmap - - replace our normal sitetree generator: + replace our normal sitetree generator: - + {{{ <map:generate type="sitetree"> - <map:parameter name="area" value="{2}"/> + <map:parameter name="area" value="{2}"/> </map:generate> - + }}} - with our selective one: + with our selective one: - + {{{ <map:generate type="sitetree-selective"> - <map:parameter name="area" value="{page-envelope:area}"/> + <map:parameter name="area" value="{page-envelope:area}"/> - <map:parameter name="documentid" value="{page-envelope:document-id}"/> + <map:parameter name="documentid" value="{page-envelope:document-id}"/> - </map:generate> + </map:generate> + }}} + 1. OK, we know only generate the sitetree that we need. + Still, the sitetreeselective generator takes on a Pentium 4 / 2.4 Ghz about 200ms to fetch the right + components out of the sitetree.xml (30 000 nodes). Unfortunatly, in the current sitemap, this fetch occurs 4 + times, which results in slowing down every click to a new document which + wasn't cached to about 1 second. This is because the "map:aggregate" construct does not cache the first + fetch of this sitetree-selective. The "cinclude" does, which speeds up the rendering of a page with about 600ms + for every page you visit (when sitetree was not cached after eg. move)! + Therefore, in your publicaton-sitemap.xml replace the part: + {{{ - - - 3) - - OK, we know only generate the sitetree that we need. - Still, the sitetreeselective generator takes on a Pentium 4 / 2.4 Ghz about 200ms to fetch the right - components out of the sitetree.xml (30 000 nodes). Unfortunatly, in the current sitemap, this fetch occurs 4 times - (breadcrumb, menu, tabs, search), which results in slowing down every click to a new document which - wasn't cached to about 1 second. This is because the "map:aggregate" construct does not cache the first - fetch of this sitetree-selective. The "cinclude" does, which speeds up the rendering of a page with about 600ms - for every page you visit (when sitetree was not cached after eg. move)! - Therefore, in your publicaton-sitemap.xml replace the part: - - <map:aggregate element="cmsbody"> + <map:aggregate element="cmsbody"> <map:part src="cocoon://navigation/{2}/{3}/breadcrumb/{5}.xml"/> <map:part src="cocoon://navigation/{2}/{3}/tabs/{5}.xml"/> <map:part src="cocoon://navigation/{2}/{3}/menu/{5}.xml"/> <map:part src="cocoon://navigation/{2}/{3}/search/{5}.xml"/> <map:part src="cocoon:/lenya-document-{1}/{3}/{4}/{page-envelope:document-path}"/> - </map:aggregate> + </map:aggregate> + }}} - - with: + with: - + {{{ <map:generate src="../../content/util/empty.xml" /> <map:transform src="xslt/custom/lenyaBodyCincludes.xsl"> @@ -91, +80 @@ </map:transform> <map:transform type="cinclude" /> - + }}} - You should add the attached XSL file lenyaBodyCincludes.xsl in "yourPUB/xslt/custom/" + You should add the attached XSL file lenyaBodyCincludes.xsl in "yourPUB/xslt/custom/" + Note that for this caching to take effect you should make sure that the sitetree-selective pipe (in navigation.xmap) is cached ! On a normal build this should be ok since the default pipe was defined to be caching (in sitemap.xmap). + + 1. Speeding up delete usecase. + the DocumentReferencesHelper looks through all the documents to find links to a document. If don't want to slow down the delete usecase (which looks for broken links in all documents !), + + * You should uncomment the part with the DocumentReferencesHelper statements in + + {{{src/webapp/lenya/content/info/delete.xsp}}} + + (insert comments after </task-id> and before </info> ) - Note that for this caching to take effect you should make sure that the sitetree-selective pipe (in navigation.xmap) is cached ! - On a normal build this should be ok since the default pipe was defined to be caching (in sitemap.xmap). - - 4) - - the DocumentReferencesHelper looks through all the documents to find links to a document. - If don't want to slow down the delete usecase (which looks for broken links in all documents !), - - You should uncomment the part with the DocumentReferencesHelper statements in - - src/webapp/lenya/content/info/delete.xsp - - (insert comments after </task-id> and before </info> ) - - - Other xsp's that use the reference helper and which you may want to disable (called via usecase.xmap). + Other xsp's that use the reference helper and which you may want to disable (called via usecase.xmap). - + {{{ src/webapp/lenya/content/info/archive.xsp src/webapp/lenya/content/info/deactivate.xsp src/webapp/lenya/content/info/delete.xsp src/webapp/lenya/content/publishing/referenced-documents.xsp src/webapp/lenya/content/publishing/screen.xsp src/webapp/lenya/content/scheduler/screen.xsp + }}} + + + [[AttachList]] + --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
