Hello Stefan, the insertion itself is fairly straightforward, but the repository structure might be not. I ran some profile tests to pinpoint where it was slow and the porblem seems to be on ItemImpl.initVersionHistories. It happens probably because, for every node inserted, i have to insert some subnodes under it, some of them versionable, so for every session.save() call, it seems that createVersion() is invoked many times.
I am copying here my nodetypes.cnd. Also, since i get the data from a DB, to provide a test case is not really simple, but it is only a loop that creates the nodes and subnodes (with no documents attached) and, every 250 ctx:registro (the type of node i am inserting) nodes, the session is saved. The node ctx:registro is inserted under ctx:registros, which is under ctx:area For such a structure, should i expect the performance to be so slow? It also doesn't seem to me that, at least for 300k nodes, the performance degrades so greatly (about 30% to 40% slower), but should i expect this to happen? Thanks ndoetypes <?xml version="1.0" encoding="UTF-8" standalone="no"?> <nodeTypes xmlns:ctx="http://www.murah.com.br/contexpress" xmlns:fn="http://www.w3.org/2005/xpath-functions" xmlns:fn_old="http://www.w3.org/2004/10/xpath-functions" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:mix="http://www.jcp.org/jcr/mix/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0" xmlns:rep="internal" xmlns:sv="http://www.jcp.org/jcr/sv/1.0" xmlns:xs="http://www.w3.org/2001/XMLSchema"> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:permissoesRegistro" primaryItemName=""> <supertypes> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="IGNORE" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="ctx:permissaoRemocaoGrupoUsuario" onParentVersion="IGNORE" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="ctx:permissaoAlteracaoGrupoUsuario" onParentVersion="IGNORE" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="ctx:permissaoPesquisaGrupoUsuario" onParentVersion="IGNORE" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="ctx:permissaoRemocaoUsuario" onParentVersion="IGNORE" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="ctx:permissaoAlteracaoUsuario" onParentVersion="IGNORE" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="ctx:permissaoPesquisaUsuario" onParentVersion="IGNORE" protected="false" requiredType="Long"/> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:documentos" primaryItemName=""> <supertypes> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="ctx:documento" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:documento</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:registros" primaryItemName=""> <supertypes> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:nivel</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:documento" primaryItemName=""> <supertypes> <supertype>mix:versionable</supertype> <supertype>nt:resource</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:algoritmoChecksum" onParentVersion="COPY" protected="false" requiredType="String"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:checksum" onParentVersion="COPY" protected="false" requiredType="String"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:texto" onParentVersion="COPY" protected="false" requiredType="String"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:idRegistro" onParentVersion="COPY" protected="false" requiredType="String"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:idArea" onParentVersion="COPY" protected="false" requiredType="String"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="true" name="ctx:documentoViolado" onParentVersion="IGNORE" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:documentoViolado</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="true" name="ctx:metaDocumento" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:metaDocumento</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:metaDocumento" primaryItemName=""> <supertypes> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:removido" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:areas" primaryItemName=""> <supertypes> <supertype>mix:referenceable</supertype> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="ctx:nome" onParentVersion="COPY" protected="false" requiredType="String"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="ctx:area" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:area</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:indicesRegistro" primaryItemName=""> <supertypes> <supertype>mix:referenceable</supertype> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="ctx:indiceRegistro" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:indice</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:indicesDocumento" primaryItemName=""> <supertypes> <supertype>mix:referenceable</supertype> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="ctx:indiceDocumento" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:indice</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:nivel" primaryItemName=""> <supertypes> <supertype>mix:referenceable</supertype> <supertype>nt:base</supertype> </supertypes> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:documentoViolado" primaryItemName=""> <supertypes> <supertype>mix:versionable</supertype> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="IGNORE" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:temDocumentoViolado" onParentVersion="IGNORE" protected="false" requiredType="Boolean"/> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:registro" primaryItemName=""> <supertypes> <supertype>nt:base</supertype> <supertype>mix:versionable</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:idUsuario" onParentVersion="COPY" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:idArea" onParentVersion="COPY" protected="false" requiredType="String"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="true" name="ctx:documentos" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:documentos</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="true" name="ctx:permissoesRegistro" onParentVersion="IGNORE" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:permissoesRegistro</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="true" name="ctx:metaRegistro" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:metaRegistro</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:indice" primaryItemName=""> <supertypes> <supertype>mix:referenceable</supertype> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="ctx:valores" onParentVersion="COPY" protected="false" requiredType="String"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="ctx:sequence" onParentVersion="COPY" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:idArea" onParentVersion="COPY" protected="false" requiredType="String"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:descricao" onParentVersion="COPY" protected="false" requiredType="String"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:identificador" onParentVersion="COPY" protected="false" requiredType="String"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:idTipoIndice" onParentVersion="COPY" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:preenchimentoHabilitado" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:preenchimentoObrigatorio" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:utilizadoParaBusca" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:utilizadoParaAssociacao" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:tamanho" onParentVersion="COPY" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:ordem" onParentVersion="COPY" protected="false" requiredType="Long"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:unico" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:alteravel" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:protegidoPeloSistema" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:metaRegistro" primaryItemName=""> <supertypes> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:removido" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> <nodeType hasOrderableChildNodes="false" isMixin="false" name="ctx:area" primaryItemName=""> <supertypes> <supertype>mix:referenceable</supertype> <supertype>nt:base</supertype> </supertypes> <propertyDefinition autoCreated="false" mandatory="false" multiple="false" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="false" multiple="true" name="*" onParentVersion="COPY" protected="false" requiredType="undefined"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:removido" onParentVersion="COPY" protected="false" requiredType="Boolean"/> <propertyDefinition autoCreated="false" mandatory="true" multiple="false" name="ctx:nome" onParentVersion="COPY" protected="false" requiredType="String"/> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="false" name="*" onParentVersion="COPY" protected="false" sameNameSiblings="true"> <requiredPrimaryTypes> <requiredPrimaryType>nt:base</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="true" name="ctx:registros" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:registros</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="true" name="ctx:indicesDocumento" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:indicesDocumento</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> <childNodeDefinition autoCreated="false" defaultPrimaryType="" mandatory="true" name="ctx:indicesRegistro" onParentVersion="COPY" protected="false" sameNameSiblings="false"> <requiredPrimaryTypes> <requiredPrimaryType>ctx:indicesRegistro</requiredPrimaryType> </requiredPrimaryTypes> </childNodeDefinition> </nodeType> </nodeTypes> Stefan Guggisberg-2 wrote: > > hi thiago > > On Mon, May 26, 2008 at 4:42 PM, Thiago Monteiro <[EMAIL PROTECTED]> > wrote: >> >> Hello Stefan, >> >> thanks for the quick reply. As for your questions: >> >> "what db are you targeting?" >> >> I am currently targeting MySQL, but the application must support other >> DBs, >> this is the reason the repository configuration is geared toward a more >> general approach. >> >> "how much memory have you allocated for the jvm?" >> >> Currently, a max of 1024mb is allocated for the jvm. I have tried to >> ensure >> that only the added nodes are in memory during the session.save() through >> cleaning the list of nodes after every call (and, thus, the end of the >> batch), but i don't really believe this has any impact, since no further >> changes are made and tehrefore registered as pending in the Session. >> >> are large binaries involved? >> >> No binaries are involved. The largest data for a single property is a 255 >> chars string, but even that is hard to come by. >> >> The repository configuration is given below > > the configuration seems to incomplete (e.g. missing PersistenceManager > config). seems to be a mailer issue, i am not sure. > > your test case seems to be trivial as far as i can tell. your save calls > should > take seconds rather than minutes. > > could you please test with jackrabbit's current default config > (embedded derby bundle pm) and report the results? this would help to > identify the bottleneck. > > could you post your test code as well, ideally a simple complete test > case? > > thanks > stefan > > >> >> <code><Repository> >> <FileSystem >> class="org.apache.jackrabbit.core.fs.local.LocalFileSystem"> >> >> </FileSystem> >> <Security appName="Jackrabbit"> >> <AccessManager >> >> class="org.apache.jackrabbit.core.security.SimpleAccessManager"> >> </AccessManager> >> >> <LoginModule >> >> class="org.apache.jackrabbit.core.security.SimpleLoginModule"> >> >> </LoginModule> >> </Security> >> <Workspaces rootPath="${rep.home}/workspaces" /> >> <Workspace name="${wsp.name}"> >> <FileSystem >> >> class="org.apache.jackrabbit.core.fs.local.LocalFileSystem"> >> >> </FileSystem> >> <PersistenceManager >> >> class="org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager"> >> >> >> >> </PersistenceManager> >> <SearchIndex >> >> class="org.apache.jackrabbit.core.query.lucene.SearchIndex"> >> >> >> >> >> >> >> >> </SearchIndex> >> </Workspace> >> >> >> <Versioning rootPath="${rep.home}/version"> >> <FileSystem >> >> class="org.apache.jackrabbit.core.fs.local.LocalFileSystem"> >> >> </FileSystem> >> >> <PersistenceManager >> >> class="org.apache.jackrabbit.core.persistence.bundle.BundleDbPersistenceManager"> >> >> >> >> </PersistenceManager> >> </Versioning> >> >> <SearchIndex >> class="org.apache.jackrabbit.core.query.lucene.SearchIndex"> >> >> </SearchIndex> >> >> <DataStore class="org.apache.jackrabbit.core.data.FileDataStore"> >> >> >> </DataStore> >> </Repository></code> >> >> >> -- >> View this message in context: >> http://www.nabble.com/session.save%28%29-takes-too-long-tp17471593p17473166.html >> Sent from the Jackrabbit - Users mailing list archive at Nabble.com. >> >> > > -- View this message in context: http://www.nabble.com/session.save%28%29-takes-too-long-tp17471593p17536269.html Sent from the Jackrabbit - Users mailing list archive at Nabble.com.
