Tiry a écrit :
> Hi,
>
> If you want to do a mass import, please take a look at Nuxeo-Shell and
> the fsimport / mtfsimport commands.
> On an average server, document creation rate is between 40 and 60 docs/s.
>
> About your problem :
> If you do a loop 600 times that creates documents and only do the save
> at the end :
> - you do a big transaction that commits 600 documents
> => this commit can be very long and consume a lot of memory
> - your folders won't be visible until the save is reached
> => until the commit is done, nothing is visible to other sessions
> - when the session save is done, it triggers reindexing for the 600
> documents
> => this can be resources consuming too
>
> Change your code to do commit in batch :
> => you will have better performances and less resources used
> => you will be able to see your data while importing
>
> Also, during batch insertion, we usually deactivate indexing : it's
> faster to batch insert and then batch index.
>
> for example, if you want to disable fulltext async indexing :
> ...
> Map<String,Object> options = new HashMap<String, Object>();
> options.put("BLOCK_JMS_PRODUCING", true);
> String docType = "Folder";
> DocumentModel doc = session.createDocumentModel(docType, options);
> doc.setPathInfo(parent.getPathAsString(), name);
> doc.setProperty("dublincore", "title", file.getName());
> if (blockJMS) {
> doc.putContextData("BLOCK_JMS_PRODUCING", true);
> }
> if (blockSyncIndexing) {
> doc.putContextData("BLOCK_SYNC_INDEXING", true); // if you want to
> block sync indexing
> }
> doc = session.createDocument(doc);
> ...
>
> NB : If you block sync indexing, you won't be able to see your folders
> in the UI until you reindex since the default nav used search service ...
NB2: if you use "BLOCK_JMS_PRODUCING" that will prevent audit logging that uses
JMS notifications too.
--
Olivier
_______________________________________________
ECM mailing list
[email protected]
http://lists.nuxeo.com/mailman/listinfo/ecm
