Olivier Grisel a écrit :
> Tiry a écrit :
>
>> Hi,
>>
>> If you want to do a mass import, please take a look at Nuxeo-Shell and
>> the fsimport / mtfsimport commands.
>> On an average server, document creation rate is between 40 and 60 docs/s.
>>
>> About your problem :
>> If you do a loop 600 times that creates documents and only do the save
>> at the end :
>> - you do a big transaction that commits 600 documents
>> => this commit can be very long and consume a lot of memory
>> - your folders won't be visible until the save is reached
>> => until the commit is done, nothing is visible to other sessions
>> - when the session save is done, it triggers reindexing for the 600
>> documents
>> => this can be resources consuming too
>>
>> Change your code to do commit in batch :
>> => you will have better performances and less resources used
>> => you will be able to see your data while importing
>>
>> Also, during batch insertion, we usually deactivate indexing : it's
>> faster to batch insert and then batch index.
>>
>> for example, if you want to disable fulltext async indexing :
>> ...
>> Map<String,Object> options = new HashMap<String, Object>();
>> options.put("BLOCK_JMS_PRODUCING", true);
>> String docType = "Folder";
>> DocumentModel doc = session.createDocumentModel(docType, options);
>> doc.setPathInfo(parent.getPathAsString(), name);
>> doc.setProperty("dublincore", "title", file.getName());
>> if (blockJMS) {
>> doc.putContextData("BLOCK_JMS_PRODUCING", true);
>> }
>> if (blockSyncIndexing) {
>> doc.putContextData("BLOCK_SYNC_INDEXING", true); // if you want to
>> block sync indexing
>> }
>> doc = session.createDocument(doc);
>> ...
>>
>> NB : If you block sync indexing, you won't be able to see your folders
>> in the UI until you reindex since the default nav used search service ...
>>
>
>
> NB2: if you use "BLOCK_JMS_PRODUCING" that will prevent audit logging that
> uses
> JMS notifications too.
>
True, that's the reason why a command named auditsync exists.
=> It recreates the audit logs for creation event (since this
information is also recorded in the dublincore)
_______________________________________________
ECM mailing list
[email protected]
http://lists.nuxeo.com/mailman/listinfo/ecm
