Hi, here is the code that reproduces the stress tests I've made.
https://github.com/nrv/FedoraCommonsPerf You have to modify the connection parameters, the number of threads and choose wheter you want a single FedoraClient instance for all the threads or an instance per thread. Both configurations leads to the same conclusions. The test has two steps. First it purges all the fake documents, then it recreates them. Run it twice and take a look at the server CPU, network and I/O wait. I'm running Fedora Commons 3.5 on a 12 cores server. I've already increased the memory for the tomcat process (-Xms1g -Xmx4g). Thanks Nicolas ________________________________________ De : Serhiy Polyakov [sp0...@gmail.com] Envoyé : mercredi 29 août 2012 20:16 À : Support and info exchange list for Fedora users. Objet : Re: [fcrepo-user] Ingest performances May be the following can give some clues... I had export performance increased after increasing memory parameters in client/bin/env-client.sh Original was: exec_cmd="exec \"$java\" -Xms64m -Xmx96m \ Serhiy Polyakov University of North Texas On Wed, Aug 22, 2012 at 4:56 AM, Nicolas Hervé <nhe...@ina.fr> wrote: > Hi, > > I'm running a 3.5 instance and use the Java fedora client library to access > it. I'm doing massive parallel operations to manipulate huge amount of > objects. I've writen a multi-threaded application that is able to ingest and > purge. > > When purging, I have all the CPU on my server that are running, that's fine. > But when ingesting, only 20% of them are used and I don't undestand where > the bottleneck is. I've modified the Tomcat connector to allow massive > parallel connections. I've done the same for MySQL. I've desactivated DC > fields indexing and Resource indexing. By the way, as it's working fine when > purging, I suspect a problem in the ingestion process rather than a "maximum > number of connections" configuration issue. Finally, I don't have I/O wait, > so the HDD shouldn't be the problem either. > > Any clue on what is happening ? > > Thanks > > Nicolas > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Fedora-commons-users mailing list > Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users > ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-users ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
