Ludovic, The -Xmx option determines the size of the heap. Did you also set the -XX:MaxPermSize option? The Sun JVM also uses this parameter to allocate memory space, in addition to the heap.
http://www.sumoc.com/blog/index.cfm?mode=entry&entry=CDCDBF8B-5004-2066-B7460CDEAB79328F http://www.javaperformancetuning.com/news/qotm036.shtml -- Scott On 07/08/2011 10:08 AM, Ludovic Deravet wrote: > Dear all, > > We are experiencing a serious issue with Fedora when ingesting thousands of > packages into the repository. Indeed, the development of our project is > almost finished and our customer is trying to perform massive load of > objects into Fedora before going into production. > > However an instability problem that manifest itself through *java heap out > of memory *errors appears when the customer wants to ingest about 10.000 of > METS envelopes. > > Internally, our application is transforming all METS envelopes into FOXML > files that are then sent to the Fedora's REST's API. And documents are > defined as content managed datastreams. > > To give some figures about these envelopes: > > - A METS envelope defines in average 80 digital objects > - 10% of the envelopes reference documents where their total size is > lower than 1MB > - 65% of the envelopes reference documents where their total size is > between 1MB and 5MB > - 20% of the envelopes reference documents where their total size is > between 5MB and 10MB > - 5% of the envelopes reference documents where their total size is > bigger than 10MB > > Fedora is installed on a production-like hardware: > > - Sun Oracle Enterprise M5000 - 2 clusters of 4 CPUs 2.53GHZ SPARC64 > USVII (so 32 cores in total) > - JVM is configured with 2GB of memory (-Xmx2048m) > - JVM is in server mode on Solaris 10 (64bits) > - Fedora-commons 3.4.2 with default configuration options activated > > In a production-like environment, our application is in fact launching 10 > threads where each thread is processing one METS envelope at a time. We > realize that this is kind of stressful for Fedora but the customer is > expecting us to ingest all their METS envelopes in a reasonable short period > of time. > > We have performed many different tests in our development environment (of > course quite less powerful than the production-like environment) and cannot > reproduce the instability problem we are experiencing in production. > > We have analyzed in our development environment how Fedora is managing the > memory and everything seems normal. The Garbage is working fine even if we > set the heap size to 255MB and try to ingest several MB of data. > > I really doubt that this problem lies within Fedora itself. Instead, I think > that the problem is more a configuration issue somewhere but right now we > don't know which one exactly. Is there someone that could share their > experience with such issue and help us try to find a solution to this > problem? > > Thank you in advance for any help. > > Kind regards, > > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2d-c2 > > > > _______________________________________________ > Fedora-commons-users mailing list > Fedora-commons-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/fedora-commons-users -- Scott Prater Library, Instructional, and Research Applications (LIRA) Division of Information Technology (DoIT) University of Wisconsin - Madison pra...@wisc.edu ------------------------------------------------------------------------------ AppSumo Presents a FREE Video for the SourceForge Community by Eric Ries, the creator of the Lean Startup Methodology on "Lean Startup Secrets Revealed." This video shows you how to validate your ideas, optimize your ideas and identify your business strategy. http://p.sf.net/sfu/appsumosfdev2dev _______________________________________________ Fedora-commons-users mailing list Fedora-commons-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
