DSpace scalability issues report, per wiki template:
1. dsp...@cambridge, The University of Cambridge, UK.
Technical contacts: Tom De Mulder, [email protected] (systems manager)
Simon Brown [email protected] (DSpace developer)
2. a. DSpace version 1.6.2 with extensive local patches, using JSPUI
Size: 137 communities, 258 collections, >200k items, 12TB, 436k
bitstreams (excluding licenses)
b. PostgreSQL 8.4.4
c. Tomcat 6.0.24 standalone
d. Separate servers for webapp, DB, storage and ancillary functions
Webapp/DB servers are HT 8-core Intel servers running Ubuntu Linux
with 16GB of memory and fast local storage
Java memory: -Xmx2048M -Xms2048M
3. a. - Unless Tomcat is restarted, it will consistently fail due to lack of
memory in less than 48 hours.
- Batch importer: will fail on large batch imports (order of thousands of
items), performance degrades with size of repository and of batch.
- Search indexer: fails on large repositories, slowing down and
eventually running out of memory.
- Assetstore: random structure causes large overhead on filesystem for no
real gain
See also our poster, presented in Gothenburg:
http://tools.dspace.cam.ac.uk/DSUG09%20A2%20poster.pdf
b. Installed vanilla DSpace 1.6.2, imported 200k randomly generated items,
ran siege against it, watched it not cope.
We've done profiling in the past, but not for 1.6. However, we've not
noticed significant changes in the code that has issues.
c. We have patches for the indexer; batch importer; thumbnail and PDF text
extraction; assetstore structure; dark item masking in OAI and browse code
4. We can't commit to volunteering unless this can be incorporated into the
work we need to undertake in our primary capacity of running the University's
Institutional Repository. However, we would be willing to try and make this
happen.
--
Tom De Mulder <[email protected]> - Cambridge University Computing Service
+44 1223 3 31843 - New Museums Site, Pembroke Street, Cambridge CB2 3QH
------------------------------------------------------------------------------
Beautiful is writing same markup. Internet Explorer 9 supports
standards for HTML5, CSS3, SVG 1.1, ECMAScript5, and DOM L2 & L3.
Spend less time writing and rewriting code and more time creating great
experiences on the web. Be a part of the beta today.
http://p.sf.net/sfu/beautyoftheweb
_______________________________________________
DSpace-tech mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspace-tech
