Has anyone noticed that the larger their DSpace repository grows, the slower
the imports seem to run?
I found a study someone had done on import times, etc and it basically
concluded that import speed has been seen to degrade in larger batches of
ingest items and it was recommended that smaller input batches be used. We've
been importing batches of 3,000 documents (with metadata) into our repository
and just recently it's really been taking longer and longer to run. Our entire
repository contains a total of 211,459 active Items and 337,091 active
documents, including *.pdf.txt files (not including *.pdf.txt files, we have
202,879 active documents).
We are working on loading thousands more documents into our repository,
but I'm concerned that import speed (and possibly even DSpace's overall
performance, particularly the browses) will continue to degrade in direct
proportion to the increase in number of records in our repository.
We have DSpace 1.5.1 installed on our Test machine and are getting ready
to migrate to Production soon, and I understand improvements were made in this
release, particularly in Browses. I also did a test import the other day into
the Test repository and 500 records only took a few seconds to load, however we
only have a small subset of data in the Test repository.
I would appreciate any insight, advice, or thoughts on this issue,
especially those who have had the same problem(s).
Thanks in advance,
Sue
Sue Walker-Thornton
ConITS Contract
NASA Langley Research Center
Integrated Library Systems Application & Database Administrator
130 Research Drive
Hampton, VA 23666
Office: (757) 224-4074
Fax: (757) 224-4001
Pager: (757) 988-2547
Email: [email protected]<mailto:[email protected]>
------------------------------------------------------------------------------
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
_______________________________________________
DSpace-tech mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dspace-tech
