This is a nice article, but I can't see if the server is dedicated to this
DSpace and how much memory is allocated to Java processes.
On a fairly loaded server, it took me 15 seconds per document on a 680,000
items DSpace (with 512MB allocated).
On Tue, Jan 27, 2009 at 4:09 PM, Stuart Lewis
Hi Ilias,
I am using the dspace import tool for batch ingesting in a Postgres
database and I am facing extremely slow feedback in each record commitment.
Initially, the speed was normal but when the items tend to be around 30
thousand, the speed of each commitment is unacceptable.
Is there
On 27 Jan 2009, at 10:27, Stuart Lewis wrote:
Hi Ilias,
I am using the dspace import tool for batch ingesting in a Postgres
database and I am facing extremely slow feedback in each record
commitment.
Initially, the speed was normal but when the items tend to be
around 30
thousand,
I would recommend two things;
1) batches of 200 documents (elsewhere their XML representation in
memory becomes very big)
2) make a PostgreSQL Maintenance (ANALYZE) after loading the first 200
and after loading a big part (first 10 thousand records)
Christophe
Stuart Lewis a écrit :
Hi
On Tue, 27 Jan 2009, Stuart Lewis wrote:
The following paper talks about this, and how DSpace performs when ingesting
1 million items:
Testing the Scalability of a DSpace-based Archive, Dharitri Misra, James
Seamans, George R. Thoma, National Library of Medicine, Bethesda, Maryland,
USA
Hi Simon,
Testing the Scalability of a DSpace-based Archive, Dharitri Misra,
James
Seamans, George R. Thoma, National Library of Medicine, Bethesda,
Maryland,
USA
http://www.dspace.org/images/stories/ist2008_paper_submitted1.pdf
Can I enquire as to where within the standard DSpace
Hi Tom,
The following paper talks about this, and how DSpace performs when ingesting
1 million items:
Testing the Scalability of a DSpace-based Archive, Dharitri Misra, James
Seamans, George R. Thoma, National Library of Medicine, Bethesda, Maryland,
USA
On 27 Jan 2009, at 14:14, Stuart Lewis wrote:
Hi Tom,
The following paper talks about this, and how DSpace performs when
ingesting
1 million items:
Testing the Scalability of a DSpace-based Archive, Dharitri Misra,
James
Seamans, George R. Thoma, National Library of Medicine,
Given that the test in the paper uses neither postgres nor the DSpace
import tool, that seems unlikely.
30,000 items shouldn't pose a big problem for any mature DBMS (e.g. Postgres
/ MySQL etc). If there are problems at that scale, they are more likely to
be in other parts of the system.
We've
On Tue, Jan 27, 2009 at 10:27 AM, Stuart Lewis s...@aber.ac.uk wrote:
Hi Ilias,
I am using the dspace import tool for batch ingesting in a Postgres
database and I am facing extremely slow feedback in each record
commitment.
The following paper talks about this, and how DSpace performs
10 matches
Mail list logo
