It is a 64-bit JVM and a pretty good size machine, but I don't think
I am anywhere near pushing the limits on it either, so don't read too
much into my numbers other than as a raw statement of how many
documents I've indexed.
When using the Hits API, documents aren't loaded until you ask for a
specific document. Even then, if you use the FieldSelector mechanism
checked into trunk, you can choose to control which fields get
loaded, which can help w/ memory issues.
-Grant
On Jan 26, 2007, at 1:13 PM, Chiradeep Vittal wrote:
Grant,
Is that on a single machine? If so, what kind of hardware specs
does the machine have? I guess you're using a 64-bit JVM?
A slightly unrelated question: if a query matches all the documents
in the index, does that cause the entire index to get loaded into
RAM ?
----- Original Message ----
From: Grant Ingersoll <[EMAIL PROTECTED]>
To: java-user@lucene.apache.org
Sent: Friday, January 26, 2007 8:03:11 AM
Subject: Re: How many documents in the biggest Lucene index to date?
I just indexed a collection w/ 15+ million docs in one index. Index
size is roughly 42 gb.
On Jan 26, 2007, at 12:45 AM, Bill Taylor wrote:
I have used Lucene to index a small collection - only a few hundred
documents. I have a potential client who wants to index a
collection which will start at about a million documents and could
easily grow to two million.
Has anyone used Lucene with an index that large?
Thank you very much.
Bill Taylor
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------
Grant Ingersoll
Center for Natural Language Processing
http://www.cnlp.org
Read the Lucene Java FAQ at http://wiki.apache.org/jakarta-lucene/
LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
------------------------------------------------------
Grant Ingersoll
http://www.grantingersoll.com/
http://www.paperoftheweek.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
