this message in context:
http://lucene.472066.n3.nabble.com/Making-lucene-indexing-multi-threaded-tp4087830p4166123.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: java-user
://lucene.472066.n3.nabble.com/Making-lucene-indexing-multi-threaded-tp4087830p4166094.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
@lucene.apache.org
Subject: Re: Making lucene indexing multi threaded
Hi Nischal,
I had similar indexing issue. My lucene indexing took 22 mins for 70 MB
docs. When i debugged the problem, i found out the
indexWriter.addDocument(doc) taking a really long time.
Have you already found the solution about
and take 22 mins.
Did you have any similar experience like the above before?
Thank you,
Jason
--
View this message in context:
http://lucene.472066.n3.nabble.com/Making-lucene-indexing-multi-threaded-tp4087830p4166116.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com
://lucene.472066.n3.nabble.com/Making-lucene-indexing-multi-threaded-tp4087830p4166116.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
:
http://lucene.472066.n3.nabble.com/Making-lucene-indexing-multi-threaded-tp4087830p4166123.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
Don't commit after adding each and every document.
On Tue, Sep 3, 2013 at 7:20 AM, nischal reddy nischal.srini...@gmail.comwrote:
Hi,
Some more update on my progress,
i have multithreaded indexing in my application, i have used thread pool
executor and used a pool size of 4 but had a
Stop. Back up. Test. G
The very _first_ thing I'd do is just comment out the bit that
actually indexes the content. I'm guessing you have some
loop like:
while (more files) {
read the file
transform the data
create a Lucene document
index the document
}
Just comment out the index
Hi,
Lucene's IndexWriter can safely accept updates coming from several
threads, just make sure to share the same IndexWriter instance across
all threads, no extrenal locking is necessary.
30 minutes sound slike a lot for 3 files unless they are large.
You can have a look at
Hi Eric,
I have commented out the indexing part (indexwriter.addDocument()) part in
my application and it is taking around 90 seconds, but when i uncomment the
indexing part it is taking lot of time.
My machine specs are
windows 7, intel i7 processor, 4gb ram and doest have an ssd harddisk.
Hi,
Some more update on my progress,
i have multithreaded indexing in my application, i have used thread pool
executor and used a pool size of 4 but had a very slight increase in the
performace very negligible, still it is taking around 20 minutes of time to
index around 30k files,
Some more
11 matches
Mail list logo