Re: Indexing slower in trunk

2011-06-16 Thread Simon Willnauer
:46 AM To: dev@lucene.apache.org; simon.willna...@gmail.com Subject: Re: Indexing slower in trunk Simon: Yep, I was asking to see if it was weird. Pursuant to our chat I tried some things, results below: All these are running on my local machine, same disk, same JVM settings re: memory

Re: Indexing slower in trunk

2011-06-16 Thread Erick Erickson
-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, June 14, 2011 3:19 PM To: dev@lucene.apache.org Subject: Re: Indexing slower in trunk Optimization used to have a very noticeable

Re: Indexing slower in trunk

2011-06-16 Thread Martijn v Groningen
-Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, June 14, 2011 3:19 PM To: dev@lucene.apache.org Subject: Re: Indexing slower in trunk Optimization used to have a very noticeable impact on search speed prior to some index format changes from

Re: Indexing slower in trunk

2011-06-16 Thread Erick Erickson
: Tuesday, June 14, 2011 3:19 PM To: dev@lucene.apache.org Subject: Re: Indexing slower in trunk Optimization used to have a very noticeable impact on search speed prior to some index format changes from quite a while ago. At this point the effect is much less noticeable, but the thing optimize

RE: Indexing slower in trunk

2011-06-14 Thread Uwe Schindler
14, 2011 2:46 AM To: dev@lucene.apache.org; simon.willna...@gmail.com Subject: Re: Indexing slower in trunk Simon: Yep, I was asking to see if it was weird. Pursuant to our chat I tried some things, results below: All these are running on my local machine, same disk, same JVM settings

Re: Indexing slower in trunk

2011-06-14 Thread Simon Willnauer
://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, June 14, 2011 2:46 AM To: dev@lucene.apache.org; simon.willna...@gmail.com Subject: Re: Indexing slower in trunk Simon: Yep, I was asking to see

Re: Indexing slower in trunk

2011-06-14 Thread Yury Kats
On 6/14/2011 4:28 AM, Uwe Schindler wrote: indexing and optimizing was only a good idea pre Lucene-2.9, now it's mostly obsolete) Could you please elaborate on this? Is optimizing obsolete in general or after indexing new documents? Is it obsolete after deletions? And what it mostly? Thanks!

Re: Indexing slower in trunk

2011-06-14 Thread Erick Erickson
Optimization used to have a very noticeable impact on search speed prior to some index format changes from quite a while ago. At this point the effect is much less noticeable, but the thing optimize does do is reclaim resources from deleted documents. If you have lots of deletions, it's a good

RE: Indexing slower in trunk

2011-06-14 Thread Uwe Schindler
.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Tuesday, June 14, 2011 3:19 PM To: dev@lucene.apache.org Subject: Re: Indexing slower in trunk Optimization used to have a very

Indexing slower in trunk

2011-06-13 Thread Erick Erickson
I half remember that this has come up before, but I couldn't find the thread. I was running some tests over the weekend that involved indexing 1.9M documents from the English Wiki dump. I'm consistently seeing that trunk takes about twice as long to index the docs as 1.4, 3.2 and 3x. Optimize is

Re: Indexing slower in trunk

2011-06-13 Thread Simon Willnauer
On Mon, Jun 13, 2011 at 8:13 PM, Erick Erickson erickerick...@gmail.com wrote: I half remember that this has come up before, but I couldn't find the thread. I was running some tests over the weekend that involved indexing 1.9M documents from the English Wiki dump. I'm consistently seeing that