Re: Indexing slower in trunk

2011-06-16 Thread Erick Erickson
d from them. >> >> - >> Uwe Schindler >> H.-H.-Meier-Allee 63, D-28213 Bremen >> http://www.thetaphi.de >> eMail: u...@thetaphi.de >> >> >>> -Original Message- >>> From: Erick Erickson [mailto:erickerick...@gmail.com] >>

Re: Indexing slower in trunk

2011-06-16 Thread Martijn v Groningen
rom them. > > > > - > > Uwe Schindler > > H.-H.-Meier-Allee 63, D-28213 Bremen > > http://www.thetaphi.de > > eMail: u...@thetaphi.de > > > > > >> -----Original Message- > >> From: Erick Erickson [mailto:erickerick...@gmail.

Re: Indexing slower in trunk

2011-06-16 Thread Erick Erickson
gt; > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > >> -Original Message- >> From: Erick Erickson [mailto:erickerick...@gmail.com] >> Sent: Tuesday, June 14, 2011 3:19 PM >> To

Re: Indexing slower in trunk

2011-06-16 Thread Simon Willnauer
n: there are many more files to >>> merge (but optimize cost should not be counted as a problem here as normally >>> you won't need to optimize after initial indexing and optimizing was only a >>> good idea pre Lucene-2.9, now it's mostly obsolete) >>> &

RE: Indexing slower in trunk

2011-06-14 Thread Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Erick Erickson [mailto:erickerick...@gmail.com] > Sent: Tuesday, June 14, 2011 3:19 PM > To: dev@lucene.apache.org > Subject: Re: Indexing slower in trunk > >

RE: Indexing slower in trunk

2011-06-14 Thread Uwe Schindler
- > From: Yury Kats [mailto:yuryk...@yahoo.com] > Sent: Tuesday, June 14, 2011 3:04 PM > To: dev@lucene.apache.org > Subject: Re: Indexing slower in trunk > > On 6/14/2011 4:28 AM, Uwe Schindler wrote: > > indexing and optimizing was only a > > good idea pre Luce

Re: Indexing slower in trunk

2011-06-14 Thread Erick Erickson
Optimization used to have a very noticeable impact on search speed prior to some index format changes from quite a while ago. At this point the effect is much less noticeable, but the thing optimize does do is reclaim resources from deleted documents. If you have lots of deletions, it's a good ide

Re: Indexing slower in trunk

2011-06-14 Thread Yury Kats
On 6/14/2011 4:28 AM, Uwe Schindler wrote: > indexing and optimizing was only a > good idea pre Lucene-2.9, now it's mostly obsolete) Could you please elaborate on this? Is optimizing obsolete in general or after indexing new documents? Is it obsolete after deletions? And what it "mostly"? Thanks

Re: Indexing slower in trunk

2011-06-14 Thread Erick Erickson
need to optimize after initial indexing and optimizing was only a >> good idea pre Lucene-2.9, now it's mostly obsolete) >> >> Uwe >> >> - >> Uwe Schindler >> H.-H.-Meier-Allee 63, D-28213 Bremen >> http://www.thetaphi.de >> eMail: u...@

Re: Indexing slower in trunk

2011-06-14 Thread Simon Willnauer
-- > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > >> -Original Message- >> From: Erick Erickson [mailto:erickerick...@gmail.com] >> Sent: Tuesday, June 14, 2011 2:46 AM >> To: dev@lucen

RE: Indexing slower in trunk

2011-06-14 Thread Uwe Schindler
> Sent: Tuesday, June 14, 2011 2:46 AM > To: dev@lucene.apache.org; simon.willna...@gmail.com > Subject: Re: Indexing slower in trunk > > Simon: > > Yep, I was asking to see if it was weird. Pursuant to our > chat I tried some things, results below: > > All these ar

Re: Indexing slower in trunk

2011-06-13 Thread Simon Willnauer
On Mon, Jun 13, 2011 at 8:13 PM, Erick Erickson wrote: > I half remember that this has come up before, but I couldn't find the > thread. I was running some tests over the weekend that involved > indexing 1.9M documents from the English Wiki dump. > > I'm consistently seeing that trunk takes about

Indexing slower in trunk

2011-06-13 Thread Erick Erickson
I half remember that this has come up before, but I couldn't find the thread. I was running some tests over the weekend that involved indexing 1.9M documents from the English Wiki dump. I'm consistently seeing that trunk takes about twice as long to index the docs as 1.4, 3.2 and 3x. Optimize is a