RE: Text storing design and performance question

2007-01-11 Thread Renaud Waldura
storing term vectors would keep the index lean and allow for fast highlighting? --Renaud -Original Message- From: Mark Miller [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 9:54 AM To: java-user@lucene.apache.org Subject: Re: Text storing design and performance question

Re: Text storing design and performance question

2007-01-11 Thread Jason Pump
-user@lucene.apache.org Subject: Re: Text storing design and performance question Renaud, one optimization you can do on this is to try the first 10kb, see if it finds text worth highlighting, if not, with a slight overlap try the next 9.9kb - 19.9kb or just 9.9kb - end if you're feeling lazy

Re: Text storing design and performance question

2007-01-11 Thread Chris Hostetter
In general, if you are having performance issues with highlighting, the first thing to do is double check what the bottleneck is: is it accessing the text to by highlighted, or is it running the highlighter? you suggested earlier in the thread that the problem was with accessing the text... :

Text storing design and performance question

2007-01-10 Thread moraleslos
. Because I have duplicated data, one in the index and the other in the db, are there other ways of handling this situation in a more efficient and performant way? Thanks in advance. -los -- View this message in context: http://www.nabble.com/Text-storing-design-and-performance-question-tf2953201

Re: Text storing design and performance question

2007-01-10 Thread Erik Hatcher
and the other in the db, are there other ways of handling this situation in a more efficient and performant way? Thanks in advance. -los -- View this message in context: http://www.nabble.com/Text-storing- design-and-performance-question-tf2953201.html#a8259883 Sent from the Lucene - Java Users

Re: Text storing design and performance question

2007-01-10 Thread moraleslos
? Thanks in advance. -los -- View this message in context: http://www.nabble.com/Text-storing- design-and-performance-question-tf2953201.html#a8259883 Sent from the Lucene - Java Users mailing list archive at Nabble.com

Re: Text storing design and performance question

2007-01-10 Thread Mark Miller
-storing- design-and-performance-question-tf2953201.html#a8259883 Sent from the Lucene - Java Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED

Re: Text storing design and performance question

2007-01-10 Thread moraleslos
duplicated data, one in the index and the other in the db, are there other ways of handling this situation in a more efficient and performant way? Thanks in advance. -los -- View this message in context: http://www.nabble.com/Text-storing- design-and-performance-question-tf2953201

Re: Text storing design and performance question

2007-01-10 Thread Mark Miller
this message in context: http://www.nabble.com/Text-storing- design-and-performance-question-tf2953201.html#a8259883 Sent from the Lucene - Java Users mailing list archive at Nabble.com. - To unsubscribe, e-mail: [EMAIL

RE: Text storing design and performance question

2007-01-10 Thread Renaud Waldura
storing term vectors would keep the index lean and allow for fast highlighting? --Renaud -Original Message- From: Mark Miller [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 9:54 AM To: java-user@lucene.apache.org Subject: Re: Text storing design and performance question

Re: Text storing design and performance question

2007-01-10 Thread moraleslos
in advance. -los -- View this message in context: http://www.nabble.com/Text-storing- design-and-performance-question-tf2953201.html#a8259883 Sent from the Lucene - Java Users mailing list archive at Nabble.com

RE: Text storing design and performance question

2007-01-10 Thread moraleslos
-Original Message- From: Mark Miller [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 9:54 AM To: java-user@lucene.apache.org Subject: Re: Text storing design and performance question Being stateless should not be much of an issue. As Erick mentioned, the highlighter just

RE: Text storing design and performance question

2007-01-10 Thread Renaud Waldura
To: java-user@lucene.apache.org Subject: RE: Text storing design and performance question Maybe keeping the data in the DB would make it quicker? Seems like the I/O performance would cause most of the performance issues you're seeing. -los Renaud Waldura-5 wrote: We used to store a big text

Re: Text storing design and performance question

2007-01-10 Thread Jason Pump
for fast highlighting? --Renaud -Original Message- From: Mark Miller [mailto:[EMAIL PROTECTED] Sent: Wednesday, January 10, 2007 9:54 AM To: java-user@lucene.apache.org Subject: Re: Text storing design and performance question Being stateless should not be much of an issue. As Erick