RE: WFST/Analyzing Suggesters: foreign keys, user-supplied filter, highlighting

2012-10-31 Thread Oliver Christ
Hi, I've added LUCENE-4516 - Suggesters: allow to associate a user-specified key (int) with a string LUCENE-4517 - Suggesters: allow to pass a user-defined predicate/filter to the completion searcher LUCENE-4518 - Suggesters: highlighting (explicit markup of user-typed portions vs. generated

Re: WFST/Analyzing Suggesters: foreign keys, user-supplied filter, highlighting

2012-10-31 Thread Michael McCandless
Thanks! Mike McCandless http://blog.mikemccandless.com On Wed, Oct 31, 2012 at 8:16 AM, Oliver Christ wrote: > Hi, > > I've added > > LUCENE-4516 - Suggesters: allow to associate a user-specified key (int) > with a string > > LUCENE-4517 - Suggesters: allow to pass a user-defined predicate/fil

Re: "read past EOF" when merge

2012-10-31 Thread Michael McCandless
Run CheckIndex on the index? What filesystem is IndexWriter using to write to the index...? Have you changed the LockFactory on the Directory? Mike McCandless http://blog.mikemccandless.com On Tue, Oct 30, 2012 at 11:27 PM, superruiye wrote: > The exception "read past EOF" Bothering me a lon

IntField vs (IntDocValuesField + StoredField)

2012-10-31 Thread Ivan Vasilev
Hy Guys, Is there some advantage in speed or index size to use this: IntDocValuesField fld = new IntDocValuesField("fldName", 1); StoredField fld = new StoredField("fldName", 1); instead of this: IntField fld = new IntField("fld", 1, Field.Store.YES); Searching, sorting and retrieving data fr

Re: IntField vs (IntDocValuesField + StoredField)

2012-10-31 Thread Michael McCandless
The big advantage of IntField is you can do NumericRangeQuery/Filter on the field. Mike McCandless http://blog.mikemccandless.com On Wed, Oct 31, 2012 at 9:42 AM, Ivan Vasilev wrote: > Hy Guys, > > Is there some advantage in speed or index size to use this: > > IntDocValuesField fld = new IntDo

Re: IntField vs (IntDocValuesField + StoredField)

2012-10-31 Thread Ivan Vasilev
Thanks Mike. On 31.10.2012 г. 15:52, Michael McCandless wrote: The big advantage of IntField is you can do NumericRangeQuery/Filter on the field. Mike McCandless http://blog.mikemccandless.com On Wed, Oct 31, 2012 at 9:42 AM, Ivan Vasilev wrote: Hy Guys, Is there some advantage in speed or

RE: IntField vs (IntDocValuesField + StoredField)

2012-10-31 Thread Uwe Schindler
And in general, using 1 as precisionStep is not always a good idea - size-wise and performance wise. I would use the default of 4. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Michael McCandless [mail

RE: IntField vs (IntDocValuesField + StoredField)

2012-10-31 Thread Uwe Schindler
The DocValues field is better for sorting, as the values do not need to be uninverted. The Field is already written to disk as a big int[] array, so sorting using SortField is faster and less memory expensive because this structure can be used for sorting. - Uwe Schindler H.-H.-Meier-Allee

Re: IntField vs (IntDocValuesField + StoredField)

2012-10-31 Thread Michael McCandless
Uwe, that '1' was the value of the field not the precision step... And that's a good point about IntDocValues being more efficient for sorting. Mike McCandless http://blog.mikemccandless.com On Wed, Oct 31, 2012 at 10:16 AM, Uwe Schindler wrote: > And in general, using 1 as precisionStep is no

Re: IntField vs (IntDocValuesField + StoredField)

2012-10-31 Thread Ivan Vasilev
Yes the "1" is the value of the field. Thanks Mike and Uwe. On 31.10.2012 г. 16:23, Michael McCandless wrote: Uwe, that '1' was the value of the field not the precision step... And that's a good point about IntDocValues being more efficient for sorting. Mike McCandless http://blog.mikemccandl

writer.tryDeleteDocument(..) does not delete document

2012-10-31 Thread Ivan Vasilev
Hy Guys, I use as suggested in question "Lucene 4.0 delete by ID" from 29.Oct - instead of reader.delete(docID) use - writer.tryDeleteDocument(..) method but for some reason it does not work. My code is: IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE

Re: writer.tryDeleteDocument(..) does not delete document

2012-10-31 Thread Michael McCandless
This looks like a real bug! Thanks for reporting it Ivan ... I'll open an issue. Mike McCandless http://blog.mikemccandless.com On Wed, Oct 31, 2012 at 11:41 AM, Ivan Vasilev wrote: > Hy Guys, > > I use as suggested in question "Lucene 4.0 delete by ID" from 29.Oct - > instead of reader.delete

Re: writer.tryDeleteDocument(..) does not delete document

2012-10-31 Thread Michael McCandless
I opened https://issues.apache.org/jira/browse/LUCENE-4521 ... Mike McCandless http://blog.mikemccandless.com On Wed, Oct 31, 2012 at 12:10 PM, Michael McCandless wrote: > This looks like a real bug! Thanks for reporting it Ivan ... I'll > open an issue. > > Mike McCandless > > http://blog.mik

Highlighting and InvalidTokenOffsetsException in Lucene 4.0

2012-10-31 Thread Scott Smith
I'm migrating code from Lucene 3.5 to 4.0. I have the following code which is supposed to highlight text. I get the exception InvalidTokenOffsetsException. I have no idea what that means. I am using a custom analyzer which seems to work for searching/indexing, so I assume it will work here (

Re: "read past EOF" when merge

2012-10-31 Thread pliu367
Hi Mike, I got the same problem as above. My search project is running on two servers(1 master for index writing and reading and 1 slaver for index reading), and the master server writes the index into a seperate disk(not belong to master) which is shared by the slaver. I have never changed the Loc