[jira] Updated: (LUCENE-1614) Add next() and skipTo() variants to DocIdSetIterator that return the current doc, instead of boolean

2009-05-28 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-1614: --- Attachment: LUCENE-1614.patch Deprecated SpanScorer.firstTime and removed the use of it. (in general

[jira] Commented: (LUCENE-1252) Avoid using positions when not all required terms are present

2009-05-28 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713918#action_12713918 ] Shai Erera commented on LUCENE-1252: I must admit that I read the issue briefly, and s

Re: upcoming Solr release on Lucene 2.9-dev

2009-05-28 Thread Michael McCandless
The IndexWriter diagnostics (LUCENE-1654: recording Lucene version, Java/OS version, etc into each segment created) also bumped the index file format. And LUCENE-1623 (fixing back-compat issue w/ field names that have non-ascii characters) also bumped the format of the fnm file. Offhand I can't t

[jira] Commented: (LUCENE-1252) Avoid using positions when not all required terms are present

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713928#action_12713928 ] Michael McCandless commented on LUCENE-1252: I agree, we would want to do a Co

[jira] Commented: (LUCENE-1614) Add next() and skipTo() variants to DocIdSetIterator that return the current doc, instead of boolean

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713931#action_12713931 ] Michael McCandless commented on LUCENE-1614: {quote} After I made add() privat

[jira] Resolved: (LUCENE-1657) Make "boolean readOnly" a required arg to IndexReader.open

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-1657. Resolution: Fixed > Make "boolean readOnly" a required arg to IndexReader.open > -

[jira] Resolved: (LUCENE-1653) Change DateTools to not create a Calendar in every call to dateToString or timeToString

2009-05-28 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller resolved LUCENE-1653. - Resolution: Fixed Lucene Fields: [New, Patch Available] (was: [New]) Thanks Shai! > Cha

Re: upcoming Solr release on Lucene 2.9-dev

2009-05-28 Thread Yonik Seeley
On Thu, May 28, 2009 at 5:35 AM, Michael McCandless wrote: > The IndexWriter diagnostics (LUCENE-1654: recording Lucene version, > Java/OS version, etc into each segment created) also bumped the index > file format. > > And LUCENE-1623 (fixing back-compat issue w/ field names that have > non-ascii

Re: upcoming Solr release on Lucene 2.9-dev

2009-05-28 Thread Yonik Seeley
On Thu, May 28, 2009 at 2:56 AM, Shai Erera wrote: > If by changes you also mean deprecated features, then take a look at > LUCENE-1614 - if you have your own Scorers/DISIs, you might want to > implement the new methods, since the current ones are deprecated. Yes, we have our own Scorers, but cha

[jira] Created: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Renaud Delbru (JIRA)
Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST -- Key: LUCENE-1666 URL: https://issues.apache.org/jira/browse/LUCEN

[jira] Updated: (LUCENE-1542) Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

2009-05-28 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-1542: Summary: Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

[jira] Updated: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Renaud Delbru (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Renaud Delbru updated LUCENE-1666: -- Attachment: LUCENE-1666.patch > Constants causing NullPointerException when fetching metadata

[jira] Updated: (LUCENE-1542) Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

2009-05-28 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-1542: Attachment: LUCENE-1542.patch something like this to fix > Lucene can incorrectly set the positio

[jira] Commented: (LUCENE-1542) Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

2009-05-28 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12713999#action_12713999 ] Mark Miller commented on LUCENE-1542: - so that appears to fix it - but i'm not sure th

Re: upcoming Solr release on Lucene 2.9-dev

2009-05-28 Thread Grant Ingersoll
On May 28, 2009, at 10:24 AM, Yonik Seeley wrote: removing all the deprecated stuff related to Token/TokenStream is for another day... I think Lucene needs to re-look at the TokenStream stuff anyway. I seem to recall talking with Michael B. about changing it at ApacheCon. I'd be inter

[jira] Assigned: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-1666: -- Assignee: Michael McCandless > Constants causing NullPointerException when fet

[jira] Resolved: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-1666. Resolution: Fixed Woops, thanks Renaud -- I just committed this. > Constants caus

[jira] Commented: (LUCENE-1542) Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714020#action_12714020 ] Michael McCandless commented on LUCENE-1542: Alas, this looks like a dup of LU

[jira] Updated: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-1666: -- Attachment: LUCENE-1666.patch Mike, you were a little bit too fast :-) I worked out a better

[jira] Updated: (LUCENE-1542) Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

2009-05-28 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-1542: Attachment: LUCENE-1542.patch with unit test > Lucene can incorrectly set the position of tokens

[jira] Updated: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-1666: -- Attachment: LUCENE-1666.patch Oh forget the "-dev" thing... Attached is fixed patch. > Consta

[jira] Updated: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-1666: -- Attachment: (was: LUCENE-1666.patch) > Constants causing NullPointerException when fetchin

[jira] Commented: (LUCENE-1542) Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

2009-05-28 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714033#action_12714033 ] Mark Miller commented on LUCENE-1542: - I don't think the fix here needs to disallow -1

[jira] Commented: (LUCENE-1542) Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

2009-05-28 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714031#action_12714031 ] Mark Miller commented on LUCENE-1542: - and Spans. if its its included in a span, it wi

[jira] Commented: (LUCENE-1255) CheckIndex should allow term position = -1

2009-05-28 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714039#action_12714039 ] Mark Miller commented on LUCENE-1255: - why should the first token have an increment of

[jira] Commented: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714046#action_12714046 ] Michael McCandless commented on LUCENE-1666: Much simpler, thanks Uwe! But, s

[jira] Updated: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-1666: -- Attachment: LUCENE-1666.patch I think the check should be simply if getPackage() returns null

Jonathan Mamou is out of the office.

2009-05-28 Thread Jonathan Mamou
I will be out of the office starting 27/05/2009 and will not return until 31/05/2009. I will respond to your message when I return. - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-ma

[jira] Commented: (LUCENE-1542) Lucene can incorrectly set the position of tokens that start a field with positonInc 0.

2009-05-28 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714065#action_12714065 ] Shai Erera commented on LUCENE-1542: Just wanted to say we've had an internal discussi

[jira] Commented: (LUCENE-1654) Include diagnostics per-segment when writing a new segment

2009-05-28 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714068#action_12714068 ] Uwe Schindler commented on LUCENE-1654: --- I found a bug for indexes with the string-b

[jira] Commented: (LUCENE-1666) Constants causing NullPointerException when fetching metadata "Implementation Version" in MANIFEST

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714074#action_12714074 ] Michael McCandless commented on LUCENE-1666: Excellent -- I just committed thi

[jira] Issue Comment Edited: (LUCENE-1654) Include diagnostics per-segment when writing a new segment

2009-05-28 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714068#action_12714068 ] Uwe Schindler edited comment on LUCENE-1654 at 5/28/09 10:50 AM: ---

[jira] Commented: (LUCENE-1654) Include diagnostics per-segment when writing a new segment

2009-05-28 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714087#action_12714087 ] Michael McCandless commented on LUCENE-1654: Woops -- thanks Uwe! > Include d

NRT getReader turnaround on large segments

2009-05-28 Thread Jason Rutherglen
An interesting discussion came up. How do we handle IW.getReader turnaround time on large new segments?

Re: NRT getReader turnaround on large segments

2009-05-28 Thread Michael McCandless
This is exactly why we added IndexReaderWarmer -- it pre-warms a newly merged segment before committing to SegmentInfos. So, while such warming is happening, if getReader() is called, the returned reader will still read the old segments. Mike On Thu, May 28, 2009 at 3:06 PM, Jason Rutherglen wr

Re: NRT getReader turnaround on large segments

2009-05-28 Thread Jason Rutherglen
And warming a segment in mergeMiddle doesn't block the addition of new segments. On Thu, May 28, 2009 at 12:30 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > This is exactly why we added IndexReaderWarmer -- it pre-warms a newly > merged segment before committing to SegmentInfos. >

Re: NRT getReader turnaround on large segments

2009-05-28 Thread Michael McCandless
Right. If you play w/ this please report back on how it goes! Mike On Thu, May 28, 2009 at 3:38 PM, Jason Rutherglen wrote: > And warming a segment in mergeMiddle doesn't block the addition of new > segments. > > On Thu, May 28, 2009 at 12:30 PM, Michael McCandless > wrote: >> >> This is exact

Re: NRT getReader turnaround on large segments

2009-05-28 Thread Jason Rutherglen
Also important to note that only with ConcurrentMergeScheduler will the segmentReader warming occur in the background. When using SerialMergeScheduler, the warming will happen in the foreground. On Thu, May 28, 2009 at 12:40 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > Right. If

[jira] Created: (LUCENE-1667) ConcurrentMergeScheduler use a thread pool (per directory)

2009-05-28 Thread Jason Rutherglen (JIRA)
ConcurrentMergeScheduler use a thread pool (per directory) -- Key: LUCENE-1667 URL: https://issues.apache.org/jira/browse/LUCENE-1667 Project: Lucene - Java Issue Type: Improvement

IndexReaderFactory for IndexWriter LUCENE-1516

2009-05-28 Thread Jason Rutherglen
Some folks may want to have IW.getReader (LUCENE-1516) use custom readers underneath, we can have IW support an IndexReaderFactory?

Re: NRT getReader turnaround on large segments

2009-05-28 Thread Yonik Seeley
On Thu, May 28, 2009 at 3:30 PM, Michael McCandless wrote: > This is exactly why we added IndexReaderWarmer -- it pre-warms a newly > merged segment before committing to SegmentInfos. > > So, while such warming is happening, if getReader() is called, the > returned reader will still read the old s

Re: NRT getReader turnaround on large segments

2009-05-28 Thread Michael McCandless
Newly added docs are still free to make new segments, and be reopened, while this warming is taking place. So, getReader() will wait for newly added/deleted docs to be flushed & reopened, but will not wait for any running merges to complete (finish writing the files & finish warming a SegmentReade

Re: NRT getReader turnaround on large segments

2009-05-28 Thread Yonik Seeley
On Thu, May 28, 2009 at 4:18 PM, Michael McCandless wrote: > Newly added docs are still free to make new segments, and be reopened, > while this warming is taking place. > > So, getReader() will wait for newly added/deleted docs to be flushed & > reopened, but will not wait for any running merges

More efficient loading of terms dictionary

2009-05-28 Thread Jason Rutherglen
This is for Marvin who previously mentioned loading the term dictionary directly from the filesystem (rather than load every Nth term into Java heap) which could improve latency of opening new readers. Were you able to take this idea any further?

[jira] Commented: (LUCENE-1629) contrib intelligent Analyzer for Chinese

2009-05-28 Thread Otis Gospodnetic (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714285#action_12714285 ] Otis Gospodnetic commented on LUCENE-1629: -- I just got to look at this code and I

[jira] Commented: (LUCENE-1629) contrib intelligent Analyzer for Chinese

2009-05-28 Thread Xiaoping Gao (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714293#action_12714293 ] Xiaoping Gao commented on LUCENE-1629: -- I think the algorithm of Hidden Markov Model

[jira] Commented: (LUCENE-1614) Add next() and skipTo() variants to DocIdSetIterator that return the current doc, instead of boolean

2009-05-28 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12714298#action_12714298 ] Shai Erera commented on LUCENE-1614: Mike, I think we may have relaxed the policy of d