Re: lucene indexing and merge process

2007-10-19 Thread Michael McCandless
It seems like there are (at least) two angles here for getting better performance from FieldCache: 1) Be incremental: with reopen() we should only have to update a subset of the array in the FieldCache, according to the changed segments. This is what Hoss is working on and Mark was r

[jira] Updated: (LUCENE-1020) Basic tool for checking & repairing an index

2007-10-19 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1020: --- Attachment: LUCENE-1020.take2.patch Attached patch: another rev of this tool, with a

[jira] Commented: (LUCENE-743) IndexReader.reopen()

2007-10-19 Thread Michael Busch (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12536353 ] Michael Busch commented on LUCENE-743: -- Hi Mike, I'm not sure if I fully understand your comment. Consider the

Re: Per-document Payloads (was: Re: lucene indexing and merge process)

2007-10-19 Thread John Wang
Hi Michael: Thanks for the info. I haven't played with payloads. Can you give me an example or point me to how it is used to solve this problem? Thanks -John On 10/19/07, Michael Busch <[EMAIL PROTECTED]> wrote: > > John Wang wrote: > > > > I can tried to get some numbers for lea

Per-document Payloads (was: Re: lucene indexing and merge process)

2007-10-19 Thread Michael Busch
John Wang wrote: > > I can tried to get some numbers for leading an int[] array vs > FieldCache.getInts(). I've had a similar performance problem when I used the FieldCache. The loading performance is apparently so slow, because each value is stored as a term in the dictionary. For loading t

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

2007-10-19 Thread Sean Timm (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Timm updated LUCENE-997: - Attachment: timeout.patch Two issues are addressed in this latest patch: 1) Timeout support was not adde

Re: Remove TermEnum.skipTo(Term target)

2007-10-19 Thread Doug Cutting
Karl Wettin wrote: So what's up with this method? Did anyone ever figure out what it is used for? I found the origin of it. It was added in 2004: http://svn.apache.org/viewvc?view=rev&revision=150206. This was to fix issue: http://issues.apache.org/bugzilla/show_bug.cgi?id=18927 But the

Re: lucene indexing and merge process

2007-10-19 Thread John Wang
Hi Mike: This is an excellent analysis. To do 2), we tried computing the field cache at indexing time to avoid "parsing" at search time. But what we've found out was that this degrades indexing (because it computes the entire fieldcache, not in segements) which was not acceptable to our

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: CachedTokenStream.java) > Extend contrib Highlighter to properly support phrase

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: CachedTokenStream.java) > Extend contrib Highlighter to properly support phrase

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: SpanHighlighterTest.java) > Extend contrib Highlighter to properly support phra

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: SpanScorer.java) > Extend contrib Highlighter to properly support phrase querie

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: WeightedSpanTerm.java) > Extend contrib Highlighter to properly support phrase

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: SpanHighlighterTest.java) > Extend contrib Highlighter to properly support phra

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: QuerySpansExtractor.java) > Extend contrib Highlighter to properly support phra

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: MemoryIndex.java) > Extend contrib Highlighter to properly support phrase queri

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: SpanScorer.java) > Extend contrib Highlighter to properly support phrase querie

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: SimpleFormatter.java) > Extend contrib Highlighter to properly support phrase q

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: QuerySpansExtractor.java) > Extend contrib Highlighter to properly support phra

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: Highlighter.java) > Extend contrib Highlighter to properly support phrase queri

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: HighlighterTest.java) > Extend contrib Highlighter to properly support phrase q

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: QuerySpansExtractor.java) > Extend contrib Highlighter to properly support phra

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: Highlighter.java) > Extend contrib Highlighter to properly support phrase queri

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: Highlighter.java) > Extend contrib Highlighter to properly support phrase queri

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: Highlighter.java) > Extend contrib Highlighter to properly support phrase queri

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: HighlighterTest.java) > Extend contrib Highlighter to properly support phrase q

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: HighlighterTest.java) > Extend contrib Highlighter to properly support phrase q

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: Highlighter.java) > Extend contrib Highlighter to properly support phrase queri

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: Formatter.java) > Extend contrib Highlighter to properly support phrase queries

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: Encoder.java) > Extend contrib Highlighter to properly support phrase queries a

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: DefaultEncoder.java) > Extend contrib Highlighter to properly support phrase qu

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: spanhighlighter12.patch Nice little addition courtesy of Michael Goddard: "...In our Luc

[jira] Updated: (LUCENE-550) InstantiatedIndex - faster but memory consuming index

2007-10-19 Thread Karl Wettin (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karl Wettin updated LUCENE-550: --- Attachment: LUCENE-550_20071019_no_core_changes.txt In this patch: * IndexReader.terms(Term) optimi

[jira] Resolved: (LUCENE-1031) Fixes a handful of misspellings/mistakes in changes.txt

2007-10-19 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-1031. Resolution: Fixed Fix Version/s: 2.3 Lucene Fields: [New, Patch Availabl

[jira] Assigned: (LUCENE-1031) Fixes a handful of misspellings/mistakes in changes.txt

2007-10-19 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-1031: -- Assignee: Michael McCandless > Fixes a handful of misspellings/mistakes in cha

[jira] Commented: (LUCENE-1031) Fixes a handful of misspellings/mistakes in changes.txt

2007-10-19 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12536205 ] Michael McCandless commented on LUCENE-1031: Sheesh, we committers really can't spell!! Thanks Mark! I

Re: Remove TermEnum.skipTo(Term target)

2007-10-19 Thread Karl Wettin
Wolfgang Hoschek wrote at Wed, 04 May 2005 20:59:24 GMT: I was considering an efficient impl of TermEnum.skipTo(Term target) for the MemoryIndex. But then I realized that nothing anywhere in Lucene calls that method. It's effectively dead code; a remainder of a previous ice age - nothing would

Re: [jira] Commented: (LUCENE-1029) Illegal character replacements in ISOLatin1AccentFilter

2007-10-19 Thread Mark Miller
If you are to compare with stemmers, consider that these creates unique tokens that does not interfere with semantic meanings. Not starting anything here again, but it took me so darn long to find something that porter stems and kills the semantic meaning that I had to share. That damn alg

[jira] Updated: (LUCENE-1031) Fixes a handful of misspellings/mistakes in changes.txt

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-1031: Attachment: changes.txt.patch > Fixes a handful of misspellings/mistakes in changes.txt >

[jira] Created: (LUCENE-1031) Fixes a handful of misspellings/mistakes in changes.txt

2007-10-19 Thread Mark Miller (JIRA)
Fixes a handful of misspellings/mistakes in changes.txt --- Key: LUCENE-1031 URL: https://issues.apache.org/jira/browse/LUCENE-1031 Project: Lucene - Java Issue Type: Improvement

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: HighlighterTest.java) > Extend contrib Highlighter to properly support phrase q

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: CachedTokenStream.java) > Extend contrib Highlighter to properly support phrase

[jira] Updated: (LUCENE-794) Extend contrib Highlighter to properly support phrase queries and span queries

2007-10-19 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-794: --- Attachment: (was: QuerySpansExtractor.java) > Extend contrib Highlighter to properly support phra