[jira] Updated: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-06 Thread Michael Busch (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Busch updated LUCENE-1473: -- Attachment: custom-externalizable-reader.patch I really wouldn't want to add another backwards

[jira] Commented: (LUCENE-1448) add getFinalOffset() to TokenStream

2008-12-06 Thread Michael Busch (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654044#action_12654044 ] Michael Busch commented on LUCENE-1448: --- {quote} But: you'd still need a way in this

[jira] Commented: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654045#action_12654045 ] Michael McCandless commented on LUCENE-1480: Shai, did you intend on changing

[jira] Commented: (LUCENE-1476) BitVector implement DocIdSet

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654047#action_12654047 ] Michael McCandless commented on LUCENE-1476: bq. Mike, you were planning on ma

[jira] Updated: (LUCENE-1479) TrecDocMaker skips over documents when "Date" is missing from documents

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1479: --- Lucene Fields: [New, Patch Available] (was: [Patch Available, New]) Fix Version/

Re: Java logging in Lucene

2008-12-06 Thread Grant Ingersoll
On Dec 5, 2008, at 11:36 PM, Shai Erera wrote: What do you have against JUL? I've used it and in my company (which is quite a large one btw) we've moved to JUL just because it's so easy to configure, comes already with the JDK and very intuitive. Perhaps it has some shortcomings which I'

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654055#action_12654055 ] Michael McCandless commented on LUCENE-831: --- [Note: my understanding of this are

[jira] Commented: (LUCENE-1478) Missing possibility to supply custom FieldParser when sorting search results

2008-12-06 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654056#action_12654056 ] Uwe Schindler commented on LUCENE-1478: --- I found a hidden bug in FieldSortedHitQueue

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654057#action_12654057 ] Michael McCandless commented on LUCENE-831: --- One more thing here... while random

[jira] Updated: (LUCENE-1478) Missing possibility to supply custom FieldParser when sorting search results

2008-12-06 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-1478: -- Attachment: LUCENE-1478.patch Here is the patch using the superinterface for all field parsers

[jira] Commented: (LUCENE-1478) Missing possibility to supply custom FieldParser when sorting search results

2008-12-06 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654062#action_12654062 ] Uwe Schindler commented on LUCENE-1478: --- I forget to mention, this patch also extend

[jira] Created: (LUCENE-1481) Sort and SortField does not have equals() and hashCode()

2008-12-06 Thread Uwe Schindler (JIRA)
Sort and SortField does not have equals() and hashCode() Key: LUCENE-1481 URL: https://issues.apache.org/jira/browse/LUCENE-1481 Project: Lucene - Java Issue Type: Bug Compon

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654064#action_12654064 ] Mark Miller commented on LUCENE-831: Ah, the dirty secret of 831 - there is plenty more

[jira] Commented: (LUCENE-1478) Missing possibility to supply custom FieldParser when sorting search results

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654068#action_12654068 ] Michael McCandless commented on LUCENE-1478: bq. The problem is, that the comp

[jira] Assigned: (LUCENE-1481) Sort and SortField does not have equals() and hashCode()

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless reassigned LUCENE-1481: -- Assignee: Michael McCandless > Sort and SortField does not have equals() and h

[jira] Commented: (LUCENE-1481) Sort and SortField does not have equals() and hashCode()

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654070#action_12654070 ] Michael McCandless commented on LUCENE-1481: This sounds reasonable. Probably

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Robert Newson (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654069#action_12654069 ] Robert Newson commented on LUCENE-831: -- This enhancement is particularly interesting t

[jira] Commented: (LUCENE-1481) Sort and SortField does not have equals() and hashCode()

2008-12-06 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654100#action_12654100 ] Uwe Schindler commented on LUCENE-1481: --- No problem, I check out a new svn trunk cop

[jira] Updated: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated LUCENE-831: --- Attachment: LUCENE-831.patch Hmmm - not sure what is up. There is already one small conflict for me (

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654105#action_12654105 ] Uwe Schindler commented on LUCENE-831: -- Maybe every asignee should tag his issues that

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654106#action_12654106 ] Uwe Schindler commented on LUCENE-831: -- Maybe we need two trunks or branches or whatev

[jira] Commented: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654108#action_12654108 ] Michael McCandless commented on LUCENE-1473: {quote} > Often in the past was e

[jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Robert Newson (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654109#action_12654109 ] Robert Newson commented on LUCENE-831: -- The conflict was easy to resolve, it was just

[jira] Commented: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-06 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654116#action_12654116 ] Shai Erera commented on LUCENE-1480: In IndexWriter, almost all the code used if (info

IndexWriter.flush performance

2008-12-06 Thread mimounl
Hello, In my application, I need to flush data each time a modification is made. So each time an entry is added in the lucene index we call IndexWriter.flush this way all data are secured on file system. We noticed that this operation is more and more time consuming while the size of the index r

Re: Java logging in Lucene

2008-12-06 Thread Shai Erera
On the performance side, I don't expect to see any different performance than what we have today, since checking if infoStream != null should be similar to logger.isLoggable (or the equivalent methods from SLF4J). I'll look at SLF4J, open an issue and work out a patch. On Sat, Dec 6, 2008 at 1:22

Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Mark Miller
Michael McCandless (JIRA) wrote: [ https://issues.apache.org/jira/browse/LUCENE-831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654057#action_12654057 ] Michael McCandless commented on LUCENE-831: --- One

Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Mark Miller
Michael McCandless (JIRA) wrote: However, stepping back, this is poor approach. We should instead be doing what MultiSearcher does, which is gather top results per-sub-reader, and then merge-sort the results. At that point, to do the merge, we only need actual field values for those docs in

[jira] Commented: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654142#action_12654142 ] Michael McCandless commented on LUCENE-1480: OK let's leave IndexWriter as-is.

[jira] Resolved: (LUCENE-1480) Wrap messages output with a check of InfoStream != null

2008-12-06 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-1480. Resolution: Fixed Fix Version/s: (was: 2.4.1) 2.9

Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Michael McCandless
Mark Miller wrote: Michael McCandless (JIRA) wrote: However, stepping back, this is poor approach. We should instead be doing what MultiSearcher does, which is gather top results per-sub-reader, and then merge-sort the results. At that point, to do the merge, we only need actual field va

[jira] Commented: (LUCENE-1473) Implement standard Serialization across Lucene versions

2008-12-06 Thread Michael Busch (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654147#action_12654147 ] Michael Busch commented on LUCENE-1473: --- {quote} Your first example is missing the r

Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Michael McCandless
Mark Miller wrote: EG when sorting by field, we could pull say an IntData iterator from the reader, and then access the int values in docID order as we visit the docs. We need random access after collecting/visiting though...do we put what we collect into a map? If a lot of docs match? I g

Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Mark Miller
Michael McCandless wrote: Today, with IndexSearcher(MultiReader), the FieldSortedHitQueue asks FieldCache to materialize the full array for each field. Whereas MultiSearcher only asks each child reader to materialize its array for the field, which is better because on reopen we only need to ini

Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Mark Miller
MultiSearcher has a few aspects I don't like. Do you mean the score differences vs IndexSearcher(MultiReader), or is there something else? And rewrite does not work properly. And to get 30 docs over 3 indexes, you ask for 90. And sort twice. Minor gripes, but bugs me non the less. - Mark

Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Marvin Humphrey
On Sat, Dec 06, 2008 at 04:21:04PM -0500, Mark Miller wrote: > And to get 30 docs over 3 indexes, you ask for 90. And sort twice. However, this scales with the number of segments, not the number of documents. Marvin Humphrey -

Re: [jira] Commented: (LUCENE-831) Complete overhaul of FieldCache API/Implementation

2008-12-06 Thread Mark Miller
Marvin Humphrey wrote: On Sat, Dec 06, 2008 at 04:21:04PM -0500, Mark Miller wrote: And to get 30 docs over 3 indexes, you ask for 90. And sort twice. However, this scales with the number of segments, not the number of documents. Marvin Humphre Right. They are all minor gripes to be s

[jira] Resolved: (LUCENE-1390) add ISOLatinAccentFilter and deprecate ISOLatin1AccentFilter

2008-12-06 Thread Mark Miller (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller resolved LUCENE-1390. - Resolution: Fixed Lucene Fields: [New, Patch Available] (was: [Patch Available, New]) Co

[jira] Commented: (LUCENE-1390) add ISOLatinAccentFilter and deprecate ISOLatin1AccentFilter

2008-12-06 Thread Andi Vajda (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654160#action_12654160 ] Andi Vajda commented on LUCENE-1390: Thanks Mark ! > add ISOLatinAccentFilter and de

JavaCC and Demo files

2008-12-06 Thread Grant Ingersoll
Anyone else seeing: javacc-notice: [echo] [echo] One or more of the JavaCC .jj files is newer than its corresponding [echo] .java file. Run the "javacc" target to regenerate the artifacts. [echo] I think the demo files are out of date for the HTML parser, bu

Re: JavaCC and Demo files

2008-12-06 Thread Mark Miller
Grant Ingersoll wrote: Anyone else seeing: javacc-notice: [echo] [echo] One or more of the JavaCC .jj files is newer than its corresponding [echo] .java file. Run the "javacc" target to regenerate the artifacts. [echo] I think the demo files are out of date f

Re: JavaCC and Demo files

2008-12-06 Thread Grant Ingersoll
I figured as much, but wanted to double check that it wasn't just caused by the fact that I'm on JavaCC 4.x and the files themselves are 3.x. On Dec 6, 2008, at 9:07 PM, Mark Miller wrote: Grant Ingersoll wrote: Anyone else seeing: javacc-notice: [echo] [echo] One or more of

Build failed in Hudson: Lucene-trunk #668

2008-12-06 Thread Apache Hudson Server
See http://hudson.zones.apache.org/hudson/job/Lucene-trunk/668/changes Changes: [gsingers] make constructors public [markrmiller] LUCENE-1390: Added ASCIIFoldingFilter, a Filter that converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (

hudson failure

2008-12-06 Thread Mark Miller
Ideas? clover: common.compile-core: [mkdir] Created dir: /export/home/hudson/hudson-slave/workspace/Lucene-trunk/trunk/build/classes/java [javac] Compiling 353 source files to /export/home/hudson/hudson-slave/workspace/Lucene-trunk/trunk/build/classes/java [clover] Clover Version 1.3.1

Re: hudson failure

2008-12-06 Thread Andi Vajda
On Sat, 6 Dec 2008, Mark Miller wrote: Ideas? clover: common.compile-core: [mkdir] Created dir: /export/home/hudson/hudson-slave/workspace/Lucene-trunk/trunk/build/classes/java [javac] Compiling 353 source files to /export/home/hudson/hudson-slave/workspace/Lucene-trunk/trunk/build/clas

Re: hudson failure

2008-12-06 Thread Mark Miller
Thanks. I gave excluding the class a shot. Clover is a code coverage analysis tool. http://hudson.zones.apache.org/hudson/job/Lucene-trunk/clover/ - mark Andi Vajda wrote: On Sat, 6 Dec 2008, Mark Miller wrote: Ideas? clover: common.compile-core: [mkdir] Created dir: /export/home/huds