Build failed in Hudson: Lucene-Nightly #150

2007-07-13 Thread hudson
See http://lucene.zones.apache.org:8080/hudson/job/Lucene-Nightly/150/changes -- started Checking out http://svn.apache.org/repos/asf/lucene/java/trunk ERROR: Failed to check out http://svn.apache.org/repos/asf/lucene/java/trunk org.tmatesoft.svn.core.SVNExc

[EMAIL PROTECTED]: Project lucene-java (in module lucene-java) failed

2007-07-13 Thread Jason van Zyl
To whom it may engage... This is an automated request, but not an unsolicited one. For more information please visit http://gump.apache.org/nagged.html, and/or contact the folk at [EMAIL PROTECTED] Project lucene-java has an issue affecting its community integration. This issue affects

[EMAIL PROTECTED]: Project lucene-java (in module lucene-java) failed

2007-07-13 Thread Jason van Zyl
To whom it may engage... This is an automated request, but not an unsolicited one. For more information please visit http://gump.apache.org/nagged.html, and/or contact the folk at [EMAIL PROTECTED] Project lucene-java has an issue affecting its community integration. This issue affects

[jira] Updated: (LUCENE-959) Document Vector->ArrayList

2007-07-13 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated LUCENE-959: Attachment: Document.patch Trivial patch attached. Any concerns? > Document Vector->ArrayList > --

[jira] Created: (LUCENE-959) Document Vector->ArrayList

2007-07-13 Thread Yonik Seeley (JIRA)
Document Vector->ArrayList -- Key: LUCENE-959 URL: https://issues.apache.org/jira/browse/LUCENE-959 Project: Lucene - Java Issue Type: Improvement Reporter: Yonik Seeley Priority: Trivial Doc

[jira] Resolved: (LUCENE-958) MultiSearcher Filtering with more than one index is broken

2007-07-13 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley resolved LUCENE-958. - Resolution: Invalid > MultiSearcher Filtering with more than one index is broken > --

[jira] Commented: (LUCENE-958) MultiSearcher Filtering with more than one index is broken

2007-07-13 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512610 ] Yonik Seeley commented on LUCENE-958: - Actually, I retract that I think it's broken. Filter.getBits() will be cal

Re: for a better spellchecker

2007-07-13 Thread Mathieu Lecarme
The SpellChecker code mix indexing function, ngram treatment, and querying functions. Extending it will not produce clean code. Is it relevant to first refactor SpellChecker code for extracting dictionary reading function and indexing/searching functions? SpellChecker will get a method to add

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread Yonik Seeley
On 7/13/07, eks dev <[EMAIL PROTECTED]> wrote: Would you expect it to be faster on 64bit jvm, is it worth digging deeper? No. It depends on the app. Certain apps do better with more registers, but Java can often be a bit slower because it's so object heavy (more bigger pointers in every object

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread eks dev
thanks for correcting me. as always, when it gets low, dirty and hardwired, call Yonik to help :) I do not know exactly why, but we tried it a while ago and all our lucene apps were faster on 32bit jvm Would you expect it to be faster on 64bit jvm, is it worth digging deeper? - Original

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread eks dev
>OK, this is a good datapoint. It sound like one has to test in their >own environment to find the optimal performance / RAM usage tradeoff. Absolutely! Especially when you are trying to find max RAM for buffers, sooner or later you hit gc(), and this is 100% environment dependant and has nothi

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread Yonik Seeley
On 7/13/07, eks dev <[EMAIL PROTECTED]> wrote: no, we have only 2G on this machine and 32bit jvm can utilize most of it (1..6G) leaving something to the OS, and is generally faster due to shorter pointers and more availyble registers on 32bit. Less available registers in 32 bit mode :-) Right

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread eks dev
no, we have only 2G on this machine and 32bit jvm can utilize most of it (1..6G) leaving something to the OS, and is generally faster due to shorter pointers and more availyble registers on 32bit. Lucene is not memory hungry for *our setup* and works perfectly fine with 32bit jvm... actually, o

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread Michael McCandless
"eks dev" <[EMAIL PROTECTED]> wrote: > >Interesting. This matches the experience Doron had where adding more > >RAM actually slowed things down a bit (posted to > >LUCENE-843). > > I know this intrigues you, so our fresh experience: Yes it does! Thanks :) > The bigger the RAM Buffer the faste

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread Peter Keegan
I see you have a 64-bit OS but 32-bit JVM. Have you tried a 64-bit JVM (with a bigger RAM buffer)? Peter On 7/13/07, eks dev <[EMAIL PROTECTED]> wrote: >Interesting. This matches the experience Doron had where adding more >RAM actually slowed things down a bit (posted to >LUCENE-843). I kno

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread Michael McCandless
"Grant Ingersoll" <[EMAIL PROTECTED]> wrote: > This is good stuff... Might be good to put a organized version of > this up on the Wiki under Best Practices I agree! I will update the ImproveIndexingSpeed page: http://wiki.apache.org/lucene-java/ImproveIndexingSpeed with these suggestion

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread eks dev
>Interesting. This matches the experience Doron had where adding more >RAM actually slowed things down a bit (posted to >LUCENE-843). I know this intrigues you, so our fresh experience: The bigger the RAM Buffer the faster indexing, this holds until you hit some limit that starts irritating gc(

[jira] Resolved: (LUCENE-938) I/O exceptions can cause loss of buffered deletes

2007-07-13 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-938. --- Resolution: Fixed Lucene Fields: [New, Patch Available] (was: [Patch Available

[jira] Commented: (LUCENE-938) I/O exceptions can cause loss of buffered deletes

2007-07-13 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512494 ] Michael McCandless commented on LUCENE-938: --- This patch looks great! Thanks Steve. I'll commit it shortly

[jira] Resolved: (LUCENE-759) Add n-gram tokenizers to contrib/analyzers

2007-07-13 Thread Otis Gospodnetic (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Gospodnetic resolved LUCENE-759. - Resolution: Fixed This should have been marked Fixed a while back. > Add n-gram tokenize

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread eks dev
Thanks Mike! well, this worked well for me :) logger.info("bufferd documents: "+ lineCntr + " Buffer size: "+ ixWriter.getRAMBufferSizeMB() + "Mb ; from that currently in use: " + ixWriter.ramSizeInBytes()); so I know the size per doc in RAM buffer now, the rest is easy now, just * 5.5 . Field

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread Grant Ingersoll
This is good stuff... Might be good to put a organized version of this up on the Wiki under Best Practices On Jul 13, 2007, at 8:13 AM, Michael McCandless wrote: Yeah it's not so easy now: Field.java does not have setters. You have to make your own class that implements Fieldable (or subcl

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread Michael McCandless
"eks dev" <[EMAIL PROTECTED]> wrote: > > Was 24M (and not more) clearly the fastest performance? > > No, this is kind of optimum. Throwing more memory up to 32M makes things > slightly faster at slow rate, having maximum at 32. After that things > start getting slower (slowly) Interesting. T

Re: Post mortem kudos for (LUCENE-843) :)

2007-07-13 Thread eks dev
Hi Mike, > Was 24M (and not more) clearly the fastest performance? No, this is kind of optimum. Throwing more memory up to 32M makes things slightly faster at slow rate, having maximum at 32. After that things start getting slower (slowly) We are not yet completely done with tuning, especia