Re: Hiding JIRA issues

2009-11-27 Thread Michael McCandless
Hi Mark, Thanks for the suggestions! I've forwarded this to infrastruct...@a.o... Some other responses below: On Fri, Nov 27, 2009 at 12:49 AM, Mark Lassau wrote: > The root cause is most likely http://jira.atlassian.com/browse/JRA-15548 > This bug was fixed in JIRA v4.0 and usually only effec

[jira] Updated: (LUCENE-2069) fix LowerCaseFilter for unicode 4.0

2009-11-27 Thread Simon Willnauer (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon Willnauer updated LUCENE-2069: Attachment: LUCENE-2069.patch I updated the patch with another testcase for a trailing sur

[jira] Commented: (LUCENE-2061) Create benchmark & approach for testing Lucene's near real-time performance

2009-11-27 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783093#action_12783093 ] Michael McCandless commented on LUCENE-2061: BTW, based on these last results

[jira] Commented: (LUCENE-2069) fix LowerCaseFilter for unicode 4.0

2009-11-27 Thread Uwe Schindler (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783094#action_12783094 ] Uwe Schindler commented on LUCENE-2069: --- Looks good, +1 to commit! > fix LowerCaseF

[jira] Updated: (LUCENE-1844) Speed up junit tests

2009-11-27 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1844: --- Attachment: LUCENE-1844.patch New patch, trims a few more tests. I'm done for now..

Re: svn commit: r884870 - /lucene/java/trunk/src/java/org/apache/lucene/search/Collector.java

2009-11-27 Thread Simon Willnauer
Very good addition Mike! Thanks! On Fri, Nov 27, 2009 at 2:36 PM, wrote: > Author: mikemccand > Date: Fri Nov 27 13:36:22 2009 > New Revision: 884870 > > URL: http://svn.apache.org/viewvc?rev=884870&view=rev > Log: > improve javadocs for Collector.acceptsDocsOutOfOrder > > Modified: >    lucene/

Fwd: Socket and file locks

2009-11-27 Thread Thomas Mueller
Hi, I'm wondering if your are interested in automatically releasing the write lock. See also my comments on https://issues.apache.org/jira/browse/LUCENE-1877 - I thought it's a problem worth solving, because it's also in the Lucene FAQ list at http://wiki.apache.org/lucene-java/LuceneFAQ#What_is_t

Re: Socket and file locks

2009-11-27 Thread Michael McCandless
I think a LockFactory for Lucene that implemented the ideas you & Marvin are discussing in LUCENE-1877, and/or the approach you implemented in the H2 DB, would be a useful addition to Lucene! For many apps, the simple LockFactory impls suffice, but for apps where multiple machines can become the

[jira] Resolved: (LUCENE-2075) Share the Term -> TermInfo cache across threads

2009-11-27 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless resolved LUCENE-2075. Resolution: Fixed Thanks all! > Share the Term -> TermInfo cache across threads >

Re: (LUCENE-1844) Speed up junit tests

2009-11-27 Thread Erick Erickson
But then I got to thinking. I admit I've only scratched the surface of the JUnit4 parallelization stuff. That said, it seems like the real benefit comes from making use of multiple cores, we don't get huge speedups just from running multiple threads at once on a single core. Which makes sense i

deadlock in TestCrash

2009-11-27 Thread Michael McCandless
RAMFile has this method: final synchronized byte[] addBuffer(int size) { byte[] buffer = newBuffer(size); if (directory!=null) synchronized (directory) { // Ensure addition of buffer and adjustment to directory size are atomic wrt directory buffers.add(buffer);

Re: deadlock in TestCrash

2009-11-27 Thread Jason Rutherglen
If syncing on directory is not important, then sizeInBytes can be an AtomicLong? On Fri, Nov 27, 2009 at 8:37 AM, Michael McCandless wrote: > RAMFile has this method: > >  final synchronized byte[] addBuffer(int size) { >    byte[] buffer = newBuffer(size); >    if (directory!=null) >      synchr

Sentence detection/extraction as Tokenizer?

2009-11-27 Thread Otis Gospodnetic
Hello, The contrib/wordnet package contains an AnalyzerUtil class with a method that extracts sentences from text/String. It is super-simplistic, so probably not very accurate, but I am wondering if *conceptually* it would make sense to have a Tokenizer that extracts sentences? I suppose that

Re: (LUCENE-1844) Speed up junit tests

2009-11-27 Thread Michael McCandless
On Fri, Nov 27, 2009 at 10:52 AM, Erick Erickson wrote: > But then I got to thinking. I admit I've only scratched the > surface of the JUnit4 parallelization stuff. That said, it > seems like the real benefit comes from making use of > multiple cores, we don't get huge speedups just from > run

Re: deadlock in TestCrash

2009-11-27 Thread Michael McCandless
True... I'll make that change too. Mike On Fri, Nov 27, 2009 at 12:56 PM, Jason Rutherglen wrote: > If syncing on directory is not important, then sizeInBytes can be an > AtomicLong? > > On Fri, Nov 27, 2009 at 8:37 AM, Michael McCandless > wrote: >> RAMFile has this method: >> >>  final synch

[jira] Commented: (LUCENE-2069) fix LowerCaseFilter for unicode 4.0

2009-11-27 Thread Robert Muir (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783158#action_12783158 ] Robert Muir commented on LUCENE-2069: - Simon you are right, there is no problem. mayb

Re: (LUCENE-1844) Speed up junit tests

2009-11-27 Thread Erick Erickson
<<>> That's the implication I took from here: http://ant-contrib.sourceforge.net/tasks/tasks/index.html Ignorance is bliss, I didn't find the ForEach by looking at Ant documentation, but by googling "ant parallel". Turns out this is in Contrib. I don't even know if it's current. Tell ya' what. I

[jira] Created: (LUCENE-2096) Investigate parallelizing Ant junit tests

2009-11-27 Thread Erick Erickson (JIRA)
Investigate parallelizing Ant junit tests - Key: LUCENE-2096 URL: https://issues.apache.org/jira/browse/LUCENE-2096 Project: Lucene - Java Issue Type: Improvement Components: Build

Re: svn commit: r884895 - in /lucene/java/trunk: ./ src/java/org/apache/lucene/index/ src/java/org/apache/lucene/util/cache/ src/test/org/apache/lucene/util/cache/

2009-11-27 Thread Michael McCandless
That is a spooky failure -- it's an assertion I added. Hmm I can also get it to happen if I run the tests enough. I'll try fixing the seed! Mike On Fri, Nov 27, 2009 at 3:57 PM, Robert Muir wrote: > I got a suspicious test error today, I think related to the change. > > I cant reproduce it by

RE: svn commit: r884895 - in /lucene/java/trunk: ./ src/java/org/apache/lucene/index/ src/java/org/apache/lucene/util/cache/ src/test/org/apache/lucene/util/cache/

2009-11-27 Thread Uwe Schindler
You can pass the printed random seed to the newRandom call in the failing testcase. I would do it now, but I wanted to go to bed, so I can look into it tomorrow. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- >

Re: svn commit: r884895 - in /lucene/java/trunk: ./ src/java/org/apache/lucene/index/ src/java/org/apache/lucene/util/cache/ src/test/org/apache/lucene/util/cache/

2009-11-27 Thread Michael McCandless
Indeed that's what I've done, and it succeeds in always reproducing the issue! The problem is when I == two TermInfos... they have a false difference in their skipOffset, because the field has omitTFAP enabled. I need to conditionalize that out of the equality... Mike On Fri, Nov 27, 2009 at 4:

RE: svn commit: r884895 - in /lucene/java/trunk: ./ src/java/org/apache/lucene/index/ src/java/org/apache/lucene/util/cache/ src/test/org/apache/lucene/util/cache/

2009-11-27 Thread Uwe Schindler
That's correct, NumericField always have omitTFAP by default (makes no sense for NRQ). But, why does the assertion failure only occurs with this random seed? Maybe because we hit a termenum that returns no terms or something like that? - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http:

Re: svn commit: r884895 - in /lucene/java/trunk: ./ src/java/org/apache/lucene/index/ src/java/org/apache/lucene/util/cache/ src/test/org/apache/lucene/util/cache/

2009-11-27 Thread Michael McCandless
Actually, I was wrong. omitTFAP has nothing to do with it. It's happening because TermInfo.equals is checking the skipOffset, but for terms whose docFreq is < skipInterval, that field is unused. And then in the way the cache gets populated, it's hit or miss what particular leftover value we have

[jira] Resolved: (LUCENE-2069) fix LowerCaseFilter for unicode 4.0

2009-11-27 Thread Robert Muir (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-2069. - Resolution: Fixed Committed revision 885024. > fix LowerCaseFilter for unicode 4.0 > --

Re: svn commit: r884895 - in /lucene/java/trunk: ./ src/java/org/apache/lucene/index/ src/java/org/apache/lucene/util/cache/ src/test/org/apache/lucene/util/cache/

2009-11-27 Thread Michael McCandless
OK should be fixed now! Thanks for catching it Robert :) Mike On Fri, Nov 27, 2009 at 4:33 PM, Michael McCandless wrote: > Actually, I was wrong.  omitTFAP has nothing to do with it. > > It's happening because TermInfo.equals is checking the skipOffset, but > for terms whose docFreq is < skipIn

[jira] Updated: (LUCENE-1785) Simple FieldCache merging

2009-11-27 Thread Jason Rutherglen (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Rutherglen updated LUCENE-1785: - Attachment: LUCENE-1785.patch This mostly works, not committable. I've noticed we're cre