Re: Deadlock when multi-threading DocumentsWriter

2008-07-30 Thread Michael McCandless
The lock acquire order for all call stacks that lock on these two classes should be IndexWriter then DocumentsWriter, as is the case with IndexWriter.doFlush calling DocumentsWriter.pauseAllThreads. So you shouldn't hit a thread deadlock. Also, doFlush is called when it's time to write

[jira] Commented: (LUCENE-1340) Make it posible not to include TF information in index

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12618272#action_12618272 ] Michael McCandless commented on LUCENE-1340: Sigh, I too missed that we broke

[jira] Commented: (LUCENE-1340) Make it posible not to include TF information in index

2008-07-30 Thread Grant Ingersoll (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12618290#action_12618290 ] Grant Ingersoll commented on LUCENE-1340: - OK, I think we should call a vote on

[VOTE] Break Back Compatibility Contract on Fieldable

2008-07-30 Thread Grant Ingersoll
As they say, rules are meant to be broken... For a variety of reasons, some outlined below, I (and others) would like us to break our back compatibility requirements and allow for modifying the Fieldable interface in 2.x releases with the 3.x plan to be to separate out write side

[jira] Created: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
IndexWriter.rollback can hang if a previous call hit an exception - Key: LUCENE-1347 URL: https://issues.apache.org/jira/browse/LUCENE-1347 Project: Lucene - Java Issue Type:

[jira] Updated: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1347: --- Attachment: LUCENE-1347.patch Attached patch. I plan to commit in a day or so, and

Re: [VOTE] Break Back Compatibility Contract on Fieldable

2008-07-30 Thread Michael McCandless
+1 to all three. Mike Grant Ingersoll wrote: As they say, rules are meant to be broken... For a variety of reasons, some outlined below, I (and others) would like us to break our back compatibility requirements and allow for modifying the Fieldable interface in 2.x releases with the 3.x

Re: [VOTE] Break Back Compatibility Contract on Fieldable

2008-07-30 Thread Erik Hatcher
+1 to all three from me. Darn you, Java, for making object- orientation kludgey. Erik On Jul 30, 2008, at 8:44 AM, Grant Ingersoll wrote: As they say, rules are meant to be broken... For a variety of reasons, some outlined below, I (and others) would like us to break our back

Re: [VOTE] Break Back Compatibility Contract on Fieldable

2008-07-30 Thread Michael Busch
+1 to all three items. This is good stuff. -Michael Grant Ingersoll wrote: As they say, rules are meant to be broken... For a variety of reasons, some outlined below, I (and others) would like us to break our back compatibility requirements and allow for modifying the Fieldable interface in

[jira] Commented: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12618356#action_12618356 ] Yonik Seeley commented on LUCENE-1347: -- Looks good Mike! Just one question it

[jira] Updated: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1347: --- Attachment: LUCENE-1347.patch Woops, you're right -- thanks for the review! Two

Re: [VOTE] Break Back Compatibility Contract on Fieldable

2008-07-30 Thread DM Smith
As a user, I have no problem with this, as I have a pulse:) If I understand that it just breaks software backward compatibility and not compatibility with the index itself. Minor software changes are no big deal to me. I would still expect that a newer API would still read earlier indexes.

Re: [VOTE] Break Back Compatibility Contract on Fieldable

2008-07-30 Thread Grant Ingersoll
On Jul 30, 2008, at 11:07 AM, DM Smith wrote: I'm not sure that the comment that this gives anyone with a pulse enough time to react is particularly accurate or helpful. It all depends upon effective communication (such as to Lucene user's mailing list and package maintainers). It was

[jira] Updated: (LUCENE-1333) Token implementation needs improvements

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1333: --- Attachment: LUCENE-1333.patch This patch looks good; thanks DM! I made a few

[jira] Commented: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12618392#action_12618392 ] Michael McCandless commented on LUCENE-1347: Super, I'll commit shortly --

RE: Deadlock when multi-threading DocumentsWriter

2008-07-30 Thread Jagadesh Nomula
Thanks again for the comments. I am trying to split the IndexWriter.addDocuent into 2 different methods getThreadState() and finishDoc(), the idea is to make them multithreaded for parallelWriter. I am running into this situation, wherein DocumentsWriter.getThreadState and

[jira] Updated: (LUCENE-1347) IndexWriter.rollback can hang if a previous call hit an exception

2008-07-30 Thread Michael McCandless (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-1347: --- Fix Version/s: (was: 2.3.2) 2.3.3 IndexWriter.rollback can

[jira] Updated: (LUCENE-973) Token of returns in CJK

2008-07-30 Thread Koji Sekiguchi (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Sekiguchi updated LUCENE-973: -- Attachment: with-patch.jpg without-patch.jpg I attached Solr analysis screen

[jira] Updated: (LUCENE-973) Token of returns in CJK

2008-07-30 Thread Steven Rowe (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steven Rowe updated LUCENE-973: --- Hi Koji, The test class in your patch is a nice addition. bq. There is no problem in CJKAnalyzer.

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley
disclaimer: this is just for fun differences should be in the noise in any complex system, and I'm not suggesting any code changes. Actually, with 32 bit registers, x0 should be faster than x==-1 by one cycle. If it doesn't test faster, then it's because of some optimizations that could be

Re: Deadlock when multi-threading DocumentsWriter

2008-07-30 Thread Michael McCandless
Could you also post your mods to DocumentsWriter? Eg doGetThreadState and finishDocWithThreadState? Or, better yet, post a full patch with a unit test showing the hang? I think it should be OK to have one thread get a ThreadState and another thread finish indexing the doc with that

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Michael McCandless
Neat! Do you know how to get the corresponding asm that the hostpot compiler produces? This way we can see if this difference survives through java... Mike Yonik Seeley wrote: disclaimer: this is just for fun differences should be in the noise in any complex system, and I'm not

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley
On Wed, Jul 30, 2008 at 3:06 PM, Michael McCandless [EMAIL PROTECTED] wrote: Neat! Do you know how to get the corresponding asm that the hostpot compiler produces? This way we can see if this difference survives through java... Unfortunately, no. I've looked in the past and couldn't find

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Stephen Green
On Jul 30, 2008, at 3:12 PM, Yonik Seeley wrote: On Wed, Jul 30, 2008 at 3:06 PM, Michael McCandless [EMAIL PROTECTED] wrote: Neat! Do you know how to get the corresponding asm that the hostpot compiler produces? This way we can see if this difference survives through java...

Re: [jira] Updated: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley
On Wed, Jul 30, 2008 at 3:17 PM, Stephen Green [EMAIL PROTECTED] wrote: Might the description here: http://weblogs.java.net/blog/kohsuke/archive/2008/03/deep_dive_into.html help? Sweet! Thanks! -Yonik - To unsubscribe,

[jira] Commented: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread Yonik Seeley (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12618513#action_12618513 ] Yonik Seeley commented on LUCENE-1345: -- Eks, I just tried your first

Re: [jira] Commented: (LUCENE-1345) Allow Filter as clause to BooleanQuery

2008-07-30 Thread eks dev
than we conclude, comparison with 0 is faster :) Maybe something on my XP machine was doing something in background I have not noticed, stealing cycles, on Windows this can not be easily controlled. or when I tested it the other day, I used comparison with -1 while((doc=it.next()) -1) could

[jira] Commented: (LUCENE-973) Token of returns in CJK

2008-07-30 Thread Koji Sekiguchi (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12618534#action_12618534 ] Koji Sekiguchi commented on LUCENE-973: --- Hi Steven, bq. The test class in your patch

[jira] Commented: (LUCENE-973) Token of returns in CJK

2008-07-30 Thread Steven Rowe (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12618540#action_12618540 ] Steven Rowe commented on LUCENE-973: Sorry Toru, I saw Koji's two most recent comments