Re: TestCodecs running time

2010-04-15 Thread Shai Erera
to reduce the number of iterations -- just make sure there's no seed to newRandom() so the distributing testing is effective. Mike On Fri, Apr 9, 2010 at 12:43 AM, Shai Erera ser...@gmail.com wrote: Hi I've noticed that TestCodecs takes an insanely long time to run on my machine

SnapshotDeletionPolicy throws NPE if no commit happened

2010-04-15 Thread Shai Erera
SDP throws NPE if the index includes no commits, but snapshot() is called. This is an extreme case, but can happen if one takes snapshots (for backup purposes for example) in a separate code segment than indexing, and does not know if commit was called or not. I think we should throw an

Re: SnapshotDeletionPolicy throws NPE if no commit happened

2010-04-15 Thread Shai Erera
: We should just let IW create a null commit on an empty directory, like it always did ;) Then a whole class of such problems disappears. On Thu, Apr 15, 2010 at 11:16, Shai Erera ser...@gmail.com wrote: SDP throws NPE if the index includes no commits, but snapshot() is called

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
for this. The question of what to call major release in X.Y.Z scheme - X or Y, is there, but immaterial :) I think it's okay to settle with X.Y, we have major releases and bugfixes, what that third number can be used for? On Thu, Apr 15, 2010 at 09:29, Shai Erera ser...@gmail.com wrote: So then I don't

Re: SnapshotDeletionPolicy throws NPE if no commit happened

2010-04-15 Thread Shai Erera
ear...@gmail.comwrote: We should just let IW create a null commit on an empty directory, like it always did ;) Then a whole class of such problems disappears. On Thu, Apr 15, 2010 at 11:16, Shai Erera ser...@gmail.com wrote: SDP throws NPE if the index includes no commits, but snapshot

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
structure back-support. Shai On Thu, Apr 15, 2010 at 1:12 PM, Michael McCandless luc...@mikemccandless.com wrote: 2010/4/15 Shai Erera ser...@gmail.com: One way is to define 'major' as X and minor X.Y, and another is to define major as 'X.Y' and minor as 'X.Y.Z'. I prefer the latter but don't

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
:57, Robert Muir rcm...@gmail.com wrote: On Thu, Apr 15, 2010 at 7:52 AM, Shai Erera ser...@gmail.com wrote: Well ... I must say that I completely disagree w/ dropping index structure back-support. Our customers will simply not hear of reindexing 10s of TBs of content because of version

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
15, 2010 at 14:57, Robert Muir rcm...@gmail.com wrote: On Thu, Apr 15, 2010 at 7:52 AM, Shai Erera ser...@gmail.com wrote: Well ... I must say that I completely disagree w/ dropping index structure back-support. Our customers will simply not hear of reindexing 10s of TBs of content

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
: On Thu, Apr 15, 2010 at 7:52 AM, Shai Erera ser...@gmail.com wrote: Well ... I must say that I completely disagree w/ dropping index structure back-support. Our customers will simply not hear of reindexing 10s of TBs of content because of version upgrades. Such a decision is key to Lucene

[jira] Commented: (LUCENE-2396) remove version from contrib/analyzers.

2010-04-15 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12857388#action_12857388 ] Shai Erera commented on LUCENE-2396: Robert I think this is great! Can we move more

[jira] Commented: (LUCENE-2396) remove version from contrib/analyzers.

2010-04-15 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12857396#action_12857396 ] Shai Erera commented on LUCENE-2396: Static? Weren't you against that!? But if we

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
I seriously don't understand the fuss around index format back compat. How many times is this changed such that it is too much to ask to keep X support X-1? I prefer to have ongoing segment merging but can live w/ a manual converter tool. Thing is - I'll probably not be able to develop one myself

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
The reason Earwin why online migration is faster is because when u finally need to *fully* migrate your index, most chances are that most of the segments are already on the newer format. Offline migration will just keep the application idle for some amount of time until ALL segments are migrated.

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
+1 on the Analyzers as well. Earwin, I think I don't mind if we introduce migrate() elsewhere rather than on IW. What I meant to say is that if we stick w/ index format back-compat and ongoing migration, then such a method would be useful on IW for customers to call to ensure they're on the

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
tool for the latest index format version. Simple as that. The rest, we can 'see what happens' ... Shai On Thu, Apr 15, 2010 at 11:29 PM, Grant Ingersoll gsing...@apache.orgwrote: On Apr 15, 2010, at 4:21 PM, Shai Erera wrote: +1 on the Analyzers as well. Earwin, I think I don't mind if we

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
at 11:33 PM, Robert Muir rcm...@gmail.com wrote: On Thu, Apr 15, 2010 at 4:21 PM, Shai Erera ser...@gmail.com wrote: Actually, I'd like to know if people like Robert (basically those who have no problem to reindex and don't understand the fuss around it) will want to change the index format

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
now. On Thu, Apr 15, 2010 at 4:50 PM, Shai Erera ser...@gmail.com wrote: Robert ... I'm sorry but changes to Analyzers don't *force* people to reindex. They can simply choose not to use the latest version. They can choose not to upgrade a Unicode version. They can copy the entire Analyzer

Re: Proposal about Version API relaxation

2010-04-15 Thread Shai Erera
control we have on that ... Shai On Fri, Apr 16, 2010 at 12:21 AM, DM Smith dmsmith...@gmail.com wrote: On Apr 15, 2010, at 4:50 PM, Shai Erera wrote: Robert ... I'm sorry but changes to Analyzers don't *force* people to reindex. They can simply choose not to use the latest version

[jira] Created: (LUCENE-2397) SnapshotDeletionPolicy.snapshot() throws NPE if no commits happened

2010-04-15 Thread Shai Erera (JIRA)
Type: Improvement Components: Index Reporter: Shai Erera Assignee: Shai Erera Priority: Minor Fix For: 3.1 SDP throws NPE if no commits occurred and snapshot() was called. I will replace it w/ throwing IllegalStateException. I'll also move

Re: Build failed in Hudson: Lucene-trunk #1157

2010-04-15 Thread Shai Erera
DB jars again ... I think this one is a false alarm. Shai On Fri, Apr 16, 2010 at 5:14 AM, Apache Hudson Server hud...@hudson.zones.apache.org wrote: See http://hudson.zones.apache.org/hudson/job/Lucene-trunk/1157/changes Changes: [mikemccand] speed up TestStressIndexing2

[jira] Resolved: (LUCENE-2316) Define clear semantics for Directory.fileLength

2010-04-14 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera resolved LUCENE-2316. Lucene Fields: [New, Patch Available] (was: [New]) Assignee: Shai Erera

[jira] Commented: (LUCENE-2159) Tool to expand the index for perf/stress testing.

2010-04-14 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12856845#action_12856845 ] Shai Erera commented on LUCENE-2159: This looks like a nice tool. But all it does

[jira] Commented: (LUCENE-2159) Tool to expand the index for perf/stress testing.

2010-04-14 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12856877#action_12856877 ] Shai Erera commented on LUCENE-2159: bq. I understand having a general performance

[jira] Commented: (LUCENE-2159) Tool to expand the index for perf/stress testing.

2010-04-14 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12856911#action_12856911 ] Shai Erera commented on LUCENE-2159: Which is fine - I think this would be a neat task

[jira] Commented: (LUCENE-2159) Tool to expand the index for perf/stress testing.

2010-04-14 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12856917#action_12856917 ] Shai Erera commented on LUCENE-2159: bq. There is an excellent section on it in LIA2

Re: Proposal about Version API relaxation

2010-04-14 Thread Shai Erera
Ahh ... a dream finally comes true ... what a great way to start a day :). +1 !!! I have some questions/comments though: * Index back compat should be maintained between major releases, like it is today, STRUCTURE-wise. So apps get a chance to incrementally upgrade their segments when they move

Re: Proposal about Version API relaxation

2010-04-14 Thread Shai Erera
Also, we will still need to maintain the Backwards section in CHANGES (or move it to API Changes), to help people upgrade from release to release. Just pointing that out as well. Shai On Thu, Apr 15, 2010 at 7:05 AM, Shai Erera ser...@gmail.com wrote: Ahh ... a dream finally comes true

Re: Proposal about Version API relaxation

2010-04-14 Thread Shai Erera
://www.lucidimagination.com (mobile) On Apr 15, 2010, at 12:05 AM, Shai Erera ser...@gmail.com wrote: Ahh ... a dream finally comes true ... what a great way to start a day :). +1 !!! I have some questions/comments though: * Index back compat should be maintained between major releases, like

[jira] Resolved: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-13 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera resolved LUCENE-2386. Resolution: Fixed Committed revision 933613. (take #2) IndexWriter commits unnecessarily

Proposal about Version API relaxation

2010-04-13 Thread Shai Erera
Hi I'd like to propose a relaxation on the Version API. Uwe, please read the entire email before you reply :). I was thinking, following a question on the user list, that the Version-based API may not be very intuitive to users, especially those who don't care about versioning, as well as very

[jira] Updated: (LUCENE-2316) Define clear semantics for Directory.fileLength

2010-04-13 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2316: --- Attachment: LUCENE-2316.patch Patch clarifies the contract, fixes the directories to adhere

Re: Proposal about Version API relaxation

2010-04-13 Thread Shai Erera
Well the no-arg ctor will be using Version.getDefault() which will throw an exception if not set, and delegate the call to the Version-aware ctor. On Tuesday, April 13, 2010, Robert Muir rcm...@gmail.com wrote: On Tue, Apr 13, 2010 at 11:27 AM, Shai Erera ser...@gmail.com wrote: I

Re: Proposal about Version API relaxation

2010-04-13 Thread Shai Erera
, this type of thing makes web applications no longer isolated. This is also a general contract for libraries: never ever rely on sysprops or statics. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de *From:* Shai Erera

Re: Proposal about Version API relaxation

2010-04-13 Thread Shai Erera
. -- DM On 04/13/2010 11:27 AM, Shai Erera wrote: Hi I'd like to propose a relaxation on the Version API. Uwe, please read the entire email before you reply :). I was thinking, following a question on the user list, that the Version-based API may not be very intuitive to users, especially those

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855870#action_12855870 ] Shai Erera commented on LUCENE-2386: I'm not sure if we're arguing about the same

[jira] Commented: (LUCENE-2316) Define clear semantics for Directory.fileLength

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855873#action_12855873 ] Shai Erera commented on LUCENE-2316: Well ... dir.fileLength is also used

[jira] Commented: (LUCENE-2392) Enable flexible scoring

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855875#action_12855875 ] Shai Erera commented on LUCENE-2392: Mike - it'll also be great if we can store

[jira] Commented: (LUCENE-2373) Change StandardTermsDictWriter to work with streaming and append-only filesystems

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855877#action_12855877 ] Shai Erera commented on LUCENE-2373: I'd rather not count on file length as well

Re: [jira] Commented: (LUCENE-2392) Enable flexible scoring

2010-04-12 Thread Shai Erera
into the index (this would be messy like how to get access to the aggregates you need anyway) is something different entirely, best left to research systems. You can't even do that with Terrier now. On Mon, Apr 12, 2010 at 3:35 AM, Shai Erera (JIRA) j...@apache.orgwrote: [ https

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855892#action_12855892 ] Shai Erera commented on LUCENE-2386: bq. what is the proper way (after this fix

[jira] Commented: (LUCENE-2392) Enable flexible scoring

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855913#action_12855913 ] Shai Erera commented on LUCENE-2392: I'd like to withdraw my request from above. I

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855924#action_12855924 ] Shai Erera commented on LUCENE-2386: I don't think that people need to write

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12856063#action_12856063 ] Shai Erera commented on LUCENE-2386: So just call new IW(), then rollback and ensure

[jira] Updated: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-12 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2386: --- Attachment: LUCENE-2386.patch Patch includes the proposed test in TestIndexWriter. I think

[jira] Resolved: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-11 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera resolved LUCENE-2386. Lucene Fields: [New, Patch Available] (was: [New]) Resolution: Fixed Committed revision

[jira] Commented: (LUCENE-1709) Parallelize Tests

2010-04-11 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855713#action_12855713 ] Shai Erera commented on LUCENE-1709: Committed revision 932878 with the following

Re: svn commit: r932873 - /lucene/dev/trunk/lucene/src/java/org/apache/lucene/index/IndexNotFoundException.java

2010-04-11 Thread Shai Erera
Sorry about that ... On Sun, Apr 11, 2010 at 3:10 PM, uschind...@apache.org wrote: Author: uschindler Date: Sun Apr 11 12:10:57 2010 New Revision: 932873 URL: http://svn.apache.org/viewvc?rev=932873view=rev Log: add missing license header Modified:

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-11 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855727#action_12855727 ] Shai Erera commented on LUCENE-2386: Committed revision 932917 for the revert

[jira] Updated: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-11 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2386: --- Attachment: LUCENE-2386.patch Fixes IndexFileDeleter, adds a proper test to TestIndexWriter

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-11 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855767#action_12855767 ] Shai Erera commented on LUCENE-2386: About IndexReader.listCommits ... the javadocs

[jira] Updated: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-11 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2386: --- Attachment: LUCENE-2386.patch Patch w/ proposed fixes. All tests pass, including Solr's

[jira] Updated: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-10 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2386: --- Attachment: LUCENE-2386.patch Patch updated to latest rev. + the proposed name change

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-09 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855344#action_12855344 ] Shai Erera commented on LUCENE-2386: Ok I've added the following to DirReader: {code

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-09 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855369#action_12855369 ] Shai Erera commented on LUCENE-2386: I already did that ... just didn't post back

[jira] Commented: (LUCENE-1879) Parallel incremental indexing

2010-04-09 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855379#action_12855379 ] Shai Erera commented on LUCENE-1879: I have found such version ... and it fails too

[jira] Updated: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-09 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2386: --- Attachment: LUCENE-2386.patch Patch fixes all tests as well as changes to IndexWriter

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-09 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855457#action_12855457 ] Shai Erera commented on LUCENE-2386: Ok sounds good. Is there a preferred package

Move NoDeletionPolicy to core

2010-04-08 Thread Shai Erera
Hi I've noticed benchmark has a NoDeletionPolicy class and I was wondering if we can move it to core. I might want to use it for the parallel index stuff, but I think it'll also fit nicely in core, together with the other No* classes. In addition, this class should be made a singleton. If moving

[jira] Commented: (LUCENE-2074) Use a separate JFlex generated Unicode 4 by Java 5 compatible StandardTokenizer

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12854885#action_12854885 ] Shai Erera commented on LUCENE-2074: Uwe, must this be coupled with that issue

[jira] Commented: (LUCENE-2074) Use a separate JFlex generated Unicode 4 by Java 5 compatible StandardTokenizer

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12854887#action_12854887 ] Shai Erera commented on LUCENE-2074: bq. I plan to commit this soon! That's great

[jira] Commented: (LUCENE-1482) Replace infoSteram by a logging framework (SLF4J)

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12854920#action_12854920 ] Shai Erera commented on LUCENE-1482: I still think that calling isDebugEnabled

[jira] Commented: (LUCENE-1709) Parallelize Tests

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855020#action_12855020 ] Shai Erera commented on LUCENE-1709: Robert, I will commit the patch, seems good to do

[jira] Created: (LUCENE-2385) Move NoDeletionPolicy from benchmark to core

2010-04-08 Thread Shai Erera (JIRA)
/benchmark, Index Reporter: Shai Erera Assignee: Shai Erera Priority: Trivial Fix For: 3.1 As the subject says, but I'll also make it a singleton + add some unit tests, as well as some documentation. I'll post a patch hopefully today

[jira] Created: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-08 Thread Shai Erera (JIRA)
: Index Reporter: Shai Erera Assignee: Shai Erera Fix For: 3.1 I've noticed IndexWriter's ctor commits a first commit (empty one) if a fresh Directory is passed, w/ OpenMode.CREATE or CREATE_OR_APPEND. This seems unnecessarily, and kind of brings back

[jira] Updated: (LUCENE-2385) Move NoDeletionPolicy from benchmark to core

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2385: --- Attachment: LUCENE-2385.patch Move NoDeletionPolicy to core, adds javadocs + TestNoDeletionPolicy

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855131#action_12855131 ] Shai Erera commented on LUCENE-2386: Took a look at IndexFileDeleter, and located

[jira] Commented: (LUCENE-2385) Move NoDeletionPolicy from benchmark to core

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855140#action_12855140 ] Shai Erera commented on LUCENE-2385: I did that first, but then remembered that when I

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855148#action_12855148 ] Shai Erera commented on LUCENE-2386: Looking at IFD again, I think a boolean ctor arg

[jira] Updated: (LUCENE-2385) Move NoDeletionPolicy from benchmark to core

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2385: --- Attachment: LUCENE-2385.patch Is it better now? Move NoDeletionPolicy from benchmark to core

[jira] Commented: (LUCENE-2385) Move NoDeletionPolicy from benchmark to core

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855155#action_12855155 ] Shai Erera commented on LUCENE-2385: Forgot to mention that the only move I made

[jira] Resolved: (LUCENE-2385) Move NoDeletionPolicy from benchmark to core

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera resolved LUCENE-2385. Resolution: Fixed Committed revision 932129. Move NoDeletionPolicy from benchmark to core

[jira] Updated: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2386: --- Attachment: LUCENE-2386.patch First stab at this. Patch still missing CHANGES entry, and I haven't

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855265#action_12855265 ] Shai Erera commented on LUCENE-2386: bq. Maybe change testImmediateDiskFull to set max

TestCodecs running time

2010-04-08 Thread Shai Erera
Hi I've noticed that TestCodecs takes an insanely long time to run on my machine - between 35-40 seconds. Is that expected? The reason why it runs so long, seems to be that its threads make (each) 4000 iterations ... is that really required to ensure correctness? Shai

Re: Controlling the maximum size of a segment during indexing

2010-04-08 Thread Shai Erera
I'm not sure .. but did you set the RAMBufferSizeMB on IWC? Doesn't look like it, and the default is 16 MB, which can explain why it doesn't flush before that. Shai On Fri, Apr 9, 2010 at 8:01 AM, Lance Norskog goks...@gmail.com wrote: Here is a Java unit test that uses the

[jira] Commented: (LUCENE-2386) IndexWriter commits unnecessarily on fresh Directory

2010-04-08 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12855277#action_12855277 ] Shai Erera commented on LUCENE-2386: Apparently, there are more tests that fail

[jira] Updated: (LUCENE-1709) Parallelize Tests

2010-04-07 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-1709: --- Attachment: LUCENE-1709-2.patch Since I had the changes on my local env. I thought it's best

[jira] Resolved: (LUCENE-2377) Enable the use of NoMergePolicy and NoMergeScheduler by Benchmark

2010-04-07 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera resolved LUCENE-2377. Resolution: Fixed Committed revision 931502. Enable the use of NoMergePolicy

[jira] Commented: (LUCENE-2353) Config incorrectly handles Windows absolute pathnames

2010-04-07 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12854588#action_12854588 ] Shai Erera commented on LUCENE-2353: Actually, we've reopened LUCENE-1709 to track

Re: Getting fsync out of the loop

2010-04-06 Thread Shai Erera
How often is fsync called? If it's just during calls to commit, then is that that expensive? I mean, how often do you call commit? If that's that expensive (do you have some numbers to share) then I think that's be a neat idea. Though losing a few minutes worth of updates may sometimes be

Re: Getting fsync out of the loop

2010-04-06 Thread Shai Erera
Earwin - do you have some numbers to share on the running time of the indexing application? You've mentioned that if you take out fsync into a BG thread, the running time improves, but I'm curious to know by how much. Shai On Wed, Apr 7, 2010 at 2:26 AM, Earwin Burrfoot ear...@gmail.com wrote:

[jira] Commented: (LUCENE-1709) Parallelize Tests

2010-04-06 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-1709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12854348#action_12854348 ] Shai Erera commented on LUCENE-1709: One more thing - change benchmark tests to run

[jira] Created: (LUCENE-2377) Enable the use of NoMergePolicy and NoMergeScheduler by Benchmark

2010-04-06 Thread Shai Erera (JIRA)
: Improvement Components: contrib/benchmark Reporter: Shai Erera Assignee: Shai Erera Priority: Minor Fix For: 3.1 Benchmark allows one to set the MP and MS to use, by defining the class name and then use reflection to instantiate them

[jira] Updated: (LUCENE-2377) Enable the use of NoMergePolicy and NoMergeScheduler by Benchmark

2010-04-06 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2377: --- Attachment: LUCENE-2377.patch Patch includes both fix to CreateIndexTask as well as relevant tests

Re: Parallel tests in Benchmark

2010-04-03 Thread Shai Erera
wrote: On Thu, Apr 1, 2010 at 12:03 AM, Shai Erera ser...@gmail.com wrote: Hi I'd like to summarize a discussion I had w/ Robert and Mike last night on IRC, about the parallelism of tasks in Benchmark: For some reason, ever since parallel tasks were introduced, when I run 'ant test' from

Re: Landing the flex branch

2010-04-03 Thread Shai Erera
bq. Try a merge back: This would let flex appear as a single commit to trunk, so the history of trunk would be preserved. +1 for that - I think the history of trunk is important to preserve. And there is also a way to ask for flex's history so everybody win? Shai On Thursday, April 1, 2010,

Re: Welcome Uwe Schindler to the Lucene PMC

2010-04-01 Thread Shai Erera
Congratulations Uwe ! Shai On Thursday, April 1, 2010, Earwin Burrfoot ear...@gmail.com wrote: Generics SpecOps made it to the top and are gonna rule us from the shadows :)  Congrats! On Thu, Apr 1, 2010 at 16:37, Robert Muir rcm...@gmail.com wrote: Congrats Uwe! On Thu, Apr 1, 2010 at

[jira] Commented: (LUCENE-2310) Reduce Fieldable, AbstractField and Field complexity

2010-03-31 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12851829#action_12851829 ] Shai Erera commented on LUCENE-2310: +1 for this simplification. Can we just name

[jira] Assigned: (LUCENE-2353) Config incorrectly handles Windows absolute pathnames

2010-03-31 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera reassigned LUCENE-2353: -- Assignee: Shai Erera Config incorrectly handles Windows absolute pathnames

[jira] Commented: (LUCENE-2353) Config incorrectly handles Windows absolute pathnames

2010-03-31 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12851836#action_12851836 ] Shai Erera commented on LUCENE-2353: Unless there are objections, I plan to commit

[jira] Commented: (LUCENE-2310) Reduce Fieldable, AbstractField and Field complexity

2010-03-31 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12851842#action_12851842 ] Shai Erera commented on LUCENE-2310: Right Earwin - agreed. I'd like to summarize

[jira] Resolved: (LUCENE-2353) Config incorrectly handles Windows absolute pathnames

2010-03-31 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera resolved LUCENE-2353. Resolution: Fixed Committed revision 929520. Config incorrectly handles Windows absolute

Parallel tests in Benchmark

2010-03-31 Thread Shai Erera
Hi I'd like to summarize a discussion I had w/ Robert and Mike last night on IRC, about the parallelism of tasks in Benchmark: For some reason, ever since parallel tasks were introduced, when I run 'ant test' from the contrib/benchmark folder (or the root), the tests just hang at some point,

[jira] Updated: (LUCENE-2353) Config incorrectly handles Windows absolute pathnames

2010-03-29 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2353: --- Attachment: LUCENE-2353.patch Updated to also match 'c:/temp' like paths, which are also accepted

[jira] Commented: (LUCENE-2353) Config incorrectly handles Windows absolute pathnames

2010-03-28 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850644#action_12850644 ] Shai Erera commented on LUCENE-2353: I don't have an account yet, so I cannot commit

[jira] Created: (LUCENE-2353) Config incorrectly handles Windows absolute pathnames

2010-03-27 Thread Shai Erera (JIRA)
: contrib/benchmark Reporter: Shai Erera Fix For: 3.1 I have no idea how no one ran into this so far, but I tried to execute an .alg file which used ReutersContentSource and referenced both docs.dir and work.dir as Windows absolute pathnames (e.g. d:\something

[jira] Updated: (LUCENE-2353) Config incorrectly handles Windows absolute pathnames

2010-03-27 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2353: --- Attachment: LUCENE-2353.patch The fix is only relevant to get(String, String) and not to all other

[jira] Commented: (LUCENE-2345) Make it possible to subclass SegmentReader

2010-03-26 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850075#action_12850075 ] Shai Erera commented on LUCENE-2345: Earwin, w/o knowing too much about the details

[jira] Commented: (LUCENE-2345) Make it possible to subclass SegmentReader

2010-03-26 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850083#action_12850083 ] Shai Erera commented on LUCENE-2345: Thanks Uwe, I know that ctor is the preferred way

[jira] Commented: (LUCENE-2215) paging collector

2010-03-26 Thread Shai Erera (JIRA)
[ https://issues.apache.org/jira/browse/LUCENE-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12850086#action_12850086 ] Shai Erera commented on LUCENE-2215: Sure let's wait for the patch and some perf

  1   2   3   4   5   6   7   8   9   10   >