date:20120618

See http://sierranevada.servebeer.com/job/Lucene-Core-4x-Beasting/2603/

--
[...truncated 882 lines...]
   [junit4] Completed on J2 in 1.20s, 7 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads
   [junit4] Completed on J3 in 1.05s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestOmitPositions
   [junit4] Completed on J0 in 0.14s, 3 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions
   [junit4] Completed on J3 in 0.15s, 6 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.store.TestHugeRamFile
   [junit4] Completed on J0 in 0.45s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestSameTokenSamePosition
   [junit4] Completed on J0 in 0.06s, 2 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestCheckIndex
   [junit4] Completed on J0 in 0.03s, 2 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.codecs.lucene3x.TestImpersonation
   [junit4] Completed on J0 in 0.01s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestSort
   [junit4] Completed on J1 in 0.91s, 25 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery
   [junit4] Completed on J1 in 0.19s, 10 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestDirectoryReader
   [junit4] Completed on J3 in 0.57s, 24 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterCommit
   [junit4] IGNOR/A 0.00s J2 | TestIndexWriterCommit.testCommitOnCloseDiskUsage
   [junit4] Assumption #1: This test cannot run with Memory codec
   [junit4] Completed on J2 in 1.09s, 12 tests, 1 skipped
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors
   [junit4] Completed on J3 in 0.26s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRewriteMethod
   [junit4] Completed on J2 in 0.34s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestMultiValuedNumericRangeQuery
   [junit4] Completed on J2 in 0.21s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestDateSort
   [junit4] Completed on J0 in 0.01s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestRandomStoredFields
   [junit4] Completed on J3 in 0.42s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestMaxTermFrequency
   [junit4] Completed on J2 in 0.13s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestFieldsReader
   [junit4] Completed on J2 in 0.22s, 4 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata
   [junit4] Completed on J2 in 0.49s, 4 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterNRTIsCurrent
   [junit4] Completed on J3 in 0.77s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestBackwardsCompatibility
   [junit4] Completed on J0 in 2.75s, 13 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.spans.TestPayloadSpans
   [junit4] Completed on J3 in 0.39s, 10 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminism
   [junit4] Completed on J0 in 0.35s, 2 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestTransactions
   [junit4] Completed on J2 in 0.56s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestPostingsOffsets
   [junit4] Completed on J2 in 0.24s, 11 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminizeLexicon
   [junit4] Completed on J3 in 0.40s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestCrash
   [junit4] Completed on J0 in 0.37s, 5 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRangeFilter
   [junit4] Completed on J3 in 0.12s, 9 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestTermVectors
   [junit4] Completed on J0 in 0.11s, 11 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.util.TestWeakIdentityMap
   [junit4] Completed on J2 in 0.37s, 2 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestAutomatonQuery
   [junit4] Completed on J3 in 0.03s, 6 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestFilteredQuery
   [junit4] Completed on J0 in 0.12s, 9 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.util.TestIndexableBinaryStringTools
   [junit4] Completed on J2 in 0.08s, 5 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestMultiLevelSkipList
   [junit4] Completed on J3 in 0.04s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanations
   [junit4] Completed on J1 in 3.78s, 69 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestSimpleSearchEquivalence
   [junit4] Completed on J0 in 0.10s, 12 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery
   [junit4] Completed on J2 in 0.05s, 5 tests

Jenkins build is back to normal : Lucene-Core-4x-Beasting #2604

See http://sierranevada.servebeer.com/job/Lucene-Core-4x-Beasting/2604/


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3406) Support grouped range and query facets.


 [ 
https://issues.apache.org/jira/browse/SOLR-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen updated SOLR-3406:


Affects Version/s: 4.0

 Support grouped range and query facets.
 ---

 Key: SOLR-3406
 URL: https://issues.apache.org/jira/browse/SOLR-3406
 Project: Solr
  Issue Type: New Feature
Affects Versions: 4.0
Reporter: David
Assignee: Martijn van Groningen
Priority: Critical
 Fix For: 4.0

 Attachments: SOLR-2898-backport.patch, SOLR-3406.patch, 
 SOLR-3406.patch, SOLR-3406.patch

   Original Estimate: 504h
  Remaining Estimate: 504h

 Need the ability to support grouped range and query facets. Grouped facet 
 fields have already been implemented in SOLR-2898 but we still need the 
 ability to compute grouped range and query facets.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3406) Support grouped range and query facets.


 [ 
https://issues.apache.org/jira/browse/SOLR-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen updated SOLR-3406:


Attachment: SOLR-3406.patch

Updated patch to also support facet.range parameter.

 Support grouped range and query facets.
 ---

 Key: SOLR-3406
 URL: https://issues.apache.org/jira/browse/SOLR-3406
 Project: Solr
  Issue Type: New Feature
Affects Versions: 4.0
Reporter: David
Assignee: Martijn van Groningen
Priority: Critical
 Fix For: 4.0

 Attachments: SOLR-2898-backport.patch, SOLR-3406.patch, 
 SOLR-3406.patch, SOLR-3406.patch

   Original Estimate: 504h
  Remaining Estimate: 504h

 Need the ability to support grouped range and query facets. Grouped facet 
 fields have already been implemented in SOLR-2898 but we still need the 
 ability to compute grouped range and query facets.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3406) Support grouped range and query facets.


 [ 
https://issues.apache.org/jira/browse/SOLR-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen resolved SOLR-3406.
-

   Resolution: Fixed
Fix Version/s: 5.0

Committed to trunk and branch4x.

 Support grouped range and query facets.
 ---

 Key: SOLR-3406
 URL: https://issues.apache.org/jira/browse/SOLR-3406
 Project: Solr
  Issue Type: New Feature
Affects Versions: 4.0
Reporter: David
Assignee: Martijn van Groningen
Priority: Critical
 Fix For: 4.0, 5.0

 Attachments: SOLR-2898-backport.patch, SOLR-3406.patch, 
 SOLR-3406.patch, SOLR-3406.patch

   Original Estimate: 504h
  Remaining Estimate: 504h

 Need the ability to support grouped range and query facets. Grouped facet 
 fields have already been implemented in SOLR-2898 but we still need the 
 ability to compute grouped range and query facets.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-4132) IndexWriterConfig live settings

2012-06-18 Thread Shai Erera (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Shai Erera resolved LUCENE-4132.

Resolution: Fixed
Lucene Fields: New,Patch Available (was: New)

Committed revision 1351225 (trunk) and revision 1351229 (4x).

Thanks all for your comments !

IndexWriterConfig live settings
---

Key: LUCENE-4132
URL: https://issues.apache.org/jira/browse/LUCENE-4132
Project: Lucene - Java
Issue Type: Improvement
Reporter: Shai Erera
Assignee: Shai Erera
Priority: Minor
Fix For: 4.0, 5.0

Attachments: LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch,
LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch

A while ago there was a discussion about making some IW settings live and I
remember that RAM buffer size was one of them. Judging from IW code, I see
that RAM buffer can be changed live as IW never caches it.
However, I don't remember which other settings were decided to be live and
I don't see any documentation in IW nor IWC for that. IW.getConfig mentions:
{code}
* bNOTE:/b some settings may be changed on the
* returned {@link IndexWriterConfig}, and will take
* effect in the current IndexWriter instance. See the
* javadocs for the specific setters in {@link
* IndexWriterConfig} for details.
{code}
But there's no text on e.g. IWC.setRAMBuffer mentioning that.
I think that it'd be good if we make it easier for users to tell which of the
settings are live ones. There are few possible ways to do it:
* Introduce a custom @live.setting tag on the relevant IWC.set methods, and
add special text for them in build.xml
** Or, drop the tag and just document it clearly.
* Separate IWC to two interfaces, LiveConfig and OneTimeConfig (name
proposals are welcome !), have IWC impl both, and introduce another
IW.getLiveConfig which will return that interface, thereby clearly letting
the user know which of the settings are live.
It'd be good if IWC itself could only expose setXYZ methods for the live
settings though. So perhaps, off the top of my head, we can do something like
this:
* Introduce a Config object, which is essentially what IWC is today, and pass
it to IW.
* IW will create a different object, IWC from that Config and IW.getConfig
will return IWC.
* IWC itself will only have setXYZ methods for the live settings.
It adds another object, but user code doesn't change - it still creates a
Config object when initializing IW, and need to handle a different type if it
ever calls IW.getConfig.
Maybe that's not such a bad idea?

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4148) _TestUtil should be able to generate random longs

2012-06-18 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395487#comment-13395487
 ] 

Dawid Weiss commented on LUCENE-4148:
-

I didn't check _TestUtil, to be honest -- I think it did have a bug with range 
overflow passed to nextInt (which is not a problem in practice, but was my 
example on how randomized testing can explore unexpected things). 

Anyway, as for randomLongBetween -- here you'd need to be more careful about 
double's representation so simple scaling won't work (there are ranges of 
double that transform to NaNs for example). I am not so strong in numerical 
analysis to be able to prove something is a correct solution to the problem. My 
(naive) take at this would be to calculate the range and then if split into a 
few cases:

1) if it doesn't exceed a positive int, use nextInt()
2) if it doesn't exceed a positive long, use nextLong()
3) if it does exceed positive long, use BigDecimal? :) 

Maybe this can be done faster without keeping the uniform distribution 
requirement so strong (i.e. use a modulo from a combination of two concatenated 
longs as BigDecimal or something like that).

 _TestUtil should be able to generate random longs
 -

 Key: LUCENE-4148
 URL: https://issues.apache.org/jira/browse/LUCENE-4148
 Project: Lucene - Java
  Issue Type: Improvement
  Components: general/test
Reporter: Adrien Grand
Priority: Trivial

 It would be helpful in TestPackedInts at least, in order to generate random 
 values (as a workaround, we currently generate a random int between 0 and 
 {{min(Integer.MAX_VALUE, PackedInts.maxValue(bitsPerValue)}}). Moreover, it 
 would help to fix {{nextInt}} for large ranges (calling {{nextInt(random, 
 -10, Integer.MAX_VALUE)}} or even {{nextInt(random, 0, Integer.MAX_VALUE)}} 
 currently fails because the range of values is {{ Integer.MAX_VALUE}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-4149) TestPackedInts test failure

2012-06-18 Thread Adrien Grand (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-4149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-4149.
--

   Resolution: Fixed
Fix Version/s: 5.0
   4.0

Committed (r1351231 on trunk and r1351239 on brand_4x).

 TestPackedInts test failure
 ---

 Key: LUCENE-4149
 URL: https://issues.apache.org/jira/browse/LUCENE-4149
 Project: Lucene - Java
  Issue Type: Bug
Affects Versions: 4.0, 5.0
Reporter: Robert Muir
Assignee: Adrien Grand
 Fix For: 4.0, 5.0

 Attachments: LUCENE-4149.patch


 ant test  -Dtestcase=TestPackedInts -Dtests.method=testBulkSet 
 -Dtests.seed=ABDA58FE51EFA72B -Dtests.locale=lt 
 -Dtests.timezone=Europe/Athens -Dargs=-Dfile.encoding=UTF-8

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4069) Segment-level Bloom filters for a 2 x speed up on rare term searches

[
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395773#comment-13395773
]

Mark Harwood commented on LUCENE-4069:
--

Interesting results, Mike - thanks for taking the time to run them.

bq. BloomFilteredFieldsProducer should just pass through intersect to the
delegate?

I have tried to make the BloomFilteredFieldsProducer get out of the way of the
client app and the delegate PostingsFormat as soon as it is safe to do so i.e.
when the user is safely focused on a non-filtered field. While there is a
chance the client may end up making a call to TermsEnum.seekExact(..) on a
filtered field then I need to have a wrapper object in place which is in a
position to intercept this call. In all other method invocations I just end up
delegating calls so I wonder if all these extra method calls are the cause of
the slowdown you see e.g. when Fuzzy is enumerating over many terms.
The only other alternatives to endlessly wrapping in this way are:
a) API change - e.g. allow TermsEnum.seekExact to have a pluggable call-out for
just this one method.
b) Mess around with byte-code manipulation techniques to weave in Bloom
filtering(the sort of thing I recall Hibernate resorts to)

Neither of these seem particularly appealing options so I think we may have to
live with fuzzy+bloom not being as fast as straight fuzzy.

For completeness sake - I don't have access to your benchmarking code but I
would hope that PostingsFormat.fieldsProducer() isn't called more than once for
the same segment as that's where the Bloom filters get loaded from disk so
there's inherent cost there too. I can't imagine this is the case.

BTW I've just finished a long-running set of tests which mixes up reads and
writes here: http://goo.gl/KJmGv
This benchmark represents how graph databases such as Neo4j use Lucene for an
index when loading (I typically use the Wikipedia links as a test set). I look
to get a 3.5 x speed up in Lucene 4 and Lucene 3.6 gets nearly 9 x speedup over
the comparatively slower 3.6 codebase.

Segment-level Bloom filters for a 2 x speed up on rare term searches

Key: LUCENE-4069
URL: https://issues.apache.org/jira/browse/LUCENE-4069
Project: Lucene - Java
Issue Type: Improvement
Components: core/index
Affects Versions: 3.6, 4.0
Reporter: Mark Harwood
Priority: Minor
Fix For: 4.0, 3.6.1

Attachments: BloomFilterPostingsBranch4x.patch,
MHBloomFilterOn3.6Branch.patch, PrimaryKeyPerfTest40.java

An addition to each segment which stores a Bloom filter for selected fields
in order to give fast-fail to term searches, helping avoid wasted disk access.
Best suited for low-frequency fields e.g. primary keys on big indexes with
many segments but also speeds up general searching in my tests.
Overview slideshow here:
http://www.slideshare.net/MarkHarwood/lucene-bloomfilteredsegments
Benchmarks based on Wikipedia content here: http://goo.gl/X7QqU
Patch based on 3.6 codebase attached.
There are no 3.6 API changes currently - to play just add a field with _blm
on the end of the name to invoke special indexing/querying capability.
Clearly a new Field or schema declaration(!) would need adding to APIs to
configure the service properly.
Also, a patch for Lucene4.0 codebase introducing a new PostingsFormat

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-2899) Add OpenNLP Analysis capabilities as a module

2012-06-18 Thread Joern Kottmann (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395787#comment-13395787
 ] 

Joern Kottmann commented on LUCENE-2899:


For NER you should try the perceptron and a cutoff of zero. For NER with a 
cutoff of 5 you need otherwise much more training data.

 Add OpenNLP Analysis capabilities as a module
 -

 Key: LUCENE-2899
 URL: https://issues.apache.org/jira/browse/LUCENE-2899
 Project: Lucene - Java
  Issue Type: New Feature
  Components: modules/analysis
Reporter: Grant Ingersoll
Assignee: Grant Ingersoll
Priority: Minor
 Attachments: LUCENE-2899.patch, opennlp_trunk.patch


 Now that OpenNLP is an ASF project and has a nice license, it would be nice 
 to have a submodule (under analysis) that exposed capabilities for it. Drew 
 Farris, Tom Morton and I have code that does:
 * Sentence Detection as a Tokenizer (could also be a TokenFilter, although it 
 would have to change slightly to buffer tokens)
 * NamedEntity recognition as a TokenFilter
 We are also planning a Tokenizer/TokenFilter that can put parts of speech as 
 either payloads (PartOfSpeechAttribute?) on a token or at the same position.
 I'd propose it go under:
 modules/analysis/opennlp

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3866) Make CompositeReader.getSequentialSubReaders() and the corresponding IndexReaderContext methods return unmodifiable ListR extends IndexReader


 [ 
https://issues.apache.org/jira/browse/LUCENE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-3866:
--

Attachment: LUCENE-3866-step2.patch

This is the final patch, now also moving IndexReaderContext to collections and 
make them unmodifiable.

For safety, IRC.leaves() now throws UnsupportedOperationException, if the 
context is not top-level (returned null before), this helps to find bugs.

All tests pass, JavaDocs are updated.

Robert suggested to me yesterday, to maybe make getSequentialSubReaders() 
protected, as it is no longer used by user code (only tests). All code should 
use the topReaderContext returned by the IndexReader to get all *atomic* 
IRC.leaves(), or IRC.children() to get the sub-contexts/readers.

For easy use it might be a good idea to add some easy-use methods in 
ReaderUtil to get a view on all AtomicReader leaves (without docBase, so 
returns ListAtomicReader). Some code not needing the whole info would get 
simplier. This is stuff for a new issue.

In my opinion we should also move and hide ReaderSlice and BitSlice to index 
package, those classes are solely privately used from there.

I think this is ready to commit to trunk and for backport to 4.x. I will not 
wait too long (max 24hrs), as the patch may get outdated very quickly. Maybe 
Mike can do some perf benchmarks with beast to show that it does not affect 
performance (some parts like creating Multi* should be much more effective now).

 Make CompositeReader.getSequentialSubReaders() and the corresponding 
 IndexReaderContext methods return unmodifiable ListR extends IndexReader
 ---

 Key: LUCENE-3866
 URL: https://issues.apache.org/jira/browse/LUCENE-3866
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3866-step1.patch, LUCENE-3866-step2.patch


 Since Lucene 2.9 we have the method getSequentialSubReader() returning 
 IndexReader[]. Based on hardly-to-debug errors in user's code, Robert and me 
 realized that returning an array from a public API is an anti-pattern. If the 
 array is intended to be modifiable (like byte[] in terms,...), it is fine to 
 use arrays in public APIs, but not, if the array must be protected from 
 modification. As IndexReaders are 100% unmodifiable in trunk code (no 
 deletions,...), the only possibility to corrumpt the reader is by modifying 
 the array returned by getSequentialSubReaders(). We should prevent this.
 The same theoretically applies to FieldCache, too - but the party that is 
 afraid of performance problems is too big to fight against that :(
 For getSequentialSubReaders there is no performance problem at all. The 
 binary search of reader-ids inside BaseCompositeReader can still be done with 
 the internal protected array, but public APIs should expose only a 
 unmodifiable List. The same applies to leaves() and children() in 
 IndexReaderContext. This change to list would also allow to make 
 CompositeReader and CompositeReaderContext IterableR extends IndexReader, 
 so some loops would look nice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3866) Make CompositeReader.getSequentialSubReaders() and the corresponding IndexReaderContext methods return unmodifiable ListR extends IndexReader


 [ 
https://issues.apache.org/jira/browse/LUCENE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-3866:
--

Attachment: LUCENE-3866-step2.patch

Samll update with JavaDocs (remove the don't modify returned array @ 
getSequentialSubReaders)

 Make CompositeReader.getSequentialSubReaders() and the corresponding 
 IndexReaderContext methods return unmodifiable ListR extends IndexReader
 ---

 Key: LUCENE-3866
 URL: https://issues.apache.org/jira/browse/LUCENE-3866
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3866-step1.patch, LUCENE-3866-step2.patch, 
 LUCENE-3866-step2.patch


 Since Lucene 2.9 we have the method getSequentialSubReader() returning 
 IndexReader[]. Based on hardly-to-debug errors in user's code, Robert and me 
 realized that returning an array from a public API is an anti-pattern. If the 
 array is intended to be modifiable (like byte[] in terms,...), it is fine to 
 use arrays in public APIs, but not, if the array must be protected from 
 modification. As IndexReaders are 100% unmodifiable in trunk code (no 
 deletions,...), the only possibility to corrumpt the reader is by modifying 
 the array returned by getSequentialSubReaders(). We should prevent this.
 The same theoretically applies to FieldCache, too - but the party that is 
 afraid of performance problems is too big to fight against that :(
 For getSequentialSubReaders there is no performance problem at all. The 
 binary search of reader-ids inside BaseCompositeReader can still be done with 
 the internal protected array, but public APIs should expose only a 
 unmodifiable List. The same applies to leaves() and children() in 
 IndexReaderContext. This change to list would also allow to make 
 CompositeReader and CompositeReaderContext IterableR extends IndexReader, 
 so some loops would look nice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Build failed in Jenkins: Lucene-Core-4x-Beasting #2798

See http://sierranevada.servebeer.com/job/Lucene-Core-4x-Beasting/2798/

--
[...truncated 840 lines...]
   [junit4] Completed on J2 in 0.02s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestRollingUpdates
   [junit4] Completed on J2 in 1.24s, 2 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter
   [junit4] Completed on J2 in 0.08s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestIndexWriter
   [junit4] Completed on J3 in 3.13s, 47 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy
   [junit4] Completed on J1 in 1.93s, 5 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestSegmentTermDocs
   [junit4] Completed on J1 in 0.42s, 5 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestBackwardsCompatibility
   [junit4] Completed on J2 in 2.42s, 13 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.util.packed.TestPackedInts
   [junit4] Completed on J0 in 7.49s, 15 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery32
   [junit4] Completed on J3 in 3.69s, 27 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecs
   [junit4] Completed on J1 in 3.83s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy
   [junit4] Completed on J0 in 3.52s, 6 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestCompoundFile
   [junit4] Completed on J0 in 0.93s, 18 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestTimeLimitingCollector
   [junit4] Completed on J1 in 1.55s, 6 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.store.TestBufferedIndexInput
   [junit4] Completed on J0 in 1.45s, 4 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads
   [junit4] Completed on J1 in 1.44s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.store.TestNRTCachingDirectory
   [junit4] Completed on J0 in 0.48s, 5 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestNorms
   [junit4] Completed on J2 in 7.87s, 3 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestForceMergeForever
   [junit4] Completed on J2 in 0.40s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize
   [junit4] Completed on J0 in 1.36s, 3 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestNeverDelete
   [junit4] Completed on J1 in 1.90s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarityBase
   [junit4] IGNOR/A 0.00s J0 | TestSimilarityBase.testHeartRanking
   [junit4] Assumption #1: PreFlex codec does not support the stats 
necessary for this test!
   [junit4] Completed on J0 in 0.25s, 29 tests, 1 skipped
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.spans.TestSpans
   [junit4] Completed on J2 in 0.63s, 25 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestParallelCompositeReader
   [junit4] Completed on J0 in 0.12s, 8 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions
   [junit4] Completed on J1 in 0.38s, 6 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors
   [junit4] Completed on J0 in 0.20s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestOmitTf
   [junit4] Completed on J1 in 0.15s, 6 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestNRTThreads
   [junit4] Completed on J3 in 7.97s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestRandomStoredFields
   [junit4] Completed on J0 in 0.30s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery
   [junit4] Completed on J2 in 0.77s, 10 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestQueryWrapperFilter
   [junit4] Completed on J3 in 0.12s, 3 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.payloads.TestPayloadNearQuery
   [junit4] Completed on J2 in 0.09s, 7 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestPrefixRandom
   [junit4] Completed on J1 in 0.53s, 1 test
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestBooleanOr
   [junit4] Completed on J3 in 0.05s, 5 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata
   [junit4] Completed on J0 in 0.41s, 4 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestCustomNorms
   [junit4] Completed on J2 in 0.21s, 3 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRangeFilter
   [junit4] Completed on J3 in 0.12s, 9 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.index.TestPrefixCodedTerms
   [junit4] Completed on J2 in 0.08s, 6 tests
   [junit4]  
   [junit4] Suite: org.apache.lucene.codecs.lucene3x.TestTermInfosReaderIndex
   [junit4] Completed on J3 in 0.14s, 3 tests

Jenkins build is back to normal : Lucene-Core-4x-Beasting #2799

See http://sierranevada.servebeer.com/job/Lucene-Core-4x-Beasting/2799/


-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4147) rollback/preparecommit thread hazard


 [ 
https://issues.apache.org/jira/browse/LUCENE-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-4147:
---

Attachment: LUCENE-4147.patch

Patch, acquiring the commitLock around close and rollback, and adding
the ensureOpen before prepareCommit.  However, the test still fails
after a few hundred beasting iterations, I think because of thread safety
issues where one thread calls docWriter.abort while another is
flushing ... not sure how to fix that one yet.  Simon maybe you can
have a look?


 rollback/preparecommit thread hazard
 

 Key: LUCENE-4147
 URL: https://issues.apache.org/jira/browse/LUCENE-4147
 Project: Lucene - Java
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Robert Muir
 Attachments: LUCENE-4147.patch, LUCENE-4147.patch


 found by 
 http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows-Java7-64/70/
 rollback should never throw this exception, as it documents it clears any 
 pendingcommits.
 but it calls closeInternal outside of any sync block, so it looks like there 
 is a race here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4152) add one-syllable method to IndexReader enumerate subreaders

Robert Muir created LUCENE-4152:
---

 Summary: add one-syllable method to IndexReader enumerate 
subreaders
 Key: LUCENE-4152
 URL: https://issues.apache.org/jira/browse/LUCENE-4152
 Project: Lucene - Java
  Issue Type: Bug
Reporter: Robert Muir
Priority: Blocker
 Fix For: 4.0


Description is exactly as written.

getSequentialSubReaders/getTopLevelReaderContext, these method names are way 
too long/unuseable. They also have tricky semantics (e.g. returning null).

In lucene 4, people cannot just use any indexreader and get a merged view. So 
we need to make this stuff easy on them:
* single-syllable method name (leaves(), subs(), i will think on this)
* supports enhanced for-loop (no returning null or anything like that)
* on indexreader (not atomic or composite, plain old indexreader)


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4147) rollback/preparecommit thread hazard

2012-06-18 Thread Simon Willnauer (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395826#comment-13395826
 ] 

Simon Willnauer commented on LUCENE-4147:
-

mike, do we really need to acquire the commit lock? from my perspective it 
would be enough to add an ensure open when we assigne pendingCommit (inside the 
sync block) so that racing threads hit already close exceptions.

regarding the thread safety issue in DocWriter can you paste the trace?


 rollback/preparecommit thread hazard
 

 Key: LUCENE-4147
 URL: https://issues.apache.org/jira/browse/LUCENE-4147
 Project: Lucene - Java
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Robert Muir
 Attachments: LUCENE-4147.patch, LUCENE-4147.patch


 found by 
 http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows-Java7-64/70/
 rollback should never throw this exception, as it documents it clears any 
 pendingcommits.
 but it calls closeInternal outside of any sync block, so it looks like there 
 is a race here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4069) Segment-level Bloom filters for a 2 x speed up on rare term searches

[
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395830#comment-13395830
]

Michael McCandless commented on LUCENE-4069:

bq. Interesting results, Mike - thanks for taking the time to run them.

You're welcome!

{quote}
bq. BloomFilteredFieldsProducer should just pass through intersect to the
delegate?

I have tried to make the BloomFilteredFieldsProducer get out of the way of the
client app and the delegate PostingsFormat as soon as it is safe to do so i.e.
when the user is safely focused on a non-filtered field. While there is a
chance the client may end up making a call to TermsEnum.seekExact(..) on a
filtered field then I need to have a wrapper object in place which is in a
position to intercept this call. In all other method invocations I just end up
delegating calls so I wonder if all these extra method calls are the cause of
the slowdown you see e.g. when Fuzzy is enumerating over many terms.
The only other alternatives to endlessly wrapping in this way are:
a) API change - e.g. allow TermsEnum.seekExact to have a pluggable call-out
for just this one method.
b) Mess around with byte-code manipulation techniques to weave in Bloom
filtering(the sort of thing I recall Hibernate resorts to)

Neither of these seem particularly appealing options so I think we may have to
live with fuzzy+bloom not being as fast as straight fuzzy.
{quote}

I think the fix is simple: you are not overriding Terms.intersect now,
in BloomFilteredTerms. I think you should override it and immediately
delegate and then FuzzyN/Respell performance should be just as good as
Lucene40 codec.

bq. For completeness sake - I don't have access to your benchmarking code

All the benchmarking code is here:
http://code.google.com/a/apache-extras.org/p/luceneutil/

I run it nightly (trunk) and publish the results here:
http://people.apache.org/~mikemccand/lucenebench/

bq. but I would hope that PostingsFormat.fieldsProducer() isn't called more
than once for the same segment as that's where the Bloom filters get loaded
from disk so there's inherent cost there too. I can't imagine this is the case.

It's only called once on init'ing the SegmentReader (or at least it
better be!).

{quote}
BTW I've just finished a long-running set of tests which mixes up reads and
writes here: http://goo.gl/KJmGv
This benchmark represents how graph databases such as Neo4j use Lucene for an
index when loading (I typically use the Wikipedia links as a test set). I look
to get a 3.5 x speed up in Lucene 4 and Lucene 3.6 gets nearly 9 x speedup over
the comparatively slower 3.6 codebase.
{quote}

Nice results! It looks like bloom(3.6) is faster than bloom(4.0)?
Why is that...

Also I wonder why you see such sizable (3.5X speedup) gains on PK
lookup but in my benchmark I see only ~13% - 24%. My index has 5
segments per level...

Segment-level Bloom filters for a 2 x speed up on rare term searches

Attachments: BloomFilterPostingsBranch4x.patch,
MHBloomFilterOn3.6Branch.patch, PrimaryKeyPerfTest40.java

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4152) add one-syllable method to IndexReader enumerate subreaders


[ 
https://issues.apache.org/jira/browse/LUCENE-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395840#comment-13395840
 ] 

Uwe Schindler commented on LUCENE-4152:
---

bq. They also have tricky semantics (e.g. returning null).

No longer since January's refactoring, leaves() returns always something != 
null. And getSeqSubReaders is only available on CompositeReaders, which always 
have subReaders. See the other issue LUCENE-3866 where I did some more 
refactoring to make all this easier.

In general I agree with that, as leaves() and children()/subs() would then just 
be a shortcut to getTopReaderContext()'s methods (leaves(), children() - both 
are Iterable, no need to change anything) (I would rename that one to 
as[Top]Context()).

Returning plain subReaders without Contexts is not really useful, as all of 
Lucene's Query logic uses AtomicReaderContext, so leaves() on IndexReader 
returning the same as getTopReaderContext().leaves() is the way to go. This 
method can be added as final easy-use method. I hope you look at my other 
patch @ LUCENE-3866, because it shows how simple the code is now without 
ReaderUtil.Gather.

 add one-syllable method to IndexReader enumerate subreaders
 ---

 Key: LUCENE-4152
 URL: https://issues.apache.org/jira/browse/LUCENE-4152
 Project: Lucene - Java
  Issue Type: Bug
Reporter: Robert Muir
Priority: Blocker
 Fix For: 4.0


 Description is exactly as written.
 getSequentialSubReaders/getTopLevelReaderContext, these method names are way 
 too long/unuseable. They also have tricky semantics (e.g. returning null).
 In lucene 4, people cannot just use any indexreader and get a merged view. So 
 we need to make this stuff easy on them:
 * single-syllable method name (leaves(), subs(), i will think on this)
 * supports enhanced for-loop (no returning null or anything like that)
 * on indexreader (not atomic or composite, plain old indexreader)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-4.x-java7 - Build # 15 - Failure

2012-06-18 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x-java7/15/

No tests ran.

Build Log:
[...truncated 451 lines...]

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/ivy-settings.xml

resolve:
 [echo] Building solr-uima...

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/ivy-settings.xml

resolve:
 [echo] Building solr-velocity...

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/ivy-settings.xml

resolve:

validate:
 [echo] License check under: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/solr
 [licenses] Scanned 82 JAR file(s) for licenses (in 0.66s.), 0 error(s).

test:

test-core:

jflex-uptodate-check:

jflex-notice:

javacc-uptodate-check:

javacc-notice:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/ivy-settings.xml

resolve:

init:

clover.setup:

clover.info:
 [echo] 
 [echo]   Clover not found. Code coverage reports disabled.
 [echo] 

clover:

common.compile-core:
[mkdir] Created dir: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build/core/classes/java
[javac] Compiling 649 source files to 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/build/core/classes/java
[javac] warning: [options] bootstrap class path not set in conjunction with 
-source 1.6
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:506:
 error: class, interface, or enum expected
[javac] package org.apache.lucene.index;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:525:
 error: class, interface, or enum expected
[javac] import org.apache.lucene.analysis.Analyzer;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:526:
 error: class, interface, or enum expected
[javac] import org.apache.lucene.codecs.Codec;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:527:
 error: class, interface, or enum expected
[javac] import 
org.apache.lucene.index.DocumentsWriterPerThread.IndexingChain;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:528:
 error: class, interface, or enum expected
[javac] import org.apache.lucene.index.IndexWriter.IndexReaderWarmer;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:529:
 error: class, interface, or enum expected
[javac] import org.apache.lucene.index.IndexWriterConfig.OpenMode;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:530:
 error: class, interface, or enum expected
[javac] import org.apache.lucene.search.IndexSearcher;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:531:
 error: class, interface, or enum expected
[javac] import org.apache.lucene.search.similarities.Similarity;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:532:
 error: class, interface, or enum expected
[javac] import org.apache.lucene.util.InfoStream;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-java7/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:533:
 error: class, interface, or enum expected
[javac] import org.apache.lucene.util.Version;
[javac] ^
[javac] 10 errors
[...truncated 14 lines...]

[...truncated 566 lines...]

[...truncated 566 lines...]

[...truncated 566 lines...]

[...truncated 566 lines...]

[jira] [Comment Edited] (LUCENE-4152) add one-syllable method to IndexReader enumerate subreaders


[ 
https://issues.apache.org/jira/browse/LUCENE-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395840#comment-13395840
 ] 

Uwe Schindler edited comment on LUCENE-4152 at 6/18/12 12:02 PM:
-

bq. They also have tricky semantics (e.g. returning null).

No longer since January's refactoring, leaves() returns always something != 
null. And getSeqSubReaders is only available on CompositeReaders, which always 
have subReaders. See the other issue LUCENE-3866 where I did some more 
refactoring to make all this easier.

In general I agree with that, as leaves() and children()/subs() would then just 
be a shortcut to getTopReaderContext()'s methods (leaves(), children() - both 
are Iterable, no need to change anything) (I would rename that one to 
as[Top]Context()).

Returning plain subReaders without Contexts is not really useful, as all of 
Lucene's Query logic uses AtomicReaderContext, so leaves() on IndexReader 
returning the same as getTopReaderContext().leaves() is the way to go. This 
method can be added as final easy-use method. I hope you look at my other 
patch @ LUCENE-3866, because it shows how simple the code is now without 
ReaderUtil.Gather.

Additionally, as noted in LUCENE-3866, getSequentialSubReaders() in 
CompositeReader should be made protected, user code does not need to use it. 
It's solely there for building the reader hierarchy, later available using 
IRC.leaves() and IRC.children(). Currently this method is only used by tests 
anymore (which can be rewritten). In CompositeReader, getSequentialSubReaders() 
should just be the protected abstract way for the API on top to get the inner 
structure, but not for the outside.

  was (Author: thetaphi):
bq. They also have tricky semantics (e.g. returning null).

No longer since January's refactoring, leaves() returns always something != 
null. And getSeqSubReaders is only available on CompositeReaders, which always 
have subReaders. See the other issue LUCENE-3866 where I did some more 
refactoring to make all this easier.

In general I agree with that, as leaves() and children()/subs() would then just 
be a shortcut to getTopReaderContext()'s methods (leaves(), children() - both 
are Iterable, no need to change anything) (I would rename that one to 
as[Top]Context()).

Returning plain subReaders without Contexts is not really useful, as all of 
Lucene's Query logic uses AtomicReaderContext, so leaves() on IndexReader 
returning the same as getTopReaderContext().leaves() is the way to go. This 
method can be added as final easy-use method. I hope you look at my other 
patch @ LUCENE-3866, because it shows how simple the code is now without 
ReaderUtil.Gather.
  
 add one-syllable method to IndexReader enumerate subreaders
 ---

 Key: LUCENE-4152
 URL: https://issues.apache.org/jira/browse/LUCENE-4152
 Project: Lucene - Java
  Issue Type: Bug
Reporter: Robert Muir
Priority: Blocker
 Fix For: 4.0


 Description is exactly as written.
 getSequentialSubReaders/getTopLevelReaderContext, these method names are way 
 too long/unuseable. They also have tricky semantics (e.g. returning null).
 In lucene 4, people cannot just use any indexreader and get a merged view. So 
 we need to make this stuff easy on them:
 * single-syllable method name (leaves(), subs(), i will think on this)
 * supports enhanced for-loop (no returning null or anything like that)
 * on indexreader (not atomic or composite, plain old indexreader)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: svn commit: r1351229 - in /lucene/dev/branches/branch_4x: ./ lucene/ lucene/core/ lucene/core/src/java/org/apache/lucene/index/ lucene/core/src/test/org/apache/lucene/index/

2012-06-18 Thread Robert Muir

Why are the CHANGES totally screwy in this commit? Please put them
under the right section (also fix trunk too)


On Mon, Jun 18, 2012 at 4:24 AM,  sh...@apache.org wrote:
 Author: shaie
 Date: Mon Jun 18 08:24:25 2012
 New Revision: 1351229

 URL: http://svn.apache.org/viewvc?rev=1351229view=rev
 Log:
 LUCENE-4132: merge from trunk

 Added:
    
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java
      - copied unchanged from r1351225, 
 lucene/dev/trunk/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java
 Modified:
    lucene/dev/branches/branch_4x/   (props changed)
    lucene/dev/branches/branch_4x/lucene/   (props changed)
    lucene/dev/branches/branch_4x/lucene/CHANGES.txt
    lucene/dev/branches/branch_4x/lucene/core/   (props changed)
    
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriter.java
    
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriterFlushControl.java
    
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriterPerThreadPool.java
    
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/FlushPolicy.java
    
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java
    
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/IndexWriterConfig.java
    
 lucene/dev/branches/branch_4x/lucene/core/src/test/org/apache/lucene/index/TestFlushByRamOrCountsPolicy.java
    
 lucene/dev/branches/branch_4x/lucene/core/src/test/org/apache/lucene/index/TestIndexWriterConfig.java

 Modified: lucene/dev/branches/branch_4x/lucene/CHANGES.txt
 URL: 
 http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/lucene/CHANGES.txt?rev=1351229r1=1351228r2=1351229view=diff
 ==
 --- lucene/dev/branches/branch_4x/lucene/CHANGES.txt (original)
 +++ lucene/dev/branches/branch_4x/lucene/CHANGES.txt Mon Jun 18 08:24:25 2012
 @@ -6,6 +6,17 @@ http://s.apache.org/luceneversions
  For contrib changes prior to 4.0, please see:
  http://svn.apache.org/repos/asf/lucene/dev/tags/lucene_solr_3_6_0/lucene/contrib/CHANGES.txt

 +New features
 +
 +* LUCENE-4108: add replaceTaxonomy to DirectoryTaxonomyWriter, which replaces
 +  the taxonomy in place with the given one. (Shai Erera)
 +
 +API Changes
 +
 +* LUCENE-4132: IndexWriter.getConfig() now returns a LiveIndexWriterConfig 
 object
 +  which can be used to change the IndexWriter's live settings. 
 IndexWriterConfig
 +  is used only for initializing the IndexWriter. (Shai Erera)
 +
  === Lucene 4.0.0-ALPHA ===
  More information about this release, including any errata related to the
  release notes, upgrade instructions, or other changes may be found online at:

 Modified: 
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriter.java
 URL: 
 http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriter.java?rev=1351229r1=1351228r2=1351229view=diff
 ==
 --- 
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriter.java
  (original)
 +++ 
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriter.java
  Mon Jun 18 08:24:25 2012
 @@ -133,7 +133,7 @@ final class DocumentsWriter {
   final DocumentsWriterFlushControl flushControl;

   final Codec codec;
 -  DocumentsWriter(Codec codec, IndexWriterConfig config, Directory 
 directory, IndexWriter writer, FieldNumbers globalFieldNumbers,
 +  DocumentsWriter(Codec codec, LiveIndexWriterConfig config, Directory 
 directory, IndexWriter writer, FieldNumbers globalFieldNumbers,
       BufferedDeletesStream bufferedDeletesStream) throws IOException {
     this.codec = codec;
     this.directory = directory;

 Modified: 
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriterFlushControl.java
 URL: 
 http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriterFlushControl.java?rev=1351229r1=1351228r2=1351229view=diff
 ==
 --- 
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriterFlushControl.java
  (original)
 +++ 
 lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/index/DocumentsWriterFlushControl.java
  Mon Jun 18 08:24:25 2012
 @@ -65,10 +65,9 @@ final class DocumentsWriterFlushControl
   private final FlushPolicy flushPolicy;
   private boolean closed = false;
   private final DocumentsWriter documentsWriter;
 -  private final IndexWriterConfig config;
 +  private final LiveIndexWriterConfig

[jira] [Reopened] (LUCENE-4108) Add replaceTaxonomy to DirectoryTaxonomyWriter


 [ 
https://issues.apache.org/jira/browse/LUCENE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir reopened LUCENE-4108:
-


CHANGES.txt is mangled

 Add replaceTaxonomy to DirectoryTaxonomyWriter
 --

 Key: LUCENE-4108
 URL: https://issues.apache.org/jira/browse/LUCENE-4108
 Project: Lucene - Java
  Issue Type: New Feature
  Components: modules/facet
Reporter: Shai Erera
Assignee: Shai Erera
Priority: Minor
 Fix For: 4.0, 5.0

 Attachments: LUCENE-4108.patch


 I'd like to add the option to replace a taxonomy with another one, live on 
 DirectoryTaxoWriter. Similar to how one can do so on IndexWriter, by calling 
 deleteAll() and addIndexes(), it is valuable to be able to do so on DirTW as 
 well.
 That way, if you need to replace a search index and taxonomy index live, 
 you will be able to, given this new API.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (LUCENE-4132) IndexWriterConfig live settings


 [ 
https://issues.apache.org/jira/browse/LUCENE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir reopened LUCENE-4132:
-


CHANGES.txt is mangled

 IndexWriterConfig live settings
 ---

 Key: LUCENE-4132
 URL: https://issues.apache.org/jira/browse/LUCENE-4132
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Shai Erera
Assignee: Shai Erera
Priority: Minor
 Fix For: 4.0, 5.0

 Attachments: LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch, 
 LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch


 A while ago there was a discussion about making some IW settings live and I 
 remember that RAM buffer size was one of them. Judging from IW code, I see 
 that RAM buffer can be changed live as IW never caches it.
 However, I don't remember which other settings were decided to be live and 
 I don't see any documentation in IW nor IWC for that. IW.getConfig mentions:
 {code}
 * bNOTE:/b some settings may be changed on the
 * returned {@link IndexWriterConfig}, and will take
 * effect in the current IndexWriter instance.  See the
 * javadocs for the specific setters in {@link
 * IndexWriterConfig} for details.
 {code}
 But there's no text on e.g. IWC.setRAMBuffer mentioning that.
 I think that it'd be good if we make it easier for users to tell which of the 
 settings are live ones. There are few possible ways to do it:
 * Introduce a custom @live.setting tag on the relevant IWC.set methods, and 
 add special text for them in build.xml
 ** Or, drop the tag and just document it clearly.
 * Separate IWC to two interfaces, LiveConfig and OneTimeConfig (name 
 proposals are welcome !), have IWC impl both, and introduce another 
 IW.getLiveConfig which will return that interface, thereby clearly letting 
 the user know which of the settings are live.
 It'd be good if IWC itself could only expose setXYZ methods for the live 
 settings though. So perhaps, off the top of my head, we can do something like 
 this:
 * Introduce a Config object, which is essentially what IWC is today, and pass 
 it to IW.
 * IW will create a different object, IWC from that Config and IW.getConfig 
 will return IWC.
 * IWC itself will only have setXYZ methods for the live settings.
 It adds another object, but user code doesn't change - it still creates a 
 Config object when initializing IW, and need to handle a different type if it 
 ever calls IW.getConfig.
 Maybe that's not such a bad idea?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4132) IndexWriterConfig live settings


[ 
https://issues.apache.org/jira/browse/LUCENE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395846#comment-13395846
 ] 

Uwe Schindler commented on LUCENE-4132:
---

The 4.x commit also breaks the builds, looks like duplicate merges to same file.

 IndexWriterConfig live settings
 ---

 Key: LUCENE-4132
 URL: https://issues.apache.org/jira/browse/LUCENE-4132
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Shai Erera
Assignee: Shai Erera
Priority: Minor
 Fix For: 4.0, 5.0

 Attachments: LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch, 
 LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch


 A while ago there was a discussion about making some IW settings live and I 
 remember that RAM buffer size was one of them. Judging from IW code, I see 
 that RAM buffer can be changed live as IW never caches it.
 However, I don't remember which other settings were decided to be live and 
 I don't see any documentation in IW nor IWC for that. IW.getConfig mentions:
 {code}
 * bNOTE:/b some settings may be changed on the
 * returned {@link IndexWriterConfig}, and will take
 * effect in the current IndexWriter instance.  See the
 * javadocs for the specific setters in {@link
 * IndexWriterConfig} for details.
 {code}
 But there's no text on e.g. IWC.setRAMBuffer mentioning that.
 I think that it'd be good if we make it easier for users to tell which of the 
 settings are live ones. There are few possible ways to do it:
 * Introduce a custom @live.setting tag on the relevant IWC.set methods, and 
 add special text for them in build.xml
 ** Or, drop the tag and just document it clearly.
 * Separate IWC to two interfaces, LiveConfig and OneTimeConfig (name 
 proposals are welcome !), have IWC impl both, and introduce another 
 IW.getLiveConfig which will return that interface, thereby clearly letting 
 the user know which of the settings are live.
 It'd be good if IWC itself could only expose setXYZ methods for the live 
 settings though. So perhaps, off the top of my head, we can do something like 
 this:
 * Introduce a Config object, which is essentially what IWC is today, and pass 
 it to IW.
 * IW will create a different object, IWC from that Config and IW.getConfig 
 will return IWC.
 * IWC itself will only have setXYZ methods for the live settings.
 It adds another object, but user code doesn't change - it still creates a 
 Config object when initializing IW, and need to handle a different type if it 
 ever calls IW.getConfig.
 Maybe that's not such a bad idea?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3866) Make CompositeReader.getSequentialSubReaders() and the corresponding IndexReaderContext methods return unmodifiable ListR extends IndexReader


[ 
https://issues.apache.org/jira/browse/LUCENE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395847#comment-13395847
 ] 

Uwe Schindler commented on LUCENE-3866:
---

I will later also upgrade MIGRATE.txt markdown.

 Make CompositeReader.getSequentialSubReaders() and the corresponding 
 IndexReaderContext methods return unmodifiable ListR extends IndexReader
 ---

 Key: LUCENE-3866
 URL: https://issues.apache.org/jira/browse/LUCENE-3866
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3866-step1.patch, LUCENE-3866-step2.patch, 
 LUCENE-3866-step2.patch


 Since Lucene 2.9 we have the method getSequentialSubReader() returning 
 IndexReader[]. Based on hardly-to-debug errors in user's code, Robert and me 
 realized that returning an array from a public API is an anti-pattern. If the 
 array is intended to be modifiable (like byte[] in terms,...), it is fine to 
 use arrays in public APIs, but not, if the array must be protected from 
 modification. As IndexReaders are 100% unmodifiable in trunk code (no 
 deletions,...), the only possibility to corrumpt the reader is by modifying 
 the array returned by getSequentialSubReaders(). We should prevent this.
 The same theoretically applies to FieldCache, too - but the party that is 
 afraid of performance problems is too big to fight against that :(
 For getSequentialSubReaders there is no performance problem at all. The 
 binary search of reader-ids inside BaseCompositeReader can still be done with 
 the internal protected array, but public APIs should expose only a 
 unmodifiable List. The same applies to leaves() and children() in 
 IndexReaderContext. This change to list would also allow to make 
 CompositeReader and CompositeReaderContext IterableR extends IndexReader, 
 so some loops would look nice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4147) rollback/preparecommit thread hazard

[
https://issues.apache.org/jira/browse/LUCENE-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395854#comment-13395854
]

Michael McCandless commented on LUCENE-4147:

bq. mike, do we really need to acquire the commit lock?

The problem is rollback forcefully clears the pendingCommit and then deletes
any files it had (alone) referenced, so if a commit is running concurrently the
fsyncs will fail since the files were deleted.

Also: it doesn't really make sense to allow rollback and commit to proceed
concurrently? Why would an app need this? Seems like we can simplify the code
by making them exclusive.

bq. regarding the thread safety issue in DocWriter can you paste the trace?

Will do ... need to re-beast.

rollback/preparecommit thread hazard

Key: LUCENE-4147
URL: https://issues.apache.org/jira/browse/LUCENE-4147
Project: Lucene - Java
Issue Type: Bug
Affects Versions: 4.0
Reporter: Robert Muir
Attachments: LUCENE-4147.patch, LUCENE-4147.patch

found by
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows-Java7-64/70/
rollback should never throw this exception, as it documents it clears any
pendingcommits.
but it calls closeInternal outside of any sync block, so it looks like there
is a race here.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4069) Segment-level Bloom filters for a 2 x speed up on rare term searches

[
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395857#comment-13395857
]

Mark Harwood commented on LUCENE-4069:
--

bq. I think the fix is simple: you are not overriding Terms.intersect now, in
BloomFilteredTerms

Good catch - a quick test indeed shows a speed up on fuzzy queries.
I'll prepare a new patch.

I'm not sure on why 3.6+Bloom is faster than 4+Bloom in my tests. I'll take a
closer look at your benchmark.

Segment-level Bloom filters for a 2 x speed up on rare term searches

Attachments: BloomFilterPostingsBranch4x.patch,
MHBloomFilterOn3.6Branch.patch, PrimaryKeyPerfTest40.java

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4147) rollback/preparecommit thread hazard


 [ 
https://issues.apache.org/jira/browse/LUCENE-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless updated LUCENE-4147:
---

Attachment: fail.log

Here's the verbose output from a failure w/ the patch ... 

 rollback/preparecommit thread hazard
 

 Key: LUCENE-4147
 URL: https://issues.apache.org/jira/browse/LUCENE-4147
 Project: Lucene - Java
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Robert Muir
 Attachments: LUCENE-4147.patch, LUCENE-4147.patch, fail.log


 found by 
 http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows-Java7-64/70/
 rollback should never throw this exception, as it documents it clears any 
 pendingcommits.
 but it calls closeInternal outside of any sync block, so it looks like there 
 is a race here.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4152) add one-syllable method to IndexReader enumerate subreaders


[ 
https://issues.apache.org/jira/browse/LUCENE-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395866#comment-13395866
 ] 

Robert Muir commented on LUCENE-4152:
-

ok, so after LUCENE-3866 is resolved maybe all that is needed here is an 
eclipse rename?

 add one-syllable method to IndexReader enumerate subreaders
 ---

 Key: LUCENE-4152
 URL: https://issues.apache.org/jira/browse/LUCENE-4152
 Project: Lucene - Java
  Issue Type: Bug
Reporter: Robert Muir
Priority: Blocker
 Fix For: 4.0


 Description is exactly as written.
 getSequentialSubReaders/getTopLevelReaderContext, these method names are way 
 too long/unuseable. They also have tricky semantics (e.g. returning null).
 In lucene 4, people cannot just use any indexreader and get a merged view. So 
 we need to make this stuff easy on them:
 * single-syllable method name (leaves(), subs(), i will think on this)
 * supports enhanced for-loop (no returning null or anything like that)
 * on indexreader (not atomic or composite, plain old indexreader)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3555) Lucene DirectoryReader can't find *.frq files created by solr

2012-06-18 Thread Nik V. Babichev (JIRA)

Nik V. Babichev created SOLR-3555:
-

 Summary: Lucene DirectoryReader can't find *.frq files created by 
solr
 Key: SOLR-3555
 URL: https://issues.apache.org/jira/browse/SOLR-3555
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Nik V. Babichev


Create index with Solr4

In index directory have such files:
_0_Lucene40_0.frq
_0_Lucene40_0.prx
_0_Lucene40_0.tim
_0_Lucene40_0.tip


When try to open index by Lucene Directory reader got:
java.io.FileNotFoundException: 
/opt/solr4/irotor/data/docs/index/_0_Lucene40.frq (No such file)

File idxLocation = new File('path to index');
DirectoryFactory df = new StandardDirectoryFactory();
DirectoryReader.open(df.get(idxLocation.getAbsolutePath(), native))

Exception generated in this method:
org.apache.lucene.codecs.lucene40.Lucene40PostingsReader(Directory dir, 
FieldInfos fieldInfos, SegmentInfo segmentInfo, IOContext ioContext, String 
segmentSuffix);

freqIn = dir.openInput(IndexFileNames.segmentFileName(segmentInfo.name, 
segmentSuffix, Lucene40PostingsFormat.FREQ_EXTENSION), ioContext);

When try to open unexisting _0_Lucene40.frq instead of _0_Lucene40_0.frq

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4152) add one-syllable method to IndexReader enumerate subreaders


[ 
https://issues.apache.org/jira/browse/LUCENE-4152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395868#comment-13395868
 ] 

Uwe Schindler commented on LUCENE-4152:
---

Yes, and some final methods in IndexReader abstract class for easy usage, so 
you can do:

{code:java}
for (AtomicReaderContext ctx : reader.leaves()) ...
{code}

 add one-syllable method to IndexReader enumerate subreaders
 ---

 Key: LUCENE-4152
 URL: https://issues.apache.org/jira/browse/LUCENE-4152
 Project: Lucene - Java
  Issue Type: Bug
Reporter: Robert Muir
Priority: Blocker
 Fix For: 4.0


 Description is exactly as written.
 getSequentialSubReaders/getTopLevelReaderContext, these method names are way 
 too long/unuseable. They also have tricky semantics (e.g. returning null).
 In lucene 4, people cannot just use any indexreader and get a merged view. So 
 we need to make this stuff easy on them:
 * single-syllable method name (leaves(), subs(), i will think on this)
 * supports enhanced for-loop (no returning null or anything like that)
 * on indexreader (not atomic or composite, plain old indexreader)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3555) Lucene DirectoryReader can't find *.frq files created by solr


 [ 
https://issues.apache.org/jira/browse/SOLR-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Muir resolved SOLR-3555.
---

Resolution: Invalid

if you are using trunk, you need to subscribe to dev@ and commits@. when a note 
says reindex, we really mean it.

 Lucene DirectoryReader can't find *.frq files created by solr
 -

 Key: SOLR-3555
 URL: https://issues.apache.org/jira/browse/SOLR-3555
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Nik V. Babichev

 Create index with Solr4
 In index directory have such files:
 _0_Lucene40_0.frq
 _0_Lucene40_0.prx
 _0_Lucene40_0.tim
 _0_Lucene40_0.tip
 When try to open index by Lucene Directory reader got:
 java.io.FileNotFoundException: 
 /opt/solr4/irotor/data/docs/index/_0_Lucene40.frq (No such file)
 File idxLocation = new File('path to index');
 DirectoryFactory df = new StandardDirectoryFactory();
 DirectoryReader.open(df.get(idxLocation.getAbsolutePath(), native))
 Exception generated in this method:
 org.apache.lucene.codecs.lucene40.Lucene40PostingsReader(Directory dir, 
 FieldInfos fieldInfos, SegmentInfo segmentInfo, IOContext ioContext, String 
 segmentSuffix);
 freqIn = dir.openInput(IndexFileNames.segmentFileName(segmentInfo.name, 
 segmentSuffix, Lucene40PostingsFormat.FREQ_EXTENSION), ioContext);
 When try to open unexisting _0_Lucene40.frq instead of _0_Lucene40_0.frq

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4153) Fast vector highlighting should be aware of field boundaries in case of mv fields.

Martijn van Groningen created LUCENE-4153:
-

 Summary: Fast vector highlighting should be aware of field 
boundaries in case of mv fields.
 Key: LUCENE-4153
 URL: https://issues.apache.org/jira/browse/LUCENE-4153
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Martijn van Groningen


Improve fast vector highlighter to respect field boundaries in the case for 
multivalued fields.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4153) Fast vector highlighting should be aware of field boundaries in case of mv fields.


 [ 
https://issues.apache.org/jira/browse/LUCENE-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martijn van Groningen updated LUCENE-4153:
--

Attachment: LUCENE-4153.patch

Initial approach to make fvh respect field boundaries when highlighting on mv 
fields.

Basically in the BaseFragmentsBuilder the fragInfos are checked if not crossing 
field boundaries. If so a fragInfo might be modified or split. This approach 
seems good to me. Maybe others have a different opinion?

This behavior can be enabled with the discreteMultiValueHighlighting boolean 
property.  

 Fast vector highlighting should be aware of field boundaries in case of mv 
 fields.
 --

 Key: LUCENE-4153
 URL: https://issues.apache.org/jira/browse/LUCENE-4153
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Martijn van Groningen
 Attachments: LUCENE-4153.patch


 Improve fast vector highlighter to respect field boundaries in the case for 
 multivalued fields.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)


 [ 
https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Han Jiang updated LUCENE-3892:
--

Attachment: LUCENE-3892_for.patch
LUCENE-3892_pfor.patch

The new 3892_pfor patch fixed some SuppressingCodec stuff since last two 
weeks. And the 3892_for lazily implements For postingsformat based on 
current codes. These two patches are temporary separated, in order to prevent 
performance reduction for the sake of method overriding.

Currently, blocksize ranges from 32 to 128 are tested on both two patches. 
However, for those skipping-intensive queries, there is no significant 
performance gain when smaller blocksize was applied. 

Here is a previous result for PFor, with blockSize=64, comparing with 128(in 
brackets):
{noformat}
TaskQPS Base StdDev BaseQPS PFor StdDev PFor  Pct 
diff
  Phrase4.930.363.100.33  -47% -  
-25%  (-47% -  -25%)
  AndHighMed   27.922.26   19.161.72  -42% -  
-18%  (-37% -  -15%)
SpanNear2.730.161.960.24  -40% -  
-14%  (-36% -  -13%)
SloppyPhrase4.190.213.200.30  -34% -  
-12%  (-30% -   -6%)
Wildcard   19.440.87   17.110.94  -20% -   
-2%  (-17% -3%)
 AndHighHigh7.500.386.610.59  -23% -
1%  (-19% -6%)
  IntNRQ4.060.523.880.35  -22% -   
19%  (-16% -   24%)
 Prefix3   31.001.69   30.452.29  -13% -   
11%  ( -6% -   20%)
  OrHighHigh4.160.474.110.34  -18% -   
20%  (-14% -   27%)
   OrHighMed4.980.594.940.41  -18% -   
22%  (-14% -   27%)
 Respell   40.292.11   40.112.13  -10% -   
10%  (-15% -2%)
TermBGroup1M   20.500.32   20.520.80   -5% -
5%  (  1% -   10%)
 TermGroup1M   13.510.43   13.610.40   -5% -
7%  (  1% -9%)
  Fuzzy1   43.201.83   44.021.95   -6% -   
11%  (-11% -1%)
PKLookup   87.161.78   89.520.940% -
5%  ( -2% -7%)
  Fuzzy2   16.090.80   16.540.77   -6% -   
13%  (-11% -6%)
Term   43.561.53   45.263.84   -8% -   
16%  (  2% -   26%)
  TermBGroup1M1P   21.330.64   22.241.23   -4% -   
13%  (  0% -   14%) 
{noformat}

Also, the For postingsformat shows few performance change. So I suppose the 
bottleneck isn't in this method: PForUtil.patchException.
Here is an example with blockSize=64:
{noformat}
TaskQPS Base StdDev Base QPS For  StdDev For  Pct 
diff
  Phrase5.030.453.300.43  -47% -  
-18%
  AndHighMed   28.052.33   18.831.77  -43% -  
-19%
SpanNear2.690.181.940.25  -40% -  
-12%
SloppyPhrase4.190.203.220.35  -34% -  
-10%
 AndHighHigh7.610.466.410.54  -27% -   
-2%
 Respell   41.361.65   37.942.42  -17% -
1%
Wildcard   19.200.77   17.890.99  -15% -
2%
  OrHighHigh4.220.373.940.32  -21% -   
10%
   OrHighMed5.060.464.730.39  -21% -   
11%
  Fuzzy1   44.151.31   42.381.74  -10% -
2%
  Fuzzy2   16.480.59   15.840.76  -11% -
4%
 TermGroup1M   13.320.35   13.440.53   -5% -
7%
PKLookup   87.701.81   88.621.22   -2% -
4%
TermBGroup1M   20.140.47   20.400.59   -3% -
6%
 Prefix3   30.311.49   31.082.26   -9% -   
15%
  TermBGroup1M1P   21.130.46   21.791.42   -5% -   
12%
  IntNRQ3.960.454.140.46  -16% -   
31%
Term   43.071.51   46.064.50   -6% -   
21%
{noformat}

 Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, 
 Simple9/16/64, etc.)
 -

 Key: LUCENE-3892
 URL: https://issues.apache.org/jira/browse/LUCENE-3892
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Michael McCandless
  Labels: gsoc2012,

[jira] [Resolved] (LUCENE-4108) Add replaceTaxonomy to DirectoryTaxonomyWriter

2012-06-18 Thread Shai Erera (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shai Erera resolved LUCENE-4108.


Resolution: Fixed

I committed a fix for that -- moved the relevant entries to the 4.0.0 section. 
Hope it's ok now.

 Add replaceTaxonomy to DirectoryTaxonomyWriter
 --

 Key: LUCENE-4108
 URL: https://issues.apache.org/jira/browse/LUCENE-4108
 Project: Lucene - Java
  Issue Type: New Feature
  Components: modules/facet
Reporter: Shai Erera
Assignee: Shai Erera
Priority: Minor
 Fix For: 4.0, 5.0

 Attachments: LUCENE-4108.patch


 I'd like to add the option to replace a taxonomy with another one, live on 
 DirectoryTaxoWriter. Similar to how one can do so on IndexWriter, by calling 
 deleteAll() and addIndexes(), it is valuable to be able to do so on DirTW as 
 well.
 That way, if you need to replace a search index and taxonomy index live, 
 you will be able to, given this new API.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4148) _TestUtil should be able to generate random longs

2012-06-18 Thread Adrien Grand (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-4148:
-

Attachment: LUCENE-4148.patch

Here is a patch proposition:
 * {{_TestUtil.nextInt}} forwards calls to {{RandomInts.randomIntBetween}}, as 
Robert suggested,
 * A new {{_TestUtil.nextLong}} uses {{Random.nextInt(int)}} when the range 
does not exceed {{Integer.MAX_VALUE}} and scales {{Double.nextDouble}} into the 
expected range of values otherwise using {{BigDecimal}} arithmetic.

I also updated a few tests to use this new method (especially a call to 
{{Math.abs(random.nextLong())}} in {{TestIndexInput}}, which is buggy since 
{{random.nextLong()}} might return Long.MIN_VALUE whose absolute value cannot 
be represented by a long).

 _TestUtil should be able to generate random longs
 -

 Key: LUCENE-4148
 URL: https://issues.apache.org/jira/browse/LUCENE-4148
 Project: Lucene - Java
  Issue Type: Improvement
  Components: general/test
Reporter: Adrien Grand
Priority: Trivial
 Attachments: LUCENE-4148.patch


 It would be helpful in TestPackedInts at least, in order to generate random 
 values (as a workaround, we currently generate a random int between 0 and 
 {{min(Integer.MAX_VALUE, PackedInts.maxValue(bitsPerValue)}}). Moreover, it 
 would help to fix {{nextInt}} for large ranges (calling {{nextInt(random, 
 -10, Integer.MAX_VALUE)}} or even {{nextInt(random, 0, Integer.MAX_VALUE)}} 
 currently fails because the range of values is {{ Integer.MAX_VALUE}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)


[ 
https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395894#comment-13395894
 ] 

Han Jiang commented on LUCENE-3892:
---

There's a potential bottleneck during method calling...Here is an example for 
PFor, with blocksize=128, exception rate = 97%, normal value = 2 bits, 
exception value = 32 bits:

{noformat}
Decoding normal values:  4703 ns
Patching exceptions: 5797 ns
Single call of PForUtil.decompress totally takes:   58318 ns
{noformat}

In addition, it costs about 4000ns to record the time span.

 Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, 
 Simple9/16/64, etc.)
 -

 Key: LUCENE-3892
 URL: https://issues.apache.org/jira/browse/LUCENE-3892
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Michael McCandless
  Labels: gsoc2012, lucene-gsoc-12
 Fix For: 4.1

 Attachments: LUCENE-3892_for.patch, LUCENE-3892_pfor.patch, 
 LUCENE-3892_pfor.patch, LUCENE-3892_pfor.patch, LUCENE-3892_settings.patch, 
 LUCENE-3892_settings.patch


 On the flex branch we explored a number of possible intblock
 encodings, but for whatever reason never brought them to completion.
 There are still a number of issues opened with patches in different
 states.
 Initial results (based on prototype) were excellent (see
 http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html
 ).
 I think this would make a good GSoC project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-tests-only-4.x - Build # 128 - Still Failing

2012-06-18 Thread Apache Jenkins Server

Build: https://builds.apache.org/job/Lucene-Solr-tests-only-4.x/128/

No tests ran.

Build Log:
[...truncated 466 lines...]
 [echo] Building solr-langid...

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/ivy-settings.xml

resolve:
 [echo] Building solr-uima...

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/ivy-settings.xml

resolve:
 [echo] Building solr-velocity...

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/ivy-settings.xml

resolve:

validate:
 [echo] License check under: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/solr
 [licenses] Scanned 82 JAR file(s) for licenses (in 0.79s.), 0 error(s).

test:

test-core:

jflex-uptodate-check:

jflex-notice:

javacc-uptodate-check:

javacc-notice:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/ivy-settings.xml

resolve:

init:

clover.setup:

clover.info:
 [echo] 
 [echo]   Clover not found. Code coverage reports disabled.
 [echo] 

clover:

common.compile-core:
[mkdir] Created dir: 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/build/core/classes/java
[javac] Compiling 649 source files to 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/build/core/classes/java
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:506:
 class, interface, or enum expected
[javac] package org.apache.lucene.index;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:525:
 class, interface, or enum expected
[javac] import org.apache.lucene.analysis.Analyzer;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:526:
 class, interface, or enum expected
[javac] import org.apache.lucene.codecs.Codec;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:527:
 class, interface, or enum expected
[javac] import 
org.apache.lucene.index.DocumentsWriterPerThread.IndexingChain;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:528:
 class, interface, or enum expected
[javac] import org.apache.lucene.index.IndexWriter.IndexReaderWarmer;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:529:
 class, interface, or enum expected
[javac] import org.apache.lucene.index.IndexWriterConfig.OpenMode;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:530:
 class, interface, or enum expected
[javac] import org.apache.lucene.search.IndexSearcher;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:531:
 class, interface, or enum expected
[javac] import org.apache.lucene.search.similarities.Similarity;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:532:
 class, interface, or enum expected
[javac] import org.apache.lucene.util.InfoStream;
[javac] ^
[javac] 
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x/checkout/lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:533:
 class, interface, or enum expected
[javac] import org.apache.lucene.util.Version;
[javac] ^
[javac] 10 errors
[...truncated 15 lines...]

[...truncated 582 lines...]

[...truncated 582 lines...]

[...truncated 582 lines...]

[...truncated 582 lines...]

[...truncated 582 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3555) Lucene DirectoryReader can't find *.frq files created by solr

2012-06-18 Thread Nik V. Babichev (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395904#comment-13395904
 ] 

Nik V. Babichev commented on SOLR-3555:
---

Thanks, 

I resolve my current problem.

I build solr.war from SVN branch_4x 
and use maven artifact from 4.0-SNAPSHOT 
(https://repository.apache.org/content/groups/snapshots)
for lucene requests.
As I see 4.0-SNAPSHOT have difference with svn version.

Will try trunk and 5.0-SNAPSHOT

 Lucene DirectoryReader can't find *.frq files created by solr
 -

 Key: SOLR-3555
 URL: https://issues.apache.org/jira/browse/SOLR-3555
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Nik V. Babichev

 Create index with Solr4
 In index directory have such files:
 _0_Lucene40_0.frq
 _0_Lucene40_0.prx
 _0_Lucene40_0.tim
 _0_Lucene40_0.tip
 When try to open index by Lucene Directory reader got:
 java.io.FileNotFoundException: 
 /opt/solr4/irotor/data/docs/index/_0_Lucene40.frq (No such file)
 File idxLocation = new File('path to index');
 DirectoryFactory df = new StandardDirectoryFactory();
 DirectoryReader.open(df.get(idxLocation.getAbsolutePath(), native))
 Exception generated in this method:
 org.apache.lucene.codecs.lucene40.Lucene40PostingsReader(Directory dir, 
 FieldInfos fieldInfos, SegmentInfo segmentInfo, IOContext ioContext, String 
 segmentSuffix);
 freqIn = dir.openInput(IndexFileNames.segmentFileName(segmentInfo.name, 
 segmentSuffix, Lucene40PostingsFormat.FREQ_EXTENSION), ioContext);
 When try to open unexisting _0_Lucene40.frq instead of _0_Lucene40_0.frq

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)


 [ 
https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Han Jiang updated LUCENE-3892:
--

Attachment: (was: LUCENE-3892_for.patch)

 Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, 
 Simple9/16/64, etc.)
 -

 Key: LUCENE-3892
 URL: https://issues.apache.org/jira/browse/LUCENE-3892
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Michael McCandless
  Labels: gsoc2012, lucene-gsoc-12
 Fix For: 4.1

 Attachments: LUCENE-3892_for.patch, LUCENE-3892_pfor.patch, 
 LUCENE-3892_pfor.patch, LUCENE-3892_pfor.patch, LUCENE-3892_settings.patch, 
 LUCENE-3892_settings.patch


 On the flex branch we explored a number of possible intblock
 encodings, but for whatever reason never brought them to completion.
 There are still a number of issues opened with patches in different
 states.
 Initial results (based on prototype) were excellent (see
 http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html
 ).
 I think this would make a good GSoC project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)


 [ 
https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Han Jiang updated LUCENE-3892:
--

Attachment: LUCENE-3892_for.patch
LUCENE-3892_pfor.patch

 Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, 
 Simple9/16/64, etc.)
 -

 Key: LUCENE-3892
 URL: https://issues.apache.org/jira/browse/LUCENE-3892
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Michael McCandless
  Labels: gsoc2012, lucene-gsoc-12
 Fix For: 4.1

 Attachments: LUCENE-3892_for.patch, LUCENE-3892_pfor.patch, 
 LUCENE-3892_pfor.patch, LUCENE-3892_pfor.patch, LUCENE-3892_settings.patch, 
 LUCENE-3892_settings.patch


 On the flex branch we explored a number of possible intblock
 encodings, but for whatever reason never brought them to completion.
 There are still a number of issues opened with patches in different
 states.
 Initial results (based on prototype) were excellent (see
 http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html
 ).
 I think this would make a good GSoC project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)


 [ 
https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Han Jiang updated LUCENE-3892:
--

Attachment: (was: LUCENE-3892_pfor.patch)

 Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, 
 Simple9/16/64, etc.)
 -

 Key: LUCENE-3892
 URL: https://issues.apache.org/jira/browse/LUCENE-3892
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Michael McCandless
  Labels: gsoc2012, lucene-gsoc-12
 Fix For: 4.1

 Attachments: LUCENE-3892_for.patch, LUCENE-3892_pfor.patch, 
 LUCENE-3892_pfor.patch, LUCENE-3892_pfor.patch, LUCENE-3892_settings.patch, 
 LUCENE-3892_settings.patch


 On the flex branch we explored a number of possible intblock
 encodings, but for whatever reason never brought them to completion.
 There are still a number of issues opened with patches in different
 states.
 Initial results (based on prototype) were excellent (see
 http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html
 ).
 I think this would make a good GSoC project.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-3555) Lucene DirectoryReader can't find *.frq files created by solr

2012-06-18 Thread Nik V. Babichev (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395904#comment-13395904
 ] 

Nik V. Babichev edited comment on SOLR-3555 at 6/18/12 2:20 PM:


Thanks, 

I resolve my current problem.

I build solr.war from SVN branch_4x 
and use maven artifact from 4.0-SNAPSHOT 
(https://repository.apache.org/content/groups/snapshots)
for lucene requests.
As I see 4.0-SNAPSHOT have difference with svn version.

Trunk and 5.0-SNAPSHOT works well

  was (Author: sharky):
Thanks, 

I resolve my current problem.

I build solr.war from SVN branch_4x 
and use maven artifact from 4.0-SNAPSHOT 
(https://repository.apache.org/content/groups/snapshots)
for lucene requests.
As I see 4.0-SNAPSHOT have difference with svn version.

Will try trunk and 5.0-SNAPSHOT
  
 Lucene DirectoryReader can't find *.frq files created by solr
 -

 Key: SOLR-3555
 URL: https://issues.apache.org/jira/browse/SOLR-3555
 Project: Solr
  Issue Type: Bug
Affects Versions: 4.0
Reporter: Nik V. Babichev

 Create index with Solr4
 In index directory have such files:
 _0_Lucene40_0.frq
 _0_Lucene40_0.prx
 _0_Lucene40_0.tim
 _0_Lucene40_0.tip
 When try to open index by Lucene Directory reader got:
 java.io.FileNotFoundException: 
 /opt/solr4/irotor/data/docs/index/_0_Lucene40.frq (No such file)
 File idxLocation = new File('path to index');
 DirectoryFactory df = new StandardDirectoryFactory();
 DirectoryReader.open(df.get(idxLocation.getAbsolutePath(), native))
 Exception generated in this method:
 org.apache.lucene.codecs.lucene40.Lucene40PostingsReader(Directory dir, 
 FieldInfos fieldInfos, SegmentInfo segmentInfo, IOContext ioContext, String 
 segmentSuffix);
 freqIn = dir.openInput(IndexFileNames.segmentFileName(segmentInfo.name, 
 segmentSuffix, Lucene40PostingsFormat.FREQ_EXTENSION), ioContext);
 When try to open unexisting _0_Lucene40.frq instead of _0_Lucene40_0.frq

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 170 - Failure!

Build: 
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-64/170/

All tests passed

Build Log:
[...truncated 4766 lines...]

[...truncated 4766 lines...]

[...truncated 4766 lines...]

[...truncated 4766 lines...]

[...truncated 4747 lines...]
   [junit4] ERROR: Forked JVM execution exception: 0, execution line: 
/mnt/ssd/jenkins/tools/java/jdk1.6.0_32/jre/bin/java 
-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/heapdumps/
 -Dtests.prefix=tests -Dtests.seed=4C54E59EBD35BEF0 -Xmx512M -Dtests.iters= 
-Dtests.verbose=false -Dtests.infostream=false 
-Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build
 -Dtests.codec=random -Dtests.postingsformat=random -Dtests.locale=random 
-Dtests.timezone=random -Dtests.directory=random 
-Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.0 
-Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/dev/null 
-Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=false 
-Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. 
-Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/test-framework/lib/junit-4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/test-framework/lib/randomizedtesting-runner-1.5.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/analysis/common/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/analysis/common/classes/test:/var/lib/jenkins/tools/ant/supported18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-netrexx.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-javamail.jar:/mnt/ssd/jenkins/tools/java/jdk1.6.0_32/lib/tools.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/test-framework/lib/junit4-ant-1.5.0.jar
 -ea:org.apache.lucene... -ea:org.apache.solr... 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/analysis/common/test/junit4-J0-1422064855254770f.events
 
@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/analysis/common/test/junit4-J0-1422064855254770f.suites
   [junit4] ERROR: Forked JVM execution exception: 1, execution line: 
/mnt/ssd/jenkins/tools/java/jdk1.6.0_32/jre/bin/java 
-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/heapdumps/
 -Dtests.prefix=tests -Dtests.seed=4C54E59EBD35BEF0 -Xmx512M -Dtests.iters= 
-Dtests.verbose=false -Dtests.infostream=false 
-Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build
 -Dtests.codec=random -Dtests.postingsformat=random -Dtests.locale=random 
-Dtests.timezone=random -Dtests.directory=random 
-Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.0 
-Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/dev/null 
-Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=false 
-Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. 
-Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath

RE: [JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 170 - Failure!

2012-06-18 Thread Uwe Schindler

This was a test by Steven and me!

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de

 -Original Message-
 From: Jenkins Server with Girls [mailto:jenk...@sd-datasolutions.de]
 Sent: Monday, June 18, 2012 4:23 PM
 To: dev@lucene.apache.org; sh...@apache.org
 Subject: [JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 170 - Failure!
 
 Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-
 64/170/
 
 All tests passed
 
 Build Log:
 [...truncated 4766 lines...]
 
 [...truncated 4766 lines...]
 
 [...truncated 4766 lines...]
 
 [...truncated 4766 lines...]
 
 [...truncated 4747 lines...]
[junit4] ERROR: Forked JVM execution exception: 0, execution line:
 /mnt/ssd/jenkins/tools/java/jdk1.6.0_32/jre/bin/java -
 XX:+HeapDumpOnOutOfMemoryError -
 XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/heapdumps/ -Dtests.prefix=tests -Dtests.seed=4C54E59EBD35BEF0 -
 Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -
 Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build -Dtests.codec=random -
 Dtests.postingsformat=random -Dtests.locale=random -
 Dtests.timezone=random -Dtests.directory=random -
 Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.0 -
 Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/dev/null -
 Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=false -
 Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. -
 Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1
 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath
 /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/test-
 framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-
 Java6-
 64/checkout/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucen
 e-Solr-4.x-Linux-Java6-64/checkout/lucene/test-framework/lib/junit-
 4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/test-framework/lib/randomizedtesting-runner-
 1.5.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/analysis/common/classes/java:/mnt/ssd/jenkins/wor
 kspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/analysis/common/classes/test:/var/lib/jenkins/tools/
 ant/supported18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-
 2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-
 net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tool
 s/ant/supported18/lib/ant-
 junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-
 logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 netrexx.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 javamail.jar:/mnt/ssd/jenkins/tools/java/jdk1.6.0_32/lib/tools.jar:/mnt/ssd/jenk
 ins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/test-
 framework/lib/junit4-ant-1.5.0.jar -ea:org.apache.lucene... -
 ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -
 eventsfile /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/analysis/common/test/junit4-J0-
 1422064855254770f.events @/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-
 Linux-Java6-64/checkout/lucene/build/analysis/common/test/junit4-J0-
 1422064855254770f.suites
[junit4] ERROR: Forked JVM execution exception: 1, execution line:
 /mnt/ssd/jenkins/tools/java/jdk1.6.0_32/jre/bin/java -
 XX:+HeapDumpOnOutOfMemoryError -
 XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/heapdumps/ -Dtests.prefix=tests -Dtests.seed=4C54E59EBD35BEF0 -
 Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -
 Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build -Dtests.codec=random -
 Dtests.postingsformat=random -Dtests.locale=random -
 Dtests.timezone=random -Dtests.directory=random -

[jira] [Created] (SOLR-3556) Provide LBHttpSolrServer based on Http Components in Solr 3.x

2012-06-18 Thread Andreas Hubold (JIRA)

Andreas Hubold created SOLR-3556:


 Summary: Provide LBHttpSolrServer based on Http Components in Solr 
3.x
 Key: SOLR-3556
 URL: https://issues.apache.org/jira/browse/SOLR-3556
 Project: Solr
  Issue Type: Bug
  Components: clients - java
Affects Versions: 3.6
Reporter: Andreas Hubold
 Fix For: 3.6.1


With SOLR-2020 Solr 3.6 provides a HttpSolrServer based on Apache Http 
Components. But the LBHttpSolrServer still uses commons-httpclient. 

It would be cool if you could provide a loadbalancing implementation based on 
HttpComponents as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3556) Provide LBHttpSolrServer based on Http Components in Solr 3.x

2012-06-18 Thread Andreas Hubold (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andreas Hubold updated SOLR-3556:
-

Priority: Minor  (was: Major)

 Provide LBHttpSolrServer based on Http Components in Solr 3.x
 -

 Key: SOLR-3556
 URL: https://issues.apache.org/jira/browse/SOLR-3556
 Project: Solr
  Issue Type: Bug
  Components: clients - java
Affects Versions: 3.6
Reporter: Andreas Hubold
Priority: Minor
 Fix For: 3.6.1


 With SOLR-2020 Solr 3.6 provides a HttpSolrServer based on Apache Http 
 Components. But the LBHttpSolrServer still uses commons-httpclient. 
 It would be cool if you could provide a loadbalancing implementation based on 
 HttpComponents as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4069) Segment-level Bloom filters for a 2 x speed up on rare term searches


 [ 
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Harwood updated LUCENE-4069:
-

Attachment: (was: BloomFilterPostingsBranch4x.patch)

 Segment-level Bloom filters for a 2 x speed up on rare term searches
 

 Key: LUCENE-4069
 URL: https://issues.apache.org/jira/browse/LUCENE-4069
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
Affects Versions: 3.6, 4.0
Reporter: Mark Harwood
Priority: Minor
 Fix For: 4.0, 3.6.1

 Attachments: BloomFilterPostingsBranch4x.patch, 
 MHBloomFilterOn3.6Branch.patch, PrimaryKeyPerfTest40.java


 An addition to each segment which stores a Bloom filter for selected fields 
 in order to give fast-fail to term searches, helping avoid wasted disk access.
 Best suited for low-frequency fields e.g. primary keys on big indexes with 
 many segments but also speeds up general searching in my tests.
 Overview slideshow here: 
 http://www.slideshare.net/MarkHarwood/lucene-bloomfilteredsegments
 Benchmarks based on Wikipedia content here: http://goo.gl/X7QqU
 Patch based on 3.6 codebase attached.
 There are no 3.6 API changes currently - to play just add a field with _blm 
 on the end of the name to invoke special indexing/querying capability. 
 Clearly a new Field or schema declaration(!) would need adding to APIs to 
 configure the service properly.
 Also, a patch for Lucene4.0 codebase introducing a new PostingsFormat

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-4069) Segment-level Bloom filters for a 2 x speed up on rare term searches


 [ 
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Harwood updated LUCENE-4069:
-

Attachment: BloomFilterPostingsBranch4x.patch

New patch with Terms.intersect overridden for faster Fuzzy queries.
Didn't get too far with running the Wikipedia perf tests due to missing data 
file (see 
http://code.google.com/a/apache-extras.org/p/luceneutil/issues/detail?id=7 )

 Segment-level Bloom filters for a 2 x speed up on rare term searches
 

 Key: LUCENE-4069
 URL: https://issues.apache.org/jira/browse/LUCENE-4069
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
Affects Versions: 3.6, 4.0
Reporter: Mark Harwood
Priority: Minor
 Fix For: 4.0, 3.6.1

 Attachments: BloomFilterPostingsBranch4x.patch, 
 MHBloomFilterOn3.6Branch.patch, PrimaryKeyPerfTest40.java


 An addition to each segment which stores a Bloom filter for selected fields 
 in order to give fast-fail to term searches, helping avoid wasted disk access.
 Best suited for low-frequency fields e.g. primary keys on big indexes with 
 many segments but also speeds up general searching in my tests.
 Overview slideshow here: 
 http://www.slideshare.net/MarkHarwood/lucene-bloomfilteredsegments
 Benchmarks based on Wikipedia content here: http://goo.gl/X7QqU
 Patch based on 3.6 codebase attached.
 There are no 3.6 API changes currently - to play just add a field with _blm 
 on the end of the name to invoke special indexing/querying capability. 
 Clearly a new Field or schema declaration(!) would need adding to APIs to 
 configure the service properly.
 Also, a patch for Lucene4.0 codebase introducing a new PostingsFormat

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Solr - ConcurrentHashMap - NullPointerException

2012-06-18 Thread Fuad Efendi


I am recalling big noise in JIRA; and Paul Nobble implemented first version
of this what can be wrong now, after few years? Solr 3.6.0


Jun 18, 2012 3:40:43 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
at org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:87)
at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:136)
at 
org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher
.java:632)
at 
org.apache.solr.search.SolrIndexSearcher.numDocs(SolrIndexSearcher.java:1743
)
at 
org.apache.solr.request.SimpleFacets.getFacetQueryCounts(SimpleFacets.java:2
34)
at 
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:194)
at 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java
:72)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHand
ler.java:186)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.
java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:3
65)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:
260)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
va:233)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
va:191)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127
)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
:109)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:680)



Fuad Efendi
416-993-2060
http://www.tokenizer.ca

[jira] [Commented] (LUCENE-3866) Make CompositeReader.getSequentialSubReaders() and the corresponding IndexReaderContext methods return unmodifiable ListR extends IndexReader


[ 
https://issues.apache.org/jira/browse/LUCENE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395968#comment-13395968
 ] 

Michael McCandless commented on LUCENE-3866:


I ran quick perf test:
{noformat}
TaskQPS base StdDev base   QPS patchStdDev patch  Pct 
diff
 Respell   92.151.40   90.302.70   -6% -
2%
PKLookup  130.684.16  128.242.28   -6% -
3%
  Fuzzy2   41.790.40   41.141.11   -5% -
2%
  Fuzzy1  108.972.40  107.332.87   -6% -
3%
 AndHighHigh   16.190.48   15.970.31   -6% -
3%
  Phrase   12.900.32   12.740.36   -6% -
4%
  AndHighMed   64.181.81   63.461.83   -6% -
4%
SloppyPhrase8.370.298.290.11   -5% -
3%
SpanNear5.510.125.460.18   -6% -
4%
 TermGroup1M   36.190.60   35.890.74   -4% -
2%
TermBGroup1M   70.640.49   70.740.57   -1% -
1%
 Prefix3   61.073.58   61.251.35   -7% -
8%
Wildcard   40.842.20   41.000.98   -7% -
8%
Term  147.323.85  149.654.87   -4% -
7%
  TermBGroup1M1P   50.321.58   51.290.88   -2% -
7%
  IntNRQ9.961.40   10.180.56  -15% -   
25%
  OrHighHigh   10.310.74   10.560.57   -9% -   
16%
   OrHighMed   12.951.01   13.260.79  -10% -   
17%
{noformat}

Basically no real change ... good!

 Make CompositeReader.getSequentialSubReaders() and the corresponding 
 IndexReaderContext methods return unmodifiable ListR extends IndexReader
 ---

 Key: LUCENE-3866
 URL: https://issues.apache.org/jira/browse/LUCENE-3866
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3866-step1.patch, LUCENE-3866-step2.patch, 
 LUCENE-3866-step2.patch


 Since Lucene 2.9 we have the method getSequentialSubReader() returning 
 IndexReader[]. Based on hardly-to-debug errors in user's code, Robert and me 
 realized that returning an array from a public API is an anti-pattern. If the 
 array is intended to be modifiable (like byte[] in terms,...), it is fine to 
 use arrays in public APIs, but not, if the array must be protected from 
 modification. As IndexReaders are 100% unmodifiable in trunk code (no 
 deletions,...), the only possibility to corrumpt the reader is by modifying 
 the array returned by getSequentialSubReaders(). We should prevent this.
 The same theoretically applies to FieldCache, too - but the party that is 
 afraid of performance problems is too big to fight against that :(
 For getSequentialSubReaders there is no performance problem at all. The 
 binary search of reader-ids inside BaseCompositeReader can still be done with 
 the internal protected array, but public APIs should expose only a 
 unmodifiable List. The same applies to leaves() and children() in 
 IndexReaderContext. This change to list would also allow to make 
 CompositeReader and CompositeReaderContext IterableR extends IndexReader, 
 so some loops would look nice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr - ConcurrentHashMap - NullPointerException

2012-06-18 Thread Fuad Efendi

Some research, line 768 of ConcurrentHashMap:

 int hash = hash(key.hashCode());


I believe we have Unit Test with null as a key very strange for Solr cache
implementation



   /**

 * Returns the value to which the specified key is mapped,

 * or {@code null} if this map contains no mapping for the key.

 *

 * pMore formally, if this map contains a mapping from a key

 * {@code k} to a value {@code v} such that {@code key.equals(k)},

 * then this method returns {@code v}; otherwise it returns

 * {@code null}.  (There can be at most one such mapping.)

 *

 * @throws NullPointerException if the specified key is null

 */

public V get(Object key) {

int hash = hash(key.hashCode());

return segmentFor(hash).get(key, hash);

}







From:  Fuad Efendi f...@efendi.ca
Reply-To:  lucene-dev dev@lucene.apache.org
Date:  Monday, 18 June, 2012 11:45 AM
To:  lucene-dev dev@lucene.apache.org
Subject:  Solr - ConcurrentHashMap - NullPointerException


I am recalling big noise in JIRA; and Paul Nobble implemented first version
of this what can be wrong now, after few years? Solr 3.6.0


Jun 18, 2012 3:40:43 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
at org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:87)
at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:136)
at 
org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher
.java:632)
at 
org.apache.solr.search.SolrIndexSearcher.numDocs(SolrIndexSearcher.java:1743
)
at 
org.apache.solr.request.SimpleFacets.getFacetQueryCounts(SimpleFacets.java:2
34)
at 
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:194)
at 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java
:72)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHand
ler.java:186)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.
java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:3
65)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:
260)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
va:233)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
va:191)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127
)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
:109)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:680)



Fuad Efendi
416-993-2060
http://www.tokenizer.ca

[jira] [Commented] (LUCENE-3866) Make CompositeReader.getSequentialSubReaders() and the corresponding IndexReaderContext methods return unmodifiable ListR extends IndexReader


[ 
https://issues.apache.org/jira/browse/LUCENE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395975#comment-13395975
 ] 

Uwe Schindler commented on LUCENE-3866:
---

Thanks, I was not expecting a change - otherwise all my believes would have 
been devastated... :-)

 Make CompositeReader.getSequentialSubReaders() and the corresponding 
 IndexReaderContext methods return unmodifiable ListR extends IndexReader
 ---

 Key: LUCENE-3866
 URL: https://issues.apache.org/jira/browse/LUCENE-3866
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3866-step1.patch, LUCENE-3866-step2.patch, 
 LUCENE-3866-step2.patch


 Since Lucene 2.9 we have the method getSequentialSubReader() returning 
 IndexReader[]. Based on hardly-to-debug errors in user's code, Robert and me 
 realized that returning an array from a public API is an anti-pattern. If the 
 array is intended to be modifiable (like byte[] in terms,...), it is fine to 
 use arrays in public APIs, but not, if the array must be protected from 
 modification. As IndexReaders are 100% unmodifiable in trunk code (no 
 deletions,...), the only possibility to corrumpt the reader is by modifying 
 the array returned by getSequentialSubReaders(). We should prevent this.
 The same theoretically applies to FieldCache, too - but the party that is 
 afraid of performance problems is too big to fight against that :(
 For getSequentialSubReaders there is no performance problem at all. The 
 binary search of reader-ids inside BaseCompositeReader can still be done with 
 the internal protected array, but public APIs should expose only a 
 unmodifiable List. The same applies to leaves() and children() in 
 IndexReaderContext. This change to list would also allow to make 
 CompositeReader and CompositeReaderContext IterableR extends IndexReader, 
 so some loops would look nice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4069) Segment-level Bloom filters for a 2 x speed up on rare term searches


[ 
https://issues.apache.org/jira/browse/LUCENE-4069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395980#comment-13395980
 ] 

Michael McCandless commented on LUCENE-4069:


bq. Didn't get too far with running the Wikipedia perf tests due to missing 
data file (see 
http://code.google.com/a/apache-extras.org/p/luceneutil/issues/detail?id=7 )

Woops sorry: I posted a comment there (new, more recent wikipedia export).

 Segment-level Bloom filters for a 2 x speed up on rare term searches
 

 Key: LUCENE-4069
 URL: https://issues.apache.org/jira/browse/LUCENE-4069
 Project: Lucene - Java
  Issue Type: Improvement
  Components: core/index
Affects Versions: 3.6, 4.0
Reporter: Mark Harwood
Priority: Minor
 Fix For: 4.0, 3.6.1

 Attachments: BloomFilterPostingsBranch4x.patch, 
 MHBloomFilterOn3.6Branch.patch, PrimaryKeyPerfTest40.java


 An addition to each segment which stores a Bloom filter for selected fields 
 in order to give fast-fail to term searches, helping avoid wasted disk access.
 Best suited for low-frequency fields e.g. primary keys on big indexes with 
 many segments but also speeds up general searching in my tests.
 Overview slideshow here: 
 http://www.slideshare.net/MarkHarwood/lucene-bloomfilteredsegments
 Benchmarks based on Wikipedia content here: http://goo.gl/X7QqU
 Patch based on 3.6 codebase attached.
 There are no 3.6 API changes currently - to play just add a field with _blm 
 on the end of the name to invoke special indexing/querying capability. 
 Clearly a new Field or schema declaration(!) would need adding to APIs to 
 configure the service properly.
 Also, a patch for Lucene4.0 codebase introducing a new PostingsFormat

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4148) _TestUtil should be able to generate random longs

2012-06-18 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395992#comment-13395992
 ] 

Dawid Weiss commented on LUCENE-4148:
-

This looks good to me, although I'm wondering how much randomness do we really 
need -- those BigIntegers are worrying. I think this can be improved later if 
somebody wishes to dive into it.

Btw. an interesting random number generator -- 

 _TestUtil should be able to generate random longs
 -

 Key: LUCENE-4148
 URL: https://issues.apache.org/jira/browse/LUCENE-4148
 Project: Lucene - Java
  Issue Type: Improvement
  Components: general/test
Reporter: Adrien Grand
Priority: Trivial
 Attachments: LUCENE-4148.patch


 It would be helpful in TestPackedInts at least, in order to generate random 
 values (as a workaround, we currently generate a random int between 0 and 
 {{min(Integer.MAX_VALUE, PackedInts.maxValue(bitsPerValue)}}). Moreover, it 
 would help to fix {{nextInt}} for large ranges (calling {{nextInt(random, 
 -10, Integer.MAX_VALUE)}} or even {{nextInt(random, 0, Integer.MAX_VALUE)}} 
 currently fails because the range of values is {{ Integer.MAX_VALUE}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-4148) _TestUtil should be able to generate random longs

2012-06-18 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-4148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395992#comment-13395992
 ] 

Dawid Weiss edited comment on LUCENE-4148 at 6/18/12 4:10 PM:
--

This looks good to me, although I'm wondering how much randomness do we really 
need -- those BigIntegers are worrying. I think this can be improved later if 
somebody wishes to dive into it.

Btw. an interesting random number generator -- 
http://home.southernct.edu/~pasqualonia1/ca/report.html

  was (Author: dweiss):
This looks good to me, although I'm wondering how much randomness do we 
really need -- those BigIntegers are worrying. I think this can be improved 
later if somebody wishes to dive into it.

Btw. an interesting random number generator -- 
  
 _TestUtil should be able to generate random longs
 -

 Key: LUCENE-4148
 URL: https://issues.apache.org/jira/browse/LUCENE-4148
 Project: Lucene - Java
  Issue Type: Improvement
  Components: general/test
Reporter: Adrien Grand
Priority: Trivial
 Attachments: LUCENE-4148.patch


 It would be helpful in TestPackedInts at least, in order to generate random 
 values (as a workaround, we currently generate a random int between 0 and 
 {{min(Integer.MAX_VALUE, PackedInts.maxValue(bitsPerValue)}}). Moreover, it 
 would help to fix {{nextInt}} for large ranges (calling {{nextInt(random, 
 -10, Integer.MAX_VALUE)}} or even {{nextInt(random, 0, Integer.MAX_VALUE)}} 
 currently fails because the range of values is {{ Integer.MAX_VALUE}}.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3866) Make CompositeReader.getSequentialSubReaders() and the corresponding IndexReaderContext methods return unmodifiable ListR extends IndexReader


[ 
https://issues.apache.org/jira/browse/LUCENE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13395995#comment-13395995
 ] 

Michael McCandless commented on LUCENE-3866:


+1, patch looks good!  Thanks Uwe.

 Make CompositeReader.getSequentialSubReaders() and the corresponding 
 IndexReaderContext methods return unmodifiable ListR extends IndexReader
 ---

 Key: LUCENE-3866
 URL: https://issues.apache.org/jira/browse/LUCENE-3866
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3866-step1.patch, LUCENE-3866-step2.patch, 
 LUCENE-3866-step2.patch


 Since Lucene 2.9 we have the method getSequentialSubReader() returning 
 IndexReader[]. Based on hardly-to-debug errors in user's code, Robert and me 
 realized that returning an array from a public API is an anti-pattern. If the 
 array is intended to be modifiable (like byte[] in terms,...), it is fine to 
 use arrays in public APIs, but not, if the array must be protected from 
 modification. As IndexReaders are 100% unmodifiable in trunk code (no 
 deletions,...), the only possibility to corrumpt the reader is by modifying 
 the array returned by getSequentialSubReaders(). We should prevent this.
 The same theoretically applies to FieldCache, too - but the party that is 
 afraid of performance problems is too big to fight against that :(
 For getSequentialSubReaders there is no performance problem at all. The 
 binary search of reader-ids inside BaseCompositeReader can still be done with 
 the internal protected array, but public APIs should expose only a 
 unmodifiable List. The same applies to leaves() and children() in 
 IndexReaderContext. This change to list would also allow to make 
 CompositeReader and CompositeReaderContext IterableR extends IndexReader, 
 so some loops would look nice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4150) Change version properties in branch_4x to 4.0-ALPHA-SNAPSHOT


[ 
https://issues.apache.org/jira/browse/LUCENE-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396001#comment-13396001
 ] 

Hoss Man commented on LUCENE-4150:
--

bq. About jira, after we release i do think we should make the version in JIRA 
so people can report bugs against it. But I don't think we should do that now 
(or people will get confused about what version to terget things).

Just to be clear, i think this absolutely MUST be done, and would be total 
chaos otherwise.

If we are going to vote and have official Alpha and Beta releases that we 
want end users to actually try out and test thoroughly, then we are going to 
need a way to record when a bug was found/fixed in 4.0-Alpha vs the 4.0-Beta vs 
the 4.0-final.

Until we have a formally voted on Alpha it's not a big deal, but as soon as 
that moment happens, ever bug that currently says Fixed in: 4.0 should be 
bulked edited to say Fixed in: 4.0-Alpha, 4.0 -- so that 9 months from now 
when someone says I'm still running 4.0-Alpha because there was an API change 
to fix a bug that made it too hard for me to upgrade to 4.0-Beta there is a 
good clean record of what bug fixes they do/don't have.

bq. No. We are targeting 4.0, these are just snapshots of that!

Calling the alpha/beta snapshots is extremely missleading and should be 
avoided.  The only usage of the term snapshot in apache (that i know of) are 
for *unofficial* nightly/jenkins/svn builds.  Since the alpha/beta releases 
will in fact be _official_ (VOTEd) releases we should not ever imply/refer to 
them as snapshots since that would convey lower status on them (and reduce 
the likely hood of enn users testing them) then they will deserve.


 Change version properties in branch_4x to 4.0-ALPHA-SNAPSHOT 
 ---

 Key: LUCENE-4150
 URL: https://issues.apache.org/jira/browse/LUCENE-4150
 Project: Lucene - Java
  Issue Type: Task
  Components: general/build
Affects Versions: 4.0
Reporter: Steven Rowe
Priority: Minor

 The next release off branch_4x will be named 4.0-ALPHA, so the current 
 version string should be 4.0-ALPHA-SNAPSHOT.
 (Similarly, after 4.0-ALPHA is released, the version string should be changed 
 to 4.0-BETA-SNAPSHOT.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-4.x-Windows-Java7-64 - Build # 94 - Failure!

Build: 
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows-Java7-64/94/

No tests ran.

Build Log:
[...truncated 10392 lines...]

[...truncated 10392 lines...]

[...truncated 10392 lines...]

[...truncated 10392 lines...]

[...truncated 10357 lines...]
FATAL: hudson.remoting.RequestAbortedException: java.net.SocketException: 
Connection reset
hudson.remoting.RequestAbortedException: 
hudson.remoting.RequestAbortedException: java.net.SocketException: Connection 
reset
at hudson.remoting.Request.call(Request.java:149)
at hudson.remoting.Channel.call(Channel.java:646)
at 
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:158)
at $Proxy38.join(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:861)
at hudson.Launcher$ProcStarter.join(Launcher.java:345)
at hudson.tasks.Ant.perform(Ant.java:217)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:717)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:160)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:499)
at hudson.model.Run.execute(Run.java:1460)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:239)
Caused by: hudson.remoting.RequestAbortedException: java.net.SocketException: 
Connection reset
at hudson.remoting.Request.abort(Request.java:273)
at hudson.remoting.Channel.terminate(Channel.java:702)
at 
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at 
java.io.ObjectInputStream$PeekInputStream.peek(ObjectInputStream.java:2248)
at 
java.io.ObjectInputStream$BlockDataInputStream.peek(ObjectInputStream.java:2541)
at 
java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2551)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1296)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:350)
at hudson.remoting.Command.readFrom(Command.java:90)
at 
hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:59)
at 
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

[...truncated 10392 lines...]



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr - ConcurrentHashMap - NullPointerException

2012-06-18 Thread Fuad Efendi

Rare problem it happened when I provided NULL facet query via solrconfig:
   str name=facet.query/str

 


We probably should add null check:

  // only handle positive (non negative) queries

  DocSet getPositiveDocSet(Query q) throws IOException {

DocSet answer;

if (filterCache != null) {

  answer = filterCache.get(q);

  if (answer!=null) return answer;

}

answer = getDocSetNC(q,null);

if (filterCache != null) filterCache.put(

q,answer);

return answer;

  }





From:  Fuad Efendi f...@efendi.ca
Reply-To:  lucene-dev dev@lucene.apache.org
Date:  Monday, 18 June, 2012 11:52 AM
To:  lucene-dev dev@lucene.apache.org
Subject:  Re: Solr - ConcurrentHashMap - NullPointerException

Some research, line 768 of ConcurrentHashMap:

 int hash = hash(key.hashCode());


I believe we have Unit Test with null as a key very strange for Solr cache
implementation



   /**

 * Returns the value to which the specified key is mapped,

 * or {@code null} if this map contains no mapping for the key.

 *

 * pMore formally, if this map contains a mapping from a key

 * {@code k} to a value {@code v} such that {@code key.equals(k)},

 * then this method returns {@code v}; otherwise it returns

 * {@code null}.  (There can be at most one such mapping.)

 *

 * @throws NullPointerException if the specified key is null

 */

public V get(Object key) {

int hash = hash(key.hashCode());

return segmentFor(hash).get(key, hash);

}







From:  Fuad Efendi f...@efendi.ca
Reply-To:  lucene-dev dev@lucene.apache.org
Date:  Monday, 18 June, 2012 11:45 AM
To:  lucene-dev dev@lucene.apache.org
Subject:  Solr - ConcurrentHashMap - NullPointerException


I am recalling big noise in JIRA; and Paul Nobble implemented first version
of this what can be wrong now, after few years? Solr 3.6.0


Jun 18, 2012 3:40:43 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:768)
at org.apache.solr.util.ConcurrentLRUCache.get(ConcurrentLRUCache.java:87)
at org.apache.solr.search.FastLRUCache.get(FastLRUCache.java:136)
at 
org.apache.solr.search.SolrIndexSearcher.getPositiveDocSet(SolrIndexSearcher
.java:632)
at 
org.apache.solr.search.SolrIndexSearcher.numDocs(SolrIndexSearcher.java:1743
)
at 
org.apache.solr.request.SimpleFacets.getFacetQueryCounts(SimpleFacets.java:2
34)
at 
org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:194)
at 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java
:72)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHand
ler.java:186)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.
java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:3
65)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:
260)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
FilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
ain.java:206)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
va:233)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
va:191)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127
)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102
)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
:109)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http
11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:680)



Fuad Efendi
416-993-2060
http://www.tokenizer.ca

[JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 174 - Failure!

Build: 
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-64/174/

All tests passed

Build Log:
[...truncated 5134 lines...]
   [junit4] ERROR: Forked JVM execution exception: 0, execution line: 
/mnt/ssd/jenkins/tools/java/jdk1.6.0_32/jre/bin/java 
-XX:+HeapDumpOnOutOfMemoryError 
-XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/heapdumps/
 -Dtests.prefix=tests -Dtests.seed=2A2493D5321058C4 -Xmx512M -Dtests.iters= 
-Dtests.verbose=false -Dtests.infostream=false 
-Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build
 -Dtests.codec=random -Dtests.postingsformat=random -Dtests.locale=random 
-Dtests.timezone=random -Dtests.directory=random 
-Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.0 
-Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/dev/null 
-Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=false 
-Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. 
-Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 
-Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/grouping/lucene-grouping-4.0-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/test-framework/lib/junit-4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/test-framework/lib/randomizedtesting-runner-1.5.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/join/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/join/classes/test:/var/lib/jenkins/tools/ant/supported18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-netrexx.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-javamail.jar:/mnt/ssd/jenkins/tools/java/jdk1.6.0_32/lib/tools.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/test-framework/lib/junit4-ant-1.5.0.jar
 -ea:org.apache.lucene... -ea:org.apache.solr... 
com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile 
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/join/test/junit4-J0-165541892e65c073d.events
 
@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build/join/test/junit4-J0-165541892e65c073d.suites

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/build.xml:29:
 The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/build.xml:456:
 The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/common-build.xml:1417:
 The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/module-build.xml:62:
 The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/common-build.xml:1019:
 The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/common-build.xml:745:
 At least one slave process threw an unexpected exception, first: Forked 
process exited with an error code: 143

Total time: 18 minutes 16 seconds
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure

RE: [JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 174 - Failure!

2012-06-18 Thread Uwe Schindler

This was another test. Steven Rowe finally fixed the bug in the 
Extended-eMail-Publisher of Jenkins, it was caused not by our regex, it was 
caused by the parsing of the user input that contained our huge regexp (regexp 
used to match regexp - LOL).

I think we will submit bug report and let the Apache people also update.

Thanks for interruption, we are just fixing bugs in all components around 
Lucene. This time the Continuous Integration Server!

Thanks @ Steven,
Uwe

-
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de


 -Original Message-
 From: Jenkins Server with Girls [mailto:jenk...@sd-datasolutions.de]
 Sent: Monday, June 18, 2012 7:00 PM
 To: dev@lucene.apache.org
 Subject: [JENKINS] Lucene-Solr-4.x-Linux-Java6-64 - Build # 174 - Failure!
 
 Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-
 64/174/
 
 All tests passed
 
 Build Log:
 [...truncated 5134 lines...]
[junit4] ERROR: Forked JVM execution exception: 0, execution line:
 /mnt/ssd/jenkins/tools/java/jdk1.6.0_32/jre/bin/java -
 XX:+HeapDumpOnOutOfMemoryError -
 XX:HeapDumpPath=/var/lib/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/heapdumps/ -Dtests.prefix=tests -Dtests.seed=2A2493D5321058C4 -
 Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -
 Dtests.lockdir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build -Dtests.codec=random -
 Dtests.postingsformat=random -Dtests.locale=random -
 Dtests.timezone=random -Dtests.directory=random -
 Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.0 -
 Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/dev/null -
 Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=false -
 Dtests.asserts.gracious=false -Dtests.multiplier=3 -DtempDir=. -
 Dlucene.version=4.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1
 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -classpath
 /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/test-
 framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-
 Java6-64/checkout/lucene/build/grouping/lucene-grouping-4.0-
 SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucen
 e-Solr-4.x-Linux-Java6-64/checkout/lucene/test-framework/lib/junit-
 4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/test-framework/lib/randomizedtesting-runner-
 1.5.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/join/classes/java:/mnt/ssd/jenkins/workspace/Lucen
 e-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/join/classes/test:/var/lib/jenkins/tools/ant/supported
 18/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-
 2.2.0.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 jsch.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 jai.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 testutil.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-
 net.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 log4j.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 jmf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 antlr.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant.jar:/var/lib/jenkins/tool
 s/ant/supported18/lib/ant-
 junit.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 junit4.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-commons-
 logging.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 swing.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 oro.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 netrexx.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 resolver.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 xalan2.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 regexp.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 jdepend.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 bcel.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-apache-
 bsf.jar:/var/lib/jenkins/tools/ant/supported18/lib/ant-
 javamail.jar:/mnt/ssd/jenkins/tools/java/jdk1.6.0_32/lib/tools.jar:/mnt/ssd/jenk
 ins/workspace/Lucene-Solr-4.x-Linux-Java6-64/checkout/lucene/test-
 framework/lib/junit4-ant-1.5.0.jar -ea:org.apache.lucene... -
 ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -
 eventsfile /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/join/test/junit4-J0-165541892e65c073d.events
 @/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build/join/test/junit4-J0-165541892e65c073d.suites
 
 BUILD FAILED
 /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/build.xml:29: The following error occurred while executing this
 line:
 /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux-Java6-
 64/checkout/lucene/build.xml:456: The

[jira] [Commented] (LUCENE-3729) Allow using FST to hold terms data in DocValues.BYTES_*_SORTED


[ 
https://issues.apache.org/jira/browse/LUCENE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396032#comment-13396032
 ] 

Michael McCandless commented on LUCENE-3729:


Very cool!

Why did you need a custom subclass of FSTEnum (instead of
BytesRefFSTEnum)?  Is it for getOutput?  Couldn't you
getCurrent().output instead?

We could probably make a static Util.getCeil (like Util.get, but
finds ceiling term if there is no exact match) ... would likely be
faster than re-using an FSTEnum ... but sorta hairy to write.

I'm curious how DV sort perf compares ... hopefully it's little loss
(going on my previous tests here...).

Awesome to fix seekCeil to return enum result ... maybe factor out the
same enum from TermsEnum so we can share it?  And you can remove the
two TODOs from FSTEnum.java...

Can ToFSTBytesRefConsumer be moved to FSTSortedBytesImpl.java?


 Allow using FST to hold terms data in DocValues.BYTES_*_SORTED
 --

 Key: LUCENE-3729
 URL: https://issues.apache.org/jira/browse/LUCENE-3729
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless
  Labels: gsoc2012, lucene-gsoc-11
 Attachments: LUCENE-3729.patch, LUCENE-3729.patch, LUCENE-3729.patch, 
 LUCENE-3729.patch, LUCENE-3729.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Linux-Java7-64 - Build # 323 - Failure!

Build: 
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux-Java7-64/323/

1 tests failed.
REGRESSION:  
org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at 
__randomizedtesting.SeedInfo.seed([1A47DEC61940914D:8264B90547362D05]:0)
at org.apache.lucene.util.fst.FST.init(FST.java:331)
at 
org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsReader.init(MemoryPostingsFormat.java:811)
at 
org.apache.lucene.codecs.memory.MemoryPostingsFormat.fieldsProducer(MemoryPostingsFormat.java:858)
at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.init(PerFieldPostingsFormat.java:186)
at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:250)
at 
org.apache.lucene.index.SegmentCoreReaders.init(SegmentCoreReaders.java:107)
at org.apache.lucene.index.SegmentReader.init(SegmentReader.java:55)
at 
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:639)
at 
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:52)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:64)
at 
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:602)
at org.apache.lucene.util.IOUtils.close(IOUtils.java:143)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:472)
at 
org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:378)
at 
org.apache.lucene.analysis.ja.TestJapaneseTokenizer.testRandomHugeStrings(TestJapaneseTokenizer.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)




Build Log:
[...truncated 4666 lines...]
   [junit4] Suite: org.apache.lucene.analysis.ja.TestJapaneseTokenizer
   [junit4] ERROR   28.6s J0 | TestJapaneseTokenizer.testRandomHugeStrings
   [junit4] Throwable #1: java.lang.OutOfMemoryError: Java heap space
   [junit4]at 
__randomizedtesting.SeedInfo.seed([1A47DEC61940914D:8264B90547362D05]:0)
   [junit4]at org.apache.lucene.util.fst.FST.init(FST.java:331)
   [junit4]at 
org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsReader.init(MemoryPostingsFormat.java:811)
   [junit4]at 
org.apache.lucene.codecs.memory.MemoryPostingsFormat.fieldsProducer(MemoryPostingsFormat.java:858)
   [junit4]at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.init(PerFieldPostingsFormat.java:186)
   [junit4]at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:250)
   [junit4]at 
org.apache.lucene.index.SegmentCoreReaders.init(SegmentCoreReaders.java:107)
   [junit4]at 
org.apache.lucene.index.SegmentReader.init(SegmentReader.java:55)
   [junit4]at 
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:62)
   [junit4]at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:639)
   [junit4]at

[jira] [Commented] (LUCENE-4150) Change version properties in branch_4x to 4.0-ALPHA-SNAPSHOT

2012-06-18 Thread Steven Rowe (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396116#comment-13396116
 ] 

Steven Rowe commented on LUCENE-4150:
-

I agree with Hoss.

I think 4.0-ALPHA and 4.0-BETA should be full-fledged releases in every way.

 Change version properties in branch_4x to 4.0-ALPHA-SNAPSHOT 
 ---

 Key: LUCENE-4150
 URL: https://issues.apache.org/jira/browse/LUCENE-4150
 Project: Lucene - Java
  Issue Type: Task
  Components: general/build
Affects Versions: 4.0
Reporter: Steven Rowe
Priority: Minor

 The next release off branch_4x will be named 4.0-ALPHA, so the current 
 version string should be 4.0-ALPHA-SNAPSHOT.
 (Similarly, after 4.0-ALPHA is released, the version string should be changed 
 to 4.0-BETA-SNAPSHOT.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4150) Change version properties in branch_4x to 4.0-ALPHA-SNAPSHOT


[ 
https://issues.apache.org/jira/browse/LUCENE-4150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396129#comment-13396129
 ] 

Robert Muir commented on LUCENE-4150:
-

Everyone has the right to his own opinion: but for these alpha/betas to me they 
are snapshots with additional guarantees: and its better to get them out sooner 
than later so we can iteratively
incorporate feedback.

If some people want these to be typical lucene solr releases each that take 
months to finish, good for them, but I strongly disagree (and will not be 
convinced otherwise!).

Fortunately its ok for us to disagree here, I only need to convince 2 other PMC 
members for us to get these things out there: its been way too long for this 
release.

My advice: if you want these alpha/beta releases to be more solid, get your 
changes in now because next week I will spin RCs every 72 hours until we get 
something out. 

I imagine with enough hassling/bribing it won't be long until 2 other people 
decide its good to just get the alpha release out there and get people testing 
it instead of dragging things out.


 Change version properties in branch_4x to 4.0-ALPHA-SNAPSHOT 
 ---

 Key: LUCENE-4150
 URL: https://issues.apache.org/jira/browse/LUCENE-4150
 Project: Lucene - Java
  Issue Type: Task
  Components: general/build
Affects Versions: 4.0
Reporter: Steven Rowe
Priority: Minor

 The next release off branch_4x will be named 4.0-ALPHA, so the current 
 version string should be 4.0-ALPHA-SNAPSHOT.
 (Similarly, after 4.0-ALPHA is released, the version string should be changed 
 to 4.0-BETA-SNAPSHOT.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-3557) Avoid NPE for distributed request when shards.tolerant=true

2012-06-18 Thread Ryan McKinley (JIRA)

Ryan McKinley created SOLR-3557:
---

 Summary: Avoid NPE for distributed request when 
shards.tolerant=true
 Key: SOLR-3557
 URL: https://issues.apache.org/jira/browse/SOLR-3557
 Project: Solr
  Issue Type: Improvement
  Components: search
Reporter: Ryan McKinley
Assignee: Ryan McKinley
Priority: Minor
 Fix For: 4.0


If a shard fails to return and shards.tolerant=true, the faceting module will 
get a null pointer.  We should avoid that NPE

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3557) Avoid NPE for distributed request when shards.tolerant=true

2012-06-18 Thread Ryan McKinley (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McKinley updated SOLR-3557:


Attachment: SOLR-3557-tolerant-faceting.patch

 Avoid NPE for distributed request when shards.tolerant=true
 ---

 Key: SOLR-3557
 URL: https://issues.apache.org/jira/browse/SOLR-3557
 Project: Solr
  Issue Type: Improvement
  Components: search
Reporter: Ryan McKinley
Assignee: Ryan McKinley
Priority: Minor
 Fix For: 4.0

 Attachments: SOLR-3557-tolerant-faceting.patch


 If a shard fails to return and shards.tolerant=true, the faceting module will 
 get a null pointer.  We should avoid that NPE

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-3522) literal function can not be parsed


 [ 
https://issues.apache.org/jira/browse/SOLR-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man resolved SOLR-3522.


   Resolution: Fixed
Fix Version/s: (was: 3.6.1)
   5.0

Committed revision 1351460. - trunk
Committed revision 1351462. - 4x

deferring the more complicated (and broken) query equalities test to SOLR-3548

 literal function can not be parsed
 

 Key: SOLR-3522
 URL: https://issues.apache.org/jira/browse/SOLR-3522
 Project: Solr
  Issue Type: Bug
Reporter: Hoss Man
Assignee: Hoss Man
 Fix For: 4.0, 5.0

 Attachments: SOLR-3522.patch


 attempting to use the literal function in the fl param causes a parse 
 error...
 Example queries with functions that works fine...
 {noformat}
 http://localhost:8983/solr/collection1/select?q=*:*fl=foo:sum%284,5%29
 http://localhost:8983/solr/collection1/select?fl=scoreq={!func}strdist%28%22foo%22,%22fo%22,edit%29
 {noformat}
 Examples using literal function that fails...
 {noformat}
 http://localhost:8983/solr/collection1/select?q=*:*fl=foo:literal%28%22foo%22%29
 http://localhost:8983/solr/collection1/select?fl=scoreq={!func}strdist%28%22foo%22,literal%28%22fo%22%29,edit%29
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4154) migrate luceneutil from apache extras into lucene/benchmark

Michael McCandless created LUCENE-4154:
--

 Summary: migrate luceneutil from apache extras into 
lucene/benchmark
 Key: LUCENE-4154
 URL: https://issues.apache.org/jira/browse/LUCENE-4154
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Michael McCandless
Assignee: Michael McCandless


I think it'd make sense to move all the performance testing scripts from 
luceneutil at http://code.google.com/a/apache-extras.org/p/luceneutil under 
lucene/benchmark ...

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-4132) IndexWriterConfig live settings

[
https://issues.apache.org/jira/browse/LUCENE-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396233#comment-13396233
]

Uwe Schindler commented on LUCENE-4132:
---

bq. The 4.x commit also breaks the builds, looks like duplicate merges to same
file.

This was solved by cleaning workspace on Apache's Jenkins. Must have been a SVN
problem.

IndexWriterConfig live settings
---

Key: LUCENE-4132
URL: https://issues.apache.org/jira/browse/LUCENE-4132
Project: Lucene - Java
Issue Type: Improvement
Reporter: Shai Erera
Assignee: Shai Erera
Priority: Minor
Fix For: 4.0, 5.0

Attachments: LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch,
LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch, LUCENE-4132.patch

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3866) Make CompositeReader.getSequentialSubReaders() and the corresponding IndexReaderContext methods return unmodifiable ListR extends IndexReader


[ 
https://issues.apache.org/jira/browse/LUCENE-3866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396237#comment-13396237
 ] 

Uwe Schindler commented on LUCENE-3866:
---

I checked CHANGES.txt and MIGRATE.txt, there are no explicit mentions of the 
datatypes. No need to change anything. But MIGRATE.txt is missing the 
documentation about how to get atomic leaves from a reader. We should do this 
in the followup issue, when we simplify some names.

I will commit this patch tomorrow morning to trunk and 4.x.

 Make CompositeReader.getSequentialSubReaders() and the corresponding 
 IndexReaderContext methods return unmodifiable ListR extends IndexReader
 ---

 Key: LUCENE-3866
 URL: https://issues.apache.org/jira/browse/LUCENE-3866
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0

 Attachments: LUCENE-3866-step1.patch, LUCENE-3866-step2.patch, 
 LUCENE-3866-step2.patch


 Since Lucene 2.9 we have the method getSequentialSubReader() returning 
 IndexReader[]. Based on hardly-to-debug errors in user's code, Robert and me 
 realized that returning an array from a public API is an anti-pattern. If the 
 array is intended to be modifiable (like byte[] in terms,...), it is fine to 
 use arrays in public APIs, but not, if the array must be protected from 
 modification. As IndexReaders are 100% unmodifiable in trunk code (no 
 deletions,...), the only possibility to corrumpt the reader is by modifying 
 the array returned by getSequentialSubReaders(). We should prevent this.
 The same theoretically applies to FieldCache, too - but the party that is 
 afraid of performance problems is too big to fight against that :(
 For getSequentialSubReaders there is no performance problem at all. The 
 binary search of reader-ids inside BaseCompositeReader can still be done with 
 the internal protected array, but public APIs should expose only a 
 unmodifiable List. The same applies to leaves() and children() in 
 IndexReaderContext. This change to list would also allow to make 
 CompositeReader and CompositeReaderContext IterableR extends IndexReader, 
 so some loops would look nice.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3367) Show Logging Events in Admin UI

2012-06-18 Thread Stefan Matheis (steffkes) (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396244#comment-13396244
 ] 

Stefan Matheis (steffkes) commented on SOLR-3367:
-

Ryan, would you mind to have a look at my last comment?

 Show Logging Events in Admin UI
 ---

 Key: SOLR-3367
 URL: https://issues.apache.org/jira/browse/SOLR-3367
 Project: Solr
  Issue Type: New Feature
  Components: web gui
Reporter: Ryan McKinley
Assignee: Stefan Matheis (steffkes)
 Fix For: 4.0

 Attachments: SOLR-3367.patch, SOLR-3367.patch, SOLR-3367.patch, 
 SOLR-3367.patch, SOLR-3367.png


 We can show logging events in the Admin UI.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3529) clarify distnction between index query tables on analysis debug pages

2012-06-18 Thread Stefan Matheis (steffkes) (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-3529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Matheis (steffkes) updated SOLR-3529:


Component/s: web gui

 clarify distnction between index  query tables on analysis debug pages
 ---

 Key: SOLR-3529
 URL: https://issues.apache.org/jira/browse/SOLR-3529
 Project: Solr
  Issue Type: Improvement
  Components: web gui
Reporter: Hoss Man
Assignee: Stefan Matheis (steffkes)
 Fix For: 4.0

 Attachments: long-side-by-side-below-fold.png, long-side-by-side.png, 
 med-just-index.png, med-query-only.png, med-side-by-side-below-fold.png, 
 med-side-by-side.png, short-side-by-side.png


 Working on the tutorial, i noticed that the analysis debug page is a little 
 hard to read when specifying both index and query values
  * if the inputs are short, it's not entirely obvious that you are looking at 
 two tables (especially compared to how the page looks using only the index 
 textbox)
  * if the inputs are longer, the query table shifts down below the fold 
 where users may not even be aware of it.
 Screenshots and ideas for improvement to follow

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-4155) Move hide ReaderSlice and BitSlice classes (and possibly others) to oal.index package; move ReaderUtil to oal.index

Uwe Schindler created LUCENE-4155:
-

 Summary: Move  hide ReaderSlice and BitSlice classes (and 
possibly others) to oal.index package; move ReaderUtil to oal.index
 Key: LUCENE-4155
 URL: https://issues.apache.org/jira/browse/LUCENE-4155
 Project: Lucene - Java
  Issue Type: Bug
  Components: core/index
Reporter: Uwe Schindler
Assignee: Uwe Schindler
 Fix For: 4.0, 5.0


Those are used solely by the index package and are very internal (just helper 
classes), so they should be hidden from the user. This can be done by adding 
the pkg-private to index package.

ReaderUtil was cleaned up in LUCENE-3866, should stay public, but is in wrong 
package since Lucene 2.9. We should move it to oal.index package, too. Its name 
suggests that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3535) Add block support for XMLLoader

2012-06-18 Thread Mikhail Khludnev (JIRA)

[
https://issues.apache.org/jira/browse/SOLR-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mikhail Khludnev updated SOLR-3535:
---

Attachment: SOLR-3535.patch

new attach highlights:
* UpdReqProc.processAdd() become protected
* FlattenerUpdateProcessorFactory has been introduced.
* it transforms the given SolrInputDocument with nested subdocs into block of
SIDs
* i.e. it transforms AddUpdCmd to AddBlockUpdCmd
* nested subdocs are placed as collection of SID fields. Hoss, excuse me. It's
not really my point - we can later switch to getChildrenDocs(). it just seems
to me easier for now. Let's polish it later.
* you can see that Flattener is placed between Distributed and Log/Run. i.e. I
addressed your guys point - make it compatible with distributed update magic.
* forgot to cover tree levels of nesting, my falut. it's trivial transitive
closure via recursion. I'll switch to iteration further.
* XMLLoader supports field
name=skillsdoc../docdoc../docdoc../docdoc../doc/field it's
a nested docs.
* there are tests, puzzling a little. test harness is harmed a little.

Please have a look, and raise your concerns!
btw it might be easier https://github.com/m-khl/solr-patches/commits/blockupdate

Add block support for XMLLoader
---

Key: SOLR-3535
URL: https://issues.apache.org/jira/browse/SOLR-3535
Project: Solr
Issue Type: Sub-task
Components: update
Affects Versions: 4.1, 5.0
Reporter: Mikhail Khludnev
Priority: Minor
Attachments: SOLR-3535.patch, SOLR-3535.patch

I'd like to add the following update xml message:
add-block
doc/doc
doc/doc
/add-block
out of scope for now:
* other update formats
* update log support (NRT), should not be a big deal
* overwrite feature support for block updates - it's more complicated, I'll
tell you why
Alt
* wdyt about adding attribute to the current tag {pre}add block=true{pre}
* or we can establish RunBlockUpdateProcessor which treat every add
/add as a block.
*Test is included!!*
How you'd suggest to improve the patch?

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3367) Show Logging Events in Admin UI

2012-06-18 Thread Ryan McKinley (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396271#comment-13396271
 ] 

Ryan McKinley commented on SOLR-3367:
-

ya, i see the difference.  Obviously it is best if we can change the messages 
so they are reasonably short, but this is not realistic for all error messages. 
 (yes, we should open an issue for this specific error, but it is not an easy 
fix)

Whatever format we pick should be able to support short or long messages (i 
think)

Perhaps overflow:hidden?  or an option to toggle overflow:hidden?





 Show Logging Events in Admin UI
 ---

 Key: SOLR-3367
 URL: https://issues.apache.org/jira/browse/SOLR-3367
 Project: Solr
  Issue Type: New Feature
  Components: web gui
Reporter: Ryan McKinley
Assignee: Stefan Matheis (steffkes)
 Fix For: 4.0

 Attachments: SOLR-3367.patch, SOLR-3367.patch, SOLR-3367.patch, 
 SOLR-3367.patch, SOLR-3367.png


 We can show logging events in the Admin UI.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3553) Make solr Plugins (FieldType, Analyser etc) be inheritable


 [ 
https://issues.apache.org/jira/browse/SOLR-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer updated SOLR-3553:
--

Attachment: (was: SOLR-3553-Inheritable-schema-elements.patch)

 Make solr Plugins (FieldType, Analyser etc) be inheritable
 --

 Key: SOLR-3553
 URL: https://issues.apache.org/jira/browse/SOLR-3553
 Project: Solr
  Issue Type: New Feature
Affects Versions: 4.0
Reporter: Greg Bowyer
 Attachments: SOLR-3553-Inheritable-schema-elements.patch


 During defining an experimental schema for trialling all the combinations of 
 scoring models for lucene 4 it struck me how much duplication is involved in 
 the schema xml
 Using the current schema I am working with as an example, 
 {code:xml}
 fieldType name=text class=solr.TextField 
 positionIncrementGap=100 
 autoGeneratePhraseQueries=true
 analyzer type=index
 tokenizer class=solr.WhitespaceTokenizerFactory/
 
 filter class=solr.TrimFilterFactory/
 filter class=solr.LowerCaseFilterFactory/
 filter class=solr.WordDelimiterFilterFactory
generateWordParts=1 generateNumberParts=1
catenateWords=0 catenateNumbers=1  catenateAll=0 
splitOnCaseChange=1 splitOnNumerics=0
stemEnglishPossesive=0 preserveOriginal=0 /
 filter class=solr.ASCIIFoldingFilterFactory/
 filter class=solr.StopFilterFactory 
ignoreCase=true words=stopwords.txt
enablePositionIncrements=true /
 
 ... And so on
 {code}
 What I would like to be able to say is
 {code:xml}
 fieldType name=text_bm25 parent=text
  similarity class=solr.BM25SimilarityFactory
 float name=k11.2/float
 float name=b0.76/float
 /similarity
 /fieldType
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3553) Make solr Plugins (FieldType, Analyser etc) be inheritable


 [ 
https://issues.apache.org/jira/browse/SOLR-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer updated SOLR-3553:
--

Attachment: SOLR-3553-Inheritable-schema-elements.patch

 Make solr Plugins (FieldType, Analyser etc) be inheritable
 --

 Key: SOLR-3553
 URL: https://issues.apache.org/jira/browse/SOLR-3553
 Project: Solr
  Issue Type: New Feature
Affects Versions: 4.0
Reporter: Greg Bowyer
 Attachments: SOLR-3553-Inheritable-schema-elements.patch


 During defining an experimental schema for trialling all the combinations of 
 scoring models for lucene 4 it struck me how much duplication is involved in 
 the schema xml
 Using the current schema I am working with as an example, 
 {code:xml}
 fieldType name=text class=solr.TextField 
 positionIncrementGap=100 
 autoGeneratePhraseQueries=true
 analyzer type=index
 tokenizer class=solr.WhitespaceTokenizerFactory/
 
 filter class=solr.TrimFilterFactory/
 filter class=solr.LowerCaseFilterFactory/
 filter class=solr.WordDelimiterFilterFactory
generateWordParts=1 generateNumberParts=1
catenateWords=0 catenateNumbers=1  catenateAll=0 
splitOnCaseChange=1 splitOnNumerics=0
stemEnglishPossesive=0 preserveOriginal=0 /
 filter class=solr.ASCIIFoldingFilterFactory/
 filter class=solr.StopFilterFactory 
ignoreCase=true words=stopwords.txt
enablePositionIncrements=true /
 
 ... And so on
 {code}
 What I would like to be able to say is
 {code:xml}
 fieldType name=text_bm25 parent=text
  similarity class=solr.BM25SimilarityFactory
 float name=k11.2/float
 float name=b0.76/float
 /similarity
 /fieldType
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3553) Make solr Plugins (FieldType, Analyser etc) be inheritable


 [ 
https://issues.apache.org/jira/browse/SOLR-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer updated SOLR-3553:
--

Attachment: (was: SOLR-3553-Inheritable-schema-elements.patch)

 Make solr Plugins (FieldType, Analyser etc) be inheritable
 --

 Key: SOLR-3553
 URL: https://issues.apache.org/jira/browse/SOLR-3553
 Project: Solr
  Issue Type: New Feature
Affects Versions: 4.0
Reporter: Greg Bowyer
 Attachments: SOLR-3553-Inheritable-schema-elements.patch


 During defining an experimental schema for trialling all the combinations of 
 scoring models for lucene 4 it struck me how much duplication is involved in 
 the schema xml
 Using the current schema I am working with as an example, 
 {code:xml}
 fieldType name=text class=solr.TextField 
 positionIncrementGap=100 
 autoGeneratePhraseQueries=true
 analyzer type=index
 tokenizer class=solr.WhitespaceTokenizerFactory/
 
 filter class=solr.TrimFilterFactory/
 filter class=solr.LowerCaseFilterFactory/
 filter class=solr.WordDelimiterFilterFactory
generateWordParts=1 generateNumberParts=1
catenateWords=0 catenateNumbers=1  catenateAll=0 
splitOnCaseChange=1 splitOnNumerics=0
stemEnglishPossesive=0 preserveOriginal=0 /
 filter class=solr.ASCIIFoldingFilterFactory/
 filter class=solr.StopFilterFactory 
ignoreCase=true words=stopwords.txt
enablePositionIncrements=true /
 
 ... And so on
 {code}
 What I would like to be able to say is
 {code:xml}
 fieldType name=text_bm25 parent=text
  similarity class=solr.BM25SimilarityFactory
 float name=k11.2/float
 float name=b0.76/float
 /similarity
 /fieldType
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3553) Make solr Plugins (FieldType, Analyser etc) be inheritable


 [ 
https://issues.apache.org/jira/browse/SOLR-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Greg Bowyer updated SOLR-3553:
--

Attachment: (was: SOLR-3553-Inheritable-schema-elements.patch)

 Make solr Plugins (FieldType, Analyser etc) be inheritable
 --

 Key: SOLR-3553
 URL: https://issues.apache.org/jira/browse/SOLR-3553
 Project: Solr
  Issue Type: New Feature
Affects Versions: 4.0
Reporter: Greg Bowyer
 Attachments: SOLR-3553-Inheritable-schema-elements.patch


 During defining an experimental schema for trialling all the combinations of 
 scoring models for lucene 4 it struck me how much duplication is involved in 
 the schema xml
 Using the current schema I am working with as an example, 
 {code:xml}
 fieldType name=text class=solr.TextField 
 positionIncrementGap=100 
 autoGeneratePhraseQueries=true
 analyzer type=index
 tokenizer class=solr.WhitespaceTokenizerFactory/
 
 filter class=solr.TrimFilterFactory/
 filter class=solr.LowerCaseFilterFactory/
 filter class=solr.WordDelimiterFilterFactory
generateWordParts=1 generateNumberParts=1
catenateWords=0 catenateNumbers=1  catenateAll=0 
splitOnCaseChange=1 splitOnNumerics=0
stemEnglishPossesive=0 preserveOriginal=0 /
 filter class=solr.ASCIIFoldingFilterFactory/
 filter class=solr.StopFilterFactory 
ignoreCase=true words=stopwords.txt
enablePositionIncrements=true /
 
 ... And so on
 {code}
 What I would like to be able to say is
 {code:xml}
 fieldType name=text_bm25 parent=text
  similarity class=solr.BM25SimilarityFactory
 float name=k11.2/float
 float name=b0.76/float
 /similarity
 /fieldType
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)

[
https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396325#comment-13396325
]

Michael McCandless commented on LUCENE-3892:

On the For patch ... we shouldn't encode/decode numInts right? It's
always 128?

Up above, in ForFactory, when we readInt() to get numBytes ... it
seems like we could stuff the header numBits into that same int and
save checking that in FORUtil.decompress

I think there are a few possible ideas to explore to get faster
PFor/For performance:

* Get more direct access to the file as an int[]; eg MMapDir could
expose an IntBuffer from its ByteBuffer (saving the initial copy
into byte[] that we now do). Or maybe we add
IndexInput.readInts(int[]) and dir impl can optimize how that's
done (MMapDir could use Unsafe.copyBytes... except for little
endian architectures ... we'd probably have to have separate
specialized decoder rather than letting Int/ByteBuffer do the byte
swapping). This would require the whole file stays aligned w/ int
(eg the header must be 0 mod 4).

* Copy/share how oal.packed works, i.e. being able to waste a bit to
have faster decode (eg storing the 7 bit case as byte[], wasting 1
bit for each value).

* Skipping: can we partially decode a block? EG if we are skipping
and we know we only want values after the 80th one, then we
shouldn't decode those first 80...

* Since doc/freq are aligned, when we store pointers to a given
spot, eg in the terms dict or in skip data, we should only store
the offset once (today we store it twice).

* Alternatively, maybe we should only save skip data on doc/freq
block boundaries (prox would still need skip-within-block).

* Maybe we should store doc frq blocks interleaved in a single
file (since they are aligned) and then skip would skip to the
start of a doc/frq block pair.

Other ideas...?

Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta,
Simple9/16/64, etc.)
-

Key: LUCENE-3892
URL: https://issues.apache.org/jira/browse/LUCENE-3892
Project: Lucene - Java
Issue Type: Improvement
Reporter: Michael McCandless
Labels: gsoc2012, lucene-gsoc-12
Fix For: 4.1

Attachments: LUCENE-3892_for.patch, LUCENE-3892_pfor.patch,
LUCENE-3892_pfor.patch, LUCENE-3892_pfor.patch, LUCENE-3892_settings.patch,
LUCENE-3892_settings.patch

On the flex branch we explored a number of possible intblock
encodings, but for whatever reason never brought them to completion.
There are still a number of issues opened with patches in different
states.
Initial results (based on prototype) were excellent (see
http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html
).
I think this would make a good GSoC project.

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3548) some queries produced by built in QParsers are not .equals() to themselves


 [ 
https://issues.apache.org/jira/browse/SOLR-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-3548:
---

Attachment: SOLR-3548.patch

it took a lot longer then i though, but here is a patch that adds QueryUtils 
checking against (at least one of) the Query objects produced by every default 
QParser and ValueSourceParser.  It includes a future proofing testCoverage 
that sets a bit informing an AfterClass method to assert that all of the 
default parsers were tested so we don't risk this probably again the next time 
someone adds a new parsers.

Currently 4 methods are failing, indicating the following problems...

* strdist func - identical query strings don't produce equals() queries
* join qparser - clone w/diff boost still has equals hashCode
* bbox qparser - clone w/diff boost still has equals hashCode
* geofilt qparser - clone w/diff boost still has equals hashCode

the hashCode equality isn't the end of the world, but it suggests a really poor 
hashCode impl (that evidently doesn't call super since Query.hashCode already 
handles the boost)



 some queries produced by built in QParsers are not .equals() to themselves
 --

 Key: SOLR-3548
 URL: https://issues.apache.org/jira/browse/SOLR-3548
 Project: Solr
  Issue Type: Bug
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Blocker
 Fix For: 4.0, 3.6.1

 Attachments: SOLR-3548.patch, SOLR-3548.patch


 working on SOLR-3522 i discovered that in some cases Solr is producing 
 queries that aren't equals to themselves, so they can't be cached.
 So far the only observed problem is in func strdist, but i want to make sure 
 we have an exhaustive test of this in case there are others

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3548) some queries produced by builtin QParsers do not satisfy QueryUtils checks


 [ 
https://issues.apache.org/jira/browse/SOLR-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-3548:
---

Summary: some queries produced by builtin QParsers do not satisfy 
QueryUtils checks  (was: some queries produced by built in QParsers are not 
.equals() to themselves)

 some queries produced by builtin QParsers do not satisfy QueryUtils checks
 --

 Key: SOLR-3548
 URL: https://issues.apache.org/jira/browse/SOLR-3548
 Project: Solr
  Issue Type: Bug
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Blocker
 Fix For: 4.0, 3.6.1

 Attachments: SOLR-3548.patch, SOLR-3548.patch


 working on SOLR-3522 i discovered that in some cases Solr is producing 
 queries that aren't equals to themselves, so they can't be cached.
 So far the only observed problem is in func strdist, but i want to make sure 
 we have an exhaustive test of this in case there are others

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Windows-Java7-64 - Build # 346 - Failure!

2012-06-18 Thread Policeman Jenkins Server

Build: 
http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Windows-Java7-64/346/

1 tests failed.
REGRESSION:  org.apache.solr.spelling.suggest.SuggesterTSTTest.testReload

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([2C36619596515F9B:EBC619965C12A789]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:459)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:426)
at 
org.apache.solr.spelling.suggest.SuggesterTest.testReload(SuggesterTest.java:91)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1969)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:814)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:889)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:32)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:821)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at 
org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at 
org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53)
at 
org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//lst[@name='spellcheck']/lst[@name='suggestions']/lst[@name='ac']/int[@name='numFound'][.='2']
xml response was: ?xml version=1.0 encoding=UTF-8?
response
lst name=responseHeaderint name=status0/intint 
name=QTime0/int/lstlst name=spellchecklst name=suggestions//lst
/response

request 
was:q=acspellcheck.count=2qt=/suggest_tstspellcheck.onlyMorePopular=true
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:452)

[jira] [Commented] (LUCENE-4145) Unhandled exception from test framework (in json parsing of test output files?)


[ 
https://issues.apache.org/jira/browse/LUCENE-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13396432#comment-13396432
 ] 

Hoss Man commented on LUCENE-4145:
--

Thanks Dawid ... i would totally expect/respect -Dtests.iters=$HUGE to be 
limited by the amount of Ram in use, it was the odd except on console followed 
by the aparent infinite hang that concerend me the most (would hate to see that 
happen in a jenkins build)

 Unhandled exception from test framework (in json parsing of test output 
 files?)
 -

 Key: LUCENE-4145
 URL: https://issues.apache.org/jira/browse/LUCENE-4145
 Project: Lucene - Java
  Issue Type: Improvement
Reporter: Hoss Man
Assignee: Dawid Weiss

 Working on SOLR-3267 i got a weird exception printed to the junit output...
 {noformat}
[junit4] Unhandled exception in thread: Thread[pumper-events,5,main]
[junit4] 
 com.carrotsearch.ant.tasks.junit4.dependencies.com.google.gson.JsonParseException:
  No such reference: id#org.apache.solr.search.TestSort[3]
 ...
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-3548) some queries produced by builtin QParsers do not satisfy QueryUtils checks


 [ 
https://issues.apache.org/jira/browse/SOLR-3548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hoss Man updated SOLR-3548:
---

Attachment: SOLR-3548.patch

Fixed JoinQuery and SpatialDistanceQuery classes to consult super.equals() and 
super.hashCode() in their corrisponding methods, and added completely new 
equals/hashCode impls to JaroWinklerDistance, NGramDistance, and 
LevensteinDistance (which never had them before aparently).

This gets all the new tests passing

 some queries produced by builtin QParsers do not satisfy QueryUtils checks
 --

 Key: SOLR-3548
 URL: https://issues.apache.org/jira/browse/SOLR-3548
 Project: Solr
  Issue Type: Bug
Reporter: Hoss Man
Assignee: Hoss Man
Priority: Blocker
 Fix For: 4.0, 3.6.1

 Attachments: SOLR-3548.patch, SOLR-3548.patch, SOLR-3548.patch


 working on SOLR-3522 i discovered that in some cases Solr is producing 
 queries that aren't equals to themselves, so they can't be cached.
 So far the only observed problem is in func strdist, but i want to make sure 
 we have an exhaustive test of this in case there are others

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (LUCENE-4146) -Dtests.iters combined with -Dtestmethod never fails?