Re: Two hour recovery in non-replica setup?
Hi Thanks, Mark. I am pretty sure SOLR-4899 is not it, but our problem might we solved after 4.0.0 anyway. We will consider if we will upgrade to latest 4.x and/or if we will dive into the problem to see if we can figure out what happens. Regards, Per Steffensen On 8/14/13 4:13 PM, Mark Miller wrote: There have been hundreds of fixes since 4.0 - many around zookeeper integration. Really hard to say what it might be - I can say nothing should take that long by design. A pretty important somewhat recent bug found was https://issues.apache.org/jira/browse/SOLR-4899 When reconnecting after ZooKeeper expiration, we need to be willing to wait forever, not for 30 seconds. Don't know that it has anything to do with what you are seeing, or if perhaps that didn't exist in 4.0 - but worth looking at. - Mark On Aug 14, 2013, at 9:10 AM, Per Steffensen st...@designware.dk wrote: Hi We have a fairly large SolrCloud installation where we continuously index a lot of documents. Users are doing searches against the system from time to time. From time to time our Solrs lose their Zookeeper connection. Guess thats what happens. But it takes two hours before a Solr that loses its ZK connection has its shards active again. I cant imagine what it needs to do for two hours? We are not using replica, so synch of data among replica is not it. At the time we havnt dived into the problem, and we do not know there the time is spent, whether it is between the connection is lost and the Solr realizes it, or whether it is from when it realizes it and until the shards are declared active again. We will dive into the problem, but before doing that I wanted to ask here if this is a known problem, and, if yes, whether or not it is solved? FYI we are using Solr 4.0.0. Regards, Per Steffensen - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5056) Further clean up of ConfigSolr interface and CoreContainer construction
[ https://issues.apache.org/jira/browse/SOLR-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated SOLR-5056: --- Fix Version/s: 4.5 Further clean up of ConfigSolr interface and CoreContainer construction --- Key: SOLR-5056 URL: https://issues.apache.org/jira/browse/SOLR-5056 Project: Solr Issue Type: Improvement Reporter: Alan Woodward Assignee: Alan Woodward Priority: Minor Fix For: 4.5 Attachments: SOLR-5056.patch, SOLR-5056.patch Makes ConfigSolr a bit more typesafe, and pushes a bunch of cloud-specific config into ZkController. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5176) remove _TestUtil.keepFullyDeletedSegments
[ https://issues.apache.org/jira/browse/LUCENE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740772#comment-13740772 ] ASF subversion and git services commented on LUCENE-5176: - Commit 1514184 from [~thetaphi] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1514184 ] Merged revision(s) 1514183 from lucene/dev/trunk: LUCENE-5176: Reuse new AllDeletedFilterReader in TestAddIndexes remove _TestUtil.keepFullyDeletedSegments - Key: LUCENE-5176 URL: https://issues.apache.org/jira/browse/LUCENE-5176 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Fix For: 5.0, 4.5 Attachments: LUCENE-5176.patch, LUCENE-5176.patch The few tests that need to test something here can use the index-writer pkg-private method. Other tests (like QueryUtils) can use a FilterAtomicReader to do their crazy tests. Otherwise, we risk lots of stupid false test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5176) remove _TestUtil.keepFullyDeletedSegments
[ https://issues.apache.org/jira/browse/LUCENE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740769#comment-13740769 ] ASF subversion and git services commented on LUCENE-5176: - Commit 1514183 from [~thetaphi] in branch 'dev/trunk' [ https://svn.apache.org/r1514183 ] LUCENE-5176: Reuse new AllDeletedFilterReader in TestAddIndexes remove _TestUtil.keepFullyDeletedSegments - Key: LUCENE-5176 URL: https://issues.apache.org/jira/browse/LUCENE-5176 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir Fix For: 5.0, 4.5 Attachments: LUCENE-5176.patch, LUCENE-5176.patch The few tests that need to test something here can use the index-writer pkg-private method. Other tests (like QueryUtils) can use a FilterAtomicReader to do their crazy tests. Otherwise, we risk lots of stupid false test failures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Problem threading JIRA emails in thunderbird?
I'm wondering if anyone else is using Thunderbird and having problems with incorrect threading of emails from Jira. A few months ago I filed a bug with mozilla that has a screenshot and header info, but nothing's happened with it: https://bugzilla.mozilla.org/show_bug.cgi?id=864545 If anyone is seeing similar behavior, could you please update the bug to let Mozilla know? If you happen to know of a solution, I'd love to hear it. Thanks, Shawn - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
[
https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740829#comment-13740829
]
Dawid Weiss edited comment on LUCENE-5168 at 8/15/13 9:21 AM:
--
Yeah, I agree. It isn't a trivial single-method issue though. nextSlice is
indeed the problem, but it's inlined heavily in a number of places (and
probably optimized wherever it's inlined). I've experimented with the following:
- -XX:-Inline - no errors
- excluding the parent method from compilation (which seems to be the root of
the problem): org/apache/lucene/index/FreqProxTermsWriterPerField,flush - no
errors
- -XX:-DoEscapeAnalysis - preventing escape analysis - no errors
- enforcing value flush in ByteSliceReader.nextSlice by modifying:
{code}
buffer = pool.buffers[bufferUpto];
upto = nextIndex ByteBlockPool.BYTE_BLOCK_MASK;
{code}
to
{code}
buffer = pool.buffers[bufferUpto];
foo = upto = nextIndex ByteBlockPool.BYTE_BLOCK_MASK;
{code}
where foo is defined as a static field with no other accesses:
{code}
static int foo;
{code}
This also results in no errors.
I also dumped the generated assembly for the flush() method (you have to dump
the parent method because of the inlines) but it's huge and, sigh, it's
sometimes different even for two runs with identical options. I'm guessing
parallel compilation tasks hit different stats and hence the difference.
was (Author: dweiss):
Yeah, I agree. It isn't a trivial single-method issue though. nextSlice is
indeed the problem, but it's inlined heavily in a number of places (and
probably optimized wherever it's inlined). I've experimented with the following:
- -XX:-Inline - no errors
- excluding the parent method from compilation (which seems to be the root of
the problem): org/apache/lucene/index/FreqProxTermsWriterPerField,flush - no
errors
- -XX:-DoEscapeAnalysis - preventing escape analysis - no errors
- enforcing value flush in ByteSliceReader.nextSlice by modifying:
{code}
buffer = pool.buffers[bufferUpto];
upto = nextIndex ByteBlockPool.BYTE_BLOCK_MASK;
{code}
to
{code}
buffer = pool.buffers[bufferUpto];
foo = upto = nextIndex ByteBlockPool.BYTE_BLOCK_MASK;
{code}
where foo is defined as a static field with no other accesses:
{code}
static int foo;
{code}
I also dumped the generated assembly for the flush() method (you have to dump
the parent method because of the inlines) but it's huge and, sigh, it's
sometimes different even for two runs with identical options. I'm guessing
parallel compilation tasks hit different stats and hence the difference.
ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
---
Key: LUCENE-5168
URL: https://issues.apache.org/jira/browse/LUCENE-5168
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: java8-windows-4x-3075-console.txt
This assertion trips (sometimes from different tests), if you run the
highlighting tests on branch_4x with r1512807.
It reproduces about half the time, always only with 32bit + G1GC (other
combinations do not seem to trip it, i didnt try looping or anything really
though).
{noformat}
rmuir@beast:~/workspace/branch_4x$ svn up -r 1512807
rmuir@beast:~/workspace/branch_4x$ ant clean
rmuir@beast:~/workspace/branch_4x$ rm -rf .caches #this is important,
otherwise master seed does not work!
rmuir@beast:~/workspace/branch_4x/lucene/highlighter$ ant test
-Dtests.jvms=2 -Dtests.seed=EBBFA6F4E80A7365 -Dargs=-server
-XX:+UseG1GC
{noformat}
Originally showed up like this:
{noformat}
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6874/
Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
1 tests failed.
REGRESSION:
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testUserFailedToIndexOffsets
Error Message:
Stack Trace:
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([EBBFA6F4E80A7365:1FBF811885F2D611]:0)
at
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
at
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
at
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly,
[jira] [Commented] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
[
https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740829#comment-13740829
]
Dawid Weiss commented on LUCENE-5168:
-
Yeah, I agree. It isn't a trivial single-method issue though. nextSlice is
indeed the problem, but it's inlined heavily in a number of places (and
probably optimized wherever it's inlined). I've experimented with the following:
- -XX:-Inline - no errors
- excluding the parent method from compilation (which seems to be the root of
the problem): org/apache/lucene/index/FreqProxTermsWriterPerField,flush - no
errors
- -XX:-DoEscapeAnalysis - preventing escape analysis - no errors
- enforcing value flush in ByteSliceReader.nextSlice by modifying:
{code}
buffer = pool.buffers[bufferUpto];
upto = nextIndex ByteBlockPool.BYTE_BLOCK_MASK;
{code}
to
{code}
buffer = pool.buffers[bufferUpto];
foo = upto = nextIndex ByteBlockPool.BYTE_BLOCK_MASK;
{code}
where foo is defined as a static field with no other accesses:
{code}
static int foo;
{code}
I also dumped the generated assembly for the flush() method (you have to dump
the parent method because of the inlines) but it's huge and, sigh, it's
sometimes different even for two runs with identical options. I'm guessing
parallel compilation tasks hit different stats and hence the difference.
ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
---
Key: LUCENE-5168
URL: https://issues.apache.org/jira/browse/LUCENE-5168
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: java8-windows-4x-3075-console.txt
This assertion trips (sometimes from different tests), if you run the
highlighting tests on branch_4x with r1512807.
It reproduces about half the time, always only with 32bit + G1GC (other
combinations do not seem to trip it, i didnt try looping or anything really
though).
{noformat}
rmuir@beast:~/workspace/branch_4x$ svn up -r 1512807
rmuir@beast:~/workspace/branch_4x$ ant clean
rmuir@beast:~/workspace/branch_4x$ rm -rf .caches #this is important,
otherwise master seed does not work!
rmuir@beast:~/workspace/branch_4x/lucene/highlighter$ ant test
-Dtests.jvms=2 -Dtests.seed=EBBFA6F4E80A7365 -Dargs=-server
-XX:+UseG1GC
{noformat}
Originally showed up like this:
{noformat}
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6874/
Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
1 tests failed.
REGRESSION:
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testUserFailedToIndexOffsets
Error Message:
Stack Trace:
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([EBBFA6F4E80A7365:1FBF811885F2D611]:0)
at
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
at
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
at
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
[
https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740832#comment-13740832
]
Uwe Schindler commented on LUCENE-5168:
---
I am glad it is not {{DataInput#readVInt()}} :-)
ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
---
Key: LUCENE-5168
URL: https://issues.apache.org/jira/browse/LUCENE-5168
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: java8-windows-4x-3075-console.txt
This assertion trips (sometimes from different tests), if you run the
highlighting tests on branch_4x with r1512807.
It reproduces about half the time, always only with 32bit + G1GC (other
combinations do not seem to trip it, i didnt try looping or anything really
though).
{noformat}
rmuir@beast:~/workspace/branch_4x$ svn up -r 1512807
rmuir@beast:~/workspace/branch_4x$ ant clean
rmuir@beast:~/workspace/branch_4x$ rm -rf .caches #this is important,
otherwise master seed does not work!
rmuir@beast:~/workspace/branch_4x/lucene/highlighter$ ant test
-Dtests.jvms=2 -Dtests.seed=EBBFA6F4E80A7365 -Dargs=-server
-XX:+UseG1GC
{noformat}
Originally showed up like this:
{noformat}
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6874/
Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
1 tests failed.
REGRESSION:
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testUserFailedToIndexOffsets
Error Message:
Stack Trace:
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([EBBFA6F4E80A7365:1FBF811885F2D611]:0)
at
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
at
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
at
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
[
https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740840#comment-13740840
]
Dawid Weiss commented on LUCENE-5168:
-
Don't be so sure. :) FreqProxTermsWriterPerField::flush inlines half of the
world (and is compiled multiple times, in fact). readVInt *is* in fact inlined
there (but it doesn't seem to be the root of the problem to me).
{code}
8206 1158
org.apache.lucene.index.FreqProxTermsWriterPerField::flush (1207 bytes)
@ 4 org.apache.lucene.index.FieldInfo::isIndexed
(5 bytes) inline (hot)
@ 16
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter::addField
(327 bytes) too big
@ 16
org.apache.lucene.codecs.BlockTreeTermsWriter::addField (53 bytes) too big
@ 23
org.apache.lucene.codecs.BlockTreeTermsWriter$TermsWriter::getComparator (4
bytes) inline (hot)
@ 23
org.apache.lucene.codecs.blockterms.BlockTermsWriter$TermsWriter::getComparator
(4 bytes) inline (hot)
@ 0
org.apache.lucene.util.BytesRef::getUTF8SortedAsUnicodeComparator (4 bytes)
inline (hot)
@ 0
org.apache.lucene.util.BytesRef::getUTF8SortedAsUnicodeComparator (4 bytes)
inline (hot)
@ 32
org.apache.lucene.index.FieldInfo::getIndexOptions (5 bytes) inline (hot)
@ 61 java.lang.Enum::compareTo (44 bytes) too
big
@ 79 java.lang.Enum::compareTo (44 bytes) too
big
@ 97 java.lang.Enum::compareTo (44 bytes) too
big
@ 238 java.util.HashMap::size (5 bytes) inline
(hot)
@ 267
org.apache.lucene.index.TermsHashPerField::sortPostings (9 bytes) executed
MinInliningThreshold times
@ 279 org.apache.lucene.util.BytesRefHash::size
(5 bytes) inline (hot)
@ 288 org.apache.lucene.util.BytesRef::init (8
bytes) inline (hot)
@ 4 org.apache.lucene.util.BytesRef::init (9
bytes) inline (hot)
@ 5 org.apache.lucene.util.BytesRef::init
(41 bytes) inline (hot)
@ 1 java.lang.Object::init (1 bytes)
inline (hot)
@ 26
org.apache.lucene.util.BytesRef::isValid (329 bytes) hot method too big
@ 309
org.apache.lucene.index.ByteSliceReader::init (5 bytes) inline (hot)
@ 1 org.apache.lucene.store.DataInput::init
(5 bytes) inline (hot)
@ 1 java.lang.Object::init (1 bytes)
inline (hot)
@ 318
org.apache.lucene.index.ByteSliceReader::init (5 bytes) inline (hot)
@ 1 org.apache.lucene.store.DataInput::init
(5 bytes) inline (hot)
@ 1 java.lang.Object::init (1 bytes)
inline (hot)
@ 331
org.apache.lucene.index.SegmentInfo::getDocCount (23 bytes) inline (hot)
@ 334 org.apache.lucene.util.FixedBitSet::init
(29 bytes) inline (hot)
@ 1 org.apache.lucene.search.DocIdSet::init
(5 bytes) inline (hot)
@ 1 java.lang.Object::init (1 bytes)
inline (hot)
@ 11
org.apache.lucene.util.FixedBitSet::bits2words (17 bytes) inline (hot)
@ 350 org.apache.lucene.index.Term::init (13
bytes) inline (hot)
@ 6 org.apache.lucene.util.BytesRef::init (8
bytes) inline (hot)
@ 4 org.apache.lucene.util.BytesRef::init
(9 bytes) inline (hot)
@ 5 org.apache.lucene.util.BytesRef::init
(41 bytes) inline (hot)
@ 1 java.lang.Object::init (1 bytes)
inline (hot)
@ 26
org.apache.lucene.util.BytesRef::isValid (329 bytes) hot method too big
@ 9 org.apache.lucene.index.Term::init (15
bytes) inline (hot)
@ 1 java.lang.Object::init (1 bytes)
inline (hot)
@ 393
org.apache.lucene.util.ByteBlockPool::setBytesRef (107 bytes) inline (hot)
@ 405
org.apache.lucene.index.TermsHashPerField::initReader (87 bytes) inline (hot)
@ 83
org.apache.lucene.index.ByteSliceReader::init (153 bytes) inline (hot)
@ 427
[jira] [Commented] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
[
https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740847#comment-13740847
]
Uwe Schindler commented on LUCENE-5168:
---
What happens if you make upto volatile? I assume the same happens like with
the static dummy field.
Maybe for now we use the static field hack. I don't think this causes
performance trouble. We can add a good comment and once the bug is fixed in JDK
we can remove it in 5 years or so :-)
To me it looks like we hit this bug more often because of optimized tests that
seem to run this method more often. This is a good sign, we are improving our
tests! :-)
ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
---
Key: LUCENE-5168
URL: https://issues.apache.org/jira/browse/LUCENE-5168
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: java8-windows-4x-3075-console.txt
This assertion trips (sometimes from different tests), if you run the
highlighting tests on branch_4x with r1512807.
It reproduces about half the time, always only with 32bit + G1GC (other
combinations do not seem to trip it, i didnt try looping or anything really
though).
{noformat}
rmuir@beast:~/workspace/branch_4x$ svn up -r 1512807
rmuir@beast:~/workspace/branch_4x$ ant clean
rmuir@beast:~/workspace/branch_4x$ rm -rf .caches #this is important,
otherwise master seed does not work!
rmuir@beast:~/workspace/branch_4x/lucene/highlighter$ ant test
-Dtests.jvms=2 -Dtests.seed=EBBFA6F4E80A7365 -Dargs=-server
-XX:+UseG1GC
{noformat}
Originally showed up like this:
{noformat}
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6874/
Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
1 tests failed.
REGRESSION:
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testUserFailedToIndexOffsets
Error Message:
Stack Trace:
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([EBBFA6F4E80A7365:1FBF811885F2D611]:0)
at
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
at
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
at
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5158) PostingsHighlighter - optionally return unhighlighted fields
Stewart Sims created SOLR-5158: -- Summary: PostingsHighlighter - optionally return unhighlighted fields Key: SOLR-5158 URL: https://issues.apache.org/jira/browse/SOLR-5158 Project: Solr Issue Type: Improvement Components: highlighter Affects Versions: 4.2.1 Reporter: Stewart Sims Priority: Minor The PostingsHighlighter mechanism currently returns all fields specified in the hl.fl parameter for each document, even if they do not have hits within them. This can lead to a lot of empty / self-closing tags if highlighting on a large number of fields which may be undesirable. It would be useful if a user could specify using a parameter such as 'hl.ignoreUnhighlightedFields=true' that these fields are not returned in the response. See http://lucene.472066.n3.nabble.com/PostingsHighlighter-returning-fields-which-don-t-match-td4084495.html for a suggestions on implementation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
[
https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740851#comment-13740851
]
Dawid Weiss commented on LUCENE-5168:
-
Yeah... we could temporarily commit the static field hack and at least see if
this proves me wrong (perhaps it just happens to work for this particular seed
but will fail elsewhere).
ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
---
Key: LUCENE-5168
URL: https://issues.apache.org/jira/browse/LUCENE-5168
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: java8-windows-4x-3075-console.txt
This assertion trips (sometimes from different tests), if you run the
highlighting tests on branch_4x with r1512807.
It reproduces about half the time, always only with 32bit + G1GC (other
combinations do not seem to trip it, i didnt try looping or anything really
though).
{noformat}
rmuir@beast:~/workspace/branch_4x$ svn up -r 1512807
rmuir@beast:~/workspace/branch_4x$ ant clean
rmuir@beast:~/workspace/branch_4x$ rm -rf .caches #this is important,
otherwise master seed does not work!
rmuir@beast:~/workspace/branch_4x/lucene/highlighter$ ant test
-Dtests.jvms=2 -Dtests.seed=EBBFA6F4E80A7365 -Dargs=-server
-XX:+UseG1GC
{noformat}
Originally showed up like this:
{noformat}
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6874/
Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
1 tests failed.
REGRESSION:
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testUserFailedToIndexOffsets
Error Message:
Stack Trace:
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([EBBFA6F4E80A7365:1FBF811885F2D611]:0)
at
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
at
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
at
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5149) Query facet to respect mincount
[ https://issues.apache.org/jira/browse/SOLR-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740912#comment-13740912 ] Markus Jelsma commented on SOLR-5149: - Yes, that makes sense. But we definately have some cases in which we would like to omit them from the result so maybe having a facet.querymincount would be a good idea? Thanks Query facet to respect mincount --- Key: SOLR-5149 URL: https://issues.apache.org/jira/browse/SOLR-5149 Project: Solr Issue Type: Bug Components: SearchComponents - other Affects Versions: 4.4 Reporter: Markus Jelsma Priority: Minor Fix For: 4.5, 5.0 Attachments: SOLR-5149-trunk.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5166) PostingsHighlighter fails with IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/LUCENE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manuel Amoabeng updated LUCENE-5166: Attachment: LUCENE-5166-revisited.patch PostingsHighlighter fails with IndexOutOfBoundsException Key: LUCENE-5166 URL: https://issues.apache.org/jira/browse/LUCENE-5166 Project: Lucene - Core Issue Type: Bug Components: modules/highlighter Affects Versions: 4.4 Reporter: Manuel Amoabeng Fix For: 5.0, 4.5 Attachments: LUCENE-5166-2.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166-revisited.patch Given a document with a match at a startIndex PostingsHighlighter.maxlength and an endIndex PostingsHighlighter.maxLength, DefaultPassageFormatter will throw an IndexOutOfBoundsException when DefaultPassageFormatter.append() is invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5166) PostingsHighlighter fails with IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/LUCENE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740925#comment-13740925 ] Manuel Amoabeng commented on LUCENE-5166: - I just found another problem here: If we have both, matches that do and matches that don't span the content boundary the formatter is asked to highlight the spanning match. Please find attached additional tests and a possible fix for this. PostingsHighlighter fails with IndexOutOfBoundsException Key: LUCENE-5166 URL: https://issues.apache.org/jira/browse/LUCENE-5166 Project: Lucene - Core Issue Type: Bug Components: modules/highlighter Affects Versions: 4.4 Reporter: Manuel Amoabeng Fix For: 5.0, 4.5 Attachments: LUCENE-5166-2.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166-revisited.patch Given a document with a match at a startIndex PostingsHighlighter.maxlength and an endIndex PostingsHighlighter.maxLength, DefaultPassageFormatter will throw an IndexOutOfBoundsException when DefaultPassageFormatter.append() is invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (LUCENE-5166) PostingsHighlighter fails with IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/LUCENE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manuel Amoabeng reopened LUCENE-5166: - PostingsHighlighter fails with IndexOutOfBoundsException Key: LUCENE-5166 URL: https://issues.apache.org/jira/browse/LUCENE-5166 Project: Lucene - Core Issue Type: Bug Components: modules/highlighter Affects Versions: 4.4 Reporter: Manuel Amoabeng Fix For: 5.0, 4.5 Attachments: LUCENE-5166-2.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166-revisited.patch Given a document with a match at a startIndex PostingsHighlighter.maxlength and an endIndex PostingsHighlighter.maxLength, DefaultPassageFormatter will throw an IndexOutOfBoundsException when DefaultPassageFormatter.append() is invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3069) Lucene should have an entirely memory resident term dictionary
[ https://issues.apache.org/jira/browse/LUCENE-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Han Jiang updated LUCENE-3069: -- Attachment: LUCENE-3069.patch Patch, update BlockTerms dict so that it follows refactored API. Lucene should have an entirely memory resident term dictionary -- Key: LUCENE-3069 URL: https://issues.apache.org/jira/browse/LUCENE-3069 Project: Lucene - Core Issue Type: Improvement Components: core/index, core/search Affects Versions: 4.0-ALPHA Reporter: Simon Willnauer Assignee: Han Jiang Labels: gsoc2013 Fix For: 5.0, 4.5 Attachments: df-ttf-estimate.txt, example.png, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch FST based TermDictionary has been a great improvement yet it still uses a delta codec file for scanning to terms. Some environments have enough memory available to keep the entire FST based term dict in memory. We should add a TermDictionary implementation that encodes all needed information for each term into the FST (custom fst.Output) and builds a FST from the entire term not just the delta. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3069) Lucene should have an entirely memory resident term dictionary
[ https://issues.apache.org/jira/browse/LUCENE-3069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740948#comment-13740948 ] ASF subversion and git services commented on LUCENE-3069: - Commit 1514253 from [~billy] in branch 'dev/branches/lucene3069' [ https://svn.apache.org/r1514253 ] LUCENE-3069: API refactoring on BlockTerms dict Lucene should have an entirely memory resident term dictionary -- Key: LUCENE-3069 URL: https://issues.apache.org/jira/browse/LUCENE-3069 Project: Lucene - Core Issue Type: Improvement Components: core/index, core/search Affects Versions: 4.0-ALPHA Reporter: Simon Willnauer Assignee: Han Jiang Labels: gsoc2013 Fix For: 5.0, 4.5 Attachments: df-ttf-estimate.txt, example.png, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch, LUCENE-3069.patch FST based TermDictionary has been a great improvement yet it still uses a delta codec file for scanning to terms. Some environments have enough memory available to keep the entire FST based term dict in memory. We should add a TermDictionary implementation that encodes all needed information for each term into the FST (custom fst.Output) and builds a FST from the entire term not just the delta. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
[
https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740969#comment-13740969
]
Dawid Weiss edited comment on LUCENE-5168 at 8/15/13 1:49 PM:
--
I've built a fastdebug binary of openjdk 1.8 (top of the master branch).
Retrying my 1000-random seed sample right now (1.7's seed doesn't reproduce on
my fresh-from-the-oven binary, so perhaps it's been fixed in between).
was (Author: dweiss):
I've built a fastdebug binary of Java 1.8. Retrying my 1000-random seed
sample right now (1.7's seed doesn't reproduce on my fresh-from-the-oven
binary, so perhaps it's been fixed in between).
ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
---
Key: LUCENE-5168
URL: https://issues.apache.org/jira/browse/LUCENE-5168
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: java8-windows-4x-3075-console.txt
This assertion trips (sometimes from different tests), if you run the
highlighting tests on branch_4x with r1512807.
It reproduces about half the time, always only with 32bit + G1GC (other
combinations do not seem to trip it, i didnt try looping or anything really
though).
{noformat}
rmuir@beast:~/workspace/branch_4x$ svn up -r 1512807
rmuir@beast:~/workspace/branch_4x$ ant clean
rmuir@beast:~/workspace/branch_4x$ rm -rf .caches #this is important,
otherwise master seed does not work!
rmuir@beast:~/workspace/branch_4x/lucene/highlighter$ ant test
-Dtests.jvms=2 -Dtests.seed=EBBFA6F4E80A7365 -Dargs=-server
-XX:+UseG1GC
{noformat}
Originally showed up like this:
{noformat}
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6874/
Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
1 tests failed.
REGRESSION:
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testUserFailedToIndexOffsets
Error Message:
Stack Trace:
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([EBBFA6F4E80A7365:1FBF811885F2D611]:0)
at
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
at
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
at
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5168) ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
[
https://issues.apache.org/jira/browse/LUCENE-5168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740969#comment-13740969
]
Dawid Weiss commented on LUCENE-5168:
-
I've built a fastdebug binary of Java 1.8. Retrying my 1000-random seed sample
right now (1.7's seed doesn't reproduce on my fresh-from-the-oven binary, so
perhaps it's been fixed in between).
ByteSliceReader assert trips with 32-bit oracle 1.7.0_25 + G1GC
---
Key: LUCENE-5168
URL: https://issues.apache.org/jira/browse/LUCENE-5168
Project: Lucene - Core
Issue Type: Bug
Reporter: Robert Muir
Attachments: java8-windows-4x-3075-console.txt
This assertion trips (sometimes from different tests), if you run the
highlighting tests on branch_4x with r1512807.
It reproduces about half the time, always only with 32bit + G1GC (other
combinations do not seem to trip it, i didnt try looping or anything really
though).
{noformat}
rmuir@beast:~/workspace/branch_4x$ svn up -r 1512807
rmuir@beast:~/workspace/branch_4x$ ant clean
rmuir@beast:~/workspace/branch_4x$ rm -rf .caches #this is important,
otherwise master seed does not work!
rmuir@beast:~/workspace/branch_4x/lucene/highlighter$ ant test
-Dtests.jvms=2 -Dtests.seed=EBBFA6F4E80A7365 -Dargs=-server
-XX:+UseG1GC
{noformat}
Originally showed up like this:
{noformat}
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6874/
Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC
1 tests failed.
REGRESSION:
org.apache.lucene.search.postingshighlight.TestPostingsHighlighter.testUserFailedToIndexOffsets
Error Message:
Stack Trace:
java.lang.AssertionError
at
__randomizedtesting.SeedInfo.seed([EBBFA6F4E80A7365:1FBF811885F2D611]:0)
at
org.apache.lucene.index.ByteSliceReader.readByte(ByteSliceReader.java:73)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:108)
at
org.apache.lucene.index.FreqProxTermsWriterPerField.flush(FreqProxTermsWriterPerField.java:453)
at
org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:85)
at org.apache.lucene.index.TermsHash.flush(TermsHash.java:116)
at org.apache.lucene.index.DocInverter.flush(DocInverter.java:53)
at
org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:81)
at
org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:501)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5166) PostingsHighlighter fails with IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/LUCENE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740975#comment-13740975 ] Robert Muir commented on LUCENE-5166: - Hi Manuel: thank you! Another bug, or a bug in my fix to the other bug :) I'll investigate deeper in a bit. PostingsHighlighter fails with IndexOutOfBoundsException Key: LUCENE-5166 URL: https://issues.apache.org/jira/browse/LUCENE-5166 Project: Lucene - Core Issue Type: Bug Components: modules/highlighter Affects Versions: 4.4 Reporter: Manuel Amoabeng Fix For: 5.0, 4.5 Attachments: LUCENE-5166-2.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166-revisited.patch Given a document with a match at a startIndex PostingsHighlighter.maxlength and an endIndex PostingsHighlighter.maxLength, DefaultPassageFormatter will throw an IndexOutOfBoundsException when DefaultPassageFormatter.append() is invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5122) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/SOLR-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740999#comment-13740999 ] James Dyer commented on SOLR-5122: -- Hoss, This looks reasonable to me. The test is more forgiving to variations caused by random merges, no more integer division, etc. I appreciate your working on this as I wouldn't have much more time until next week. I think your method of estimating the hits would be at least as good of what I attempted to do before. spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero Key: SOLR-5122 URL: https://issues.apache.org/jira/browse/SOLR-5122 Project: Solr Issue Type: Bug Affects Versions: 4.4 Reporter: Hoss Man Assignee: James Dyer Attachments: SOLR-5122.patch, SOLR-5122.patch, SOLR-5122.patch As part of SOLR-4952 SpellCheckCollatorTest started using RandomMergePolicy, and this (aparently) led to a failure in testEstimatedHitCounts. As far as i can tell: the test assumes that specific values would be returned as the _estimated_ hits for a colleation, and it appears that the change in MergePolicy however resulted in different segments with different term stats, causing the estimation code to produce different values then what is expected. I made a quick attempt to improve the test to: * expect explicit exact values only when spellcheck.collateMaxCollectDocs is set such that the estimate' should actually be exact (ie: collateMaxCollectDocs == 0 or collateMaxCollectDocs greater then the num docs in the index * randomize the values used for collateMaxCollectDocs and confirm that the estimates are never more then the num docs in the index This lead to an odd ArithmeticException: / by zero error in the test, which seems to suggest that there is a genuine bug in the code for estimating the hits that only gets tickled in certain mergepolicy/segment/collateMaxCollectDocs combinations. *Update:* This appears to be a general problem with collecting docs out of order and the estimation of hits -- i believe even if there is no divide by zero error, the estimates are largely meaningless since the docs are collected out of order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5159) Manifest includes non-parsed maven variables
Artem Karpenko created SOLR-5159:
Summary: Manifest includes non-parsed maven variables
Key: SOLR-5159
URL: https://issues.apache.org/jira/browse/SOLR-5159
Project: Solr
Issue Type: Bug
Components: Build
Affects Versions: 4.4, 4.5, 5.0
Environment: Apache Maven 3.0.5
Reporter: Artem Karpenko
Priority: Minor
When building Lucene/Solr with Apache Maven 3, all MANIFEST.MF files included
into JAR artifacts contain non-parsed POM variables: namely, there are entries
like
Specification-Version: 5.0.0.${now.version}
In the end, Solr displays these values on admin page in Versions section.
This is caused by unresolved bug in maven-bundle-plugin (FELIX-3392).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5159) Manifest includes non-parsed maven variables
[
https://issues.apache.org/jira/browse/SOLR-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Karpenko updated SOLR-5159:
-
Attachment: SOLR-5159.patch
Patch moves declaration of Manifest headers from archive configuration of
maven-jar-plugin into instructions section of maven-bundle-plugin, which
latter plugin does parse for variables correctly.
Manifest includes non-parsed maven variables
Key: SOLR-5159
URL: https://issues.apache.org/jira/browse/SOLR-5159
Project: Solr
Issue Type: Bug
Components: Build
Affects Versions: 4.4, 4.5, 5.0
Environment: Apache Maven 3.0.5
Reporter: Artem Karpenko
Priority: Minor
Labels: maven-bundle-plugin, maven3,
Attachments: SOLR-5159.patch
When building Lucene/Solr with Apache Maven 3, all MANIFEST.MF files included
into JAR artifacts contain non-parsed POM variables: namely, there are
entries like
Specification-Version: 5.0.0.${now.version}
In the end, Solr displays these values on admin page in Versions section.
This is caused by unresolved bug in maven-bundle-plugin (FELIX-3392).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5159) Manifest includes non-parsed maven variables
[
https://issues.apache.org/jira/browse/SOLR-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Artem Karpenko updated SOLR-5159:
-
Description:
When building Lucene/Solr with Apache Maven 3, all MANIFEST.MF files included
into JAR artifacts contain non-parsed POM variables: namely, there are entries
like
Specification-Version: 5.0.0.$\{now.version}
In the end, Solr displays these values on admin page in Versions section.
This is caused by unresolved bug in maven-bundle-plugin (FELIX-3392).
was:
When building Lucene/Solr with Apache Maven 3, all MANIFEST.MF files included
into JAR artifacts contain non-parsed POM variables: namely, there are entries
like
Specification-Version: 5.0.0.${now.version}
In the end, Solr displays these values on admin page in Versions section.
This is caused by unresolved bug in maven-bundle-plugin (FELIX-3392).
Manifest includes non-parsed maven variables
Key: SOLR-5159
URL: https://issues.apache.org/jira/browse/SOLR-5159
Project: Solr
Issue Type: Bug
Components: Build
Affects Versions: 4.4, 4.5, 5.0
Environment: Apache Maven 3.0.5
Reporter: Artem Karpenko
Priority: Minor
Labels: maven-bundle-plugin, maven3,
Attachments: SOLR-5159.patch
When building Lucene/Solr with Apache Maven 3, all MANIFEST.MF files included
into JAR artifacts contain non-parsed POM variables: namely, there are
entries like
Specification-Version: 5.0.0.$\{now.version}
In the end, Solr displays these values on admin page in Versions section.
This is caused by unresolved bug in maven-bundle-plugin (FELIX-3392).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5159) Manifest includes non-parsed maven variables
[
https://issues.apache.org/jira/browse/SOLR-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741061#comment-13741061
]
Artem Karpenko commented on SOLR-5159:
--
I've checked maven 2 builds as well - after applying changes manifest files
stay the same (correct).
Manifest includes non-parsed maven variables
Key: SOLR-5159
URL: https://issues.apache.org/jira/browse/SOLR-5159
Project: Solr
Issue Type: Bug
Components: Build
Affects Versions: 4.4, 4.5, 5.0
Environment: Apache Maven 3.0.5
Reporter: Artem Karpenko
Priority: Minor
Labels: maven-bundle-plugin, maven3,
Attachments: SOLR-5159.patch
When building Lucene/Solr with Apache Maven 3, all MANIFEST.MF files included
into JAR artifacts contain non-parsed POM variables: namely, there are
entries like
Specification-Version: 5.0.0.$\{now.version}
In the end, Solr displays these values on admin page in Versions section.
This is caused by unresolved bug in maven-bundle-plugin (FELIX-3392).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5149) Query facet to respect mincount
[ https://issues.apache.org/jira/browse/SOLR-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741112#comment-13741112 ] Erik Hatcher commented on SOLR-5149: I'm curious about the use cases - why not filter out zero counts in the client? I'm ok with another parameter, thought that feels odd too, but at least one would have the option to filter zeros from fields without filtering out facet.query's too. facet.query.mincount? (like facet.range.*) Query facet to respect mincount --- Key: SOLR-5149 URL: https://issues.apache.org/jira/browse/SOLR-5149 Project: Solr Issue Type: Bug Components: SearchComponents - other Affects Versions: 4.4 Reporter: Markus Jelsma Priority: Minor Fix For: 4.5, 5.0 Attachments: SOLR-5149-trunk.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: MoreLikeThis (MLT) - AND operator between the fields
I was looking at the code and found that it is hard coded to Occur.SHOULD in MoreLikeThisQuery. I customized the code to pass a new parameter *mlt.operator*=AND/OR based on that it computes the MLT documents. Default operator is set to OR. And I also want to have *mlt.sort* option, So I will be trying for that as well. Do you guys think, we should make this part of the MLT feature? Please share your ideas. I can submit this change. Thanks Regards, Kranti K Parisa http://www.linkedin.com/in/krantiparisa On Thu, Aug 15, 2013 at 12:05 AM, Kranti Parisa kranti.par...@gmail.comwrote: Hi, It seems that when we pass multiple field names with mlt.fl parameter, it is ORing them to find the MLT documents. Is there a way to specify AND operator? Means if mlt.fl=language,year, then we should return back the MLT documents which has language AND year field values as same as the main query result document. http://localhost:8180/solr/mltCore/mlt?q=id:1wt=jsonmlt=truemlt.fl=language,yearfl=*,scoremlt.mindf=0mlt.mintf=0mlt.match.include=false The above query should return those documents whose field values (language, year) are exactly matching with the document id:1. Is this possible thru any config or param? If not, I think it's worth having as a feature because we don't know the values of those fields to apply as FQ. Thanks Regards, Kranti K Parisa http://www.linkedin.com/in/krantiparisa
[jira] [Commented] (LUCENE-5166) PostingsHighlighter fails with IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/LUCENE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741140#comment-13741140 ] Robert Muir commented on LUCENE-5166: - Manuel: your fix is correct, thank you. To explain: I had totally forgotten about this little loop on tf within the passage (i had removed this optimization in LUCENE-4909, which didnt turn out to work that great, so wasn't committed). We might at some point want to still just remove the optimization just based on the reason that it makes this thing more complicated, it was just intended to speed up the worst case (where someone has very common stopwords and stuff like that). But for now to complete the bugfix, we should commit your patch (LUCENE-5166-revisited.patch). PostingsHighlighter fails with IndexOutOfBoundsException Key: LUCENE-5166 URL: https://issues.apache.org/jira/browse/LUCENE-5166 Project: Lucene - Core Issue Type: Bug Components: modules/highlighter Affects Versions: 4.4 Reporter: Manuel Amoabeng Fix For: 5.0, 4.5 Attachments: LUCENE-5166-2.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166-revisited.patch Given a document with a match at a startIndex PostingsHighlighter.maxlength and an endIndex PostingsHighlighter.maxLength, DefaultPassageFormatter will throw an IndexOutOfBoundsException when DefaultPassageFormatter.append() is invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5160) MoreLikeThis (MLT) - New Optional Params - mlt.operator mlt.sort
Kranti Parisa created SOLR-5160: --- Summary: MoreLikeThis (MLT) - New Optional Params - mlt.operator mlt.sort Key: SOLR-5160 URL: https://issues.apache.org/jira/browse/SOLR-5160 Project: Solr Issue Type: Improvement Components: MoreLikeThis Reporter: Kranti Parisa Priority: Minor It seems that when we pass multiple field names with mlt.fl parameter, it is ORing them to find the MLT documents. Is there a way to specify AND operator? Means if mlt.fl=language,year, then we should return back the MLT documents which has language AND year field values as same as the main query result document. http://localhost:8180/solr/mltCore/mlt?q=id:1wt=jsonmlt=truemlt.fl=language,yearfl=*,scoremlt.mindf=0mlt.mintf=0mlt.match.include=false The above query should return those documents whose field values (language, year) are exactly matching with the document id:1. Is this possible thru any config or param? If not, I think it's worth having as a feature because we don't know the values of those fields to apply as FQ. was looking at the code and found that it is hard coded to Occur.SHOULD in MoreLikeThisQuery. I customized the code to pass a new parameter mlt.operator=AND/OR based on that it computes the MLT documents. Default operator is set to OR. And I also want to have mlt.sort option, So I will be trying for that as well. Do you guys think, we should make this part of the MLT feature? Please share your ideas. I can submit this change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5166) PostingsHighlighter fails with IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/LUCENE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741174#comment-13741174 ] ASF subversion and git services commented on LUCENE-5166: - Commit 1514367 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1514367 ] LUCENE-5166: also fix and test this case where tf 1 within the passage for a term PostingsHighlighter fails with IndexOutOfBoundsException Key: LUCENE-5166 URL: https://issues.apache.org/jira/browse/LUCENE-5166 Project: Lucene - Core Issue Type: Bug Components: modules/highlighter Affects Versions: 4.4 Reporter: Manuel Amoabeng Fix For: 5.0, 4.5 Attachments: LUCENE-5166-2.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166-revisited.patch Given a document with a match at a startIndex PostingsHighlighter.maxlength and an endIndex PostingsHighlighter.maxLength, DefaultPassageFormatter will throw an IndexOutOfBoundsException when DefaultPassageFormatter.append() is invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5160) MoreLikeThis (MLT) - New Optional Params - mlt.operator mlt.sort
[ https://issues.apache.org/jira/browse/SOLR-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741172#comment-13741172 ] Jack Krupansky commented on SOLR-5160: -- In truth, a more-like-this feature should have two modes: 1) Discovery mode, to broaden the search or increase recall (OR), and 2) Refinement mode, to narrow the search or increase precision (AND). MoreLikeThis (MLT) - New Optional Params - mlt.operator mlt.sort --- Key: SOLR-5160 URL: https://issues.apache.org/jira/browse/SOLR-5160 Project: Solr Issue Type: Improvement Components: MoreLikeThis Reporter: Kranti Parisa Priority: Minor Labels: features Original Estimate: 24h Remaining Estimate: 24h It seems that when we pass multiple field names with mlt.fl parameter, it is ORing them to find the MLT documents. Is there a way to specify AND operator? Means if mlt.fl=language,year, then we should return back the MLT documents which has language AND year field values as same as the main query result document. http://localhost:8180/solr/mltCore/mlt?q=id:1wt=jsonmlt=truemlt.fl=language,yearfl=*,scoremlt.mindf=0mlt.mintf=0mlt.match.include=false The above query should return those documents whose field values (language, year) are exactly matching with the document id:1. Is this possible thru any config or param? If not, I think it's worth having as a feature because we don't know the values of those fields to apply as FQ. was looking at the code and found that it is hard coded to Occur.SHOULD in MoreLikeThisQuery. I customized the code to pass a new parameter mlt.operator=AND/OR based on that it computes the MLT documents. Default operator is set to OR. And I also want to have mlt.sort option, So I will be trying for that as well. Do you guys think, we should make this part of the MLT feature? Please share your ideas. I can submit this change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-5159) Manifest includes non-parsed maven variables
[
https://issues.apache.org/jira/browse/SOLR-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Rowe reassigned SOLR-5159:
Assignee: Steve Rowe
Manifest includes non-parsed maven variables
Key: SOLR-5159
URL: https://issues.apache.org/jira/browse/SOLR-5159
Project: Solr
Issue Type: Bug
Components: Build
Affects Versions: 4.4, 4.5, 5.0
Environment: Apache Maven 3.0.5
Reporter: Artem Karpenko
Assignee: Steve Rowe
Priority: Minor
Labels: maven-bundle-plugin, maven3,
Attachments: SOLR-5159.patch
When building Lucene/Solr with Apache Maven 3, all MANIFEST.MF files included
into JAR artifacts contain non-parsed POM variables: namely, there are
entries like
Specification-Version: 5.0.0.$\{now.version}
In the end, Solr displays these values on admin page in Versions section.
This is caused by unresolved bug in maven-bundle-plugin (FELIX-3392).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5159) Manifest includes non-parsed maven variables
[
https://issues.apache.org/jira/browse/SOLR-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741176#comment-13741176
]
Steve Rowe commented on SOLR-5159:
--
Hi Artem, I've run and compared with and without your patch on trunk with
Maven3, looks good so far: ${svn.revision} and ${now.timestamp} and one other
timestazmp that weren't being interpolated now are. I want to verify Maven2
locally, and I also want to compare all manifest entries with the Ant-produced
ones - the solr entries were changed recently, and I want to keep them in sync.
Once these issues are settled, I'll commit.
Manifest includes non-parsed maven variables
Key: SOLR-5159
URL: https://issues.apache.org/jira/browse/SOLR-5159
Project: Solr
Issue Type: Bug
Components: Build
Affects Versions: 4.4, 4.5, 5.0
Environment: Apache Maven 3.0.5
Reporter: Artem Karpenko
Assignee: Steve Rowe
Priority: Minor
Labels: maven-bundle-plugin, maven3,
Attachments: SOLR-5159.patch
When building Lucene/Solr with Apache Maven 3, all MANIFEST.MF files included
into JAR artifacts contain non-parsed POM variables: namely, there are
entries like
Specification-Version: 5.0.0.$\{now.version}
In the end, Solr displays these values on admin page in Versions section.
This is caused by unresolved bug in maven-bundle-plugin (FELIX-3392).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_25) - Build # 3148 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/3148/ Java: 32bit/jdk1.7.0_25 -server -XX:+UseG1GC No tests ran. Build Log: [...truncated 1326 lines...] FATAL: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41) at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34) at hudson.remoting.Request.call(Request.java:174) at hudson.remoting.Channel.call(Channel.java:714) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:167) at com.sun.proxy.$Proxy84.join(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:925) at hudson.Launcher$ProcStarter.join(Launcher.java:360) at hudson.tasks.Ant.perform(Ant.java:217) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:160) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:586) at hudson.model.Run.execute(Run.java:1597) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:247) Caused by: hudson.remoting.RequestAbortedException: hudson.remoting.Channel$OrderlyShutdown at hudson.remoting.Request.abort(Request.java:299) at hudson.remoting.Channel.terminate(Channel.java:774) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:919) at hudson.remoting.Channel$2.handle(Channel.java:461) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:60) Caused by: hudson.remoting.Channel$OrderlyShutdown ... 3 more Caused by: Command close created at at hudson.remoting.Command.init(Command.java:56) at hudson.remoting.Channel$CloseCommand.init(Channel.java:913) at hudson.remoting.Channel$CloseCommand.init(Channel.java:911) at hudson.remoting.Channel.close(Channel.java:994) at hudson.remoting.Channel.close(Channel.java:977) at hudson.remoting.Channel$CloseCommand.execute(Channel.java:918) ... 2 more - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5160) MoreLikeThis (MLT) - New Optional Params - mlt.operator mlt.sort
[ https://issues.apache.org/jira/browse/SOLR-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741184#comment-13741184 ] Kranti Parisa commented on SOLR-5160: - Yes, AND will help to get MLT documents based on the criteria. the criteria could be I want to get similar documents whose values are same as the original document. for example: all products manufactured in 2013 + mobiles category (these are attributes of the original document for which we are trying to find exact matches). And if the use case is to get either manufactured in 2013 OR mobiles category, then we can ask for OR. Hence, I think allowing the user to specify the operator (mlt.operator or something) will help them get MLT documents based on their criteria and the results can be sorted based on mlt.sort MoreLikeThis (MLT) - New Optional Params - mlt.operator mlt.sort --- Key: SOLR-5160 URL: https://issues.apache.org/jira/browse/SOLR-5160 Project: Solr Issue Type: Improvement Components: MoreLikeThis Reporter: Kranti Parisa Priority: Minor Labels: features Original Estimate: 24h Remaining Estimate: 24h It seems that when we pass multiple field names with mlt.fl parameter, it is ORing them to find the MLT documents. Is there a way to specify AND operator? Means if mlt.fl=language,year, then we should return back the MLT documents which has language AND year field values as same as the main query result document. http://localhost:8180/solr/mltCore/mlt?q=id:1wt=jsonmlt=truemlt.fl=language,yearfl=*,scoremlt.mindf=0mlt.mintf=0mlt.match.include=false The above query should return those documents whose field values (language, year) are exactly matching with the document id:1. Is this possible thru any config or param? If not, I think it's worth having as a feature because we don't know the values of those fields to apply as FQ. was looking at the code and found that it is hard coded to Occur.SHOULD in MoreLikeThisQuery. I customized the code to pass a new parameter mlt.operator=AND/OR based on that it computes the MLT documents. Default operator is set to OR. And I also want to have mlt.sort option, So I will be trying for that as well. Do you guys think, we should make this part of the MLT feature? Please share your ideas. I can submit this change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4952) audit test configs to use solrconfig.snippet.randomindexconfig.xml in more tests
[
https://issues.apache.org/jira/browse/SOLR-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741185#comment-13741185
]
ASF subversion and git services commented on SOLR-4952:
---
Commit 1514377 from hoss...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1514377 ]
SOLR-4952: SolrCmdDistributorTest needs to force MergePolicy for consistent
optimize() behavior
audit test configs to use solrconfig.snippet.randomindexconfig.xml in more
tests
Key: SOLR-4952
URL: https://issues.apache.org/jira/browse/SOLR-4952
Project: Solr
Issue Type: Sub-task
Reporter: Hoss Man
Assignee: Hoss Man
Fix For: 4.5, 5.0
in SOLR-4942 i updated every solrconfig.xml to either...
* include solrconfig.snippet.randomindexconfig.xml where it was easy to do so
* use the useCompoundFile sys prop if it already had an {{indexConfig}}
section, or if including the snippet wasn't going to be easy (ie: contrib
tests)
As an improvment on this:
* audit all core configs not already using
solrconfig.snippet.randomindexconfig.xml and either:
** make them use it, ignoring any previously unimportant explicit
incdexConfig settings
** make them use it, using explicit sys props to overwrite random values in
cases were explicit indexConfig values are important for test
** add a comment why it's not using the include snippet in cases where the
explicit parsing is part of hte test
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5166) PostingsHighlighter fails with IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/LUCENE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741188#comment-13741188 ] ASF subversion and git services commented on LUCENE-5166: - Commit 1514379 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1514379 ] LUCENE-5166: also fix and test this case where tf 1 within the passage for a term PostingsHighlighter fails with IndexOutOfBoundsException Key: LUCENE-5166 URL: https://issues.apache.org/jira/browse/LUCENE-5166 Project: Lucene - Core Issue Type: Bug Components: modules/highlighter Affects Versions: 4.4 Reporter: Manuel Amoabeng Fix For: 5.0, 4.5 Attachments: LUCENE-5166-2.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166-revisited.patch Given a document with a match at a startIndex PostingsHighlighter.maxlength and an endIndex PostingsHighlighter.maxLength, DefaultPassageFormatter will throw an IndexOutOfBoundsException when DefaultPassageFormatter.append() is invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5166) PostingsHighlighter fails with IndexOutOfBoundsException
[ https://issues.apache.org/jira/browse/LUCENE-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved LUCENE-5166. - Resolution: Fixed Thank you again! PostingsHighlighter fails with IndexOutOfBoundsException Key: LUCENE-5166 URL: https://issues.apache.org/jira/browse/LUCENE-5166 Project: Lucene - Core Issue Type: Bug Components: modules/highlighter Affects Versions: 4.4 Reporter: Manuel Amoabeng Fix For: 5.0, 4.5 Attachments: LUCENE-5166-2.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166.patch, LUCENE-5166-revisited.patch Given a document with a match at a startIndex PostingsHighlighter.maxlength and an endIndex PostingsHighlighter.maxLength, DefaultPassageFormatter will throw an IndexOutOfBoundsException when DefaultPassageFormatter.append() is invoked. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5161) Distributed MoreLikeThis has problems when q.op=AND
Shawn Heisey created SOLR-5161: -- Summary: Distributed MoreLikeThis has problems when q.op=AND Key: SOLR-5161 URL: https://issues.apache.org/jira/browse/SOLR-5161 Project: Solr Issue Type: Bug Components: MoreLikeThis Affects Versions: 4.2.1 Reporter: Shawn Heisey Fix For: 4.5, 5.0 Distributed MoreLikeThis seems to result in regular distributed queries using the termvectors found in the example document. If q.op=AND in the initial request, then the distributed queries with the termvectors will ALSO have q.op=AND, which for my tests, has resulted in no matches. If I send the MLT request to a handler using edismax, or include q.op=OR, then it works as expected. It's very slow, which I'll bring up in another issue. Should it remove that parameter or assign it to OR? Should it ultimately pay attention to the mlt.operator mentioned in SOLR-5160, which is not implemented at this time? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5122) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/SOLR-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741262#comment-13741262 ] ASF subversion and git services commented on SOLR-5122: --- Commit 1514402 from hoss...@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1514402 ] SOLR-5122: Fixed bug in spellcheck.collateMaxCollectDocs. Eliminates risk of divide by zero, and makes estimated hit counts meaningful in non-optimized indexes. spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero Key: SOLR-5122 URL: https://issues.apache.org/jira/browse/SOLR-5122 Project: Solr Issue Type: Bug Affects Versions: 4.4 Reporter: Hoss Man Assignee: James Dyer Attachments: SOLR-5122.patch, SOLR-5122.patch, SOLR-5122.patch As part of SOLR-4952 SpellCheckCollatorTest started using RandomMergePolicy, and this (aparently) led to a failure in testEstimatedHitCounts. As far as i can tell: the test assumes that specific values would be returned as the _estimated_ hits for a colleation, and it appears that the change in MergePolicy however resulted in different segments with different term stats, causing the estimation code to produce different values then what is expected. I made a quick attempt to improve the test to: * expect explicit exact values only when spellcheck.collateMaxCollectDocs is set such that the estimate' should actually be exact (ie: collateMaxCollectDocs == 0 or collateMaxCollectDocs greater then the num docs in the index * randomize the values used for collateMaxCollectDocs and confirm that the estimates are never more then the num docs in the index This lead to an odd ArithmeticException: / by zero error in the test, which seems to suggest that there is a genuine bug in the code for estimating the hits that only gets tickled in certain mergepolicy/segment/collateMaxCollectDocs combinations. *Update:* This appears to be a general problem with collecting docs out of order and the estimation of hits -- i believe even if there is no divide by zero error, the estimates are largely meaningless since the docs are collected out of order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5122) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/SOLR-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741270#comment-13741270 ] ASF subversion and git services commented on SOLR-5122: --- Commit 1514408 from hoss...@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1514408 ] SOLR-5122: Fixed bug in spellcheck.collateMaxCollectDocs. Eliminates risk of divide by zero, and makes estimated hit counts meaningful in non-optimized indexes. (merge r1514402) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero Key: SOLR-5122 URL: https://issues.apache.org/jira/browse/SOLR-5122 Project: Solr Issue Type: Bug Affects Versions: 4.4 Reporter: Hoss Man Assignee: James Dyer Attachments: SOLR-5122.patch, SOLR-5122.patch, SOLR-5122.patch As part of SOLR-4952 SpellCheckCollatorTest started using RandomMergePolicy, and this (aparently) led to a failure in testEstimatedHitCounts. As far as i can tell: the test assumes that specific values would be returned as the _estimated_ hits for a colleation, and it appears that the change in MergePolicy however resulted in different segments with different term stats, causing the estimation code to produce different values then what is expected. I made a quick attempt to improve the test to: * expect explicit exact values only when spellcheck.collateMaxCollectDocs is set such that the estimate' should actually be exact (ie: collateMaxCollectDocs == 0 or collateMaxCollectDocs greater then the num docs in the index * randomize the values used for collateMaxCollectDocs and confirm that the estimates are never more then the num docs in the index This lead to an odd ArithmeticException: / by zero error in the test, which seems to suggest that there is a genuine bug in the code for estimating the hits that only gets tickled in certain mergepolicy/segment/collateMaxCollectDocs combinations. *Update:* This appears to be a general problem with collecting docs out of order and the estimation of hits -- i believe even if there is no divide by zero error, the estimates are largely meaningless since the docs are collected out of order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4952) audit test configs to use solrconfig.snippet.randomindexconfig.xml in more tests
[
https://issues.apache.org/jira/browse/SOLR-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741263#comment-13741263
]
ASF subversion and git services commented on SOLR-4952:
---
Commit 1514404 from hoss...@apache.org in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1514404 ]
SOLR-4952: SolrCmdDistributorTest needs to force MergePolicy for consistent
optimize() behavior (merge r1514377)
audit test configs to use solrconfig.snippet.randomindexconfig.xml in more
tests
Key: SOLR-4952
URL: https://issues.apache.org/jira/browse/SOLR-4952
Project: Solr
Issue Type: Sub-task
Reporter: Hoss Man
Assignee: Hoss Man
Fix For: 4.5, 5.0
in SOLR-4942 i updated every solrconfig.xml to either...
* include solrconfig.snippet.randomindexconfig.xml where it was easy to do so
* use the useCompoundFile sys prop if it already had an {{indexConfig}}
section, or if including the snippet wasn't going to be easy (ie: contrib
tests)
As an improvment on this:
* audit all core configs not already using
solrconfig.snippet.randomindexconfig.xml and either:
** make them use it, ignoring any previously unimportant explicit
incdexConfig settings
** make them use it, using explicit sys props to overwrite random values in
cases were explicit indexConfig values are important for test
** add a comment why it's not using the include snippet in cases where the
explicit parsing is part of hte test
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5122) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/SOLR-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man resolved SOLR-5122. Resolution: Fixed Fix Version/s: 5.0 4.5 Assignee: Hoss Man (was: James Dyer) Committed revision 1514402. Committed revision 1514408. spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero Key: SOLR-5122 URL: https://issues.apache.org/jira/browse/SOLR-5122 Project: Solr Issue Type: Bug Affects Versions: 4.4 Reporter: Hoss Man Assignee: Hoss Man Fix For: 4.5, 5.0 Attachments: SOLR-5122.patch, SOLR-5122.patch, SOLR-5122.patch As part of SOLR-4952 SpellCheckCollatorTest started using RandomMergePolicy, and this (aparently) led to a failure in testEstimatedHitCounts. As far as i can tell: the test assumes that specific values would be returned as the _estimated_ hits for a colleation, and it appears that the change in MergePolicy however resulted in different segments with different term stats, causing the estimation code to produce different values then what is expected. I made a quick attempt to improve the test to: * expect explicit exact values only when spellcheck.collateMaxCollectDocs is set such that the estimate' should actually be exact (ie: collateMaxCollectDocs == 0 or collateMaxCollectDocs greater then the num docs in the index * randomize the values used for collateMaxCollectDocs and confirm that the estimates are never more then the num docs in the index This lead to an odd ArithmeticException: / by zero error in the test, which seems to suggest that there is a genuine bug in the code for estimating the hits that only gets tickled in certain mergepolicy/segment/collateMaxCollectDocs combinations. *Update:* This appears to be a general problem with collecting docs out of order and the estimation of hits -- i believe even if there is no divide by zero error, the estimates are largely meaningless since the docs are collected out of order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741379#comment-13741379
]
Michael McCandless commented on LUCENE-5177:
+1 to cutover to non-weak HashMap.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1506 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1506/ All tests passed Build Log: [...truncated 28316 lines...] [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6 [javadoc] Loading source files for package org.apache.lucene... [javadoc] Loading source files for package org.apache.lucene.analysis... [javadoc] Loading source files for package org.apache.lucene.analysis.tokenattributes... [javadoc] Loading source files for package org.apache.lucene.codecs... [javadoc] Loading source files for package org.apache.lucene.codecs.compressing... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene3x... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene40... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene41... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene42... [javadoc] Loading source files for package org.apache.lucene.codecs.perfield... [javadoc] Loading source files for package org.apache.lucene.document... [javadoc] Loading source files for package org.apache.lucene.index... [javadoc] Loading source files for package org.apache.lucene.search... [javadoc] Loading source files for package org.apache.lucene.search.payloads... [javadoc] Loading source files for package org.apache.lucene.search.similarities... [javadoc] Loading source files for package org.apache.lucene.search.spans... [javadoc] Loading source files for package org.apache.lucene.store... [javadoc] Loading source files for package org.apache.lucene.util... [javadoc] Loading source files for package org.apache.lucene.util.automaton... [javadoc] Loading source files for package org.apache.lucene.util.fst... [javadoc] Loading source files for package org.apache.lucene.util.mutable... [javadoc] Loading source files for package org.apache.lucene.util.packed... [javadoc] Constructing Javadoc information... [javadoc] Standard Doclet version 1.7.0_25 [javadoc] Building tree for all the packages and classes... [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/package-summary.html... [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-2.png to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/doc-files... [javadoc] Copying file /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-1.png to directory /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/doc-files... [javadoc] Building index for all the packages and classes... [javadoc] Building index for all classes... [javadoc] Generating /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/help-doc.html... [javadoc] 1 warning [...truncated 27 lines...] [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6 [javadoc] Loading source files for package org.apache.lucene.analysis.ar... [javadoc] Loading source files for package org.apache.lucene.analysis.bg... [javadoc] Loading source files for package org.apache.lucene.analysis.br... [javadoc] Loading source files for package org.apache.lucene.analysis.ca... [javadoc] Loading source files for package org.apache.lucene.analysis.charfilter... [javadoc] Loading source files for package org.apache.lucene.analysis.cjk... [javadoc] Loading source files for package org.apache.lucene.analysis.cn... [javadoc] Loading source files for package org.apache.lucene.analysis.commongrams... [javadoc] Loading source files for package org.apache.lucene.analysis.compound... [javadoc] Loading source files for package org.apache.lucene.analysis.compound.hyphenation... [javadoc] Loading source files for package org.apache.lucene.analysis.core... [javadoc] Loading source files for package org.apache.lucene.analysis.cz... [javadoc] Loading source files for package org.apache.lucene.analysis.da... [javadoc] Loading source files for package org.apache.lucene.analysis.de... [javadoc] Loading source files for package org.apache.lucene.analysis.el... [javadoc] Loading source files for package org.apache.lucene.analysis.en... [javadoc] Loading source files for package org.apache.lucene.analysis.es... [javadoc] Loading source files for package org.apache.lucene.analysis.eu... [javadoc] Loading source files for package org.apache.lucene.analysis.fa... [javadoc] Loading source files for package
[jira] [Commented] (SOLR-3280) to many / sometimes stale CLOSE_WAIT connections from SnapPuller during / after replication
[ https://issues.apache.org/jira/browse/SOLR-3280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741426#comment-13741426 ] David Fu commented on SOLR-3280: I am facing the same problem. Any update on when this will be resolved? to many / sometimes stale CLOSE_WAIT connections from SnapPuller during / after replication --- Key: SOLR-3280 URL: https://issues.apache.org/jira/browse/SOLR-3280 Project: Solr Issue Type: Bug Affects Versions: 3.5, 3.6, 4.0-ALPHA Reporter: Bernd Fehling Assignee: Robert Muir Priority: Minor Attachments: SOLR-3280.patch There are sometimes to many and also stale CLOSE_WAIT connections during/after replication left over on SLAVE server. Normally GC should clean up this but this is not always the case. Also if a CLOSE_WAIT is hanging then the new replication won't load. Dirty work around so far is to fake a TCP connection as root to that connection and close it. After that the new replication will load, the old index and searcher released and the system will return to normal operation. Background: The SnapPuller is using Apache httpclient 3.x and uses the MultiThreadedHttpConnectionManager. The manager holds a connection in CLOSE_WAIT after its use for further requests. This is done by calling releaseConnection. But if a connection is stuck it is not available any more and a new connection from the pool is used. Solution: After calling releaseConnection clean up with closeIdleConnections(0). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1506 - Failure
The email notification regex didn't catch the cause of the build failure (I'll
fix the regex):
---
-ecj-javadoc-lint-src:
[ecj-lint] Compiling 617 source files
[ecj-lint] invalid Class-Path header in manifest of jar file:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/lib/org.restlet-2.1.1.jar
[ecj-lint] invalid Class-Path header in manifest of jar file:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar
[ecj-lint] --
[ecj-lint] 1. ERROR in
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/java/org/apache/solr/search/EarlyTerminatingCollectorException.java
(at line 39)
[ecj-lint] * the index when collecting the {@see #getNumberCollected()}
documents
[ecj-lint] ^^^
[ecj-lint] Javadoc: Unexpected tag
[ecj-lint] --
[ecj-lint] 1 problem (1 error)
---
On Aug 15, 2013, at 4:07 PM, Apache Jenkins Server jenk...@builds.apache.org
wrote:
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1506/
All tests passed
Build Log:
[...truncated 28316 lines...]
[javadoc] Generating Javadoc
[javadoc] Javadoc execution
[javadoc] warning: [options] bootstrap class path not set in conjunction
with -source 1.6
[javadoc] Loading source files for package org.apache.lucene...
[javadoc] Loading source files for package org.apache.lucene.analysis...
[javadoc] Loading source files for package
org.apache.lucene.analysis.tokenattributes...
[javadoc] Loading source files for package org.apache.lucene.codecs...
[javadoc] Loading source files for package
org.apache.lucene.codecs.compressing...
[javadoc] Loading source files for package
org.apache.lucene.codecs.lucene3x...
[javadoc] Loading source files for package
org.apache.lucene.codecs.lucene40...
[javadoc] Loading source files for package
org.apache.lucene.codecs.lucene41...
[javadoc] Loading source files for package
org.apache.lucene.codecs.lucene42...
[javadoc] Loading source files for package
org.apache.lucene.codecs.perfield...
[javadoc] Loading source files for package org.apache.lucene.document...
[javadoc] Loading source files for package org.apache.lucene.index...
[javadoc] Loading source files for package org.apache.lucene.search...
[javadoc] Loading source files for package
org.apache.lucene.search.payloads...
[javadoc] Loading source files for package
org.apache.lucene.search.similarities...
[javadoc] Loading source files for package org.apache.lucene.search.spans...
[javadoc] Loading source files for package org.apache.lucene.store...
[javadoc] Loading source files for package org.apache.lucene.util...
[javadoc] Loading source files for package
org.apache.lucene.util.automaton...
[javadoc] Loading source files for package org.apache.lucene.util.fst...
[javadoc] Loading source files for package org.apache.lucene.util.mutable...
[javadoc] Loading source files for package org.apache.lucene.util.packed...
[javadoc] Constructing Javadoc information...
[javadoc] Standard Doclet version 1.7.0_25
[javadoc] Building tree for all the packages and classes...
[javadoc] Generating
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/package-summary.html...
[javadoc] Copying file
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-2.png
to directory
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/doc-files...
[javadoc] Copying file
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-1.png
to directory
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/doc-files...
[javadoc] Building index for all the packages and classes...
[javadoc] Building index for all classes...
[javadoc] Generating
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/help-doc.html...
[javadoc] 1 warning
[...truncated 27 lines...]
[javadoc] Generating Javadoc
[javadoc] Javadoc execution
[javadoc] warning: [options] bootstrap class path not set in conjunction
with -source 1.6
[javadoc] Loading source files for package org.apache.lucene.analysis.ar...
[javadoc] Loading source files for package org.apache.lucene.analysis.bg...
[javadoc] Loading source files for package org.apache.lucene.analysis.br...
[javadoc] Loading source files for package org.apache.lucene.analysis.ca...
[javadoc] Loading source files for package
org.apache.lucene.analysis.charfilter...
[javadoc] Loading source files for package org.apache.lucene.analysis.cjk...
[javadoc] Loading source
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741469#comment-13741469
]
Hoss Man commented on LUCENE-5177:
--
I understand the value of the improved testing -- but is there any tangible
benefit to users to converting the WeakHashMap - HashMap?
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1506 - Failure
: [ecj-lint] * the index when collecting the {@see #getNumberCollected()}
documents
Bah ... Sorry, fix in progress.
: On Aug 15, 2013, at 4:07 PM, Apache Jenkins Server
jenk...@builds.apache.org wrote:
:
: Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1506/
:
: All tests passed
:
: Build Log:
: [...truncated 28316 lines...]
: [javadoc] Generating Javadoc
: [javadoc] Javadoc execution
: [javadoc] warning: [options] bootstrap class path not set in conjunction
with -source 1.6
: [javadoc] Loading source files for package org.apache.lucene...
: [javadoc] Loading source files for package org.apache.lucene.analysis...
: [javadoc] Loading source files for package
org.apache.lucene.analysis.tokenattributes...
: [javadoc] Loading source files for package org.apache.lucene.codecs...
: [javadoc] Loading source files for package
org.apache.lucene.codecs.compressing...
: [javadoc] Loading source files for package
org.apache.lucene.codecs.lucene3x...
: [javadoc] Loading source files for package
org.apache.lucene.codecs.lucene40...
: [javadoc] Loading source files for package
org.apache.lucene.codecs.lucene41...
: [javadoc] Loading source files for package
org.apache.lucene.codecs.lucene42...
: [javadoc] Loading source files for package
org.apache.lucene.codecs.perfield...
: [javadoc] Loading source files for package org.apache.lucene.document...
: [javadoc] Loading source files for package org.apache.lucene.index...
: [javadoc] Loading source files for package org.apache.lucene.search...
: [javadoc] Loading source files for package
org.apache.lucene.search.payloads...
: [javadoc] Loading source files for package
org.apache.lucene.search.similarities...
: [javadoc] Loading source files for package
org.apache.lucene.search.spans...
: [javadoc] Loading source files for package org.apache.lucene.store...
: [javadoc] Loading source files for package org.apache.lucene.util...
: [javadoc] Loading source files for package
org.apache.lucene.util.automaton...
: [javadoc] Loading source files for package org.apache.lucene.util.fst...
: [javadoc] Loading source files for package
org.apache.lucene.util.mutable...
: [javadoc] Loading source files for package org.apache.lucene.util.packed...
: [javadoc] Constructing Javadoc information...
: [javadoc] Standard Doclet version 1.7.0_25
: [javadoc] Building tree for all the packages and classes...
: [javadoc] Generating
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/package-summary.html...
: [javadoc] Copying file
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-2.png
to directory
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/doc-files...
: [javadoc] Copying file
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-1.png
to directory
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/org/apache/lucene/search/doc-files...
: [javadoc] Building index for all the packages and classes...
: [javadoc] Building index for all classes...
: [javadoc] Generating
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/docs/core/help-doc.html...
: [javadoc] 1 warning
:
: [...truncated 27 lines...]
: [javadoc] Generating Javadoc
: [javadoc] Javadoc execution
: [javadoc] warning: [options] bootstrap class path not set in conjunction
with -source 1.6
: [javadoc] Loading source files for package org.apache.lucene.analysis.ar...
: [javadoc] Loading source files for package org.apache.lucene.analysis.bg...
: [javadoc] Loading source files for package org.apache.lucene.analysis.br...
: [javadoc] Loading source files for package org.apache.lucene.analysis.ca...
: [javadoc] Loading source files for package
org.apache.lucene.analysis.charfilter...
: [javadoc] Loading source files for package
org.apache.lucene.analysis.cjk...
: [javadoc] Loading source files for package org.apache.lucene.analysis.cn...
: [javadoc] Loading source files for package
org.apache.lucene.analysis.commongrams...
: [javadoc] Loading source files for package
org.apache.lucene.analysis.compound...
: [javadoc] Loading source files for package
org.apache.lucene.analysis.compound.hyphenation...
: [javadoc] Loading source files for package
org.apache.lucene.analysis.core...
: [javadoc] Loading source files for package org.apache.lucene.analysis.cz...
: [javadoc] Loading source files for package org.apache.lucene.analysis.da...
: [javadoc] Loading source files for package org.apache.lucene.analysis.de...
: [javadoc] Loading source files for package org.apache.lucene.analysis.el...
: [javadoc] Loading
[jira] [Created] (SOLR-5162) Implicit core properties are no longer available
Alan Woodward created SOLR-5162: --- Summary: Implicit core properties are no longer available Key: SOLR-5162 URL: https://issues.apache.org/jira/browse/SOLR-5162 Project: Solr Issue Type: Bug Affects Versions: 4.5 Reporter: Alan Woodward Assignee: Alan Woodward Priority: Minor Fix For: 4.5 As noted by ~elyograg on IRC, implicit property substitution doesn't work any more in 4.5. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5122) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/SOLR-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741477#comment-13741477 ] ASF subversion and git services commented on SOLR-5122: --- Commit 1514494 from hoss...@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1514494 ] SOLR-5122: fix javadocs spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero Key: SOLR-5122 URL: https://issues.apache.org/jira/browse/SOLR-5122 Project: Solr Issue Type: Bug Affects Versions: 4.4 Reporter: Hoss Man Assignee: Hoss Man Fix For: 4.5, 5.0 Attachments: SOLR-5122.patch, SOLR-5122.patch, SOLR-5122.patch As part of SOLR-4952 SpellCheckCollatorTest started using RandomMergePolicy, and this (aparently) led to a failure in testEstimatedHitCounts. As far as i can tell: the test assumes that specific values would be returned as the _estimated_ hits for a colleation, and it appears that the change in MergePolicy however resulted in different segments with different term stats, causing the estimation code to produce different values then what is expected. I made a quick attempt to improve the test to: * expect explicit exact values only when spellcheck.collateMaxCollectDocs is set such that the estimate' should actually be exact (ie: collateMaxCollectDocs == 0 or collateMaxCollectDocs greater then the num docs in the index * randomize the values used for collateMaxCollectDocs and confirm that the estimates are never more then the num docs in the index This lead to an odd ArithmeticException: / by zero error in the test, which seems to suggest that there is a genuine bug in the code for estimating the hits that only gets tickled in certain mergepolicy/segment/collateMaxCollectDocs combinations. *Update:* This appears to be a general problem with collecting docs out of order and the estimation of hits -- i believe even if there is no divide by zero error, the estimates are largely meaningless since the docs are collected out of order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5162) Implicit core properties are no longer available
[ https://issues.apache.org/jira/browse/SOLR-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741482#comment-13741482 ] Alan Woodward commented on SOLR-5162: - I managed to chop off the 'solr.' prefix of all these substitutions in SOLR-4914. The tricksy part here is that these prefixes are not present in the core.properties file, so we need to add them in/strip them out at the appropriate times. The only call where the 'solr.' prefix needs to be present is in the Properties object returned by CoreDescriptor.getCoreProperties(), and this method is only ever called when creating ResourceLoaders, so we should be able to safely isolate the changes here. Implicit core properties are no longer available Key: SOLR-5162 URL: https://issues.apache.org/jira/browse/SOLR-5162 Project: Solr Issue Type: Bug Affects Versions: 4.5 Reporter: Alan Woodward Assignee: Alan Woodward Priority: Minor Fix For: 4.5 As noted by ~elyograg on IRC, implicit property substitution doesn't work any more in 4.5. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5122) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/SOLR-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741477#comment-13741477 ] ASF subversion and git services commented on SOLR-5122: --- Commit 1514494 from hoss...@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1514494 ] SOLR-5122: fix javadocs spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero Key: SOLR-5122 URL: https://issues.apache.org/jira/browse/SOLR-5122 Project: Solr Issue Type: Bug Affects Versions: 4.4 Reporter: Hoss Man Assignee: Hoss Man Fix For: 4.5, 5.0 Attachments: SOLR-5122.patch, SOLR-5122.patch, SOLR-5122.patch As part of SOLR-4952 SpellCheckCollatorTest started using RandomMergePolicy, and this (aparently) led to a failure in testEstimatedHitCounts. As far as i can tell: the test assumes that specific values would be returned as the _estimated_ hits for a colleation, and it appears that the change in MergePolicy however resulted in different segments with different term stats, causing the estimation code to produce different values then what is expected. I made a quick attempt to improve the test to: * expect explicit exact values only when spellcheck.collateMaxCollectDocs is set such that the estimate' should actually be exact (ie: collateMaxCollectDocs == 0 or collateMaxCollectDocs greater then the num docs in the index * randomize the values used for collateMaxCollectDocs and confirm that the estimates are never more then the num docs in the index This lead to an odd ArithmeticException: / by zero error in the test, which seems to suggest that there is a genuine bug in the code for estimating the hits that only gets tickled in certain mergepolicy/segment/collateMaxCollectDocs combinations. *Update:* This appears to be a general problem with collecting docs out of order and the estimation of hits -- i believe even if there is no divide by zero error, the estimates are largely meaningless since the docs are collected out of order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741491#comment-13741491
]
Robert Muir commented on LUCENE-5177:
-
there need not be tangible benefits to users for us to make a change.
we can make a change because its refactoring our codebase and making it cleaner
or many other reasons.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741500#comment-13741500
]
Hoss Man commented on LUCENE-5177:
--
bq. its refactoring our codebase and making it cleaner or many other reasons.
How does this do that?
I'm not trying to be confrontational, i'm genuinely not understanding what is
improved by switching away from a WeakHashMap and i just want to make sure i'm
not missunderstanding something about the big picture.
(If you proposed to get rid of the Map completely and have the Caches hang
directly off the readers (something i remember discussing a LNG time ago
that people seemed to think was a good idea but no one seemd to have bandwidth
for) then i could totally understand arguments that doing so would be making
the codebase cleaner -- but i'm not understanding what's clearner/better about
using a global static HashMap instead of a WeakHashMap.)
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-trunk-Java7 - Build # 4237 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java7/4237/ All tests passed Build Log: [...truncated 32721 lines...] BUILD FAILED /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:389: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/build.xml:60: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/solr/build.xml:583: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1657: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-Java7/lucene/common-build.xml:1691: Compile failed; see the compiler error output for details. Total time: 76 minutes 10 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5122) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero
[ https://issues.apache.org/jira/browse/SOLR-5122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741506#comment-13741506 ] ASF subversion and git services commented on SOLR-5122: --- Commit 1514504 from hoss...@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1514504 ] SOLR-5122: fix javadocs (merge r1514494) spellcheck.collateMaxCollectDocs estimates seem to be meaninless -- can lead to ArithmeticException: / by zero Key: SOLR-5122 URL: https://issues.apache.org/jira/browse/SOLR-5122 Project: Solr Issue Type: Bug Affects Versions: 4.4 Reporter: Hoss Man Assignee: Hoss Man Fix For: 4.5, 5.0 Attachments: SOLR-5122.patch, SOLR-5122.patch, SOLR-5122.patch As part of SOLR-4952 SpellCheckCollatorTest started using RandomMergePolicy, and this (aparently) led to a failure in testEstimatedHitCounts. As far as i can tell: the test assumes that specific values would be returned as the _estimated_ hits for a colleation, and it appears that the change in MergePolicy however resulted in different segments with different term stats, causing the estimation code to produce different values then what is expected. I made a quick attempt to improve the test to: * expect explicit exact values only when spellcheck.collateMaxCollectDocs is set such that the estimate' should actually be exact (ie: collateMaxCollectDocs == 0 or collateMaxCollectDocs greater then the num docs in the index * randomize the values used for collateMaxCollectDocs and confirm that the estimates are never more then the num docs in the index This lead to an odd ArithmeticException: / by zero error in the test, which seems to suggest that there is a genuine bug in the code for estimating the hits that only gets tickled in certain mergepolicy/segment/collateMaxCollectDocs combinations. *Update:* This appears to be a general problem with collecting docs out of order and the estimation of hits -- i believe even if there is no divide by zero error, the estimates are largely meaningless since the docs are collected out of order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5162) Implicit core properties are no longer available
[ https://issues.apache.org/jira/browse/SOLR-5162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Woodward updated SOLR-5162: Attachment: SOLR-5162.patch Quick patch, which should solve the problem. I need to think of a decent way of testing it, though - is there a reasonably non-invasive solrconfig parameter I can use here? Implicit core properties are no longer available Key: SOLR-5162 URL: https://issues.apache.org/jira/browse/SOLR-5162 Project: Solr Issue Type: Bug Affects Versions: 4.5 Reporter: Alan Woodward Assignee: Alan Woodward Priority: Minor Fix For: 4.5 Attachments: SOLR-5162.patch As noted by ~elyograg on IRC, implicit property substitution doesn't work any more in 4.5. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741514#comment-13741514
]
Robert Muir commented on LUCENE-5177:
-
I don't think we should hang fieldcaches directly off readers.
Fieldcache is a broken design: users who want to sort/facet on a field should
index their content correctly with docvalues instead.
Its fine for it to be wrapped underneath an UninvertingFilterReader, that also
takes a MapString,DocValuesType up front though, but by no means should we
have this broken shit tightly coupled with stuff in lucene/core
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.6.0_45) - Build # 6944 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6944/ Java: 32bit/jdk1.6.0_45 -client -XX:+UseParallelGC 3 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.impl.CloudSolrServerTest Error Message: 2 threads leaked from SUITE scope at org.apache.solr.client.solrj.impl.CloudSolrServerTest: 1) Thread[id=15, name=TEST-CloudSolrServerTest.testDistribSearch-seed#[9B250AF16AC5B6E4]-SendThread(localhost.localdomain:58694), state=TIMED_WAITING, group=TGRP-CloudSolrServerTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) 2) Thread[id=16, name=TEST-CloudSolrServerTest.testDistribSearch-seed#[9B250AF16AC5B6E4]-EventThread, state=WAITING, group=TGRP-CloudSolrServerTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.client.solrj.impl.CloudSolrServerTest: 1) Thread[id=15, name=TEST-CloudSolrServerTest.testDistribSearch-seed#[9B250AF16AC5B6E4]-SendThread(localhost.localdomain:58694), state=TIMED_WAITING, group=TGRP-CloudSolrServerTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) 2) Thread[id=16, name=TEST-CloudSolrServerTest.testDistribSearch-seed#[9B250AF16AC5B6E4]-EventThread, state=WAITING, group=TGRP-CloudSolrServerTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491) at __randomizedtesting.SeedInfo.seed([9B250AF16AC5B6E4]:0) FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.impl.CloudSolrServerTest Error Message: There are still zombie threads that couldn't be terminated:1) Thread[id=15, name=TEST-CloudSolrServerTest.testDistribSearch-seed#[9B250AF16AC5B6E4]-SendThread(localhost.localdomain:58694), state=TIMED_WAITING, group=TGRP-CloudSolrServerTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated: 1) Thread[id=15, name=TEST-CloudSolrServerTest.testDistribSearch-seed#[9B250AF16AC5B6E4]-SendThread(localhost.localdomain:58694), state=TIMED_WAITING, group=TGRP-CloudSolrServerTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993) at __randomizedtesting.SeedInfo.seed([9B250AF16AC5B6E4]:0) REGRESSION: org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch Error Message: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:58694 within 3 ms Stack Trace: java.lang.RuntimeException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:58694 within 3 ms at __randomizedtesting.SeedInfo.seed([9B250AF16AC5B6E4:1AC384E91D9AD6D8]:0) at org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:130) at org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:93) at org.apache.solr.common.cloud.SolrZkClient.init(SolrZkClient.java:84) at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:89) at org.apache.solr.cloud.AbstractZkTestCase.buildZooKeeper(AbstractZkTestCase.java:83) at
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741547#comment-13741547
]
Hoss Man commented on LUCENE-5177:
--
bq. by no means should we have this broken shit tightly coupled with stuff in
lucene/core
ok fine, fieldcaches as a concept is fundementally broken, and the suggestion
of hanging the caches of index readers is stupid in this day and age of
docvalues -- forget it, then. It was simply an aside comment.
Can you (or anyone else) please help me understand why keeping these broken
caches in global static HashMaps is cleaner/better then keeping them in global
static WeakHashMaps?
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5163) edismax gobbles exceptions for unrelated issues
Steven Bower created SOLR-5163: -- Summary: edismax gobbles exceptions for unrelated issues Key: SOLR-5163 URL: https://issues.apache.org/jira/browse/SOLR-5163 Project: Solr Issue Type: Bug Reporter: Steven Bower query: q=foo AND bar qf=field1 qf=field2 defType=edismax Where field1 exists and field2 doesn't.. will treat the AND as a term vs and operator -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4956) make maxBufferedAddsPerServer configurable
[ https://issues.apache.org/jira/browse/SOLR-4956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741556#comment-13741556 ] Otis Gospodnetic commented on SOLR-4956: +1 for what [~gpendleb] wrote. Let me be the master of my own destiny. make maxBufferedAddsPerServer configurable -- Key: SOLR-4956 URL: https://issues.apache.org/jira/browse/SOLR-4956 Project: Solr Issue Type: Improvement Affects Versions: 4.3, 5.0 Reporter: Erick Erickson Anecdotal user's list evidence indicates that in high-throughput situations, the default of 10 docs/batch for inter-shard batching can generate significant CPU load. See the thread titled Sharding and Replication on June 19th, but the gist is below. I haven't poked around, but it's a little surprising on the surface that Asif is seeing this kind of difference. So I'm wondering if this change indicates some other underlying issue. Regardless, this seems like it would be good to investigate. Here's the gist of Asif's experience from the thread: Its a completely practical problem - we are exploring Solr to build a real time analytics/data solution for a system handling about 1000 qps. We have various metrics that are stored as different collections on the cloud, which means very high amount of writes. The cloud also needs to support about 300-400 qps. We initially tested with a single Solr node on a 16 core / 24 GB box for a single metric. We saw that writes were not a issue at all - Solr was handling it extremely well. We were also able to achieve about 200 qps from a single node. When we set up the cloud ( a ensemble on 6 boxes), we saw very high CPU usage on the replicas. Up to 10 cores were getting used for writes on the replicas. Hence my concern with respect to batch updates for the replicas. BTW, I altered the maxBufferedAddsPerServer to 1000 - and now CPU usage is very similar to single node installation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741555#comment-13741555
]
Yonik Seeley commented on LUCENE-5177:
--
bq. ok fine, fieldcaches as a concept is fundementally broken, and the
suggestion of hanging the caches of index readers is stupid in this day and age
of docvalues
Meh... I happen to disagree with both of those assertions.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741560#comment-13741560
]
Robert Muir commented on LUCENE-5177:
-
Thats fine, users who want to have slow reopen times and inefficient use of
memory can use an FilterReader that uninverts and exposes stuff via the
AtomicReader docValues APIs.
Such a FilterReader is useful as a transition mechanism anyway: users could
pass it to addIndexes to 'upgrade' to docvalues.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741580#comment-13741580
]
Yonik Seeley commented on LUCENE-5177:
--
bq. users who want to have slow reopen times and inefficient use of memory can
use an FilterReader that uninverts and exposes stuff via the AtomicReader
docValues APIs
As long as it's not slower / more inefficient than the field cache stuff we
have today, that's fine. Just a different API to access it.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5164) Can not create a collection via collections API
Erick Erickson created SOLR-5164: Summary: Can not create a collection via collections API Key: SOLR-5164 URL: https://issues.apache.org/jira/browse/SOLR-5164 Project: Solr Issue Type: Bug Affects Versions: 4.5, 5.0 Reporter: Erick Erickson Assignee: Erick Erickson Priority: Blocker When you try to create a collection in SolrCloud, the instanceDir that gets created has an extra solr in it which messes up the pathing for all the lib directives in solrconfig.xml as they're all relative. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741583#comment-13741583
]
Yonik Seeley commented on LUCENE-5177:
--
Hoss - wrt WeakHashMap, I guess if it's not needed, lookups could be very
slightly faster, debugging maybe slightly easier (if you're looking at weak
references on the heap for example), and the code easier to understand (since
we are not in fact relying on a weak reference to clean anything up anymore).
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5175) Add parameter to lower-bound TF normalization for BM25 (for long documents)
[ https://issues.apache.org/jira/browse/LUCENE-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741582#comment-13741582 ] Tom Burton-West commented on LUCENE-5175: - Hi Robert, I tried running luceneutils with the default wikimedium10m collection and tasks. I ran it first on the DefaultSimilarity, which shouldn't be affected by the patch to BM25Similarity and it showed about -2.3% difference. I'm guessing there is some inaccuracy in the tests. When I changed DEFAULT_SIMILARITY to BM25Similarity, the worst change was a difference of -8.8%. Is there a separate mailing list for questions about luceneutils or should I write to the java-dev list? or directly to Mike or you? Tom Add parameter to lower-bound TF normalization for BM25 (for long documents) --- Key: LUCENE-5175 URL: https://issues.apache.org/jira/browse/LUCENE-5175 Project: Lucene - Core Issue Type: Improvement Components: core/search Reporter: Tom Burton-West Priority: Minor Attachments: LUCENE-5175.patch In the article When Documents Are Very Long, BM25 Fails! a fix for the problem is documented. There was a TODO note in BM25Similarity to add this fix. I will attach a patch that implements the fix shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741586#comment-13741586
]
Robert Muir commented on LUCENE-5177:
-
Yes, i dont think we should make it slower. but with such a filterreader, it
could be implemented cleaner/differently more easily, because people can access
it thru DV apis rather than thru FieldCache.DEFAULT.xxx
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5175) Add parameter to lower-bound TF normalization for BM25 (for long documents)
[
https://issues.apache.org/jira/browse/LUCENE-5175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741595#comment-13741595
]
Robert Muir commented on LUCENE-5175:
-
Tom: there is some variation on the test.
in my localconstants.py i have the following which I found reduces variation
significantly:
{noformat}
JAVA_COMMAND = 'java -Xms4g -Xmx4g -server'
SEARCH_NUM_THREADS = 1
{noformat}
As far as questions, just send them to the dev@lucene list i think? a lot of
committers use this thing so its probably your best bet.
Add parameter to lower-bound TF normalization for BM25 (for long documents)
---
Key: LUCENE-5175
URL: https://issues.apache.org/jira/browse/LUCENE-5175
Project: Lucene - Core
Issue Type: Improvement
Components: core/search
Reporter: Tom Burton-West
Priority: Minor
Attachments: LUCENE-5175.patch
In the article When Documents Are Very Long, BM25 Fails! a fix for the
problem is documented. There was a TODO note in BM25Similarity to add this
fix. I will attach a patch that implements the fix shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5164) Can not create a collection via collections API
[
https://issues.apache.org/jira/browse/SOLR-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741601#comment-13741601
]
Shawn Heisey commented on SOLR-5164:
In a cloud (or manually sharded) environment, IMHO it's better to put jars in
${solr.solr.home}/lib because they'll be automatically loaded at startup and
available to all cores.
Because we can't assume symlink support, it's really difficult to use this
method with the example.
Can not create a collection via collections API
---
Key: SOLR-5164
URL: https://issues.apache.org/jira/browse/SOLR-5164
Project: Solr
Issue Type: Bug
Affects Versions: 4.5, 5.0
Reporter: Erick Erickson
Assignee: Erick Erickson
Priority: Blocker
When you try to create a collection in SolrCloud, the instanceDir that gets
created has an extra solr in it which messes up the pathing for all the
lib directives in solrconfig.xml as they're all relative.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-5164) Can not create a collection via collections API
[
https://issues.apache.org/jira/browse/SOLR-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741601#comment-13741601
]
Shawn Heisey edited comment on SOLR-5164 at 8/15/13 10:39 PM:
--
In a cloud (or manually sharded) environment, IMHO it's better to put jars in
$solrhome/lib because they'll be automatically loaded at startup and available
to all cores.
Because we can't assume symlink support, it's really difficult to use this
method with the example.
was (Author: elyograg):
In a cloud (or manually sharded) environment, IMHO it's better to put jars
in ${solr.solr.home}/lib because they'll be automatically loaded at startup and
available to all cores.
Because we can't assume symlink support, it's really difficult to use this
method with the example.
Can not create a collection via collections API
---
Key: SOLR-5164
URL: https://issues.apache.org/jira/browse/SOLR-5164
Project: Solr
Issue Type: Bug
Affects Versions: 4.5, 5.0
Reporter: Erick Erickson
Assignee: Erick Erickson
Priority: Blocker
When you try to create a collection in SolrCloud, the instanceDir that gets
created has an extra solr in it which messes up the pathing for all the
lib directives in solrconfig.xml as they're all relative.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5152) Lucene FST is not immutable
[ https://issues.apache.org/jira/browse/LUCENE-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741604#comment-13741604 ] ASF subversion and git services commented on LUCENE-5152: - Commit 1514520 from [~mikemccand] in branch 'dev/trunk' [ https://svn.apache.org/r1514520 ] LUCENE-5152: make assert less costly Lucene FST is not immutable --- Key: LUCENE-5152 URL: https://issues.apache.org/jira/browse/LUCENE-5152 Project: Lucene - Core Issue Type: Bug Components: core/FSTs Affects Versions: 4.4 Reporter: Simon Willnauer Priority: Blocker Fix For: 5.0, 4.5 Attachments: LUCENE-5152.patch, LUCENE-5152.patch, LUCENE-5152.patch, LUCENE-5152.patch a spinnoff from LUCENE-5120 where the analyzing suggester modified a returned output from and FST (BytesRef) which caused sideffects in later execution. I added an assertion into the FST that checks if a cached root arc is modified and in-fact this happens for instance in our MemoryPostingsFormat and I bet we find more places. We need to think about how to make this less trappy since it can cause bugs that are super hard to find. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5152) Lucene FST is not immutable
[ https://issues.apache.org/jira/browse/LUCENE-5152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741609#comment-13741609 ] ASF subversion and git services commented on LUCENE-5152: - Commit 1514521 from [~mikemccand] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1514521 ] LUCENE-5152: make assert less costly Lucene FST is not immutable --- Key: LUCENE-5152 URL: https://issues.apache.org/jira/browse/LUCENE-5152 Project: Lucene - Core Issue Type: Bug Components: core/FSTs Affects Versions: 4.4 Reporter: Simon Willnauer Priority: Blocker Fix For: 5.0, 4.5 Attachments: LUCENE-5152.patch, LUCENE-5152.patch, LUCENE-5152.patch, LUCENE-5152.patch a spinnoff from LUCENE-5120 where the analyzing suggester modified a returned output from and FST (BytesRef) which caused sideffects in later execution. I added an assertion into the FST that checks if a cached root arc is modified and in-fact this happens for instance in our MemoryPostingsFormat and I bet we find more places. We need to think about how to make this less trappy since it can cause bugs that are super hard to find. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741611#comment-13741611
]
Shawn Heisey commented on LUCENE-5177:
--
How is Solr affected by the idea of not using fieldcache? From what I
understand, docValues have the same data that would go into a stored field, not
the indexed field ... so they might not behave exactly the same when it comes
to sorting. Although it doesn't make any sense to sort on a fully tokenized
field, it can make sense to sort (or facet) on a field that uses the keyword
tokenizer, the lowercase filter, and the trim filter. I don't think that's
possible with docValues.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741613#comment-13741613
]
Robert Muir commented on LUCENE-5177:
-
Thats all solr issues, its not related here.
DocValues fields take a byte[]. you can put whatever you want in there, it
doesnt have to be the same as what goes in a stored field, you can run it thru
an analysis chain yourself in some solr document processor or something like
that if you really want to do that... you don't need indexwriter's help.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741622#comment-13741622
]
Yonik Seeley commented on LUCENE-5177:
--
bq. How is Solr affected by the idea of not using fieldcache?
AFAIK, fieldcache-type functionality isn't going away, and any Lucene API
changes (like FieldCache.DEFAULT, etc) will be hidden by Solr.
DocValues adds additional functionality, and does not take away from anything
we already have.
There are currently one or two disadvantages to using docvalues in solr
currently. The major disadvantage is being forced to specify a default value
in Solr, thus making them impossible to use from dynamic fields. Ideally we'd
be able to specify a missing value (i.e. the value used when there is no
value for that document... or rather the default at the DocValues layer), and
that would allow us to remove the currently mandated default value at the Solr
layer.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741631#comment-13741631
]
Robert Muir commented on LUCENE-5177:
-
But really this should just be an option on the solr fieldtype, you know when
the value is missing, solr just puts that missing value in the dv field for
that document (nowhere else like stored fields or anything like that).
same with if you want the fieldtype to run stuff thru an analyzer or anything
like that. I dont think this stuff really has to do with lucene, it can just be
options in solrs update process/fieldtypes.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.7.0) - Build # 715 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/715/ Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseG1GC All tests passed Build Log: [...truncated 28349 lines...] [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6 [javadoc] Loading source files for package org.apache.lucene... [javadoc] Loading source files for package org.apache.lucene.analysis... [javadoc] Loading source files for package org.apache.lucene.analysis.tokenattributes... [javadoc] Loading source files for package org.apache.lucene.codecs... [javadoc] Loading source files for package org.apache.lucene.codecs.compressing... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene3x... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene40... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene41... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene42... [javadoc] Loading source files for package org.apache.lucene.codecs.perfield... [javadoc] Loading source files for package org.apache.lucene.document... [javadoc] Loading source files for package org.apache.lucene.index... [javadoc] Loading source files for package org.apache.lucene.search... [javadoc] Loading source files for package org.apache.lucene.search.payloads... [javadoc] Loading source files for package org.apache.lucene.search.similarities... [javadoc] Loading source files for package org.apache.lucene.search.spans... [javadoc] Loading source files for package org.apache.lucene.store... [javadoc] Loading source files for package org.apache.lucene.util... [javadoc] Loading source files for package org.apache.lucene.util.automaton... [javadoc] Loading source files for package org.apache.lucene.util.fst... [javadoc] Loading source files for package org.apache.lucene.util.mutable... [javadoc] Loading source files for package org.apache.lucene.util.packed... [javadoc] Constructing Javadoc information... [javadoc] Standard Doclet version 1.7.0_25 [javadoc] Building tree for all the packages and classes... [javadoc] Generating /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/docs/core/org/apache/lucene/search/package-summary.html... [javadoc] Copying file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-1.png to directory /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/docs/core/org/apache/lucene/search/doc-files... [javadoc] Copying file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/core/src/java/org/apache/lucene/search/doc-files/nrq-formula-2.png to directory /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/docs/core/org/apache/lucene/search/doc-files... [javadoc] Building index for all the packages and classes... [javadoc] Building index for all classes... [javadoc] Generating /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-4.x-MacOSX/lucene/build/docs/core/help-doc.html... [javadoc] 1 warning [...truncated 27 lines...] [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.6 [javadoc] Loading source files for package org.apache.lucene.analysis.ar... [javadoc] Loading source files for package org.apache.lucene.analysis.bg... [javadoc] Loading source files for package org.apache.lucene.analysis.br... [javadoc] Loading source files for package org.apache.lucene.analysis.ca... [javadoc] Loading source files for package org.apache.lucene.analysis.charfilter... [javadoc] Loading source files for package org.apache.lucene.analysis.cjk... [javadoc] Loading source files for package org.apache.lucene.analysis.cn... [javadoc] Loading source files for package org.apache.lucene.analysis.commongrams... [javadoc] Loading source files for package org.apache.lucene.analysis.compound... [javadoc] Loading source files for package org.apache.lucene.analysis.compound.hyphenation... [javadoc] Loading source files for package org.apache.lucene.analysis.core... [javadoc] Loading source files for package org.apache.lucene.analysis.cz... [javadoc] Loading source files for package org.apache.lucene.analysis.da... [javadoc] Loading source files for package org.apache.lucene.analysis.de... [javadoc] Loading source files for package org.apache.lucene.analysis.el... [javadoc] Loading source files for package org.apache.lucene.analysis.en... [javadoc] Loading source files for package org.apache.lucene.analysis.es... [javadoc] Loading source files for package org.apache.lucene.analysis.eu... [javadoc] Loading source files for package org.apache.lucene.analysis.fa... [javadoc] Loading source files for package
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741703#comment-13741703
]
Yonik Seeley commented on LUCENE-5177:
--
bq. But really this should just be an option on the solr fieldtype, you know
when the value is missing, solr just puts that missing value in the dv field
for that document.
But doing it at the solr level means that you can't define a field without
using it.
Defining a non-docvalues field in solr and not using it currently incurs no
overhead. This isn't the case with docvalues, and I don't believe docvalues
allows one to currently specify a default (it always defaults to 0 for missing
values?)
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741707#comment-13741707
]
Robert Muir commented on LUCENE-5177:
-
why would have fields defined you arent using? I dont understnad the use case
here.
and to say there is no overhead with fieldcache is wrong: it makes huge arrays
even if one document has the field. so I'm really missing something: in both
cases its a column-stride field, just one is built at index-time.
I dont understand why docvalues needs to allow you to specify a default, when
you can just specify your own for the document with missing value (if you
specify 0, its no different than if it fills that for you).
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5164) Can not create a collection via collections API (cloud mode)
[ https://issues.apache.org/jira/browse/SOLR-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-5164: - Summary: Can not create a collection via collections API (cloud mode) (was: Can not create a collection via collections API) Can not create a collection via collections API (cloud mode) Key: SOLR-5164 URL: https://issues.apache.org/jira/browse/SOLR-5164 Project: Solr Issue Type: Bug Affects Versions: 4.5, 5.0 Reporter: Erick Erickson Assignee: Erick Erickson Priority: Blocker When you try to create a collection in SolrCloud, the instanceDir that gets created has an extra solr in it which messes up the pathing for all the lib directives in solrconfig.xml as they're all relative. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741714#comment-13741714
]
Yonik Seeley commented on LUCENE-5177:
--
bq. why would have fields defined you arent using?
Why not? Other fields take no resources if you don't use them, but docvalues do.
Dynamic fields represent an almost infinite number of fields you aren't using
until you do.
BTW, this is why the only uses of docvalues in the example schema are commented
out. Who want's to incur index overhead for fields you aren't even using?
If we want anything other than 0 for a value for missing, we must do it in
DocValues.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741715#comment-13741715
]
Robert Muir commented on LUCENE-5177:
-
Thats so wrong: if you want a value other than 0 (say 20) you just set the
missing value yourself by setting it in the o.a.l.Document you add to
indexwriter.
There is absolutely no reason for indexwriter to do things for you that you can
do yourself.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Assigned] (SOLR-3936) QueryElevationComponent: Wrong order when result grouping is activated
[
https://issues.apache.org/jira/browse/SOLR-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man reassigned SOLR-3936:
--
Assignee: Hoss Man
QueryElevationComponent: Wrong order when result grouping is activated
--
Key: SOLR-3936
URL: https://issues.apache.org/jira/browse/SOLR-3936
Project: Solr
Issue Type: Bug
Components: SearchComponents - other
Affects Versions: 4.0
Reporter: Michael Berger
Assignee: Hoss Man
Attachments: SOLR-3936.patch, SOLR-3936.patch
When I use elevation together with grouping I got not the expected result
order.
I tried it with the standard solr example:
http://localhost:8983/solr/elevate?enableElevation=truefl=score%2C[elevated]%2Cid%2CnameforceElevation=truegroup.field=manugroup=onindent=onq=ipodwt=json
but the results ignored the elevation:
{
responseHeader:{
status:0,
QTime:2,
params:{
enableElevation:true,
fl:score,[elevated],id,name,
indent:on,
q:ipod,
forceElevation:true,
group.field:manu,
group:on,
wt:json}},
grouped:{
manu:{
matches:2,
groups:[{
groupValue:belkin,
doclist:{numFound:1,start:0,maxScore:0.7698604,docs:[
{
id:F8V7067-APL-KIT,
name:Belkin Mobile Power Cord for iPod w/ Dock,
score:0.7698604,
[elevated]:false}]
}},
{
groupValue:inc,
doclist:{numFound:1,start:0,maxScore:0.28869766,docs:[
{
id:MA147LL/A,
name:Apple 60 GB iPod with Video Playback Black,
score:0.28869766,
[elevated]:true}]
}}]}}}
the elevate.xml defines the following rules :
query text=ipod
doc id=MA147LL/A / !-- put the actual ipod at the top --
doc id=IW-02 exclude=true / !-- exclude this cable --
/query
/elevate
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3936) QueryElevationComponent: Wrong order when result grouping is activated
[
https://issues.apache.org/jira/browse/SOLR-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hoss Man updated SOLR-3936:
---
Attachment: SOLR-3936.patch
[~mgarski]: Thanks for tracking this down and providing a patch, and
esspecially thanks for including tests!
Everything you have looks great -- i've just updated your patch to include
additional permutations (forceElevate, group.sort, etc...)
I'll try to commit this tomorrow (running it through full regression now to
make sure it doesn't break any other tests unexpectedly)
QueryElevationComponent: Wrong order when result grouping is activated
--
Key: SOLR-3936
URL: https://issues.apache.org/jira/browse/SOLR-3936
Project: Solr
Issue Type: Bug
Components: SearchComponents - other
Affects Versions: 4.0
Reporter: Michael Berger
Attachments: SOLR-3936.patch, SOLR-3936.patch
When I use elevation together with grouping I got not the expected result
order.
I tried it with the standard solr example:
http://localhost:8983/solr/elevate?enableElevation=truefl=score%2C[elevated]%2Cid%2CnameforceElevation=truegroup.field=manugroup=onindent=onq=ipodwt=json
but the results ignored the elevation:
{
responseHeader:{
status:0,
QTime:2,
params:{
enableElevation:true,
fl:score,[elevated],id,name,
indent:on,
q:ipod,
forceElevation:true,
group.field:manu,
group:on,
wt:json}},
grouped:{
manu:{
matches:2,
groups:[{
groupValue:belkin,
doclist:{numFound:1,start:0,maxScore:0.7698604,docs:[
{
id:F8V7067-APL-KIT,
name:Belkin Mobile Power Cord for iPod w/ Dock,
score:0.7698604,
[elevated]:false}]
}},
{
groupValue:inc,
doclist:{numFound:1,start:0,maxScore:0.28869766,docs:[
{
id:MA147LL/A,
name:Apple 60 GB iPod with Video Playback Black,
score:0.28869766,
[elevated]:true}]
}}]}}}
the elevate.xml defines the following rules :
query text=ipod
doc id=MA147LL/A / !-- put the actual ipod at the top --
doc id=IW-02 exclude=true / !-- exclude this cable --
/query
/elevate
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741728#comment-13741728
]
Robert Muir commented on LUCENE-5177:
-
By the way: just to explain how it works in FieldCache when you supply these
sort missing/first/last stuff.
The way that works is with a separate fieldcache field (a bitset) which marks
documents that have a value.
So its really a separate fieldcache'd boolean[maxdoc] telling you if there is
anything there or not for the original field.
You can easily do the exact same parallel with docvalues yourself (e.g. in a
solr fieldtype) if you want to support those options (a numericdvfield only 1
or 0, takes 1 bit per document, same situation).
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3936) QueryElevationComponent: Wrong order when result grouping is activated
[
https://issues.apache.org/jira/browse/SOLR-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741729#comment-13741729
]
Michael Garski commented on SOLR-3936:
--
Great! Thanks [~hossman]!
QueryElevationComponent: Wrong order when result grouping is activated
--
Key: SOLR-3936
URL: https://issues.apache.org/jira/browse/SOLR-3936
Project: Solr
Issue Type: Bug
Components: SearchComponents - other
Affects Versions: 4.0
Reporter: Michael Berger
Assignee: Hoss Man
Attachments: SOLR-3936.patch, SOLR-3936.patch
When I use elevation together with grouping I got not the expected result
order.
I tried it with the standard solr example:
http://localhost:8983/solr/elevate?enableElevation=truefl=score%2C[elevated]%2Cid%2CnameforceElevation=truegroup.field=manugroup=onindent=onq=ipodwt=json
but the results ignored the elevation:
{
responseHeader:{
status:0,
QTime:2,
params:{
enableElevation:true,
fl:score,[elevated],id,name,
indent:on,
q:ipod,
forceElevation:true,
group.field:manu,
group:on,
wt:json}},
grouped:{
manu:{
matches:2,
groups:[{
groupValue:belkin,
doclist:{numFound:1,start:0,maxScore:0.7698604,docs:[
{
id:F8V7067-APL-KIT,
name:Belkin Mobile Power Cord for iPod w/ Dock,
score:0.7698604,
[elevated]:false}]
}},
{
groupValue:inc,
doclist:{numFound:1,start:0,maxScore:0.28869766,docs:[
{
id:MA147LL/A,
name:Apple 60 GB iPod with Video Playback Black,
score:0.28869766,
[elevated]:true}]
}}]}}}
the elevate.xml defines the following rules :
query text=ipod
doc id=MA147LL/A / !-- put the actual ipod at the top --
doc id=IW-02 exclude=true / !-- exclude this cable --
/query
/elevate
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4734) FastVectorHighlighter Overlapping Proximity Queries Do Not Highlight
[
https://issues.apache.org/jira/browse/LUCENE-4734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741735#comment-13741735
]
Jack Krupansky commented on LUCENE-4734:
Looking at some highlighter code, I see this constructor in
org.apache.lucene.search.vectorhighlight.FieldPhraseList.java of branch_4x:
{code}
/**
* a constructor.
*
* @param fieldTermStack FieldTermStack object
* @param fieldQuery FieldQuery object
* @param phraseLimit maximum size of phraseList
*/
public FieldPhraseList( FieldTermStack fieldTermStack, FieldQuery fieldQuery,
int phraseLimit ){
final String field = fieldTermStack.getFieldName();
QueryPhraseMap qpm = fieldQuery.getRootMap(field);
if (qpm != null) {
LinkedListTermInfo phraseCandidate = new LinkedListTermInfo();
extractPhrases(fieldTermStack.termList, qpm, phraseCandidate, 0);
assert phraseCandidate.size() == 0;
}
}
{code}
Clearly phraseLimit is no longer used. Is it being deprecated, or is this
simply work in progress that will use it again eventually?
This parameter is passed over several layers of code, ultimately it is set up
in Solr using the hl.phraseLimit parameter.
Seems like a dead parameter that should be cleaned up now or deprecated for
future cleanup, but I can't say that I have been able to follow all of the work
that has transpired in the highlighters.
The change occurred in Revision 1505732 (related to this Jira.) Before then,
this parameter was used.
Comments? Or should this be a separate Jira issue?
FastVectorHighlighter Overlapping Proximity Queries Do Not Highlight
Key: LUCENE-4734
URL: https://issues.apache.org/jira/browse/LUCENE-4734
Project: Lucene - Core
Issue Type: Bug
Components: modules/highlighter
Affects Versions: 4.0, 4.1, 5.0
Reporter: Ryan Lauck
Assignee: Adrien Grand
Labels: fastvectorhighlighter, highlighter
Fix For: 5.0, 4.5
Attachments: LUCENE-4734-2.patch, lucene-4734.patch, LUCENE-4734.patch
If a proximity phrase query overlaps with any other query term it will not be
highlighted.
Example Text: A B C D E F G
Example Queries:
B E~10 D
(D will be highlighted instead of B C D E)
B E~10 C F~10
(nothing will be highlighted)
This can be traced to the FieldPhraseList constructor's inner while loop.
From the first example query, the first TermInfo popped off the stack will be
B. The second TermInfo will be D which will not be found in the submap
for B E~10 and will trigger a failed match.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741738#comment-13741738
]
Yonik Seeley commented on LUCENE-5177:
--
bq. Thats so wrong: if you want a value other than 0 (say 20) you just set the
missing value yourself by setting it in the o.a.l.Document you add to
indexwriter.
And how exactly could we do that with dynamic fields?
Face it - if we want any other defaults than 0 (for numerics) it needs to
somehow be configurable.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741747#comment-13741747
]
Robert Muir commented on LUCENE-5177:
-
It doesnt need to be configurable. its not configurable in fieldcache today!
It has a separate bitset cached on fieldcache indicating 1 or 0.
so you just do the same thing with docvalues, as I explained earlier.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5164) Can not create a collection via collections API (cloud mode)
[ https://issues.apache.org/jira/browse/SOLR-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-5164: - Attachment: SOLR-5164.patch [~romseygeek] Well, this has to be the most time I've spent changing about 4 lines. If you can please take a glance at it. These seem fairly safe changes, they both just bring back what was in the code a while ago. Besides the solr/solr issue, when creating a properties file in Cloud mode, it appears that the parent directory hasn't been created first, so that step was failing. If you don't apply it (and backport it to 4.4) I'll do it in the morning. But you have a 5 hour head start G... Can not create a collection via collections API (cloud mode) Key: SOLR-5164 URL: https://issues.apache.org/jira/browse/SOLR-5164 Project: Solr Issue Type: Bug Affects Versions: 4.5, 5.0 Reporter: Erick Erickson Assignee: Erick Erickson Priority: Blocker Attachments: SOLR-5164.patch When you try to create a collection in SolrCloud, the instanceDir that gets created has an extra solr in it which messes up the pathing for all the lib directives in solrconfig.xml as they're all relative. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-5178) doc values should allow configurable defaults
Yonik Seeley created LUCENE-5178: Summary: doc values should allow configurable defaults Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5177) remove fieldcache weakmap or at least see what relies on GC for purging today
[
https://issues.apache.org/jira/browse/LUCENE-5177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741770#comment-13741770
]
Yonik Seeley commented on LUCENE-5177:
--
I've opened LUCENE-5178 for this. It's not strictly related to the FieldCache.
remove fieldcache weakmap or at least see what relies on GC for purging today
-
Key: LUCENE-5177
URL: https://issues.apache.org/jira/browse/LUCENE-5177
Project: Lucene - Core
Issue Type: Test
Reporter: Robert Muir
Attachments: LUCENE-5177.patch
If we are registering close listeners why does this need to be weak?
But really i dont care about that, here is what i said to Hoss on the solr
mailing list:
{quote}
(In any case: it looks like a WeakHashMap is still used in case the
listeners never get called, correct?)
I think it might be the other way around: i think it was weakmap
before always, the close listeners were then added sometime in 3.x
series, so we registered purge events as an optimization.
But one way to look at it is: readers should really get closed, so why
have the weak map and not just a regular hashmap.
Even if we want to keep the weak map (seriously i dont care, and i
dont want to be the guy fielding complaints on this), I'm going to
open with an issue with a patch that removes it and fails tests in
@afterclass if there is any entries. This way its totally clear
if/when/where anything is relying on GC today here and we can at
least look at that.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-5178) doc values should allow configurable defaults
[ https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley updated LUCENE-5178: - Description: DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. was: DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. doc values should allow configurable defaults - Key: LUCENE-5178 URL: https://issues.apache.org/jira/browse/LUCENE-5178 Project: Lucene - Core Issue Type: Improvement Reporter: Yonik Seeley DocValues should somehow allow a configurable default per-field. Possible implementations include setting it on the field in the document or registration of an IndexWriter callback. If we don't make the default configurable, then another option is to have DocValues fields keep track of whether a value was indexed for that document or not. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5164) Can not create a collection via collections API (cloud mode)
[ https://issues.apache.org/jira/browse/SOLR-5164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741783#comment-13741783 ] Erick Erickson commented on SOLR-5164: -- [~elyograg] It's not a matter of putting the jars in the right place at all. The core was just being created in the wrong place so the relative pathing from the stock solrconfig.xml couldn't work. The Carrotsearch classes that couldn't be found are part of the stock solrconfig distribution Can not create a collection via collections API (cloud mode) Key: SOLR-5164 URL: https://issues.apache.org/jira/browse/SOLR-5164 Project: Solr Issue Type: Bug Affects Versions: 4.5, 5.0 Reporter: Erick Erickson Assignee: Erick Erickson Priority: Blocker Attachments: SOLR-5164.patch When you try to create a collection in SolrCloud, the instanceDir that gets created has an extra solr in it which messes up the pathing for all the lib directives in solrconfig.xml as they're all relative. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should allow configurable defaults
[
https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741786#comment-13741786
]
Robert Muir commented on LUCENE-5178:
-
I dont think lucene needs to do anything here. As i explained on LUCENE-5177
(but you refused to listen), you just have a separate numericdocvalues field
just like fieldcache does (it has a separate bitset).
So you write a 1 there, when the document has a value for the field. Otherwise
it will contain a 0 (because it gets filled with that).
This is very easy to do and will use ~ 1 bit per document just like fieldcache.
Then its just a matter of making it easy to pass this NumericDV to
FIeldComparator (currently: it always pulls a Bits directly from FC). this can
be something like:
{code}
if (ndv instanceof Bits) { // codec already specializes here
docsWithField = (Bits) ndv;
} else {
docsWithField = new Bits() {
boolean get(int index) {
return ndv.get(index) != 0;
}
}
{code}
doc values should allow configurable defaults
-
Key: LUCENE-5178
URL: https://issues.apache.org/jira/browse/LUCENE-5178
Project: Lucene - Core
Issue Type: Improvement
Reporter: Yonik Seeley
DocValues should somehow allow a configurable default per-field.
Possible implementations include setting it on the field in the document or
registration of an IndexWriter callback.
If we don't make the default configurable, then another option is to have
DocValues fields keep track of whether a value was indexed for that document
or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5178) doc values should allow configurable defaults
[
https://issues.apache.org/jira/browse/LUCENE-5178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13741786#comment-13741786
]
Robert Muir commented on LUCENE-5178:
-
I dont think lucene needs to do anything here. As i explained on LUCENE-5177
(but you refused to listen), you just have a separate numericdocvalues field
just like fieldcache does (it has a separate bitset).
So you write a 1 there, when the document has a value for the field. Otherwise
it will contain a 0 (because it gets filled with that).
This is very easy to do and will use ~ 1 bit per document just like fieldcache.
Then its just a matter of making it easy to pass this NumericDV to
FIeldComparator (currently: it always pulls a Bits directly from FC). this can
be something like:
{code}
if (ndv instanceof Bits) { // codec already specializes here
docsWithField = (Bits) ndv;
} else {
docsWithField = new Bits() {
boolean get(int index) {
return ndv.get(index) != 0;
}
}
{code}
doc values should allow configurable defaults
-
Key: LUCENE-5178
URL: https://issues.apache.org/jira/browse/LUCENE-5178
Project: Lucene - Core
Issue Type: Improvement
Reporter: Yonik Seeley
DocValues should somehow allow a configurable default per-field.
Possible implementations include setting it on the field in the document or
registration of an IndexWriter callback.
If we don't make the default configurable, then another option is to have
DocValues fields keep track of whether a value was indexed for that document
or not.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
