[jira] [Commented] (SOLR-3177) Excluding tagged filter in StatsComponent
[
https://issues.apache.org/jira/browse/SOLR-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835247#comment-13835247
]
Vitaly Lazutin commented on SOLR-3177:
--
Grouping is not the best solution.
If you will remove fq with this field, then your facet results will be wrong.
Facets on other fields will return more results, then actually need.
Excluding tagged filter in StatsComponent
-
Key: SOLR-3177
URL: https://issues.apache.org/jira/browse/SOLR-3177
Project: Solr
Issue Type: Improvement
Components: SearchComponents - other
Affects Versions: 3.5, 3.6, 4.0-ALPHA, 4.1
Reporter: Mathias H.
Priority: Minor
Labels: localparams, stats, statscomponent
Attachments: SOLR-3177.patch
It would be useful to exclude the effects of some fq params from the set of
documents used to compute stats -- similar to
how you can exclude tagged filters when generating facet counts...
https://wiki.apache.org/solr/SimpleFacetParameters#Tagging_and_excluding_Filters
So that it's possible to do something like this...
http://localhost:8983/solr/select?fq={!tag=priceFilter}price:[1 TO
20]q=*:*stats=truestats.field={!ex=priceFilter}price
If you want to create a price slider this is very useful because then you can
filter the price ([1 TO 20) and nevertheless get the lower and upper bound of
the unfiltered price (min=0, max=100):
{noformat}
|-[---]--|
$0 $1 $20$100
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5351) DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
[
https://issues.apache.org/jira/browse/LUCENE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835259#comment-13835259
]
Daniel Collins commented on LUCENE-5351:
Hmm, I have noticed this error when using RAMDirectoryFactory, I often get an
AlreadyClosedException when I shutdown my instance that uses that Will try
to see if this is the same issue.
DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
-
Key: LUCENE-5351
URL: https://issues.apache.org/jira/browse/LUCENE-5351
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Affects Versions: 4.6
Reporter: Simon Willnauer
Assignee: Simon Willnauer
Fix For: 5.0, 4.7
in StandartDirectoryReader#doClose we do this:
{noformat}
if (writer != null) {
// Since we just closed, writer may now be able to
// delete unused files:
writer.deletePendingFiles();
}
{noformat}
which can throw AlreadyClosedException from the directory if the Direcotory
has already closed. To me this looks like a bug and we should catch this
exception from the directory.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Patch proposal - LanguageIdentifierUpdateProcessor uses only firstValue() on multivalued fields
Hello list. After discussing the thread LanguageIdentifierUpdateProcessor uses only firstValue() on multivalued fields on solr-user, I like to propose a patch to add the following feature: LanguageIdentifierUpdateProcessor should use all (String) values of a multivalued field for language detection. By now, the LUP imlicitely only retieves the first-value of a multivalued field. This leads to omitting any other values of such field. Furthermore, if for some reason, the first-value is not a String but following values would be Strings, there's no language detection at all for such a multi-valued field. I propose this patch here, following your contribution guidelines. It is unclear to me if this scenario was just overlooked or if this was a conscious design decission. So, let me hear what you think of this feature. Maybe you are already working on it. If not, I'm eager to file my (probably first) feature request and patch on JIRA. I have a working trunk checkout in IDEA setup on OSX and ant clean install claims SUCCESS. Looking forward to hear from you! Regards, Stephan - srm - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-MacOSX (64bit/jdk1.7.0) - Build # 1046 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-MacOSX/1046/ Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.solr.search.TestCustomSort.testSortableBinary Error Message: java.lang.UnsupportedOperationException: this codec cannot write docvalues Stack Trace: org.apache.solr.common.SolrException: java.lang.UnsupportedOperationException: this codec cannot write docvalues at __randomizedtesting.SeedInfo.seed([B922FFD2A4DAAD3:571767E81ABE3016]:0) at org.apache.solr.util.TestHarness.update(TestHarness.java:240) at org.apache.solr.util.BaseTestHarness.checkUpdateStatus(BaseTestHarness.java:261) at org.apache.solr.util.BaseTestHarness.validateUpdate(BaseTestHarness.java:231) at org.apache.solr.SolrTestCaseJ4.checkUpdateU(SolrTestCaseJ4.java:595) at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:574) at org.apache.solr.SolrTestCaseJ4.assertU(SolrTestCaseJ4.java:568) at org.apache.solr.search.TestCustomSort.testSortableBinary(TestCustomSort.java:51) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) at
[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica
[ https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835368#comment-13835368 ] Rafał Kuć commented on SOLR-4260: - Happened to me two, collection with two four shards, each having a single replica. The replicas were our of sync. Inconsistent numDocs between leader and replica --- Key: SOLR-4260 URL: https://issues.apache.org/jira/browse/SOLR-4260 Project: Solr Issue Type: Bug Components: SolrCloud Environment: 5.0.0.2013.01.04.15.31.51 Reporter: Markus Jelsma Assignee: Mark Miller Priority: Critical Fix For: 5.0, 4.7 Attachments: 192.168.20.102-replica1.png, 192.168.20.104-replica2.png, clusterstate.png After wiping all cores and reindexing some 3.3 million docs from Nutch using CloudSolrServer we see inconsistencies between the leader and replica for some shards. Each core hold about 3.3k documents. For some reason 5 out of 10 shards have a small deviation in then number of documents. The leader and slave deviate for roughly 10-20 documents, not more. Results hopping ranks in the result set for identical queries got my attention, there were small IDF differences for exactly the same record causing a record to shift positions in the result set. During those tests no records were indexed. Consecutive catch all queries also return different number of numDocs. We're running a 10 node test cluster with 10 shards and a replication factor of two and frequently reindex using a fresh build from trunk. I've not seen this issue for quite some time until a few days ago. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5354) Distributed sort is broken with CUSTOM FieldType
[
https://issues.apache.org/jira/browse/SOLR-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835397#comment-13835397
]
ASF subversion and git services commented on SOLR-5354:
---
Commit 1546571 from [~rcmuir] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1546571 ]
SOLR-5354: don't try to write docvalues with 3.x codec in these tests
Distributed sort is broken with CUSTOM FieldType
Key: SOLR-5354
URL: https://issues.apache.org/jira/browse/SOLR-5354
Project: Solr
Issue Type: Bug
Components: SearchComponents - other
Affects Versions: 4.4, 4.5, 5.0
Reporter: Jessica Cheng
Assignee: Steve Rowe
Labels: custom, query, sort
Fix For: 5.0, 4.7
Attachments: SOLR-5354.patch, SOLR-5354.patch, SOLR-5354.patch,
SOLR-5354.patch
We added a custom field type to allow an indexed binary field type that
supports search (exact match), prefix search, and sort as unsigned bytes
lexicographical compare. For sort, BytesRef's UTF8SortedAsUnicodeComparator
accomplishes what we want, and even though the name of the comparator
mentions UTF8, it doesn't actually assume so and just does byte-level
operation, so it's good. However, when we do this across different nodes, we
run into an issue where in QueryComponent.doFieldSortValues:
// Must do the same conversion when sorting by a
// String field in Lucene, which returns the terms
// data as BytesRef:
if (val instanceof BytesRef) {
UnicodeUtil.UTF8toUTF16((BytesRef)val, spare);
field.setStringValue(spare.toString());
val = ft.toObject(field);
}
UnicodeUtil.UTF8toUTF16 is called on our byte array,which isn't actually
UTF8. I did a hack where I specified our own field comparator to be
ByteBuffer based to get around that instanceof check, but then the field
value gets transformed into BYTEARR in JavaBinCodec, and when it's
unmarshalled, it gets turned into byte[]. Then, in QueryComponent.mergeIds, a
ShardFieldSortedHitQueue is constructed with ShardDoc.getCachedComparator,
which decides to give me comparatorNatural in the else of the TODO for
CUSTOM, which barfs because byte[] are not Comparable...
From Chris Hostetter:
I'm not very familiar with the distributed sorting code, but based on your
comments, and a quick skim of the functions you pointed to, it definitely
seems like there are two problems here for people trying to implement
custom sorting in custom FieldTypes...
1) QueryComponent.doFieldSortValues - this definitely seems like it should
be based on the FieldType, not an instanceof BytesRef check (oddly: the
comment event suggestsion that it should be using the FieldType's
indexedToReadable() method -- but it doesn't do that. If it did, then
this part of hte logic should work for you as long as your custom
FieldType implemented indexedToReadable in a sane way.
2) QueryComponent.mergeIds - that TODO definitely looks like a gap that
needs filled. I'm guessing the sanest thing to do in the CUSTOM case
would be to ask the FieldComparatorSource (which should be coming from the
SortField that the custom FieldType produced) to create a FieldComparator
(via newComparator - the numHits sortPos could be anything) and then
wrap that up in a Comparator facade that delegates to
FieldComparator.compareValues
That way a custom FieldType could be in complete control of the sort
comparisons (even when merging ids).
...But as i said: i may be missing something, i'm not super familia with
that code. Please try it out and let us know if thta works -- either way
please open a Jira pointing out the problems trying to implement
distributed sorting in a custom FieldType.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: svn commit: r1546571 - in /lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr: handler/component/DistributedQueryComponentCustomSortTest.java search/TestCustomSort.java
Thanks Robert!
On Nov 29, 2013 9:37 AM, rm...@apache.org wrote:
Author: rmuir
Date: Fri Nov 29 14:37:01 2013
New Revision: 1546571
URL: http://svn.apache.org/r1546571
Log:
SOLR-5354: don't try to write docvalues with 3.x codec in these tests
Modified:
lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/handler/component/DistributedQueryComponentCustomSortTest.java
lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/search/TestCustomSort.java
Modified:
lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/handler/component/DistributedQueryComponentCustomSortTest.java
URL:
http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/handler/component/DistributedQueryComponentCustomSortTest.java?rev=1546571r1=1546570r2=1546571view=diff
==
---
lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/handler/component/DistributedQueryComponentCustomSortTest.java
(original)
+++
lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/handler/component/DistributedQueryComponentCustomSortTest.java
Fri Nov 29 14:37:01 2013
@@ -17,6 +17,7 @@ package org.apache.solr.handler.componen
* limitations under the License.
*/
+import org.apache.lucene.util.LuceneTestCase.SuppressCodecs;
import org.apache.solr.BaseDistributedSearchTestCase;
import org.apache.solr.client.solrj.response.QueryResponse;
import org.apache.solr.common.SolrDocument;
@@ -30,6 +31,7 @@ import java.nio.ByteBuffer;
*
* @see org.apache.solr.handler.component.QueryComponent
*/
+@SuppressCodecs({Lucene3x})
public class DistributedQueryComponentCustomSortTest extends
BaseDistributedSearchTestCase {
public DistributedQueryComponentCustomSortTest() {
Modified:
lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/search/TestCustomSort.java
URL:
http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/search/TestCustomSort.java?rev=1546571r1=1546570r2=1546571view=diff
==
---
lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/search/TestCustomSort.java
(original)
+++
lucene/dev/branches/branch_4x/solr/core/src/test/org/apache/solr/search/TestCustomSort.java
Fri Nov 29 14:37:01 2013
@@ -17,6 +17,7 @@ package org.apache.solr.search;
* limitations under the License.
*/
+import org.apache.lucene.util.LuceneTestCase.SuppressCodecs;
import org.apache.solr.SolrTestCaseJ4;
import org.junit.BeforeClass;
@@ -26,6 +27,7 @@ import java.nio.ByteBuffer;
/**
* Test SortField.CUSTOM sorts
*/
+@SuppressCodecs({Lucene3x})
public class TestCustomSort extends SolrTestCaseJ4 {
@BeforeClass
[jira] [Commented] (SOLR-5354) Distributed sort is broken with CUSTOM FieldType
[
https://issues.apache.org/jira/browse/SOLR-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835434#comment-13835434
]
ASF subversion and git services commented on SOLR-5354:
---
Commit 1546589 from [~steve_rowe] in branch 'dev/trunk'
[ https://svn.apache.org/r1546589 ]
SOLR-5354: fix attribution
Distributed sort is broken with CUSTOM FieldType
Key: SOLR-5354
URL: https://issues.apache.org/jira/browse/SOLR-5354
Project: Solr
Issue Type: Bug
Components: SearchComponents - other
Affects Versions: 4.4, 4.5, 5.0
Reporter: Jessica Cheng
Assignee: Steve Rowe
Labels: custom, query, sort
Fix For: 5.0, 4.7
Attachments: SOLR-5354.patch, SOLR-5354.patch, SOLR-5354.patch,
SOLR-5354.patch
We added a custom field type to allow an indexed binary field type that
supports search (exact match), prefix search, and sort as unsigned bytes
lexicographical compare. For sort, BytesRef's UTF8SortedAsUnicodeComparator
accomplishes what we want, and even though the name of the comparator
mentions UTF8, it doesn't actually assume so and just does byte-level
operation, so it's good. However, when we do this across different nodes, we
run into an issue where in QueryComponent.doFieldSortValues:
// Must do the same conversion when sorting by a
// String field in Lucene, which returns the terms
// data as BytesRef:
if (val instanceof BytesRef) {
UnicodeUtil.UTF8toUTF16((BytesRef)val, spare);
field.setStringValue(spare.toString());
val = ft.toObject(field);
}
UnicodeUtil.UTF8toUTF16 is called on our byte array,which isn't actually
UTF8. I did a hack where I specified our own field comparator to be
ByteBuffer based to get around that instanceof check, but then the field
value gets transformed into BYTEARR in JavaBinCodec, and when it's
unmarshalled, it gets turned into byte[]. Then, in QueryComponent.mergeIds, a
ShardFieldSortedHitQueue is constructed with ShardDoc.getCachedComparator,
which decides to give me comparatorNatural in the else of the TODO for
CUSTOM, which barfs because byte[] are not Comparable...
From Chris Hostetter:
I'm not very familiar with the distributed sorting code, but based on your
comments, and a quick skim of the functions you pointed to, it definitely
seems like there are two problems here for people trying to implement
custom sorting in custom FieldTypes...
1) QueryComponent.doFieldSortValues - this definitely seems like it should
be based on the FieldType, not an instanceof BytesRef check (oddly: the
comment event suggestsion that it should be using the FieldType's
indexedToReadable() method -- but it doesn't do that. If it did, then
this part of hte logic should work for you as long as your custom
FieldType implemented indexedToReadable in a sane way.
2) QueryComponent.mergeIds - that TODO definitely looks like a gap that
needs filled. I'm guessing the sanest thing to do in the CUSTOM case
would be to ask the FieldComparatorSource (which should be coming from the
SortField that the custom FieldType produced) to create a FieldComparator
(via newComparator - the numHits sortPos could be anything) and then
wrap that up in a Comparator facade that delegates to
FieldComparator.compareValues
That way a custom FieldType could be in complete control of the sort
comparisons (even when merging ids).
...But as i said: i may be missing something, i'm not super familia with
that code. Please try it out and let us know if thta works -- either way
please open a Jira pointing out the problems trying to implement
distributed sorting in a custom FieldType.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5354) Distributed sort is broken with CUSTOM FieldType
[
https://issues.apache.org/jira/browse/SOLR-5354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835435#comment-13835435
]
ASF subversion and git services commented on SOLR-5354:
---
Commit 1546591 from [~steve_rowe] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1546591 ]
SOLR-5354: fix attribution (merged trunk r1546589)
Distributed sort is broken with CUSTOM FieldType
Key: SOLR-5354
URL: https://issues.apache.org/jira/browse/SOLR-5354
Project: Solr
Issue Type: Bug
Components: SearchComponents - other
Affects Versions: 4.4, 4.5, 5.0
Reporter: Jessica Cheng
Assignee: Steve Rowe
Labels: custom, query, sort
Fix For: 5.0, 4.7
Attachments: SOLR-5354.patch, SOLR-5354.patch, SOLR-5354.patch,
SOLR-5354.patch
We added a custom field type to allow an indexed binary field type that
supports search (exact match), prefix search, and sort as unsigned bytes
lexicographical compare. For sort, BytesRef's UTF8SortedAsUnicodeComparator
accomplishes what we want, and even though the name of the comparator
mentions UTF8, it doesn't actually assume so and just does byte-level
operation, so it's good. However, when we do this across different nodes, we
run into an issue where in QueryComponent.doFieldSortValues:
// Must do the same conversion when sorting by a
// String field in Lucene, which returns the terms
// data as BytesRef:
if (val instanceof BytesRef) {
UnicodeUtil.UTF8toUTF16((BytesRef)val, spare);
field.setStringValue(spare.toString());
val = ft.toObject(field);
}
UnicodeUtil.UTF8toUTF16 is called on our byte array,which isn't actually
UTF8. I did a hack where I specified our own field comparator to be
ByteBuffer based to get around that instanceof check, but then the field
value gets transformed into BYTEARR in JavaBinCodec, and when it's
unmarshalled, it gets turned into byte[]. Then, in QueryComponent.mergeIds, a
ShardFieldSortedHitQueue is constructed with ShardDoc.getCachedComparator,
which decides to give me comparatorNatural in the else of the TODO for
CUSTOM, which barfs because byte[] are not Comparable...
From Chris Hostetter:
I'm not very familiar with the distributed sorting code, but based on your
comments, and a quick skim of the functions you pointed to, it definitely
seems like there are two problems here for people trying to implement
custom sorting in custom FieldTypes...
1) QueryComponent.doFieldSortValues - this definitely seems like it should
be based on the FieldType, not an instanceof BytesRef check (oddly: the
comment event suggestsion that it should be using the FieldType's
indexedToReadable() method -- but it doesn't do that. If it did, then
this part of hte logic should work for you as long as your custom
FieldType implemented indexedToReadable in a sane way.
2) QueryComponent.mergeIds - that TODO definitely looks like a gap that
needs filled. I'm guessing the sanest thing to do in the CUSTOM case
would be to ask the FieldComparatorSource (which should be coming from the
SortField that the custom FieldType produced) to create a FieldComparator
(via newComparator - the numHits sortPos could be anything) and then
wrap that up in a Comparator facade that delegates to
FieldComparator.compareValues
That way a custom FieldType could be in complete control of the sort
comparisons (even when merging ids).
...But as i said: i may be missing something, i'm not super familia with
that code. Please try it out and let us know if thta works -- either way
please open a Jira pointing out the problems trying to implement
distributed sorting in a custom FieldType.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 1072 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/1072/ Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC All tests passed Build Log: [...truncated 9849 lines...] [junit4] JVM J0: stdout was not empty, see: /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20131129_161258_469.sysout [junit4] JVM J0: stdout (verbatim) [junit4] # [junit4] # A fatal error has been detected by the Java Runtime Environment: [junit4] # [junit4] # SIGSEGV (0xb) at pc=0x00010994da2b, pid=184, tid=126983 [junit4] # [junit4] # JRE version: Java(TM) SE Runtime Environment (7.0_45-b18) (build 1.7.0_45-b18) [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.45-b08 mixed mode bsd-amd64 compressed oops) [junit4] # Problematic frame: [junit4] # C [libjava.dylib+0x9a2b] JNU_NewStringPlatform+0x1d3 [junit4] # [junit4] # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try ulimit -c unlimited before starting Java again [junit4] # [junit4] # An error report file with more information is saved as: [junit4] # /Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/hs_err_pid184.log [junit4] [thread 83975 also had an error] [junit4] [thread 46083 also had an error] [junit4] # [junit4] # If you would like to submit a bug report, please visit: [junit4] # http://bugreport.sun.com/bugreport/crash.jsp [junit4] # The crash happened outside the Java Virtual Machine in native code. [junit4] # See problematic frame for where to report the bug. [junit4] # [junit4] JVM J0: EOF [...truncated 1 lines...] [junit4] ERROR: JVM J0 ended with an exception, command line: /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/jre/bin/java -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/heapdumps -Dtests.prefix=tests -Dtests.seed=384C63980E651D23 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=5.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=1 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/temp -Dclover.db.dir=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/Users/jenkins/workspace/Lucene-Solr-trunk-MacOSX/lucene/tools/junit4/tests.policy -Dlucene.version=5.0-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.disableHdfs=true -classpath
[jira] [Commented] (SOLR-4260) Inconsistent numDocs between leader and replica
[ https://issues.apache.org/jira/browse/SOLR-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835436#comment-13835436 ] Mark Miller commented on SOLR-4260: --- What's the exact version / checkout? Inconsistent numDocs between leader and replica --- Key: SOLR-4260 URL: https://issues.apache.org/jira/browse/SOLR-4260 Project: Solr Issue Type: Bug Components: SolrCloud Environment: 5.0.0.2013.01.04.15.31.51 Reporter: Markus Jelsma Assignee: Mark Miller Priority: Critical Fix For: 5.0, 4.7 Attachments: 192.168.20.102-replica1.png, 192.168.20.104-replica2.png, clusterstate.png After wiping all cores and reindexing some 3.3 million docs from Nutch using CloudSolrServer we see inconsistencies between the leader and replica for some shards. Each core hold about 3.3k documents. For some reason 5 out of 10 shards have a small deviation in then number of documents. The leader and slave deviate for roughly 10-20 documents, not more. Results hopping ranks in the result set for identical queries got my attention, there were small IDF differences for exactly the same record causing a record to shift positions in the result set. During those tests no records were indexed. Consecutive catch all queries also return different number of numDocs. We're running a 10 node test cluster with 10 shards and a replication factor of two and frequently reindex using a fresh build from trunk. I've not seen this issue for quite some time until a few days ago. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5285) FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields
[ https://issues.apache.org/jira/browse/LUCENE-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835448#comment-13835448 ] ASF subversion and git services commented on LUCENE-5285: - Commit 1546603 from [~jpountz] in branch 'dev/trunk' [ https://svn.apache.org/r1546603 ] LUCENE-5285: Improved highlighting of multi-valued fields with FastVectorHighlighter. FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields --- Key: LUCENE-5285 URL: https://issues.apache.org/jira/browse/LUCENE-5285 Project: Lucene - Core Issue Type: Bug Reporter: Nik Everett Priority: Minor Attachments: LUCENE-5285.patch, LUCENE-5285.patch FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields. This is only a problem when you want to sort the fragments by score. Technically BaseFragmentsBuilder (line 261 in my copy of the source) does the copying. Rather than copying the score I _think_ it'd be more right to pull that copying logic into a protected method that child classes (such as ScoreOrderFragmentsBuilder) can override to do more intelligent things. Exactly what that means isn't clear to me at the moment. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5285) FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields
[ https://issues.apache.org/jira/browse/LUCENE-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835452#comment-13835452 ] ASF subversion and git services commented on LUCENE-5285: - Commit 1546607 from [~jpountz] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1546607 ] LUCENE-5285: Improved highlighting of multi-valued fields with FastVectorHighlighter. FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields --- Key: LUCENE-5285 URL: https://issues.apache.org/jira/browse/LUCENE-5285 Project: Lucene - Core Issue Type: Bug Reporter: Nik Everett Priority: Minor Attachments: LUCENE-5285.patch, LUCENE-5285.patch FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields. This is only a problem when you want to sort the fragments by score. Technically BaseFragmentsBuilder (line 261 in my copy of the source) does the copying. Rather than copying the score I _think_ it'd be more right to pull that copying logic into a protected method that child classes (such as ScoreOrderFragmentsBuilder) can override to do more intelligent things. Exactly what that means isn't clear to me at the moment. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5352) PackedInts.fastestFormatAndBits returns slower format for Direct
[ https://issues.apache.org/jira/browse/LUCENE-5352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835460#comment-13835460 ] Adrien Grand commented on LUCENE-5352: -- I ran tests as well and got similar results. It may be possible to optimize this reader (maybe reversing the bytes in the call to readLong takes substantial time on little-endian computers (just guessing)?) but taking a step back, I'm wondering that maybe this aligned format should be deprecated. As [~toke] noted last year, the in-memory impl is not faster than Packed64 on all CPUs, so if the speed-up is not clear for disk-based structures either, I think we should just deprecate it? PackedInts.fastestFormatAndBits returns slower format for Direct Key: LUCENE-5352 URL: https://issues.apache.org/jira/browse/LUCENE-5352 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir from some simple testing, DirectPacked64SingleBlockReader seems slower than DirectPackedReader (at least in a few cases i tested, more testing needed, but look at the code). this is unintuitive, because by passing e.g. FASTER you are making it slower. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-5285) FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields
[ https://issues.apache.org/jira/browse/LUCENE-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adrien Grand resolved LUCENE-5285. -- Resolution: Fixed Fix Version/s: 4.7 Assignee: Adrien Grand Committed, thanks Nik! FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields --- Key: LUCENE-5285 URL: https://issues.apache.org/jira/browse/LUCENE-5285 Project: Lucene - Core Issue Type: Bug Reporter: Nik Everett Assignee: Adrien Grand Priority: Minor Fix For: 4.7 Attachments: LUCENE-5285.patch, LUCENE-5285.patch FastVectorHighlighter copies segments scores when splitting segments across multi-valued fields. This is only a problem when you want to sort the fragments by score. Technically BaseFragmentsBuilder (line 261 in my copy of the source) does the copying. Rather than copying the score I _think_ it'd be more right to pull that copying logic into a protected method that child classes (such as ScoreOrderFragmentsBuilder) can override to do more intelligent things. Exactly what that means isn't clear to me at the moment. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5352) PackedInts.fastestFormatAndBits returns slower format for Direct
[ https://issues.apache.org/jira/browse/LUCENE-5352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835465#comment-13835465 ] Robert Muir commented on LUCENE-5352: - Well at the least I think the reader just does way more work than is necessary: its calling readLong() each time even for a low BPV. PackedInts.fastestFormatAndBits returns slower format for Direct Key: LUCENE-5352 URL: https://issues.apache.org/jira/browse/LUCENE-5352 Project: Lucene - Core Issue Type: Bug Reporter: Robert Muir from some simple testing, DirectPacked64SingleBlockReader seems slower than DirectPackedReader (at least in a few cases i tested, more testing needed, but look at the code). this is unintuitive, because by passing e.g. FASTER you are making it slower. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5473) Make one state.json per collection
[ https://issues.apache.org/jira/browse/SOLR-5473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noble Paul updated SOLR-5473: - Attachment: SOLR-5473.patch Don't run the tests yet What does the patch do? * collection create accepts an extra parameter external=true/false . default=false. * If external=true. The collection will have no entry in the clusterstate.json * The clusterstate for all such 'external' collections will be stored at /collections/collection-name/state * DocCollection#isExternal() can tell you if it is external or not * Use DocCollection#getVersion() to get the zk version of the cluterstate of that object in ZK * A node will only listen to the collections where it i a member of Make one state.json per collection -- Key: SOLR-5473 URL: https://issues.apache.org/jira/browse/SOLR-5473 Project: Solr Issue Type: Sub-task Components: SolrCloud Reporter: Noble Paul Assignee: Noble Paul Attachments: SOLR-5473.patch As defined in the parent issue, store the states of each collection under /collections/collectionname/state.json node -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5351) DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
[
https://issues.apache.org/jira/browse/LUCENE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835543#comment-13835543
]
Michael McCandless commented on LUCENE-5351:
Hmm, but, shouldn't the app not close the Directory until the reader is closed?
DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
-
Key: LUCENE-5351
URL: https://issues.apache.org/jira/browse/LUCENE-5351
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Affects Versions: 4.6
Reporter: Simon Willnauer
Assignee: Simon Willnauer
Fix For: 5.0, 4.7
in StandartDirectoryReader#doClose we do this:
{noformat}
if (writer != null) {
// Since we just closed, writer may now be able to
// delete unused files:
writer.deletePendingFiles();
}
{noformat}
which can throw AlreadyClosedException from the directory if the Direcotory
has already closed. To me this looks like a bug and we should catch this
exception from the directory.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5351) DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
[
https://issues.apache.org/jira/browse/LUCENE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835545#comment-13835545
]
Simon Willnauer commented on LUCENE-5351:
-
Well the semantics of our Directory allow to read from an already opened IR. I
don't think this necessarily wrong. The only think that I am arguing is that
IR#close() should not have these semantics I think it's really confusing.
DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
-
Key: LUCENE-5351
URL: https://issues.apache.org/jira/browse/LUCENE-5351
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Affects Versions: 4.6
Reporter: Simon Willnauer
Assignee: Simon Willnauer
Fix For: 5.0, 4.7
in StandartDirectoryReader#doClose we do this:
{noformat}
if (writer != null) {
// Since we just closed, writer may now be able to
// delete unused files:
writer.deletePendingFiles();
}
{noformat}
which can throw AlreadyClosedException from the directory if the Direcotory
has already closed. To me this looks like a bug and we should catch this
exception from the directory.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5350) Add Context Aware Suggester
[ https://issues.apache.org/jira/browse/LUCENE-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835546#comment-13835546 ] Michael McCandless commented on LUCENE-5350: I think a context aware suggester is a great idea! I wonder ... how this approach (stuff the context in front of each suggestion then build normally) compares to simply creating N separate suggesters, one per context? Like, I wonder how much better FST compression we get by using a single FST vs N? Lookup wise, it seems like we are just doing N separate lookups so that part should be similar? Add Context Aware Suggester --- Key: LUCENE-5350 URL: https://issues.apache.org/jira/browse/LUCENE-5350 Project: Lucene - Core Issue Type: New Feature Components: core/search Reporter: Areek Zillur Fix For: 5.0, 4.7 Attachments: LUCENE-5350.patch It would be nice to have a Context Aware Suggester (i.e. a suggester that could return suggestions depending on some specified context(s)). Use-cases: - location-based suggestions: -- returns suggestions which 'match' the context of a particular area --- suggest restaurants names which are in Palo Alto (context - Palo Alto) - category-based suggestions: -- returns suggestions for items that are only in certain categories/genres (contexts) --- suggest movies that are of the genre sci-fi and adventure (context - [sci-fi, adventure]) -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5351) DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
[
https://issues.apache.org/jira/browse/LUCENE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835548#comment-13835548
]
Michael McCandless commented on LUCENE-5351:
I don't think an app should close the Directory while there are any readers or
writers still open against it?
I mean, either that or ... maybe we should remove Directory.close? If you can
still use a Directory after it's closed (via an open'd IndexReader), I'm not
really sure why it has a close method ...
DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
-
Key: LUCENE-5351
URL: https://issues.apache.org/jira/browse/LUCENE-5351
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Affects Versions: 4.6
Reporter: Simon Willnauer
Assignee: Simon Willnauer
Fix For: 5.0, 4.7
in StandartDirectoryReader#doClose we do this:
{noformat}
if (writer != null) {
// Since we just closed, writer may now be able to
// delete unused files:
writer.deletePendingFiles();
}
{noformat}
which can throw AlreadyClosedException from the directory if the Direcotory
has already closed. To me this looks like a bug and we should catch this
exception from the directory.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5339) Simplify the facet module APIs
[ https://issues.apache.org/jira/browse/LUCENE-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835549#comment-13835549 ] ASF subversion and git services commented on LUCENE-5339: - Commit 1546653 from [~mikemccand] in branch 'dev/branches/lucene5339' [ https://svn.apache.org/r1546653 ] LUCENE-5339: fix some nocommits; move taxoWriter out of FacetsConfig; move search + collect utility methods to FacetsCollector Simplify the facet module APIs -- Key: LUCENE-5339 URL: https://issues.apache.org/jira/browse/LUCENE-5339 Project: Lucene - Core Issue Type: Improvement Components: modules/facet Reporter: Michael McCandless Assignee: Michael McCandless Attachments: LUCENE-5339.patch, LUCENE-5339.patch I'd like to explore simplifications to the facet module's APIs: I think the current APIs are complex, and the addition of a new feature (sparse faceting, LUCENE-5333) threatens to add even more classes (e.g., FacetRequestBuilder). I think we can do better. So, I've been prototyping some drastic changes; this is very early/exploratory and I'm not sure where it'll wind up but I think the new approach shows promise. The big changes are: * Instead of *FacetRequest/Params/Result, you directly instantiate the classes that do facet counting (currently TaxonomyFacetCounts, RangeFacetCounts or SortedSetDVFacetCounts), passing in the SimpleFacetsCollector, and then you interact with those classes to pull labels + values (topN under a path, sparse, specific labels). * At index time, no more FacetIndexingParams/CategoryListParams; instead, you make a new SimpleFacetFields and pass it the field it should store facets + drill downs under. If you want more than one CLI you create more than one instance of SimpleFacetFields. * I added a simple schema, where you state which dimensions are hierarchical or multi-valued. From this we decide how to index the ordinals (no more OrdinalPolicy). Sparse faceting is just another method (getAllDims), on both taxonomy ssdv facet classes. I haven't created a common base class / interface for all of the search-time facet classes, but I think this may be possible/clean, and perhaps useful for drill sideways. All the new classes are under oal.facet.simple.*. Lots of things that don't work yet: drill sideways, complements, associations, sampling, partitions, etc. This is just a start ... -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5351) DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
[
https://issues.apache.org/jira/browse/LUCENE-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835551#comment-13835551
]
Simon Willnauer commented on LUCENE-5351:
-
bq. I don't think an app should close the Directory while there are any readers
or writers still open against it?
You are not using the Directory you are using the IndexInput which is totally
ok we are not closing the file handles for you even if we close the directory.
I still think IR#close should not bubble up the exception here. On the other
hand removing Directory#close makes sense on a first glance
DirectoryReader#close can throw AlreadyClosedException if it's and NRT reader
-
Key: LUCENE-5351
URL: https://issues.apache.org/jira/browse/LUCENE-5351
Project: Lucene - Core
Issue Type: Bug
Components: core/index
Affects Versions: 4.6
Reporter: Simon Willnauer
Assignee: Simon Willnauer
Fix For: 5.0, 4.7
in StandartDirectoryReader#doClose we do this:
{noformat}
if (writer != null) {
// Since we just closed, writer may now be able to
// delete unused files:
writer.deletePendingFiles();
}
{noformat}
which can throw AlreadyClosedException from the directory if the Direcotory
has already closed. To me this looks like a bug and we should catch this
exception from the directory.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-5350) Add Context Aware Suggester
[ https://issues.apache.org/jira/browse/LUCENE-5350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835566#comment-13835566 ] Pradeep commented on LUCENE-5350: - There should be a framework to inject different contexts. Better to think solution around deep learning algorithms. But, this is a very good idea. Add Context Aware Suggester --- Key: LUCENE-5350 URL: https://issues.apache.org/jira/browse/LUCENE-5350 Project: Lucene - Core Issue Type: New Feature Components: core/search Reporter: Areek Zillur Fix For: 5.0, 4.7 Attachments: LUCENE-5350.patch It would be nice to have a Context Aware Suggester (i.e. a suggester that could return suggestions depending on some specified context(s)). Use-cases: - location-based suggestions: -- returns suggestions which 'match' the context of a particular area --- suggest restaurants names which are in Palo Alto (context - Palo Alto) - category-based suggestions: -- returns suggestions for items that are only in certain categories/genres (contexts) --- suggest movies that are of the genre sci-fi and adventure (context - [sci-fi, adventure]) -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5509) ChaosMonkeyNothingIsSafeTest rare fail.
[
https://issues.apache.org/jira/browse/SOLR-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835581#comment-13835581
]
ASF subversion and git services commented on SOLR-5509:
---
Commit 1546670 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1546670 ]
SOLR-5509: allow retry updates to self again, don't retry on remote solrserver
exceptions
ChaosMonkeyNothingIsSafeTest rare fail.
---
Key: SOLR-5509
URL: https://issues.apache.org/jira/browse/SOLR-5509
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 5.0, 4.7
Attachments: cmns-test-cloud-off-by1-control-2.log
{noformat}
[junit4] 2 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{add=[50086 (1452880907553734656)]} 0 142
[junit4] 2 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2} {add=[50086
(1452880908206997504)]} 0 254
[junit4] 2 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{delete=[50086 (-1452880908537298944)]} 0 2
[junit4] 2 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{delete=[50086 (-1452880908542541824)]} 0 1
[junit4] 2 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=TOLEADERwt=javabinversion=2} {add=[50086
(1452880908850823168)]} 0 1
[junit4] 2 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{add=[50086]} 0 1223
[junit4] 2 ## Only in cloudDocList: [{id=50086}]
[junit4] 2 cloudClient
:{numFound=1,start=0,docs=[SolrDocument{id=50086,
_version_=1452880908850823168}]}
h
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5509) ChaosMonkeyNothingIsSafeTest rare fail.
[
https://issues.apache.org/jira/browse/SOLR-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835582#comment-13835582
]
ASF subversion and git services commented on SOLR-5509:
---
Commit 1546672 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1546672 ]
SOLR-5509: allow retry updates to self again, don't retry on remote solrserver
exceptions
ChaosMonkeyNothingIsSafeTest rare fail.
---
Key: SOLR-5509
URL: https://issues.apache.org/jira/browse/SOLR-5509
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 5.0, 4.7
Attachments: cmns-test-cloud-off-by1-control-2.log
{noformat}
[junit4] 2 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{add=[50086 (1452880907553734656)]} 0 142
[junit4] 2 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2} {add=[50086
(1452880908206997504)]} 0 254
[junit4] 2 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{delete=[50086 (-1452880908537298944)]} 0 2
[junit4] 2 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{delete=[50086 (-1452880908542541824)]} 0 1
[junit4] 2 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=TOLEADERwt=javabinversion=2} {add=[50086
(1452880908850823168)]} 0 1
[junit4] 2 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{add=[50086]} 0 1223
[junit4] 2 ## Only in cloudDocList: [{id=50086}]
[junit4] 2 cloudClient
:{numFound=1,start=0,docs=[SolrDocument{id=50086,
_version_=1452880908850823168}]}
h
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5512) Optimize DocValuesFacets
[ https://issues.apache.org/jira/browse/SOLR-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835587#comment-13835587 ] ASF subversion and git services commented on SOLR-5512: --- Commit 1546675 from [~rcmuir] in branch 'dev/trunk' [ https://svn.apache.org/r1546675 ] SOLR-5512: Optimize DocValuesFacets Optimize DocValuesFacets - Key: SOLR-5512 URL: https://issues.apache.org/jira/browse/SOLR-5512 Project: Solr Issue Type: Improvement Reporter: Robert Muir Attachments: SOLR-5512.patch This works well in the general case (esp with huge numbers of unique values), but the SortedSetDocValuesAccumulator in lucene/facets does the algorithm better for typical cases (smaller number of unique values wrt the size of the document set). In this case, it collects directly with per-segment local ords, then remaps as a second step. So this is a lot less remapping. Its too bad the code is separate at the moment, for now lets steal the heuristic. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5512) Optimize DocValuesFacets
[ https://issues.apache.org/jira/browse/SOLR-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835593#comment-13835593 ] ASF subversion and git services commented on SOLR-5512: --- Commit 1546676 from [~rcmuir] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1546676 ] SOLR-5512: Optimize DocValuesFacets Optimize DocValuesFacets - Key: SOLR-5512 URL: https://issues.apache.org/jira/browse/SOLR-5512 Project: Solr Issue Type: Improvement Reporter: Robert Muir Attachments: SOLR-5512.patch This works well in the general case (esp with huge numbers of unique values), but the SortedSetDocValuesAccumulator in lucene/facets does the algorithm better for typical cases (smaller number of unique values wrt the size of the document set). In this case, it collects directly with per-segment local ords, then remaps as a second step. So this is a lot less remapping. Its too bad the code is separate at the moment, for now lets steal the heuristic. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-5512) Optimize DocValuesFacets
[ https://issues.apache.org/jira/browse/SOLR-5512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved SOLR-5512. --- Resolution: Fixed Fix Version/s: 4.7 5.0 Optimize DocValuesFacets - Key: SOLR-5512 URL: https://issues.apache.org/jira/browse/SOLR-5512 Project: Solr Issue Type: Improvement Reporter: Robert Muir Fix For: 5.0, 4.7 Attachments: SOLR-5512.patch This works well in the general case (esp with huge numbers of unique values), but the SortedSetDocValuesAccumulator in lucene/facets does the algorithm better for typical cases (smaller number of unique values wrt the size of the document set). In this case, it collects directly with per-segment local ords, then remaps as a second step. So this is a lot less remapping. Its too bad the code is separate at the moment, for now lets steal the heuristic. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5516) ChaosMonkeyNothingIsSafeTest rare inconsistency fails.
Mark Miller created SOLR-5516: - Summary: ChaosMonkeyNothingIsSafeTest rare inconsistency fails. Key: SOLR-5516 URL: https://issues.apache.org/jira/browse/SOLR-5516 Project: Solr Issue Type: Bug Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Fix For: 5.0, 4.7 -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5509) ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
[
https://issues.apache.org/jira/browse/SOLR-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Miller updated SOLR-5509:
--
Summary: ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
(was: ChaosMonkeyNothingIsSafeTest rare fail.)
ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
Key: SOLR-5509
URL: https://issues.apache.org/jira/browse/SOLR-5509
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 5.0, 4.7
Attachments: cmns-test-cloud-off-by1-control-2.log
{noformat}
[junit4] 2 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{add=[50086 (1452880907553734656)]} 0 142
[junit4] 2 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2} {add=[50086
(1452880908206997504)]} 0 254
[junit4] 2 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{delete=[50086 (-1452880908537298944)]} 0 2
[junit4] 2 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{delete=[50086 (-1452880908542541824)]} 0 1
[junit4] 2 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=TOLEADERwt=javabinversion=2} {add=[50086
(1452880908850823168)]} 0 1
[junit4] 2 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{add=[50086]} 0 1223
[junit4] 2 ## Only in cloudDocList: [{id=50086}]
[junit4] 2 cloudClient
:{numFound=1,start=0,docs=[SolrDocument{id=50086,
_version_=1452880908850823168}]}
h
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5516) ChaosMonkeyNothingIsSafeTest rare inconsistency fails.
[ https://issues.apache.org/jira/browse/SOLR-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835599#comment-13835599 ] ASF subversion and git services commented on SOLR-5516: --- Commit 1546683 from [~markrmil...@gmail.com] in branch 'dev/trunk' [ https://svn.apache.org/r1546683 ] SOLR-5516: wait a moment before trying to Sync ChaosMonkeyNothingIsSafeTest rare inconsistency fails. -- Key: SOLR-5516 URL: https://issues.apache.org/jira/browse/SOLR-5516 Project: Solr Issue Type: Bug Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Fix For: 5.0, 4.7 -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5516) ChaosMonkeyNothingIsSafeTest rare inconsistency fails.
[ https://issues.apache.org/jira/browse/SOLR-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835600#comment-13835600 ] ASF subversion and git services commented on SOLR-5516: --- Commit 1546684 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1546684 ] SOLR-5516: wait a moment before trying to Sync ChaosMonkeyNothingIsSafeTest rare inconsistency fails. -- Key: SOLR-5516 URL: https://issues.apache.org/jira/browse/SOLR-5516 Project: Solr Issue Type: Bug Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Fix For: 5.0, 4.7 -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5509) ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
[
https://issues.apache.org/jira/browse/SOLR-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835605#comment-13835605
]
Mark Miller commented on SOLR-5509:
---
This is a hard nut to crack. I've been going over the retry code a lot. Another
interesting fail:
{noformat}
[junit4] 2 119996 T28 C115 P55124 oasup.LogUpdateProcessor.finish
[collection1] webapp=/_umo/h path=/update
params={CONTROL=TRUEversion=2wt=javabin} {add=[50678 (1453094753545486336)]}
0 1
[junit4] 2 120040 T151 C117 P59797 oasc.SolrException.log ERROR
forwarding update to http://127.0.0.1:47574/_umo/h/collection1/ failed -
retrying ... retries: 1 add{,id=50678}
rsp:404:org.apache.solr.common.SolrException: Can not find:
/_umo/h/collection1/update
[junit4] 2 120544 T151 C117 P59797 oasc.SolrException.log ERROR
forwarding update to http://127.0.0.1:47574/_umo/h/collection1/ failed -
retrying ... retries: 2 add{,id=50678}
rsp:404:org.apache.solr.common.SolrException: Can not find:
/_umo/h/collection1/update
[junit4] 2 121246 T151 C117 P59797 oasc.SolrException.log ERROR
forwarding update to http://127.0.0.1:47574/_umo/h/collection1/ failed -
retrying ... retries: 3 add{,id=50678}
rsp:-1:org.apache.http.conn.HttpHostConnectException: Connection to
http://127.0.0.1:47574 refused
[junit4] 2 121246 T150 C117 P59797 oasup.LogUpdateProcessor.finish
[collection1] webapp=/_umo/h path=/update params={version=2wt=javabin}
{add=[50678 (1453094754828943360)]} 0 579
[junit4] 2 121252 T26 C115 P55124 oasup.LogUpdateProcessor.finish
[collection1] webapp=/_umo/h path=/update
params={CONTROL=TRUEversion=2wt=javabin} {delete=[50678
(-1453094754863546368)]} 0 1
[junit4] 2 121254 T152 C117 P59797 oasup.LogUpdateProcessor.finish
[collection1] webapp=/_umo/h path=/update params={version=2wt=javabin}
{delete=[50678 (-1453094754865643520)]} 0 0
[junit4] 2 121841 T149 C117 P59797 oasup.LogUpdateProcessor.finish
[collection1] webapp=/_umo/h path=/update
params={update.distrib=TOLEADERversion=2wt=javabindistrib.from=http://127.0.0.1:59797/_umo/h/collection1/}
{add=[50678 (1453094755480109056)]} 0 93
[junit4] 2 121841 T151 C117 P59797 oasup.LogUpdateProcessor.finish
[collection1] webapp=/_umo/h path=/update params={version=2wt=javabin}
{add=[50678]} 0 1842
[junit4] 2 ## Only in cloudDocList: [{id=50678}]
[junit4] 2cloudClient
:{numFound=1,start=0,docs=[SolrDocument{id=50678,
_version_=1453094755480109056}]}
{noformat}
ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
Key: SOLR-5509
URL: https://issues.apache.org/jira/browse/SOLR-5509
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 5.0, 4.7
Attachments: cmns-test-cloud-off-by1-control-2.log
{noformat}
[junit4] 2 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{add=[50086 (1452880907553734656)]} 0 142
[junit4] 2 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2} {add=[50086
(1452880908206997504)]} 0 254
[junit4] 2 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{delete=[50086 (-1452880908537298944)]} 0 2
[junit4] 2 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{delete=[50086 (-1452880908542541824)]} 0 1
[junit4] 2 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=TOLEADERwt=javabinversion=2} {add=[50086
(1452880908850823168)]} 0 1
[junit4] 2 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{add=[50086]} 0 1223
[junit4] 2 ## Only in cloudDocList: [{id=50086}]
[junit4] 2 cloudClient
:{numFound=1,start=0,docs=[SolrDocument{id=50086,
_version_=1452880908850823168}]}
h
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5509) ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
[
https://issues.apache.org/jira/browse/SOLR-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835607#comment-13835607
]
ASF subversion and git services commented on SOLR-5509:
---
Commit 1546685 from [~markrmil...@gmail.com] in branch 'dev/trunk'
[ https://svn.apache.org/r1546685 ]
SOLR-5509: More logging
ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
Key: SOLR-5509
URL: https://issues.apache.org/jira/browse/SOLR-5509
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 5.0, 4.7
Attachments: cmns-test-cloud-off-by1-control-2.log
{noformat}
[junit4] 2 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{add=[50086 (1452880907553734656)]} 0 142
[junit4] 2 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2} {add=[50086
(1452880908206997504)]} 0 254
[junit4] 2 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{delete=[50086 (-1452880908537298944)]} 0 2
[junit4] 2 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{delete=[50086 (-1452880908542541824)]} 0 1
[junit4] 2 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=TOLEADERwt=javabinversion=2} {add=[50086
(1452880908850823168)]} 0 1
[junit4] 2 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{add=[50086]} 0 1223
[junit4] 2 ## Only in cloudDocList: [{id=50086}]
[junit4] 2 cloudClient
:{numFound=1,start=0,docs=[SolrDocument{id=50086,
_version_=1452880908850823168}]}
h
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5509) ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
[
https://issues.apache.org/jira/browse/SOLR-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835608#comment-13835608
]
ASF subversion and git services commented on SOLR-5509:
---
Commit 1546686 from [~markrmil...@gmail.com] in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1546686 ]
SOLR-5509: More logging
ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
Key: SOLR-5509
URL: https://issues.apache.org/jira/browse/SOLR-5509
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 5.0, 4.7
Attachments: cmns-test-cloud-off-by1-control-2.log
{noformat}
[junit4] 2 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{add=[50086 (1452880907553734656)]} 0 142
[junit4] 2 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2} {add=[50086
(1452880908206997504)]} 0 254
[junit4] 2 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{delete=[50086 (-1452880908537298944)]} 0 2
[junit4] 2 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{delete=[50086 (-1452880908542541824)]} 0 1
[junit4] 2 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=TOLEADERwt=javabinversion=2} {add=[50086
(1452880908850823168)]} 0 1
[junit4] 2 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{add=[50086]} 0 1223
[junit4] 2 ## Only in cloudDocList: [{id=50086}]
[junit4] 2 cloudClient
:{numFound=1,start=0,docs=[SolrDocument{id=50086,
_version_=1452880908850823168}]}
h
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Collections API
If the patch is applied the work around must not be required
On 29 Nov 2013 08:17, Steve Molloy smol...@opentext.com wrote:
Thanks, I already had the genericCoreNodeNames=true in solrcloud section
of solr.xml, new format. But I had a str entry instead of a bool, which
apparently is simply treated as false. Anyhow, in my case the fix works if
I move the bit setting the coreNodeName after the publish, not before. If
it's before, I get a timeout error while it waits for a replica that is
never set in waitForShardId.
I'll both apply the modified patch and switch from str to bool. :)
Thanks for the help,
Steve
From: Alexey Serba [ase...@gmail.com]
Sent: November 28, 2013 2:10 AM
To: dev@lucene.apache.org
Subject: Re: Collections API
https://issues.apache.org/jira/browse/SOLR-5510
I don't really understand all the details why is that happening, but the
workaround is to add genericCoreNodeNames=${genericCoreNodeNames:true}
attribute to cores element in your solr.xml file.
On Tue, Nov 26, 2013 at 10:10 PM, Steve Molloy
smol...@opentext.com wrote:
I'm trying to reconcile our fork with 4.6 tag and I'm getting weird
behaviour in Collections API, more specifically in ZkController's
preRegister method after calling the create method of the collections API.
When it checks if a slice has a replica for current
node name, there is never any because at this stage, the slice has no
replica. This is the new code that seems to be causing my issue, I can
force the autoCreated to be always true to avoid the issue, but would
like a cleaner way if there is one.
if(cd.getCloudDescriptor().getCollectionName() !=null
cd.getCloudDescriptor().getCoreNodeName() != null ) {
//we were already registered
if(zkStateReader.getClusterState().hasCollection(cd.getCloudDescriptor().getCollectionName())){
DocCollection coll =
zkStateReader.getClusterState().getCollection(cd.getCloudDescriptor().getCollectionName());
if(!true.equals(coll.getStr(autoCreated))){
Slice slice =
coll.getSlice(cd.getCloudDescriptor().getShardId());
if(slice != null){
== if(slice.getReplica(cd.getCloudDescriptor().getCoreNodeName()) ==
null) {
log.info(core_removed This core is removed from ZK);
throw new SolrException(ErrorCode.NOT_FOUND,coreNodeName +
is removed);
}
}
}
}
}
Thanks.
Steve
-
To unsubscribe, e-mail:
dev-unsubscr...@lucene.apache.org
For additional commands, e-mail:
dev-h...@lucene.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5516) ChaosMonkeyNothingIsSafeTest rare inconsistency fails.
[ https://issues.apache.org/jira/browse/SOLR-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835609#comment-13835609 ] Mark Miller commented on SOLR-5516: --- I think the key here is that we have to ensure we are not somehow sending documents during the leader sync process. We are counting on the fact that there is no leader means no updates will be coming in - but it seems depending on how the leader goes down (hard fail, graceful stop, zk expiration), there can rarely be a brief overlap. I suppose I'd have to guess zk expiration... ChaosMonkeyNothingIsSafeTest rare inconsistency fails. -- Key: SOLR-5516 URL: https://issues.apache.org/jira/browse/SOLR-5516 Project: Solr Issue Type: Bug Components: SolrCloud Reporter: Mark Miller Assignee: Mark Miller Fix For: 5.0, 4.7 -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5502) A / in the ID itself throws an ArrayIndexOutOfBoundsException when using the composite id router
[ https://issues.apache.org/jira/browse/SOLR-5502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835612#comment-13835612 ] Mark Miller commented on SOLR-5502: --- Thanks! Got a test as well? A / in the ID itself throws an ArrayIndexOutOfBoundsException when using the composite id router -- Key: SOLR-5502 URL: https://issues.apache.org/jira/browse/SOLR-5502 Project: Solr Issue Type: Bug Components: SolrCloud Affects Versions: 4.6 Reporter: Anshum Gupta Attachments: SOLR-5502.patch While using the composite-id router, if the routing-id contains a / in the id part, the code throws an ArrayIndexOutOfBoundsException. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 4696 - Failure!
Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/4696/ All tests passed Build Log: [...truncated 975 lines...] [junit4] JVM J7: stdout was not empty, see: /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build/core/test/temp/junit4-J7-20131130_055407_181.sysout [junit4] JVM J7: stdout (verbatim) [junit4] # [junit4] # A fatal error has been detected by the Java Runtime Environment: [junit4] # [junit4] # SIGSEGV (0xb) at pc=0x7f25444e41ce, pid=20290, tid=139797982811904 [junit4] # [junit4] # JRE version: 6.0_45-b06 [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.45-b01 mixed mode linux-amd64 compressed oops) [junit4] # Problematic frame: [junit4] # V [libjvm.so+0x3551ce] ciObjectFactory::find_non_perm(oopDesc*)+0x4e [junit4] # [junit4] # An error report file with more information is saved as: [junit4] # /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build/core/test/J7/hs_err_pid20290.log [junit4] # [junit4] # If you would like to submit a bug report, please visit: [junit4] # http://java.sun.com/webapps/bugreport/crash.jsp [junit4] # [junit4] JVM J7: EOF [...truncated 430 lines...] [junit4] ERROR: JVM J7 ended with an exception, command line: /var/lib/jenkins/tools/hudson.model.JDK/Java_6_64bit_u45/jre/bin/java -Dtests.prefix=tests -Dtests.seed=BA912A3C36112475 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.7 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=1 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build/core/test/temp -Dclover.db.dir=/var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/tools/junit4/tests.policy -Dlucene.version=4.7-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dfile.encoding=ISO-8859-1 -classpath
[jira] [Commented] (SOLR-5509) ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
[
https://issues.apache.org/jira/browse/SOLR-5509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13835615#comment-13835615
]
Mark Miller commented on SOLR-5509:
---
This may actually be related to SOLR-5516...
ChaosMonkeyNothingIsSafeTest rare fails due to TOLEADER retries.
Key: SOLR-5509
URL: https://issues.apache.org/jira/browse/SOLR-5509
Project: Solr
Issue Type: Bug
Components: SolrCloud
Reporter: Mark Miller
Assignee: Mark Miller
Fix For: 5.0, 4.7
Attachments: cmns-test-cloud-off-by1-control-2.log
{noformat}
[junit4] 2 41386 T28 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{add=[50086 (1452880907553734656)]} 0 142
[junit4] 2 42009 T133 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2} {add=[50086
(1452880908206997504)]} 0 254
[junit4] 2 42323 T27 C21 P57194 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinCONTROL=TRUEversion=2}
{delete=[50086 (-1452880908537298944)]} 0 2
[junit4] 2 42327 T131 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{delete=[50086 (-1452880908542541824)]} 0 1
[junit4] 2 42622 T132 C27 P60411 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update
params={update.distrib=TOLEADERwt=javabinversion=2} {add=[50086
(1452880908850823168)]} 0 1
[junit4] 2 42623 T48 C22 P57136 oasup.LogUpdateProcessor.finish
[collection1] webapp= path=/update params={wt=javabinversion=2}
{add=[50086]} 0 1223
[junit4] 2 ## Only in cloudDocList: [{id=50086}]
[junit4] 2 cloudClient
:{numFound=1,start=0,docs=[SolrDocument{id=50086,
_version_=1452880908850823168}]}
h
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5517) Treat POST with no Content-Type as application/x-www-form-urlencoded
Ryan Ernst created SOLR-5517: Summary: Treat POST with no Content-Type as application/x-www-form-urlencoded Key: SOLR-5517 URL: https://issues.apache.org/jira/browse/SOLR-5517 Project: Solr Issue Type: Improvement Reporter: Ryan Ernst Attachments: SOLR-5517.patch While the http spec states requests without a content-type should be treated as application/octet-stream, the html spec says instead that post requests without a content-type should be treated as a form (http://www.w3.org/MarkUp/html-spec/html-spec_8.html#SEC8.2.1). It would be nice to allow large search requests from html forms, and not have to rely on the browser to set the content type (since the spec says it doesn't have to). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-5517) Treat POST with no Content-Type as application/x-www-form-urlencoded
[ https://issues.apache.org/jira/browse/SOLR-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Ernst updated SOLR-5517: - Attachment: SOLR-5517.patch Treat POST with no Content-Type as application/x-www-form-urlencoded Key: SOLR-5517 URL: https://issues.apache.org/jira/browse/SOLR-5517 Project: Solr Issue Type: Improvement Reporter: Ryan Ernst Attachments: SOLR-5517.patch While the http spec states requests without a content-type should be treated as application/octet-stream, the html spec says instead that post requests without a content-type should be treated as a form (http://www.w3.org/MarkUp/html-spec/html-spec_8.html#SEC8.2.1). It would be nice to allow large search requests from html forms, and not have to rely on the browser to set the content type (since the spec says it doesn't have to). -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-4x-Linux-Java7-64-test-only - Build # 4696 - Failure!
This crash is on 1.6/64 bit. I searched openjdk bugs database and there's this entry: https://bugs.openjdk.java.net/browse/JDK-6681372 64-bit VM CompilerThread received SEGV in ciObjectFactory::find_non_perm() Which seems to match our bug. The bug is dated 2008-03-28, so it's very, very old, but then it's 1.6 so it may still be somewhere in the codebase. I can't access jenkins at the moment to save hs_err_pid20290.log -- could somebody do it when jenkins is up again? Dawid On Sat, Nov 30, 2013 at 5:59 AM, buil...@flonkings.com wrote: Build: builds.flonkings.com/job/Lucene-4x-Linux-Java7-64-test-only/4696/ All tests passed Build Log: [...truncated 975 lines...] [junit4] JVM J7: stdout was not empty, see: /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build/core/test/temp/junit4-J7-20131130_055407_181.sysout [junit4] JVM J7: stdout (verbatim) [junit4] # [junit4] # A fatal error has been detected by the Java Runtime Environment: [junit4] # [junit4] # SIGSEGV (0xb) at pc=0x7f25444e41ce, pid=20290, tid=139797982811904 [junit4] # [junit4] # JRE version: 6.0_45-b06 [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.45-b01 mixed mode linux-amd64 compressed oops) [junit4] # Problematic frame: [junit4] # V [libjvm.so+0x3551ce] ciObjectFactory::find_non_perm(oopDesc*)+0x4e [junit4] # [junit4] # An error report file with more information is saved as: [junit4] # /var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build/core/test/J7/hs_err_pid20290.log [junit4] # [junit4] # If you would like to submit a bug report, please visit: [junit4] # http://java.sun.com/webapps/bugreport/crash.jsp [junit4] # [junit4] JVM J7: EOF [...truncated 430 lines...] [junit4] ERROR: JVM J7 ended with an exception, command line: /var/lib/jenkins/tools/hudson.model.JDK/Java_6_64bit_u45/jre/bin/java -Dtests.prefix=tests -Dtests.seed=BA912A3C36112475 -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=4.7 -Dtests.cleanthreads=perMethod -Djava.util.logging.config.file=/var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.slow=true -Dtests.asserts.gracious=false -Dtests.multiplier=1 -DtempDir=. -Djava.io.tmpdir=. -Djunit4.tempDir=/var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build/core/test/temp -Dclover.db.dir=/var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/build/clover/db -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Djava.security.policy=/var/lib/jenkins/workspace/Lucene-4x-Linux-Java7-64-test-only/checkout/lucene/tools/junit4/tests.policy -Dlucene.version=4.7-SNAPSHOT -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dfile.encoding=ISO-8859-1 -classpath
