[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-10.0.1) - Build # 3008 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3008/
Java: 64bit/jdk-10.0.1 -XX:+UseCompressedOops -XX:+UseParallelGC

8 tests failed.
FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:44245/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:44245/solr
at 
__randomizedtesting.SeedInfo.seed([525CC9489E970BC7:93ACB0E4B3C7C160]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at 
org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:44813/solr

Stack Trace:

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 886 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/886/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRouting

Error Message:
Error from server at http://127.0.0.1:48203/solr/collection1_shard2_replica_n3: 
Expected mime type application/octet-stream but got text/html.   
 
Error 404 Can not find: 
/solr/collection1_shard2_replica_n3/update  HTTP ERROR 
404 Problem accessing /solr/collection1_shard2_replica_n3/update. 
Reason: Can not find: 
/solr/collection1_shard2_replica_n3/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605  
  

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:48203/solr/collection1_shard2_replica_n3: Expected 
mime type application/octet-stream but got text/html. 


Error 404 Can not find: 
/solr/collection1_shard2_replica_n3/update

HTTP ERROR 404
Problem accessing /solr/collection1_shard2_replica_n3/update. Reason:
Can not find: 
/solr/collection1_shard2_replica_n3/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605




at 
__randomizedtesting.SeedInfo.seed([45C2132673B47709:87752F4E70F48771]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:237)
at 
org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRouting(CloudSolrClientTest.java:269)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   

[GitHub] lucene-solr pull request #:

[moshebla]

Github user moshebla commented on the pull request:


https://github.com/apache/lucene-solr/commit/43d7f5d104802bc3281d5995c6c2d71bebf1f369#commitcomment-31114379
  
Perhaps a better name for it would be "isNestedUpdate".
This method checks whether the update changes a block.
Beforehand, the previous version was looked up, causing ConvertedLegacyTest 
to fail.
Now the test passes, since the look up is only executed when needed.


---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12875) ArrayIndexOutOfBoundsException when using uniqueBlock(_root_) in JSON Facets

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669623#comment-16669623
 ] 

ASF subversion and git services commented on SOLR-12875:


Commit 5aa8ad5b14ab7f6f8f3191cba39285c3a0bf9112 in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~mkhludnev]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5aa8ad5 ]

SOLR-12875: AIOOBE fix when unique()/uniqueBlock() is used with DVHASH method 
in json.facet


> ArrayIndexOutOfBoundsException when using uniqueBlock(_root_) in JSON Facets
> 
>
> Key: SOLR-12875
> URL: https://issues.apache.org/jira/browse/SOLR-12875
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Assignee: Mikhail Khludnev
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-12875.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I'm seeing java.lang.ArrayIndexOutOfBoundsException exceptions for some 
> requests when trying to make use of
> {noformat}
> uniqueBlock(_root_){noformat}
> within JSON Facets.
> Here are some example Stack Traces:
> {noformat}
> 2018-10-12 14:08:50.587 ERROR (qtp215078753-3353) [   x:my_core] 
> o.a.s.s.HttpSolrCall null:java.lang.ArrayIndexOutOfBoundsException: Index 13 
> out of bounds for length 8
> at 
> org.apache.solr.search.facet.UniqueBlockAgg$UniqueBlockSlotAcc.collectOrdToSlot(UniqueBlockAgg.java:40)
> at 
> org.apache.solr.search.facet.UniqueSinglevaluedSlotAcc.collect(UniqueSinglevaluedSlotAcc.java:85)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.collectFirstPhase(FacetFieldProcessor.java:243)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.collectValFirstPhase(FacetFieldProcessorByHashDV.java:432)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.access$100(FacetFieldProcessorByHashDV.java:50)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV$5.collect(FacetFieldProcessorByHashDV.java:395)
> at 
> org.apache.solr.search.DocSetUtil.collectSortedDocSet(DocSetUtil.java:284)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.collectDocs(FacetFieldProcessorByHashDV.java:376)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.calcFacets(FacetFieldProcessorByHashDV.java:247)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.process(FacetFieldProcessorByHashDV.java:214)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:472)
> at 
> org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:429)
> at 
> org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetModule.process(FacetModule.java:139)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> {noformat}
>  
> Here is another one at a different location in UniqueBlockAgg:
>   
> {noformat}
> 2018-10-12 21:37:57.322 ERROR (qtp215078753-4072) [   x:my_core] 
> o.a.s.h.RequestHandlerBase java.lang.ArrayIndexOutOfBoundsException: Index 23 
> out of bounds for length 16
> at 
> org.apache.solr.search.facet.UniqueBlockAgg$UniqueBlockSlotAcc.getValue(UniqueBlockAgg.java:59)
> at org.apache.solr.search.facet.SlotAcc.setValues(SlotAcc.java:146)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.fillBucket(FacetFieldProcessor.java:431)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.findTopSlots(FacetFieldProcessor.java:381)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.calcFacets(FacetFieldProcessorByHashDV.java:249)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.process(FacetFieldProcessorByHashDV.java:214)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:472)
> at 
> org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:429)
> at 
> org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
> at 
> 

[jira] [Commented] (SOLR-11572) Add recip Stream Evaluator to support reciprocal transformations

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669624#comment-16669624
 ] 

ASF subversion and git services commented on SOLR-11572:


Commit 856e28d8cf07cc34bc1361784bf00e7aceb3af97 in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=856e28d ]

SOLR-11572: Add recip Stream Evaluator to support reciprocal transformations


> Add recip Stream Evaluator to support reciprocal transformations
> 
>
> Key: SOLR-11572
> URL: https://issues.apache.org/jira/browse/SOLR-11572
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-11572.patch
>
>
> This ticket adds the recip Stream Evaluator to support inverse/reciprocal 
> scalar and vector transformations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12936) Allow percentiles Stream Evaluator to accept an array of percentiles to calculate

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669622#comment-16669622
 ] 

ASF subversion and git services commented on SOLR-12936:


Commit ac1925045d137d4762c6cb5d6940bfec784bdd4f in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ac19250 ]

SOLR-12936: Allow percentiles Stream Evaluator to accept an array of 
percentiles to calculate


> Allow percentiles Stream Evaluator to accept an array of percentiles to 
> calculate
> -
>
> Key: SOLR-12936
> URL: https://issues.apache.org/jira/browse/SOLR-12936
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: SOLR-12936.patch
>
>
> Currently the *percentiles* Stream Evaluator only accepts a request for a 
> single percentile. This ticket will allow the percentiles function to 
> calculate an array of percentiles in the same request. This will make 
> *qq-plots* easier and more efficient.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12935) Streaming Expression and Math Expression tests fail too frequently

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669621#comment-16669621
 ] 

ASF subversion and git services commented on SOLR-12935:


Commit e618e831d3ac040081769ebd27324a1d190fbc5d in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e618e83 ]

SOLR-12935: Suppress SSL for StreamExpressionTest and StreamDecoratorTest


> Streaming Expression and Math Expression tests fail too frequently
> --
>
> Key: SOLR-12935
> URL: https://issues.apache.org/jira/browse/SOLR-12935
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
>
> There are a number of Streaming Expression and Math Expression tests which 
> have already been badappled. There are others that are failing much too 
> frequently that still run as part of the jenkins test runs. This ticket will 
> fix these issues and remove the badapples.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12811) Add enclosingDisk and associated geometric Stream Evaluators

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669615#comment-16669615
 ] 

ASF subversion and git services commented on SOLR-12811:


Commit f8ffc1afd66f22906a4011045ba105da8ee9ed97 in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f8ffc1a ]

SOLR-12811: Update CHANGES.txt


> Add enclosingDisk and associated geometric Stream Evaluators
> 
>
> Key: SOLR-12811
> URL: https://issues.apache.org/jira/browse/SOLR-12811
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-12811.patch
>
>
> This ticket will add the *enclosingDisk*, *getRadius, getCenter* and 
> *getSupportPoints* Stream Evaluators. The enclosingDisk function will 
> calculate the smallest circle that encloses a 2D data set. The implementation 
> is provided by *Apache Commons Math.*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12861) Add Solr factory for new ByteBuffersDirectory

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669620#comment-16669620
 ] 

ASF subversion and git services commented on SOLR-12861:


Commit c7c7b00ff99f5e0e10123f7de4d7b820a23a9733 in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c7c7b00 ]

SOLR-12861: In the upgrade note, specify the Solr version in which RAMDirectory 
will be removed.


> Add Solr factory for new ByteBuffersDirectory
> -
>
> Key: SOLR-12861
> URL: https://issues.apache.org/jira/browse/SOLR-12861
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-12861.patch
>
>
> LUCENE-8438 and sub-tasks introduced ByteBuffersDirectory, a RAMDirectory 
> replacement.  Solr needs a factory class in order to expose it to end-users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12862) Add log10 Stream Evaluator and allow the pow Stream Evaluator to accept a vector of exponents

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669617#comment-16669617
 ] 

ASF subversion and git services commented on SOLR-12862:


Commit 5fc4d516b11c1c203d5affbdaeaf0d63e070f617 in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5fc4d51 ]

SOLR-12862: Update CHANGES.txt


> Add log10 Stream Evaluator and allow the pow Stream Evaluator to accept a 
> vector of exponents
> -
>
> Key: SOLR-12862
> URL: https://issues.apache.org/jira/browse/SOLR-12862
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-12862.patch
>
>
> This ticket adds the *log10* Stream Evaluator to support base 10 log 
> transformations. It also adds support for passing a vector of exponents to 
> the *pow* Stream Evaluator to support reverse log transformations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12840) Add pairSort Stream Evaluator

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669616#comment-16669616
 ] 

ASF subversion and git services commented on SOLR-12840:


Commit 416cc163bad78c3abd516c805ce4e3f764421f02 in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=416cc16 ]

SOLR-12840: Update CHANGES.txt


> Add pairSort Stream Evaluator
> -
>
> Key: SOLR-12840
> URL: https://issues.apache.org/jira/browse/SOLR-12840
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-12840.patch
>
>
> The *pairSort* Stream evaluator takes two paired arrays of numeric data and 
> sorts both arrays based on the natural sort order of the first array. The 
> pairSort function returns a matrix with two rows containing the pair sorted 
> arrays. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12861) Add Solr factory for new ByteBuffersDirectory

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669619#comment-16669619
 ] 

ASF subversion and git services commented on SOLR-12861:


Commit d362439e277c63a0e3be539179d3b18cf96df617 in lucene-solr's branch 
refs/heads/jira/http2_benchmark from [~steve_rowe]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d362439 ]

SOLR-12861: Add Solr factory for new ByteBuffersDirectory


> Add Solr factory for new ByteBuffersDirectory
> -
>
> Key: SOLR-12861
> URL: https://issues.apache.org/jira/browse/SOLR-12861
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Steve Rowe
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-12861.patch
>
>
> LUCENE-8438 and sub-tasks introduced ByteBuffersDirectory, a RAMDirectory 
> replacement.  Solr needs a factory class in order to expose it to end-users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 1168 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1168/

No tests ran.

Build Log:
[...truncated 23412 lines...]
[asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: 
invalid part, must have at least one section (e.g., chapter, appendix, etc.)
[asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid 
part, must have at least one section (e.g., chapter, appendix, etc.)
 [java] Processed 2436 links (1988 relative) to 3199 anchors in 248 files
 [echo] Validated Links & Anchors via: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr-ref-guide/bare-bones-html/

-dist-changes:
 [copy] Copying 4 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/changes

package:

-unpack-solr-tgz:

-ensure-solr-tgz-exists:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked
[untar] Expanding: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/solr-8.0.0.tgz
 into 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked

generate-maven-artifacts:

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

[jira] [Commented] (SOLR-12878) FacetFieldProcessorByHashDV is reconstructing FieldInfos on every instantiation

[Tim Underwood (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669596#comment-16669596
 ] 

Tim Underwood commented on SOLR-12878:
--

It looks like maybe FacetFieldProcessorByHashDV is created once per field that 
is being faceted on?  In my case for the query I've been testing I'm faceting 
on 41 different fields.  So 
fcontext.searcher.getSlowAtomicReader().getFieldInfos() was being called 41 
times for each query.

> FacetFieldProcessorByHashDV is reconstructing FieldInfos on every 
> instantiation
> ---
>
> Key: SOLR-12878
> URL: https://issues.apache.org/jira/browse/SOLR-12878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Assignee: Mikhail Khludnev
>Priority: Major
>  Labels: performance
> Fix For: 7.6, master (8.0)
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV constructor is currently calling:
> {noformat}
> FieldInfo fieldInfo = 
> fcontext.searcher.getSlowAtomicReader().getFieldInfos().fieldInfo(sf.getName());
> {noformat}
> Which is reconstructing FieldInfos each time.  Simply switching it to:
> {noformat}
> FieldInfo fieldInfo = 
> fcontext.searcher.getFieldInfos().fieldInfo(sf.getName());
> {noformat}
>  
> causes it to use the cached version of FieldInfos in the SolrIndexSearcher.
> On my index the FacetFieldProcessorByHashDV is 2-3 times slower than the 
> legacy facets without this fix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12882) Eliminate excessive lambda allocation in FacetFieldProcessorByHashDV.collectValFirstPhase

[Tim Underwood (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669586#comment-16669586
 ] 

Tim Underwood commented on SOLR-12882:
--

It's not a huge improvement but I noticed it showing up in YourKit memory 
allocation profiling:

!start_-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png|width=600px!

https://issues.apache.org/jira/secure/attachment/12946329/start_-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png

!start-2018-10-31_snapshot___Users_tim_Snapshots__-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png|width=600px!

 
https://issues.apache.org/jira/secure/attachment/12946330/start-2018-10-31_snapshot___Users_tim_Snapshots__-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png

My informal benchmarking (I should really setup JMH for this stuff) for one of 
my facet heavy queries went from ~270-275 requests/second to ~285-293 
requests/second for my setup.

So it's a very minor improvement.

> Eliminate excessive lambda allocation in 
> FacetFieldProcessorByHashDV.collectValFirstPhase
> -
>
> Key: SOLR-12882
> URL: https://issues.apache.org/jira/browse/SOLR-12882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Priority: Major
> Attachments: 
> start-2018-10-31_snapshot___Users_tim_Snapshots__-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png,
>  start_-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV.collectValFirstPhase method looks like this:
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>  int slot = table.add(val); // this can trigger a rehash
>  // Our countAcc is virtual, so this is not needed:
>  // countAcc.incrementCount(slot, 1);
> super.collectFirstPhase(segDoc, slot, slotNum ->
> { Comparable value = calc.bitsToValue(val); return new 
> SlotContext(sf.getType().getFieldQuery(null, sf, calc.formatValue(value))); }
> );
> }
> {noformat}
>  
> For each value that is being iterated over there is a lambda allocation that 
> is passed as the slotContext argument to the super.collectFirstPhase method. 
> The lambda can be factored out such that there is only a single instance per 
> FacetFieldProcessorByHashDV instance. 
> The only tradeoff being that the value needs to be looked up from the table 
> in the lambda.  However looking the value up in the table is going to be less 
> expensive than a memory allocation and also the slotContext lambda is only 
> used in RelatednessAgg and not for any of the field faceting or metrics.
>  
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>   int slot = table.add(val); // this can trigger a rehash
>   // Our countAcc is virtual, so this is not needed:
>   // countAcc.incrementCount(slot, 1);
>   super.collectFirstPhase(segDoc, slot, slotContext);
> }
> /**
>  * SlotContext to use during all {@link SlotAcc} collection.
>  *
>  * This avoids a memory allocation for each invocation of 
> collectValFirstPhase.
>  */
> private IntFunction slotContext = (slotNum) -> {
>   long val = table.vals[slotNum];
>   Comparable value = calc.bitsToValue(val);
>   return new SlotContext(sf.getType().getFieldQuery(null, sf, 
> calc.formatValue(value)));
> };
> {noformat}
>  
> FacetFieldProcessorByArray already follows this same pattern



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12882) Eliminate excessive lambda allocation in FacetFieldProcessorByHashDV.collectValFirstPhase

[Tim Underwood (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Underwood updated SOLR-12882:
-
Attachment: 
start-2018-10-31_snapshot___Users_tim_Snapshots__-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png

> Eliminate excessive lambda allocation in 
> FacetFieldProcessorByHashDV.collectValFirstPhase
> -
>
> Key: SOLR-12882
> URL: https://issues.apache.org/jira/browse/SOLR-12882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Priority: Major
> Attachments: 
> start-2018-10-31_snapshot___Users_tim_Snapshots__-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png,
>  start_-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV.collectValFirstPhase method looks like this:
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>  int slot = table.add(val); // this can trigger a rehash
>  // Our countAcc is virtual, so this is not needed:
>  // countAcc.incrementCount(slot, 1);
> super.collectFirstPhase(segDoc, slot, slotNum ->
> { Comparable value = calc.bitsToValue(val); return new 
> SlotContext(sf.getType().getFieldQuery(null, sf, calc.formatValue(value))); }
> );
> }
> {noformat}
>  
> For each value that is being iterated over there is a lambda allocation that 
> is passed as the slotContext argument to the super.collectFirstPhase method. 
> The lambda can be factored out such that there is only a single instance per 
> FacetFieldProcessorByHashDV instance. 
> The only tradeoff being that the value needs to be looked up from the table 
> in the lambda.  However looking the value up in the table is going to be less 
> expensive than a memory allocation and also the slotContext lambda is only 
> used in RelatednessAgg and not for any of the field faceting or metrics.
>  
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>   int slot = table.add(val); // this can trigger a rehash
>   // Our countAcc is virtual, so this is not needed:
>   // countAcc.incrementCount(slot, 1);
>   super.collectFirstPhase(segDoc, slot, slotContext);
> }
> /**
>  * SlotContext to use during all {@link SlotAcc} collection.
>  *
>  * This avoids a memory allocation for each invocation of 
> collectValFirstPhase.
>  */
> private IntFunction slotContext = (slotNum) -> {
>   long val = table.vals[slotNum];
>   Comparable value = calc.bitsToValue(val);
>   return new SlotContext(sf.getType().getFieldQuery(null, sf, 
> calc.formatValue(value)));
> };
> {noformat}
>  
> FacetFieldProcessorByArray already follows this same pattern



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12882) Eliminate excessive lambda allocation in FacetFieldProcessorByHashDV.collectValFirstPhase

[Tim Underwood (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Underwood updated SOLR-12882:
-
Attachment: start_-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png

> Eliminate excessive lambda allocation in 
> FacetFieldProcessorByHashDV.collectValFirstPhase
> -
>
> Key: SOLR-12882
> URL: https://issues.apache.org/jira/browse/SOLR-12882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Priority: Major
> Attachments: start_-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV.collectValFirstPhase method looks like this:
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>  int slot = table.add(val); // this can trigger a rehash
>  // Our countAcc is virtual, so this is not needed:
>  // countAcc.incrementCount(slot, 1);
> super.collectFirstPhase(segDoc, slot, slotNum ->
> { Comparable value = calc.bitsToValue(val); return new 
> SlotContext(sf.getType().getFieldQuery(null, sf, calc.formatValue(value))); }
> );
> }
> {noformat}
>  
> For each value that is being iterated over there is a lambda allocation that 
> is passed as the slotContext argument to the super.collectFirstPhase method. 
> The lambda can be factored out such that there is only a single instance per 
> FacetFieldProcessorByHashDV instance. 
> The only tradeoff being that the value needs to be looked up from the table 
> in the lambda.  However looking the value up in the table is going to be less 
> expensive than a memory allocation and also the slotContext lambda is only 
> used in RelatednessAgg and not for any of the field faceting or metrics.
>  
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>   int slot = table.add(val); // this can trigger a rehash
>   // Our countAcc is virtual, so this is not needed:
>   // countAcc.incrementCount(slot, 1);
>   super.collectFirstPhase(segDoc, slot, slotContext);
> }
> /**
>  * SlotContext to use during all {@link SlotAcc} collection.
>  *
>  * This avoids a memory allocation for each invocation of 
> collectValFirstPhase.
>  */
> private IntFunction slotContext = (slotNum) -> {
>   long val = table.vals[slotNum];
>   Comparable value = calc.bitsToValue(val);
>   return new SlotContext(sf.getType().getFieldQuery(null, sf, 
> calc.formatValue(value)));
> };
> {noformat}
>  
> FacetFieldProcessorByArray already follows this same pattern



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-12-ea+12) - Build # 23125 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23125/
Java: 64bit/jdk-12-ea+12 -XX:+UseCompressedOops -XX:+UseParallelGC

16 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestCloudRecovery

Error Message:
Could not find collection:collection1

Stack Trace:
java.lang.AssertionError: Could not find collection:collection1
at __randomizedtesting.SeedInfo.seed([49D96CB8D7AAABEA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at 
org.apache.solr.cloud.TestCloudRecovery.setupCluster(TestCloudRecovery.java:70)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:835)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestCloudRecovery

Error Message:
Could not find collection:collection1

Stack Trace:
java.lang.AssertionError: Could not find collection:collection1
at __randomizedtesting.SeedInfo.seed([49D96CB8D7AAABEA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at 
org.apache.solr.cloud.TestCloudRecovery.setupCluster(TestCloudRecovery.java:70)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 

[jira] [Commented] (SOLR-12932) ant test (without badapples=false) should pass easily for developers.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669571#comment-16669571
 ] 

Mark Miller commented on SOLR-12932:


Thanks! No log necessary, really just the test, but also the message lets me 
know for sure if I got that specific issue.

I've actually seen testCollectionCreateSearchDelete and have been trying to 
correct it. I'll move to awaits fix and keep working on it.

I've also seen the Overseer test failure. I had beasted that pretty well on a 
slightly older different checkout, but a lot of changes since then. I'll just 
try and fix that one directly.



> ant test (without badapples=false) should pass easily for developers.
> -
>
> Key: SOLR-12932
> URL: https://issues.apache.org/jira/browse/SOLR-12932
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
>
> If we fix the tests we will end up here anyway, but we can shortcut this.
> Once I get my first patch in, anyone who mentions a test that fails locally 
> for them at any time (not jenkins), I will fix it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12932) ant test (without badapples=false) should pass easily for developers.

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669562#comment-16669562
 ] 

Kevin Risden edited comment on SOLR-12932 at 10/31/18 3:46 AM:
---

Below did not reproduce the the reproduce line. Saved logs off in case more 
details needed.

Test: TestCollectionsAPIViaSolrCloudCluster#testCollectionCreateSearchDelete
 Fail message:
{noformat}
testCollectionCreateSearchDelete
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestCollectionsAPIViaSolrCloudCluster 
-Dtests.method=testCollectionCreateSearchDelete -Dtests.seed=69334EE3FF38A1CC 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=it-IT 
-Dtests.timezone=Africa/Tunis -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
   [junit4] ERROR   13.3s J7 | 
TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete <<<
   [junit4]> Throwable #1: 
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:33783/solr/testcollection_shard1_replica_n2: 
Expected mime type application/octet-stream but got text/html. 
   [junit4]> 
   [junit4]> 
   [junit4]> Error 404 Can not find: 
/solr/testcollection_shard1_replica_n2/update
   [junit4]> 
   [junit4]> HTTP ERROR 404
   [junit4]> Problem accessing 
/solr/testcollection_shard1_replica_n2/update. Reason:
   [junit4]> Can not find: 
/solr/testcollection_shard1_replica_n2/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605
   [junit4]> 
   [junit4]> 
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([69334EE3FF38A1CC:CAC9E04678D04B69]:0)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
   [junit4]>at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
   [junit4]>at 
org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:237)
   [junit4]>at 
org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:181)
   [junit4]>at java.lang.Thread.run(Thread.java:748)
   [junit4]> Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:33783/solr/testcollection_shard1_replica_n2: 
Expected mime type application/octet-stream but got text/html. 
   [junit4]> 
   [junit4]> 
   [junit4]> Error 404 Can not find: 
/solr/testcollection_shard1_replica_n2/update
   [junit4]> 
   [junit4]> HTTP ERROR 404
   [junit4]> Problem accessing 
/solr/testcollection_shard1_replica_n2/update. Reason:
   [junit4]> Can not find: 
/solr/testcollection_shard1_replica_n2/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605
   [junit4]> 
   [junit4]> 
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]>at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:484)
   [junit4]>at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
   [junit4]>at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]>at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
   [junit4]>at 

[jira] [Commented] (SOLR-12932) ant test (without badapples=false) should pass easily for developers.

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669568#comment-16669568
 ] 

Kevin Risden commented on SOLR-12932:
-

Separate test run. Seed does not reproduce.

Test: OverseerTest#testBadQueueItem
Fail message:
{noformat}
23:37:34    [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=OverseerTest -Dtests.method=testBadQueueItem 
-Dtests.seed=34970C10FE2A115D -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=fr -Dtests.timezone=Canada/Yukon -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
23:37:34    [junit4] FAILURE 0.88s J6 | OverseerTest.testBadQueueItem <<<
23:37:34    [junit4]    > Throwable #1: java.lang.AssertionError: expected:<1> 
but was:<0>
23:37:34    [junit4]    > at 
__randomizedtesting.SeedInfo.seed([34970C10FE2A115D:843AC65BA75F47C5]:0)
23:37:34    [junit4]    > at 
org.apache.solr.cloud.OverseerTest.testBadQueueItem(OverseerTest.java:424)
23:37:34    [junit4]    > at java.lang.Thread.run(Thread.java:748){noformat}

> ant test (without badapples=false) should pass easily for developers.
> -
>
> Key: SOLR-12932
> URL: https://issues.apache.org/jira/browse/SOLR-12932
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
>
> If we fix the tests we will end up here anyway, but we can shortcut this.
> Once I get my first patch in, anyone who mentions a test that fails locally 
> for them at any time (not jenkins), I will fix it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12932) ant test (without badapples=false) should pass easily for developers.

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669562#comment-16669562
 ] 

Kevin Risden commented on SOLR-12932:
-

Below did not reproduce the the reproduce line. Saved logs off in case more 
details needed.

Test: TestCollectionsAPIViaSolrCloudCluster#testCollectionCreateSearchDelete
Fail message:
{noformat}
testCollectionCreateSearchDelete
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=TestCollectionsAPIViaSolrCloudCluster 
-Dtests.method=testCollectionCreateSearchDelete -Dtests.seed=69334EE3FF38A1CC 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=it-IT 
-Dtests.timezone=Africa/Tunis -Dtests.asserts=true 
-Dtests.file.encoding=US-ASCII
   [junit4] ERROR   13.3s J7 | 
TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete <<<
   [junit4]> Throwable #1: 
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:33783/solr/testcollection_shard1_replica_n2: 
Expected mime type application/octet-stream but got text/html. 
   [junit4]> 
   [junit4]> 
   [junit4]> Error 404 Can not find: 
/solr/testcollection_shard1_replica_n2/update
   [junit4]> 
   [junit4]> HTTP ERROR 404
   [junit4]> Problem accessing 
/solr/testcollection_shard1_replica_n2/update. Reason:
   [junit4]> Can not find: 
/solr/testcollection_shard1_replica_n2/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605
   [junit4]> 
   [junit4]> 
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([69334EE3FF38A1CC:CAC9E04678D04B69]:0)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
   [junit4]>at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
   [junit4]>at 
org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:237)
   [junit4]>at 
org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:181)
   [junit4]>at java.lang.Thread.run(Thread.java:748)
   [junit4]> Caused by: 
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:33783/solr/testcollection_shard1_replica_n2: 
Expected mime type application/octet-stream but got text/html. 
   [junit4]> 
   [junit4]> 
   [junit4]> Error 404 Can not find: 
/solr/testcollection_shard1_replica_n2/update
   [junit4]> 
   [junit4]> HTTP ERROR 404
   [junit4]> Problem accessing 
/solr/testcollection_shard1_replica_n2/update. Reason:
   [junit4]> Can not find: 
/solr/testcollection_shard1_replica_n2/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605
   [junit4]> 
   [junit4]> 
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]>at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]>at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:484)
   [junit4]>at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414)
   [junit4]>at 
org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
   [junit4]>at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]>at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
   [junit4]>at 

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 197 - Still unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/197/

3 tests failed.
FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRouting

Error Message:
Error from server at 
https://127.0.0.1:46403/solr/collection1_shard2_replica_n3: Expected mime type 
application/octet-stream but got text/html.Error 404 
Can not find: /solr/collection1_shard2_replica_n3/update  
HTTP ERROR 404 Problem accessing 
/solr/collection1_shard2_replica_n3/update. Reason: Can not find: 
/solr/collection1_shard2_replica_n3/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605  
  

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at https://127.0.0.1:46403/solr/collection1_shard2_replica_n3: Expected 
mime type application/octet-stream but got text/html. 


Error 404 Can not find: 
/solr/collection1_shard2_replica_n3/update

HTTP ERROR 404
Problem accessing /solr/collection1_shard2_replica_n3/update. Reason:
Can not find: 
/solr/collection1_shard2_replica_n3/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605




at 
__randomizedtesting.SeedInfo.seed([88D461272AF68EF4:4A635D4F29B67E8C]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:237)
at 
org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRouting(CloudSolrClientTest.java:269)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-12801) Fix the tests, remove BadApples and AwaitsFix annotations, improve env for test development.

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669532#comment-16669532
 ] 

Erick Erickson commented on SOLR-12801:
---

I get a compilation error: 

{quote}
compile-core:
 [mkdir] Created dir: 
/Users/Erick/apache/solrJiras/master/solr/build/solr-test-framework/classes/java
 [javac] Compiling 53 source files to 
/Users/Erick/apache/solrJiras/master/solr/build/solr-test-framework/classes/java
 [javac] 
/Users/Erick/apache/solrJiras/master/solr/test-framework/src/java/org/apache/solr/cloud/AbstractFullDistribZkTestBase.java:398:
 error: variable numOtherReplicas is already defined in method createJettys(int)
 [javac] int numOtherReplicas = numJettys - getPullReplicaCount() * sliceCount;
 [javac] ^
 [javac] Note: 
/Users/Erick/apache/solrJiras/master/solr/test-framework/src/java/org/apache/solr/SolrTestCaseJ4.java
 uses or overrides a deprecated API.
 [javac] Note: Recompile with -Xlint:deprecation for details.
 [javac] Note: Some input files use unchecked or unsafe operations.
 [javac] Note: Recompile with -Xlint:unchecked for details.
 [javac] 1 error
{quote}

Trivially fixable, I'm mostly wondering if there's something in your pull 
request that got messed up Or if I didn't apply it correctly. I fetched it 
to a patch file and applied the patch FWIW.

Running tests now.

> Fix the tests, remove BadApples and AwaitsFix annotations, improve env for 
> test development.
> 
>
> Key: SOLR-12801
> URL: https://issues.apache.org/jira/browse/SOLR-12801
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Critical
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> A single issue to counteract the single issue adding tons of annotations, the 
> continued addition of new flakey tests, and the continued addition of 
> flakiness to existing tests.
> Lots more to come.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12259) Robustly upgrade indexes

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669525#comment-16669525
 ] 

Erick Erickson commented on SOLR-12259:
---

I'm having a design problem here. At the Solr level, we have 
MergePolicyFactory, while at the Lucene level we have MergePolicy. All well and 
good.

However, I need to mix them up a bit and it's *A Bad Idea* to have Lucene be 
aware of anything having to do with Solr.

So I'm coding up a new MergePolicy and associated Factory. I can assign the 
MergePolicy to an IndexWriter and call writer.forceMerge and the writer calls 
findForcedMerges in my new policy. At that point I need to make decisions on 
which if any of the segments passed to findForcedMerges needs to be 
merged/rewritten. And some of the information I need is only available at the 
Solr level, in this case I need to compare the schema definitions against the 
info in the segments.

Ideally, I want to do something in findForcedMerges like:
{code:java}
for (each segment in segmentInfos) {      
   if (Solr thinks it should be rewritten)  { put it in the real merge list }
}
{code}
What I'm having trouble with is figuring out how to reach back out into Solr 
and evaluate the "if (Solr thinks it should be rewritten)" part without doing 
violence to Lucene.

It seems like a function pointer that I could set in my MergePolicy could do 
the trick, but that seems complicated. And even if I could, is that even 
acceptable architecturally since it's code in Solr? Albeit, there's no 
dependency on Solr here, a Java function in anybody's calling code could set 
it. It would take a seginfo and return true or false. Oh for the old C days 
when a function pointer was just an int

Or is the right thing to do just put any new MergePolicy in Solr where it _can_ 
be aware of other "Solr stuff"?

Any comments [~mikemccand] [~rcmuir] [~jpountz] [~romseygeek] ?

> Robustly upgrade indexes
> 
>
> Key: SOLR-12259
> URL: https://issues.apache.org/jira/browse/SOLR-12259
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Erick Erickson
>Assignee: Erick Erickson
>Priority: Major
>
> The general problem statement is that the current upgrade path is trappy and 
> cumbersome.  It would be a great help "in the field" to make the upgrade 
> process less painful.
> Additionally one of the most common things users want to do is enable 
> docValues, but currently they often have to re-index.
> Issues:
> 1> if I upgrade from 5x to 6x and then 7x, theres no guarantee that when I go 
> to 7x all the segments have been rewritten in 6x format. Say I have a segment 
> at max size that has no deletions. It'll never be rewritten until it has 
> deleted docs. And perhaps 50% deleted docs currently.
> 2> IndexUpgraderTool explicitly does a forcemerge to 1 segment, which is bad.
> 3> in a large distributed system, running IndexUpgraderTool on all the nodes 
> is cumbersome even if <2> is acceptable.
> 4> Users who realize specifying docValues on a field would be A Good Thing 
> have to re-index. We have UninvertDocValuesMergePolicyFactory. Wouldn't it be 
> nice to be able to have this done all at once without forceMerging to one 
> segment.
> Proposal:
> Somehow avoid the above. Currently LUCENE-7976 is a start in that direction. 
> It will make TMP respect max segments size so can avoid forceMerges that 
> result in one segment. What it does _not_ do is rewrite segments with zero 
> (or a small percentage) deleted documents.
> So it  doesn't seem like a huge stretch to be able to specify to TMP the 
> option to rewrite segments that have no deleted documents. Perhaps a new 
> parameter to optimize?
> This would likely require another change to TMP or whatever.
> So upgrading to a new solr would look like
> 1> install the new Solr
> 2> execute 
> "http://node:port/solr/collection_or_core/update?optimize=true=true;
> What's not clear to me is whether we'd require 
> UninvertDocValuesMergePolicyFactory to be specified and wrap TMP or not.
> Anyway, let's discuss. I'll create yet another LUCENE JIRA for TMP do rewrite 
> all segments that I'll link.
> I'll also link several other JIRAs in here, they're coalescing.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-11522) Suggestions/recommendations to rebalance replicas

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-11522:
-

Assignee: Noble Paul

> Suggestions/recommendations to rebalance replicas
> -
>
> Key: SOLR-11522
> URL: https://issues.apache.org/jira/browse/SOLR-11522
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: 7.6
>
>
> It is possible that a cluster is unbalanced even if it is not breaking any of 
> the policy rules. Some nodes may have very little load while some others may 
> be heavily loaded. So, it is possible to move replicas around so that the 
> load is more evenly distributed. This is going to be driven by preferences. 
> The way we arrive at these suggestions is going to be as follows
>  # Sort the nodes according to the given preferences
>  # Choose a replica from the most loaded node ({{source-node}}) 
>  # try adding them to the least loaded node ({{target-node}})
>  # See if it breaks any policy rules. If yes , try another {{target-node}} 
> (go to #3)
>  # If no policy rules are being broken, present this as a {{suggestion}} . 
> The suggestion contains the following information
>  #* The {{source-node}} and {{target-node}} names
>  #* The actual v2 command that can be run to effect the operation
>  # Go to step #1
>  # Do this until the a replicas can be moved in such a way that the {{target 
> node}} is more loaded than the {{source-node}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-11522) Suggestions/recommendations to rebalance replicas

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul resolved SOLR-11522.
---
   Resolution: Fixed
Fix Version/s: 7.6

> Suggestions/recommendations to rebalance replicas
> -
>
> Key: SOLR-11522
> URL: https://issues.apache.org/jira/browse/SOLR-11522
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: 7.6
>
>
> It is possible that a cluster is unbalanced even if it is not breaking any of 
> the policy rules. Some nodes may have very little load while some others may 
> be heavily loaded. So, it is possible to move replicas around so that the 
> load is more evenly distributed. This is going to be driven by preferences. 
> The way we arrive at these suggestions is going to be as follows
>  # Sort the nodes according to the given preferences
>  # Choose a replica from the most loaded node ({{source-node}}) 
>  # try adding them to the least loaded node ({{target-node}})
>  # See if it breaks any policy rules. If yes , try another {{target-node}} 
> (go to #3)
>  # If no policy rules are being broken, present this as a {{suggestion}} . 
> The suggestion contains the following information
>  #* The {{source-node}} and {{target-node}} names
>  #* The actual v2 command that can be run to effect the operation
>  # Go to step #1
>  # Do this until the a replicas can be moved in such a way that the {{target 
> node}} is more loaded than the {{source-node}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12938) ClusterStatus should not spew an exception trace if it gets an alias name

[Gus Heck (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669485#comment-16669485
 ] 

Gus Heck commented on SOLR-12938:
-

Attaching patch that does the following:

a) If its an alias, grab full cluster status and filter out any collections not 
in the alias, then continue as if it were full cluster status (i.e. null was 
passed for collection)

b) If the value passed for collection is neither a collection nor an alias, 
still throw the prior exception

c) Fix several spots in HttpClusterStateProvider that threw null pointer 
exceptions or had similar problems after the change due to an implicit 
assumption that there MUST be a collection available if CLUSTERSTATUS didn't 
throw.

d) Introduce a local exception meant only to be thrown between the private 
fetchClusterState methods and it's callers in the same class. This avoids the 
parsing of the exception message. The intent is the exception never escape from 
HttpClusterStateProvider, so it's a checked exception to make sure folks 
remember to catch it.

 

This introduces a small but I'm hoping tolerable backwards incompatibility for 
any code that relied on CLUSTERSTATUS to throw an exception for non-collection 
and non-null input

[~ab], [~ichattopadhyaya] I see your names on much of HttpClusterStateProvider 
so I would appreciate your review.

 

> ClusterStatus should not spew an exception trace if it gets an alias name
> -
>
> Key: SOLR-12938
> URL: https://issues.apache.org/jira/browse/SOLR-12938
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.5
>Reporter: Gus Heck
>Priority: Minor
> Attachments: SOLR-12938.patch
>
>
> This has been a lingering irritant in debugging tests for time routed 
> aliases, previously mentioned in SOLR-11949 and can be seen frequently in 
> logs attached to SOLR-12928. Basically what happens is for one reason or 
> another cluster status is called on an alias rather than a collection and 
> this is treated identically to a collection name that doesn't exist. 
> This also has lead this bit of lovely exception message parsing in 
> HttpClusteStateProvider.java
> {code:java}
>   } catch (SolrServerException | RemoteSolrException | IOException e) {
> if (e.getMessage().contains(collection + " not found")) {
>   // Cluster state for the given collection was not found.
>   // Lets fetch/update our aliases:
>   getAliases(true);
>   return null;
> }
> log.warn("Attempt to fetch cluster state from " +
> Utils.getBaseUrlForNodeName(nodeName, urlScheme) + " failed.", e);
>   }
> {code}
> Cluster status is already handled in the case of no collection name provided 
> by returning status on all collections. It would make more sense if this 
> command returned status on the component collections for the alias. 
> If that turns out to be difficult or cause too many problems this should at 
> least be downgraded to a non-stack trace warning message since this situation 
> does not represent a failure of the system. The error/stack should of course 
> be retained if neither a collection nor an alias exist.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8497) Rethink multi-term analysis handling

[Lucene/Solr QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669476#comment-16669476
 ] 

Lucene/Solr QA commented on LUCENE-8497:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 7 new or modified test 
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m  
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m  
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} | 
{color:green}  0m 43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} | 
{color:green}  0m 27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} | 
{color:green}  0m 27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  3m 
12s{color} | {color:green} common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
28s{color} | {color:green} icu in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  0m 
50s{color} | {color:green} kuromoji in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 48m 
35s{color} | {color:green} core in the patch passed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 10s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-8497 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12946196/LUCENE-8497.patch |
| Optional Tests |  compile  javac  unit  ratsources  checkforbiddenapis  
validatesourcepatterns  |
| uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon 
Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality | 
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
 |
| git revision | master / 856e28d |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | 1.8.0_172 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/114/testReport/ |
| modules | C: lucene/analysis/common lucene/analysis/icu 
lucene/analysis/kuromoji solr/core U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/114/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Rethink multi-term analysis handling
> 
>
> Key: LUCENE-8497
> URL: https://issues.apache.org/jira/browse/LUCENE-8497
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8497.patch, LUCENE-8497.patch, LUCENE-8497.patch, 
> LUCENE-8497.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> The current framework for handling term normalisation works via instanceof 
> checks for MultiTermAwareComponent and casts.  MultiTermAwareComponent itself 
> deals in AbstractAnalysisComponents, and so callers need to cast to the 
> correct component type before use, which is ripe for misuse.
> We should re-organise all this to be type-safe and usable without casts.  One 
> possibility is to add `normalize` methods to CharFilterFactory and 
> TokenFilterFactory that mirror their existing `create` methods.  The default 
> implementation would return the input unchanged, while filters that should 
> apply at normalization time can delegate to `create`.
> Related to this, we should deprecate and remove LowerCaseTokenizer, which 
> combines tokenization and normalization in a way that will break this API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12938) ClusterStatus should not spew an exception trace if it gets an alias name

[Gus Heck (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gus Heck updated SOLR-12938:

Attachment: SOLR-12938.patch

> ClusterStatus should not spew an exception trace if it gets an alias name
> -
>
> Key: SOLR-12938
> URL: https://issues.apache.org/jira/browse/SOLR-12938
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.5
>Reporter: Gus Heck
>Priority: Minor
> Attachments: SOLR-12938.patch
>
>
> This has been a lingering irritant in debugging tests for time routed 
> aliases, previously mentioned in SOLR-11949 and can be seen frequently in 
> logs attached to SOLR-12928. Basically what happens is for one reason or 
> another cluster status is called on an alias rather than a collection and 
> this is treated identically to a collection name that doesn't exist. 
> This also has lead this bit of lovely exception message parsing in 
> HttpClusteStateProvider.java
> {code:java}
>   } catch (SolrServerException | RemoteSolrException | IOException e) {
> if (e.getMessage().contains(collection + " not found")) {
>   // Cluster state for the given collection was not found.
>   // Lets fetch/update our aliases:
>   getAliases(true);
>   return null;
> }
> log.warn("Attempt to fetch cluster state from " +
> Utils.getBaseUrlForNodeName(nodeName, urlScheme) + " failed.", e);
>   }
> {code}
> Cluster status is already handled in the case of no collection name provided 
> by returning status on all collections. It would make more sense if this 
> command returned status on the component collections for the alias. 
> If that turns out to be difficult or cause too many problems this should at 
> least be downgraded to a non-stack trace warning message since this situation 
> does not represent a failure of the system. The error/stack should of course 
> be retained if neither a collection nor an alias exist.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12945) Cluster endpoint doesn't work

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-12945:
-
Component/s: v2 API

> Cluster endpoint doesn't work
> -
>
> Key: SOLR-12945
> URL: https://issues.apache.org/jira/browse/SOLR-12945
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: v2 API
>Reporter: Varun Thacker
>Priority: Major
>
> I have a 2 node cluster and then when I hit this URL  ( 
> [http://localhost:8983/api/cluster/collections]  ) with Solr 7.5 I run into 
> this error
> {code:java}
> 2018-10-31 00:03:42.565 ERROR (qtp690521419-111) [   ] o.a.s.a.V2HttpCall >> 
> path: '/cluster/collections/test_empty'
> 2018-10-31 00:03:42.565 ERROR (qtp690521419-111) [   ] o.a.s.a.V2HttpCall 
> Error in init()
> org.apache.solr.common.SolrException: no core retrieved for null
>     at org.apache.solr.api.V2HttpCall.init(V2HttpCall.java:141) 
> [solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi 
> - 2018-09-18 13:07:55]
>     at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469) 
> [solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi 
> - 2018-09-18 13:07:55]
>     at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
>  [solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi 
> - 2018-09-18 13:07:55]
>     at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
>  [solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi 
> - 2018-09-18 13:07:55]
>     at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
>  [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) 
> [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) 
> [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) 
> [jetty-security-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) 
> [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) 
> [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
>  [jetty-rewrite-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
>  [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at org.eclipse.jetty.server.Server.handle(Server.java:531) 
> [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
>     at 

[jira] [Created] (SOLR-12945) Cluster endpoint doesn't work

[Varun Thacker (JIRA)]

Varun Thacker created SOLR-12945:


 Summary: Cluster endpoint doesn't work
 Key: SOLR-12945
 URL: https://issues.apache.org/jira/browse/SOLR-12945
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Varun Thacker


I have a 2 node cluster and then when I hit this URL  ( 
[http://localhost:8983/api/cluster/collections]  ) with Solr 7.5 I run into 
this error
{code:java}
2018-10-31 00:03:42.565 ERROR (qtp690521419-111) [   ] o.a.s.a.V2HttpCall >> 
path: '/cluster/collections/test_empty'
2018-10-31 00:03:42.565 ERROR (qtp690521419-111) [   ] o.a.s.a.V2HttpCall Error 
in init()
org.apache.solr.common.SolrException: no core retrieved for null
    at org.apache.solr.api.V2HttpCall.init(V2HttpCall.java:141) 
[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 
2018-09-18 13:07:55]
    at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469) 
[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 
2018-09-18 13:07:55]
    at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:377)
 [solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 
2018-09-18 13:07:55]
    at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:323)
 [solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 
2018-09-18 13:07:55]
    at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1634)
 [jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) 
[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146) 
[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) 
[jetty-security-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) 
[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) 
[jetty-servlet-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) 
[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
 [jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) 
[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
 [jetty-rewrite-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) 
[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at org.eclipse.jetty.server.Server.handle(Server.java:531) 
[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352) 
[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) 
[jetty-server-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
 [jetty-io-9.4.11.v20180605.jar:9.4.11.v20180605]
    at 

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-9) - Build # 4903 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/4903/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseParallelGC

6 tests failed.
FAILED:  org.apache.lucene.store.TestByteBuffersDirectory.testSeekPastEOF 
{impl=byte array (heap)}

Error Message:
Unexpected exception type, expected EOFException but got 
java.lang.ArrayIndexOutOfBoundsException: 1770

Stack Trace:
junit.framework.AssertionFailedError: Unexpected exception type, expected 
EOFException but got java.lang.ArrayIndexOutOfBoundsException: 1770
at 
__randomizedtesting.SeedInfo.seed([BDFA8CEDB7C93AC1:5DBC4714B74C4450]:0)
at 
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2680)
at 
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2669)
at 
org.apache.lucene.store.BaseDirectoryTestCase.testSeekPastEOF(BaseDirectoryTestCase.java:516)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1770
at 
org.apache.lucene.store.ByteArrayIndexInput.readByte(ByteArrayIndexInput.java:145)
at 
org.apache.lucene.store.BaseDirectoryTestCase.lambda$testSeekPastEOF$12(BaseDirectoryTestCase.java:518)
at 
org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2675)
... 37 more


FAILED:  

[GitHub] lucene-solr pull request #486: SOLR-12801: Fix the tests, remove BadApples a...

[markrmiller]

Github user markrmiller commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/486#discussion_r229525959
  
--- Diff: 
solr/test-framework/src/java/org/apache/solr/cloud/MiniSolrCloudCluster.java ---
@@ -265,10 +281,30 @@ public MiniSolrCloudCluster(int numServers, Path 
baseDir, String solrXml, JettyC
 solrClient = buildSolrClient();
   }
 
-  private void waitForAllNodes(int numServers, int timeout) throws 
IOException, InterruptedException {
+  private void waitForAllNodes(int numServers, int timeoutSeconds) throws 
IOException, InterruptedException {
--- End diff --

Cool, I'll add, as well as a note about how you don't need to use this 
after starting the cluster with the builder - just when adding new jetties.


---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #486: SOLR-12801: Fix the tests, remove BadApples a...

[markrmiller]

Github user markrmiller commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/486#discussion_r229525878
  
--- Diff: solr/core/src/test-files/solr/solr.xml ---
@@ -40,12 +40,12 @@
 127.0.0.1
 ${hostPort:8983}
 ${hostContext:solr}
-${solr.zkclienttimeout:3}
+${solr.zkclienttimeout:45000} 
 ${genericCoreNodeNames:true}
-${leaderVoteWait:1}
-${leaderConflictResolveWait:18}
-${distribUpdateConnTimeout:45000}
-${distribUpdateSoTimeout:34}
+${leaderVoteWait:15000}   
--- End diff --

No, didn't necessarily want to lower it, just explain why it's set so low 
compared to what would be recommended in production (minutes).


---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #486: SOLR-12801: Fix the tests, remove BadApples a...

[vthacker]

Github user vthacker commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/486#discussion_r229523672
  
--- Diff: 
solr/test-framework/src/java/org/apache/solr/cloud/MiniSolrCloudCluster.java ---
@@ -265,10 +281,30 @@ public MiniSolrCloudCluster(int numServers, Path 
baseDir, String solrXml, JettyC
 solrClient = buildSolrClient();
   }
 
-  private void waitForAllNodes(int numServers, int timeout) throws 
IOException, InterruptedException {
+  private void waitForAllNodes(int numServers, int timeoutSeconds) throws 
IOException, InterruptedException {
--- End diff --

Some Javadocs that we could add here :

"This method wait till all Solr JVMs ( Jettys ) are running . It waits upto 
the timeout (in seconds) for the JVMs to be up before throwing 
IllegalStateException"


---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #486: SOLR-12801: Fix the tests, remove BadApples a...

[vthacker]

Github user vthacker commented on a diff in the pull request:

https://github.com/apache/lucene-solr/pull/486#discussion_r229522519
  
--- Diff: solr/core/src/test-files/solr/solr.xml ---
@@ -40,12 +40,12 @@
 127.0.0.1
 ${hostPort:8983}
 ${hostContext:solr}
-${solr.zkclienttimeout:3}
+${solr.zkclienttimeout:45000} 
 ${genericCoreNodeNames:true}
-${leaderVoteWait:1}
-${leaderConflictResolveWait:18}
-${distribUpdateConnTimeout:45000}
-${distribUpdateSoTimeout:34}
+${leaderVoteWait:15000}   
--- End diff --

The older explicitly set value for  leaderVoteWait was 1 and now it's 
increased to 15000 but the comment seems to indicate that we wanted to lower it.

Just wanted to make sure it isn't a typo


---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-12866) Reproducing TestLocalFSCloudBackupRestore and TestHdfsCloudBackupRestore failures

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669415#comment-16669415
 ] 

Varun Thacker edited comment on SOLR-12866 at 10/30/18 11:23 PM:
-

Here is a patch which tries to write a mock testing testing replica asssingment 
under the following conditions -
 # We have created a collection with createNodeSet=EMPTY , so clusterstate has 
already been created but no replicas are present ( testEmptyCollection.json )
 # Create one replica for all 3 shards of the collection on a 2 node cluster

I want to test if the assignment engine places all of them on only one node. 
This is what I am seeing in RestoreCmd as posted on my earlier comment so I 
want to see where are we going wrong.

 

Unfortunately as of now, I'm stuck in getting the mock to work. When i run the 
test I get this error
{code:java}
16:06:45.053 
[TEST-TestPolicy.testPolicyForEmptyCollection-seed#[33EAB712F76AB404]] ERROR 
org.apache.solr.client.solrj.cloud.autoscaling.Policy - Exception! prefs = [{
  "minimize":"cores",
  "precision":1}, {"maximize":"freedisk"}], recent r1 = node2, r2 = node1, 
matrix = 2


...
Caused by: org.apache.solr.common.SolrException
    at 
org.apache.solr.client.solrj.cloud.autoscaling.Policy.setApproxValuesAndSortNodes(Policy.java:314)
    at 
org.apache.solr.client.solrj.cloud.autoscaling.Policy$Session.applyRules(Policy.java:606)
...{code}
 The test doesn't like the fact that I have "replicaInfo" as empty.  So the 
preferences sort algorithm runs into a NullPointerException and throws the 
RuntimeException.


was (Author: varunthacker):
Here is a patch which tries to write a mock testing testing replica asssingment 
under the following conditions -
 # We have created a collection with createNodeSet=EMPTY , so clusterstate has 
already been created but no replicas are present ( testEmptyCollection.json )
 # Create one replica for all 3 shards of the collection on a 2 node cluster

I want to test if the assignment engine places all of them on only one node. 
This is what I am seeing in RestoreCmd as posted on my earlier comment so I 
want to see where are we going wrong.

 

Unfortunately as of now, I'm stuck in getting the mock to work. When i run the 
test I get this error

 
{code:java}
16:06:45.053 
[TEST-TestPolicy.testPolicyForEmptyCollection-seed#[33EAB712F76AB404]] ERROR 
org.apache.solr.client.solrj.cloud.autoscaling.Policy - Exception! prefs = [{
  "minimize":"cores",
  "precision":1}, {"maximize":"freedisk"}], recent r1 = node2, r2 = node1, 
matrix = 2


...
Caused by: org.apache.solr.common.SolrException
    at 
org.apache.solr.client.solrj.cloud.autoscaling.Policy.setApproxValuesAndSortNodes(Policy.java:314)
    at 
org.apache.solr.client.solrj.cloud.autoscaling.Policy$Session.applyRules(Policy.java:606)
...{code}
 

> Reproducing TestLocalFSCloudBackupRestore and TestHdfsCloudBackupRestore 
> failures
> -
>
> Key: SOLR-12866
> URL: https://issues.apache.org/jira/browse/SOLR-12866
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12866.patch
>
>
> From [https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/185/], 
> both tests failed 10/10 iterations for me on branch_7x with the seed:
> {noformat}
> Checking out Revision 37fdcb02d87ec44293ec4942c75a3cb709c45418 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestLocalFSCloudBackupRestore -Dtests.method=test 
> -Dtests.seed=3CD4284489C09DB4 -Dtests.multiplier=2 -Dtests.slow=true 
> -Dtests.badapples=true -Dtests.locale=mk-MK 
> -Dtests.timezone=Pacific/Kiritimati -Dtests.asserts=true 
> -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 10.8s J2 | TestLocalFSCloudBackupRestore.test <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Node 
> 127.0.0.1:43864_solr has 3 replicas. Expected num replicas : 2. state: 
>[junit4]> 
> DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/9)={
>[junit4]>   "pullReplicas":0,
>[junit4]>   "replicationFactor":1,
>[junit4]>   "shards":{
>[junit4]> "shard2":{
>[junit4]>   "range":"0-7fff",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node62":{
>[junit4]>   "core":"backuprestore_restored_shard2_replica_n61",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>

[jira] [Comment Edited] (SOLR-12866) Reproducing TestLocalFSCloudBackupRestore and TestHdfsCloudBackupRestore failures

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669415#comment-16669415
 ] 

Varun Thacker edited comment on SOLR-12866 at 10/30/18 11:07 PM:
-

Here is a patch which tries to write a mock testing testing replica asssingment 
under the following conditions -
 # We have created a collection with createNodeSet=EMPTY , so clusterstate has 
already been created but no replicas are present ( testEmptyCollection.json )
 # Create one replica for all 3 shards of the collection on a 2 node cluster

I want to test if the assignment engine places all of them on only one node. 
This is what I am seeing in RestoreCmd as posted on my earlier comment so I 
want to see where are we going wrong.

 

Unfortunately as of now, I'm stuck in getting the mock to work. When i run the 
test I get this error

 
{code:java}
16:06:45.053 
[TEST-TestPolicy.testPolicyForEmptyCollection-seed#[33EAB712F76AB404]] ERROR 
org.apache.solr.client.solrj.cloud.autoscaling.Policy - Exception! prefs = [{
  "minimize":"cores",
  "precision":1}, {"maximize":"freedisk"}], recent r1 = node2, r2 = node1, 
matrix = 2


...
Caused by: org.apache.solr.common.SolrException
    at 
org.apache.solr.client.solrj.cloud.autoscaling.Policy.setApproxValuesAndSortNodes(Policy.java:314)
    at 
org.apache.solr.client.solrj.cloud.autoscaling.Policy$Session.applyRules(Policy.java:606)
...{code}
 


was (Author: varunthacker):
Here is a patch which tries to write a mock testing testing replica asssingment 
under the following conditions -
 # We have created a collection with createNodeSet=EMPTY , so clusterstate has 
already been created but no replicas are present ( testEmptyCollection.json )
 # Create one replica for all 3 shards of the collection on a 2 node cluster

I want to test if the assignment engine places all of them on only one node. 
This is what I am seeing in RestoreCmd as posted on my earlier comment so I 
want to see where are we going wrong.

> Reproducing TestLocalFSCloudBackupRestore and TestHdfsCloudBackupRestore 
> failures
> -
>
> Key: SOLR-12866
> URL: https://issues.apache.org/jira/browse/SOLR-12866
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12866.patch
>
>
> From [https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/185/], 
> both tests failed 10/10 iterations for me on branch_7x with the seed:
> {noformat}
> Checking out Revision 37fdcb02d87ec44293ec4942c75a3cb709c45418 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestLocalFSCloudBackupRestore -Dtests.method=test 
> -Dtests.seed=3CD4284489C09DB4 -Dtests.multiplier=2 -Dtests.slow=true 
> -Dtests.badapples=true -Dtests.locale=mk-MK 
> -Dtests.timezone=Pacific/Kiritimati -Dtests.asserts=true 
> -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 10.8s J2 | TestLocalFSCloudBackupRestore.test <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Node 
> 127.0.0.1:43864_solr has 3 replicas. Expected num replicas : 2. state: 
>[junit4]> 
> DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/9)={
>[junit4]>   "pullReplicas":0,
>[junit4]>   "replicationFactor":1,
>[junit4]>   "shards":{
>[junit4]> "shard2":{
>[junit4]>   "range":"0-7fff",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node62":{
>[junit4]>   "core":"backuprestore_restored_shard2_replica_n61",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>[junit4]>   "force_set_state":"false",
>[junit4]>   "leader":"true"}},
>[junit4]>   "stateTimestamp":"1539459703266853250"},
>[junit4]> "shard1_1":{
>[junit4]>   "range":"c000-",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node64":{
>[junit4]>   
> "core":"backuprestore_restored_shard1_1_replica_n63",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>[junit4]>   "force_set_state":"false",
>[junit4]>   "leader":"true"}},
>[junit4]>   "stateTimestamp":"1539459703266887720"},
>

[jira] [Commented] (SOLR-12866) Reproducing TestLocalFSCloudBackupRestore and TestHdfsCloudBackupRestore failures

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669415#comment-16669415
 ] 

Varun Thacker commented on SOLR-12866:
--

Here is a patch which tries to write a mock testing testing replica asssingment 
under the following conditions -
 # We have created a collection with createNodeSet=EMPTY , so clusterstate has 
already been created but no replicas are present ( testEmptyCollection.json )
 # Create one replica for all 3 shards of the collection on a 2 node cluster

I want to test if the assignment engine places all of them on only one node. 
This is what I am seeing in RestoreCmd as posted on my earlier comment so I 
want to see where are we going wrong.

> Reproducing TestLocalFSCloudBackupRestore and TestHdfsCloudBackupRestore 
> failures
> -
>
> Key: SOLR-12866
> URL: https://issues.apache.org/jira/browse/SOLR-12866
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12866.patch
>
>
> From [https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/185/], 
> both tests failed 10/10 iterations for me on branch_7x with the seed:
> {noformat}
> Checking out Revision 37fdcb02d87ec44293ec4942c75a3cb709c45418 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestLocalFSCloudBackupRestore -Dtests.method=test 
> -Dtests.seed=3CD4284489C09DB4 -Dtests.multiplier=2 -Dtests.slow=true 
> -Dtests.badapples=true -Dtests.locale=mk-MK 
> -Dtests.timezone=Pacific/Kiritimati -Dtests.asserts=true 
> -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 10.8s J2 | TestLocalFSCloudBackupRestore.test <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Node 
> 127.0.0.1:43864_solr has 3 replicas. Expected num replicas : 2. state: 
>[junit4]> 
> DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/9)={
>[junit4]>   "pullReplicas":0,
>[junit4]>   "replicationFactor":1,
>[junit4]>   "shards":{
>[junit4]> "shard2":{
>[junit4]>   "range":"0-7fff",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node62":{
>[junit4]>   "core":"backuprestore_restored_shard2_replica_n61",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>[junit4]>   "force_set_state":"false",
>[junit4]>   "leader":"true"}},
>[junit4]>   "stateTimestamp":"1539459703266853250"},
>[junit4]> "shard1_1":{
>[junit4]>   "range":"c000-",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node64":{
>[junit4]>   
> "core":"backuprestore_restored_shard1_1_replica_n63",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>[junit4]>   "force_set_state":"false",
>[junit4]>   "leader":"true"}},
>[junit4]>   "stateTimestamp":"1539459703266887720"},
>[junit4]> "shard1_0":{
>[junit4]>   "range":"8000-bfff",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node66":{
>[junit4]>   
> "core":"backuprestore_restored_shard1_0_replica_n65",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>[junit4]>   "force_set_state":"false",
>[junit4]>   "leader":"true"}},
>[junit4]>   "stateTimestamp":"1539459703266910800"}},
>[junit4]>   "router":{
>[junit4]> "name":"compositeId",
>[junit4]> "field":"shard_s"},
>[junit4]>   "maxShardsPerNode":"-1",
>[junit4]>   "autoAddReplicas":"false",
>[junit4]>   "nrtReplicas":1,
>[junit4]>   "tlogReplicas":0}
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([3CD4284489C09DB4:B480179E273CF04C]:0)
>[junit4]>  at 
> org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
>[junit4]>  at 

[jira] [Updated] (SOLR-12866) Reproducing TestLocalFSCloudBackupRestore and TestHdfsCloudBackupRestore failures

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-12866:
-
Attachment: SOLR-12866.patch

> Reproducing TestLocalFSCloudBackupRestore and TestHdfsCloudBackupRestore 
> failures
> -
>
> Key: SOLR-12866
> URL: https://issues.apache.org/jira/browse/SOLR-12866
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12866.patch
>
>
> From [https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/185/], 
> both tests failed 10/10 iterations for me on branch_7x with the seed:
> {noformat}
> Checking out Revision 37fdcb02d87ec44293ec4942c75a3cb709c45418 
> (refs/remotes/origin/branch_7x)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestLocalFSCloudBackupRestore -Dtests.method=test 
> -Dtests.seed=3CD4284489C09DB4 -Dtests.multiplier=2 -Dtests.slow=true 
> -Dtests.badapples=true -Dtests.locale=mk-MK 
> -Dtests.timezone=Pacific/Kiritimati -Dtests.asserts=true 
> -Dtests.file.encoding=US-ASCII
>[junit4] FAILURE 10.8s J2 | TestLocalFSCloudBackupRestore.test <<<
>[junit4]> Throwable #1: java.lang.AssertionError: Node 
> 127.0.0.1:43864_solr has 3 replicas. Expected num replicas : 2. state: 
>[junit4]> 
> DocCollection(backuprestore_restored//collections/backuprestore_restored/state.json/9)={
>[junit4]>   "pullReplicas":0,
>[junit4]>   "replicationFactor":1,
>[junit4]>   "shards":{
>[junit4]> "shard2":{
>[junit4]>   "range":"0-7fff",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node62":{
>[junit4]>   "core":"backuprestore_restored_shard2_replica_n61",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>[junit4]>   "force_set_state":"false",
>[junit4]>   "leader":"true"}},
>[junit4]>   "stateTimestamp":"1539459703266853250"},
>[junit4]> "shard1_1":{
>[junit4]>   "range":"c000-",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node64":{
>[junit4]>   
> "core":"backuprestore_restored_shard1_1_replica_n63",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>[junit4]>   "force_set_state":"false",
>[junit4]>   "leader":"true"}},
>[junit4]>   "stateTimestamp":"1539459703266887720"},
>[junit4]> "shard1_0":{
>[junit4]>   "range":"8000-bfff",
>[junit4]>   "state":"active",
>[junit4]>   "replicas":{"core_node66":{
>[junit4]>   
> "core":"backuprestore_restored_shard1_0_replica_n65",
>[junit4]>   "base_url":"https://127.0.0.1:43864/solr;,
>[junit4]>   "node_name":"127.0.0.1:43864_solr",
>[junit4]>   "state":"active",
>[junit4]>   "type":"NRT",
>[junit4]>   "force_set_state":"false",
>[junit4]>   "leader":"true"}},
>[junit4]>   "stateTimestamp":"1539459703266910800"}},
>[junit4]>   "router":{
>[junit4]> "name":"compositeId",
>[junit4]> "field":"shard_s"},
>[junit4]>   "maxShardsPerNode":"-1",
>[junit4]>   "autoAddReplicas":"false",
>[junit4]>   "nrtReplicas":1,
>[junit4]>   "tlogReplicas":0}
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([3CD4284489C09DB4:B480179E273CF04C]:0)
>[junit4]>  at 
> org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.lambda$testBackupAndRestore$1(AbstractCloudBackupRestoreTestCase.java:339)
>[junit4]>  at java.util.HashMap.forEach(HashMap.java:1289)
>[junit4]>  at 
> org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:338)
>[junit4]>  at 
> org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:144)
>[junit4]>  at 
> org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test(TestLocalFSCloudBackupRestore.java:64)
>[junit4]>  at java.lang.Thread.run(Thread.java:748)
> 

[JENKINS] Lucene-Solr-BadApples-NightlyTests-7.x - Build # 33 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-NightlyTests-7.x/33/

11 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
Timeout occured while waiting response from server at: 
http://127.0.0.1:36095/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting 
response from server at: http://127.0.0.1:36095/collection1
at 
__randomizedtesting.SeedInfo.seed([27819D609D1F050:8A2C260CA72D9DA8]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1107)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:504)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:479)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1590)
at 
org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:215)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Created] (SOLR-12944) Bugs around createNodeSet=EMPTY

[Varun Thacker (JIRA)]

Varun Thacker created SOLR-12944:


 Summary: Bugs around createNodeSet=EMPTY 
 Key: SOLR-12944
 URL: https://issues.apache.org/jira/browse/SOLR-12944
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Varun Thacker


Firstly, As of today we cannot create an empty collection from SolrJ

I have a two node cluster and this API call fails with an error
{code:java}
    //Create a coreless collection of 3 shards
    CollectionAdminRequest.Create create = CollectionAdminRequest
    .createCollection("test_collection", "conf1", 3,   0)
    
.setCreateNodeSet(OverseerCollectionMessageHandler.CREATE_NODE_SET_EMPTY)
    .setMaxShardsPerNode(-1);{code}


Secondly if I use the API directly , 
{{[http://localhost:8983/solr/admin/collections?action=create=test_coll=5=EMPTY]
 }}the state.json has replicationFactor = nrtReplicas = 1 instead of 0


{code:java}
"test_coll":{
    "pullReplicas":"0",
    "replicationFactor":"1",
    "router":{"name":"compositeId"},
    "maxShardsPerNode":"1",
    "autoAddReplicas":"false",
    "nrtReplicas":"1",
    "tlogReplicas":"0",
    "shards":{
  "shard1":{
    "range":"8000-b332",
    "state":"active",
    "replicas":{}},
  "shard2":{{code}

On a side note , I think we should rethink how 
replicationFactor/nrtReplicas/tlogReplicas/pullReplicas are stored.
These values could be stored at a per shard level such that adding a replica 
will actually refelct the total replication count



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 202 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/202/

5 tests failed.
FAILED:  org.apache.solr.cloud.TestLockTree.testLocks

Error Message:
expected:<2> but was:<3>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at 
__randomizedtesting.SeedInfo.seed([A3279212BDE40515:DAEDE8EBB534EEFE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.TestLockTree.testLocks(TestLockTree.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete

Error Message:
Error from server at 
http://127.0.0.1:45994/solr/testcollection_shard1_replica_n3: 

[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk-10.0.1) - Build # 3006 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3006/
Java: 64bit/jdk-10.0.1 -XX:+UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:42563/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:42563/solr
at 
__randomizedtesting.SeedInfo.seed([D762B8612E640B0F:1692C1CD0334C1A8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at 
org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:43281/solr

Stack Trace:

[jira] [Updated] (SOLR-12939) standardise test class naming

[Christine Poerschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke updated SOLR-12939:
---
Attachment: SOLR-12939.02.patch

> standardise test class naming
> -
>
> Key: SOLR-12939
> URL: https://issues.apache.org/jira/browse/SOLR-12939
> Project: Solr
>  Issue Type: Test
>Reporter: Christine Poerschke
>Priority: Major
> Attachments: SOLR-12939.01.patch, SOLR-12939.02.patch
>
>
> This was mentioned and proposed on the dev mailing list. Starting this ticket 
> here to start to make it happen?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12939) standardise test class naming

[Christine Poerschke (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669330#comment-16669330
 ] 

Christine Poerschke commented on SOLR-12939:


bq. ... the Test suffix rather than prefix. ...

I have no strong views either way. Instead of choosing one or the other a 
hybrid solution could also be to permit both but to enforce that within a given 
directory/sub-package/package there is consistency.

Illustrative SOLR-12939.02.patch patch to follow on how {{ant 
validate-source-patterns}} (which runs as part of {{ant precommit}}) could 
incrementally enforce that.

> standardise test class naming
> -
>
> Key: SOLR-12939
> URL: https://issues.apache.org/jira/browse/SOLR-12939
> Project: Solr
>  Issue Type: Test
>Reporter: Christine Poerschke
>Priority: Major
> Attachments: SOLR-12939.01.patch
>
>
> This was mentioned and proposed on the dev mailing list. Starting this ticket 
> here to start to make it happen?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Update on the test fixing effort.

[Mark Miller]

Okay, here is my current work:
https://github.com/apache/lucene-solr/pull/486

Please check it out and try 'ant clean test' in the solr directory. Report
your fails in SOLR-12932 .

I'll work on those fails. If you want to help, you could do the same.


Mark

On Tue, Oct 30, 2018 at 2:57 PM Mark Miller  wrote:

> Here is something that would be useful someone might want to help with.
>
> I'm working on LUCENE-8541: Fix ant beast to not overwrite junit xml
> results for each beast.iters iteration.
>
> You can see there are a few linked issues around that.
>
> One thing that would be cool that I am not working on:
>
> Most junit xml result reporters don't handle multiple results for the same
> test correctly.
>
> I'm making it really easy to run 'ant beast' for a whole module or based
> on test matching patterns, which then leaves a bunch of folders with result
> xml files in them, one folder per beast round. We really could use a junit
> reporter that scans all those folders and processes all the xml files so
> it's easy to browse which test runs failed and what the fail message was
> for each test that was beasted. Same as the result html files that we
> produce now, but that that take into account multiple result xml files per
> test#method combination.
>
> - Mark
>
> On Tue, Oct 30, 2018 at 1:33 PM Mark Miller  wrote:
>
>> I've gotten a few pings on more concrete ideas on helping out. I'll keep
>> working on that over time, but I'll share a starting blurb here. I'm about
>> to share my first patch. I'll ping this thread when it's up. One thing you
>> can do is check that out and try 'ant test'. Report any fails you see
>> to SOLR-12932. If you have time, try to fix it. It's hard to do this with
>> any confidence without beasting the test.
>>
>> Also, in no particular order:
>>
>> tlog replica types rely on waitForInSyncWithLeader - this wait will often
>> wait even when indexes are in sync though, and just exhaust the timeout
>> every time. This needs to be fixed for these tests to be re-enabled.
>>
>> I've done some work to stabilize the auto scaling tests, but more needs
>> to be done. They need to be beasted and awaitsfix annotations removed after
>> fixing.
>>
>> @Nightly tests need to be beasted and addressed.
>>
>> Please report any ant tests fails you see on the command line
>> here: SOLR-12932
>>
>> Review tests with @AwaitsFix, try to address them.
>>
>> Beast tests that you write or change. Improvements and doc coming to help.
>>
>> Take your time committing. Run ant test more than once. When you see test
>> fails, report them. Better yet, look into them. If you are making quick
>> commits, you will almost certainly contribute to our test problem unless
>> you have some super tight and isolated unit test.
>>
>> Know how long it takes to run the tests on your machine! Pay attention to
>> fluctuation after making non trivial changes! On my old 6 core machine
>> running 8jvms at a time, I get this with my patch:
>>
>> BUILD SUCCESSFUL
>> Total time: 22 minutes 37 seconds
>>
>> When I see it say 40 minutes, I'll know something has gone wrong!
>>
>> As things progress, I'll bring up more. This is focusing on short term
>> firefighting. For that to be meaningful, there will be a lot of prevention
>> work to do as well. I've still got some work to do before I have more to
>> say on that.
>>
>> - Mark
>>
>> On Tue, Oct 30, 2018 at 11:03 AM Erick Erickson 
>> wrote:
>>
>>> Sounds great, I'm anxious to see it in action.
>>>
>>> Let me know if I can help with beasting. I have two machines I can use.
>>>
>>> Of course having volunteered just about when things are getting under
>>> way I'll be gone Friday->Monday. Siii.
>>>
>>> Erick
>>> On Tue, Oct 30, 2018 at 7:34 AM Mark Miller 
>>> wrote:
>>> >
>>> > We are off to the races ...
>>> >
>>> > With any luck I'll put my first patch up on SOLR-12801 today. It works
>>> towards addressing 3-4 of the sub issues. Most importantly to me, it gives
>>> ant test a chance to pass.
>>> >
>>> > This was about 40 hours of almost continuous work and it is really
>>> just the start. I've addressed a ton of test stuff, but we are not even
>>> close to done. All of these tests need to be beasted thoroughly and
>>> defended for a start. That is the effort I'll be taking on by package.
>>> >
>>> > I'm working on making beasting super simple and adding doc for it. I'm
>>> also working on making precommit do a light beasting to any new or modified
>>> tests. I'm also working on some other things.
>>> >
>>> > ant test was really, really bad. I'm going to need a lot of help to
>>> make all this effort not turn into nothing. Without some real effort and
>>> change, I promise these improvements will melt away faster than they are
>>> felt.
>>> >
>>> > - Mark
>>> > --
>>> > - Mark
>>> > about.me/markrmiller
>>>
>> --
>> - Mark
>> about.me/markrmiller
>>
> --
> - Mark
> about.me/markrmiller
>
-- 
- Mark
about.me/markrmiller

[jira] [Commented] (SOLR-12801) Fix the tests, remove BadApples and AwaitsFix annotations, improve env for test development.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669299#comment-16669299
 ] 

Mark Miller commented on SOLR-12801:


Okay, here is my current work: https://github.com/apache/lucene-solr/pull/486

Please check it out and try 'ant clean test' in the solr directory. Report your 
fails in SOLR-12932.

I'll work on those fails. If you want to help, you could do the same.


> Fix the tests, remove BadApples and AwaitsFix annotations, improve env for 
> test development.
> 
>
> Key: SOLR-12801
> URL: https://issues.apache.org/jira/browse/SOLR-12801
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Critical
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> A single issue to counteract the single issue adding tons of annotations, the 
> continued addition of new flakey tests, and the continued addition of 
> flakiness to existing tests.
> Lots more to come.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12932) ant test (without badapples=false) should pass easily for developers.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669293#comment-16669293
 ] 

Mark Miller commented on SOLR-12932:


Test Fail: HttpPartitionTest.test
Fail message:
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HttpPartitionTest 
-Dtests.method=test -Dtests.seed=3C00D32E6729C090 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=ar-OM 
-Dtests.timezone=Australia/Melbourne -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8
   [junit4] FAILURE 42.1s J1 | HttpPartitionTest.test <<<
   [junit4]> Throwable #1: java.lang.AssertionError: max version bucket 
seed not updated after recovery!
   [junit4]>  at 
__randomizedtesting.SeedInfo.seed([3C00D32E6729C090:B454ECF4C9D5AD68]:0)
   [junit4]>  at 
org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:364)
   [junit4]>  at 
org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:148)

> ant test (without badapples=false) should pass easily for developers.
> -
>
> Key: SOLR-12932
> URL: https://issues.apache.org/jira/browse/SOLR-12932
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
>
> If we fix the tests we will end up here anyway, but we can shortcut this.
> Once I get my first patch in, anyone who mentions a test that fails locally 
> for them at any time (not jenkins), I will fix it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12932) ant test (without badapples=false) should pass easily for developers.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669294#comment-16669294
 ] 

Mark Miller commented on SOLR-12932:


Test: OverseerTest
Fail message:
Unknown.

> ant test (without badapples=false) should pass easily for developers.
> -
>
> Key: SOLR-12932
> URL: https://issues.apache.org/jira/browse/SOLR-12932
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Tests
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
>
> If we fix the tests we will end up here anyway, but we can shortcut this.
> Once I get my first patch in, anyone who mentions a test that fails locally 
> for them at any time (not jenkins), I will fix it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #486: SOLR-12801: Fix the tests, remove BadApples a...

[markrmiller]

GitHub user markrmiller opened a pull request:

https://github.com/apache/lucene-solr/pull/486

SOLR-12801: Fix the tests, remove BadApples and AwaitsFix annotations…

…, improve env for test development.

SOLR-12804: Remove static modifier from Overseer queue access.

SOLR-12896: Introduce more checks for shutdown and closed to improve clean 
close and shutdown. (Partial)

SOLR-12897: Introduce AlreadyClosedException to clean up silly close / 
shutdown logging. (Partial)

SOLR-12898: Replace cluster state polling with ZkStateReader#waitFor. 
(Partial)

SOLR-12923: The new AutoScaling tests are way to flaky and need special 
attention. (Partial)

SOLR-12932: ant test (without badapples=false) should pass easily for 
developers. (Partial)

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/markrmiller/starburst SOLR-12801

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/486.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #486


commit 01c58d67d4e55251346f99f414f4181947e7024f
Author: markrmiller 
Date:   2018-10-30T20:26:23Z

SOLR-12801: Fix the tests, remove BadApples and AwaitsFix annotations, 
improve env for test development.

SOLR-12804: Remove static modifier from Overseer queue access.

SOLR-12896: Introduce more checks for shutdown and closed to improve clean 
close and shutdown. (Partial)

SOLR-12897: Introduce AlreadyClosedException to clean up silly close / 
shutdown logging. (Partial)

SOLR-12898: Replace cluster state polling with ZkStateReader#waitFor. 
(Partial)

SOLR-12923: The new AutoScaling tests are way to flaky and need special 
attention. (Partial)

SOLR-12932: ant test (without badapples=false) should pass easily for 
developers. (Partial)




---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_172) - Build # 7596 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7596/
Java: 64bit/jdk1.8.0_172 -XX:+UseCompressedOops -XX:+UseG1GC

5 tests failed.
FAILED:  org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly

Error Message:
Unexpected number of elements in the group for intGSF: 8

Stack Trace:
java.lang.AssertionError: Unexpected number of elements in the group for 
intGSF: 8
at 
__randomizedtesting.SeedInfo.seed([16384BF24166E9AB:8D8325AA0C3EDBF5]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.testGroupingDVOnly(DocValuesNotIndexedTest.java:381)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.TestSkipOverseerOperations.testSkipLeaderOperations

Error Message:
Expected 2x1 for collection: collection1 null Live Nodes: 
[127.0.0.1:62781_solr, 127.0.0.1:62784_solr, 127.0.0.1:62785_solr] Last 
available state: null

Stack Trace:

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-9.0.4) - Build # 23123 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23123/
Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

7 tests failed.
FAILED:  
org.apache.solr.client.solrj.impl.CloudSolrClientTest.testParallelUpdateQTime

Error Message:
Error from server at http://127.0.0.1:34579/solr/collection1_shard2_replica_n3: 
Expected mime type application/octet-stream but got text/html.   
 
Error 404 Can not find: 
/solr/collection1_shard2_replica_n3/update  HTTP ERROR 
404 Problem accessing /solr/collection1_shard2_replica_n3/update. 
Reason: Can not find: 
/solr/collection1_shard2_replica_n3/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605  
  

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:34579/solr/collection1_shard2_replica_n3: Expected 
mime type application/octet-stream but got text/html. 


Error 404 Can not find: 
/solr/collection1_shard2_replica_n3/update

HTTP ERROR 404
Problem accessing /solr/collection1_shard2_replica_n3/update. Reason:
Can not find: 
/solr/collection1_shard2_replica_n3/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605




at 
__randomizedtesting.SeedInfo.seed([6A381C6697DF6554:84E067F7D9A0B0C7]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.impl.CloudSolrClientTest.testParallelUpdateQTime(CloudSolrClientTest.java:146)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 

Re: Update on the test fixing effort.

[Mark Miller]

Here is something that would be useful someone might want to help with.

I'm working on LUCENE-8541: Fix ant beast to not overwrite junit xml
results for each beast.iters iteration.

You can see there are a few linked issues around that.

One thing that would be cool that I am not working on:

Most junit xml result reporters don't handle multiple results for the same
test correctly.

I'm making it really easy to run 'ant beast' for a whole module or based on
test matching patterns, which then leaves a bunch of folders with result
xml files in them, one folder per beast round. We really could use a junit
reporter that scans all those folders and processes all the xml files so
it's easy to browse which test runs failed and what the fail message was
for each test that was beasted. Same as the result html files that we
produce now, but that that take into account multiple result xml files per
test#method combination.

- Mark

On Tue, Oct 30, 2018 at 1:33 PM Mark Miller  wrote:

> I've gotten a few pings on more concrete ideas on helping out. I'll keep
> working on that over time, but I'll share a starting blurb here. I'm about
> to share my first patch. I'll ping this thread when it's up. One thing you
> can do is check that out and try 'ant test'. Report any fails you see
> to SOLR-12932. If you have time, try to fix it. It's hard to do this with
> any confidence without beasting the test.
>
> Also, in no particular order:
>
> tlog replica types rely on waitForInSyncWithLeader - this wait will often
> wait even when indexes are in sync though, and just exhaust the timeout
> every time. This needs to be fixed for these tests to be re-enabled.
>
> I've done some work to stabilize the auto scaling tests, but more needs to
> be done. They need to be beasted and awaitsfix annotations removed after
> fixing.
>
> @Nightly tests need to be beasted and addressed.
>
> Please report any ant tests fails you see on the command line
> here: SOLR-12932
>
> Review tests with @AwaitsFix, try to address them.
>
> Beast tests that you write or change. Improvements and doc coming to help.
>
> Take your time committing. Run ant test more than once. When you see test
> fails, report them. Better yet, look into them. If you are making quick
> commits, you will almost certainly contribute to our test problem unless
> you have some super tight and isolated unit test.
>
> Know how long it takes to run the tests on your machine! Pay attention to
> fluctuation after making non trivial changes! On my old 6 core machine
> running 8jvms at a time, I get this with my patch:
>
> BUILD SUCCESSFUL
> Total time: 22 minutes 37 seconds
>
> When I see it say 40 minutes, I'll know something has gone wrong!
>
> As things progress, I'll bring up more. This is focusing on short term
> firefighting. For that to be meaningful, there will be a lot of prevention
> work to do as well. I've still got some work to do before I have more to
> say on that.
>
> - Mark
>
> On Tue, Oct 30, 2018 at 11:03 AM Erick Erickson 
> wrote:
>
>> Sounds great, I'm anxious to see it in action.
>>
>> Let me know if I can help with beasting. I have two machines I can use.
>>
>> Of course having volunteered just about when things are getting under
>> way I'll be gone Friday->Monday. Siii.
>>
>> Erick
>> On Tue, Oct 30, 2018 at 7:34 AM Mark Miller 
>> wrote:
>> >
>> > We are off to the races ...
>> >
>> > With any luck I'll put my first patch up on SOLR-12801 today. It works
>> towards addressing 3-4 of the sub issues. Most importantly to me, it gives
>> ant test a chance to pass.
>> >
>> > This was about 40 hours of almost continuous work and it is really just
>> the start. I've addressed a ton of test stuff, but we are not even close to
>> done. All of these tests need to be beasted thoroughly and defended for a
>> start. That is the effort I'll be taking on by package.
>> >
>> > I'm working on making beasting super simple and adding doc for it. I'm
>> also working on making precommit do a light beasting to any new or modified
>> tests. I'm also working on some other things.
>> >
>> > ant test was really, really bad. I'm going to need a lot of help to
>> make all this effort not turn into nothing. Without some real effort and
>> change, I promise these improvements will melt away faster than they are
>> felt.
>> >
>> > - Mark
>> > --
>> > - Mark
>> > about.me/markrmiller
>>
> --
> - Mark
> about.me/markrmiller
>
-- 
- Mark
about.me/markrmiller

[jira] [Commented] (SOLR-12875) ArrayIndexOutOfBoundsException when using uniqueBlock(_root_) in JSON Facets

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669191#comment-16669191
 ] 

Mikhail Khludnev commented on SOLR-12875:
-

Note, it make sense to check that all aggregations implement {{resize()}} to 
work with DVHASH. 

> ArrayIndexOutOfBoundsException when using uniqueBlock(_root_) in JSON Facets
> 
>
> Key: SOLR-12875
> URL: https://issues.apache.org/jira/browse/SOLR-12875
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Assignee: Mikhail Khludnev
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-12875.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I'm seeing java.lang.ArrayIndexOutOfBoundsException exceptions for some 
> requests when trying to make use of
> {noformat}
> uniqueBlock(_root_){noformat}
> within JSON Facets.
> Here are some example Stack Traces:
> {noformat}
> 2018-10-12 14:08:50.587 ERROR (qtp215078753-3353) [   x:my_core] 
> o.a.s.s.HttpSolrCall null:java.lang.ArrayIndexOutOfBoundsException: Index 13 
> out of bounds for length 8
> at 
> org.apache.solr.search.facet.UniqueBlockAgg$UniqueBlockSlotAcc.collectOrdToSlot(UniqueBlockAgg.java:40)
> at 
> org.apache.solr.search.facet.UniqueSinglevaluedSlotAcc.collect(UniqueSinglevaluedSlotAcc.java:85)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.collectFirstPhase(FacetFieldProcessor.java:243)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.collectValFirstPhase(FacetFieldProcessorByHashDV.java:432)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.access$100(FacetFieldProcessorByHashDV.java:50)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV$5.collect(FacetFieldProcessorByHashDV.java:395)
> at 
> org.apache.solr.search.DocSetUtil.collectSortedDocSet(DocSetUtil.java:284)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.collectDocs(FacetFieldProcessorByHashDV.java:376)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.calcFacets(FacetFieldProcessorByHashDV.java:247)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.process(FacetFieldProcessorByHashDV.java:214)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:472)
> at 
> org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:429)
> at 
> org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetModule.process(FacetModule.java:139)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> {noformat}
>  
> Here is another one at a different location in UniqueBlockAgg:
>   
> {noformat}
> 2018-10-12 21:37:57.322 ERROR (qtp215078753-4072) [   x:my_core] 
> o.a.s.h.RequestHandlerBase java.lang.ArrayIndexOutOfBoundsException: Index 23 
> out of bounds for length 16
> at 
> org.apache.solr.search.facet.UniqueBlockAgg$UniqueBlockSlotAcc.getValue(UniqueBlockAgg.java:59)
> at org.apache.solr.search.facet.SlotAcc.setValues(SlotAcc.java:146)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.fillBucket(FacetFieldProcessor.java:431)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.findTopSlots(FacetFieldProcessor.java:381)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.calcFacets(FacetFieldProcessorByHashDV.java:249)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.process(FacetFieldProcessorByHashDV.java:214)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:472)
> at 
> org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:429)
> at 
> org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetModule.process(FacetModule.java:139)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
> at 
> 

[jira] [Commented] (SOLR-12878) FacetFieldProcessorByHashDV is reconstructing FieldInfos on every instantiation

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669185#comment-16669185
 ] 

Mikhail Khludnev commented on SOLR-12878:
-

I wonder why FacetFieldProcessorByHashDV instantiated so often? I guess it 
should be significant for comparatively short queries? Anyway caching fieldInfo 
make sense. Not sure if it should asserted anyhow. 

> FacetFieldProcessorByHashDV is reconstructing FieldInfos on every 
> instantiation
> ---
>
> Key: SOLR-12878
> URL: https://issues.apache.org/jira/browse/SOLR-12878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Assignee: Mikhail Khludnev
>Priority: Major
>  Labels: performance
> Fix For: 7.6, master (8.0)
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV constructor is currently calling:
> {noformat}
> FieldInfo fieldInfo = 
> fcontext.searcher.getSlowAtomicReader().getFieldInfos().fieldInfo(sf.getName());
> {noformat}
> Which is reconstructing FieldInfos each time.  Simply switching it to:
> {noformat}
> FieldInfo fieldInfo = 
> fcontext.searcher.getFieldInfos().fieldInfo(sf.getName());
> {noformat}
>  
> causes it to use the cached version of FieldInfos in the SolrIndexSearcher.
> On my index the FacetFieldProcessorByHashDV is 2-3 times slower than the 
> legacy facets without this fix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Update on the test fixing effort.

[Mark Miller]

I've gotten a few pings on more concrete ideas on helping out. I'll keep
working on that over time, but I'll share a starting blurb here. I'm about
to share my first patch. I'll ping this thread when it's up. One thing you
can do is check that out and try 'ant test'. Report any fails you see
to SOLR-12932. If you have time, try to fix it. It's hard to do this with
any confidence without beasting the test.

Also, in no particular order:

tlog replica types rely on waitForInSyncWithLeader - this wait will often
wait even when indexes are in sync though, and just exhaust the timeout
every time. This needs to be fixed for these tests to be re-enabled.

I've done some work to stabilize the auto scaling tests, but more needs to
be done. They need to be beasted and awaitsfix annotations removed after
fixing.

@Nightly tests need to be beasted and addressed.

Please report any ant tests fails you see on the command line
here: SOLR-12932

Review tests with @AwaitsFix, try to address them.

Beast tests that you write or change. Improvements and doc coming to help.

Take your time committing. Run ant test more than once. When you see test
fails, report them. Better yet, look into them. If you are making quick
commits, you will almost certainly contribute to our test problem unless
you have some super tight and isolated unit test.

Know how long it takes to run the tests on your machine! Pay attention to
fluctuation after making non trivial changes! On my old 6 core machine
running 8jvms at a time, I get this with my patch:

BUILD SUCCESSFUL
Total time: 22 minutes 37 seconds

When I see it say 40 minutes, I'll know something has gone wrong!

As things progress, I'll bring up more. This is focusing on short term
firefighting. For that to be meaningful, there will be a lot of prevention
work to do as well. I've still got some work to do before I have more to
say on that.

- Mark

On Tue, Oct 30, 2018 at 11:03 AM Erick Erickson 
wrote:

> Sounds great, I'm anxious to see it in action.
>
> Let me know if I can help with beasting. I have two machines I can use.
>
> Of course having volunteered just about when things are getting under
> way I'll be gone Friday->Monday. Siii.
>
> Erick
> On Tue, Oct 30, 2018 at 7:34 AM Mark Miller  wrote:
> >
> > We are off to the races ...
> >
> > With any luck I'll put my first patch up on SOLR-12801 today. It works
> towards addressing 3-4 of the sub issues. Most importantly to me, it gives
> ant test a chance to pass.
> >
> > This was about 40 hours of almost continuous work and it is really just
> the start. I've addressed a ton of test stuff, but we are not even close to
> done. All of these tests need to be beasted thoroughly and defended for a
> start. That is the effort I'll be taking on by package.
> >
> > I'm working on making beasting super simple and adding doc for it. I'm
> also working on making precommit do a light beasting to any new or modified
> tests. I'm also working on some other things.
> >
> > ant test was really, really bad. I'm going to need a lot of help to make
> all this effort not turn into nothing. Without some real effort and change,
> I promise these improvements will melt away faster than they are felt.
> >
> > - Mark
> > --
> > - Mark
> > about.me/markrmiller
>
-- 
- Mark
about.me/markrmiller

[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 885 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/885/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds

Error Message:
did not finish processing in time

Stack Trace:
java.lang.AssertionError: did not finish processing in time
at 
__randomizedtesting.SeedInfo.seed([81DC68B6650E8B45:8B5FD71B28B5801F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMixedBounds(IndexSizeTriggerTest.java:572)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12543 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
   [junit4]   2> Creating dataDir: 

[jira] [Commented] (SOLR-12746) Ref Guide HTML output should adhere to more standard HTML5

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669132#comment-16669132
 ] 

Cassandra Targett commented on SOLR-12746:
--

Thanks again Alexandre, I'm glad to figure out those lines were causing the 
problem.

> Ref Guide HTML output should adhere to more standard HTML5
> --
>
> Key: SOLR-12746
> URL: https://issues.apache.org/jira/browse/SOLR-12746
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
>
> The default HTML produced by Jekyll/Asciidoctor adds a lot of extra {{}} 
> tags to the content which break up our content into very small chunks. This 
> is acceptable to a casual website reader as far as it goes, but any Reader 
> view in a browser or another type of content extraction system that uses a 
> similar "readability" scoring algorithm is going to either miss a lot of 
> content or fail to display the page entirely.
> To see what I mean, take a page like 
> https://lucene.apache.org/solr/guide/7_4/language-analysis.html and enable 
> Reader View in your browser (I used Firefox; Steve Rowe told me offline 
> Safari would not even offer the option on the page for him). You will notice 
> a lot of missing content. It's almost like someone selected sentences at 
> random.
> Asciidoctor has a long-standing issue to provide a better more 
> semantic-oriented HTML5 output, but it has not been resolved yet: 
> https://github.com/asciidoctor/asciidoctor/issues/242
> Asciidoctor does provide a way to override the default output templates by 
> providing your own in Slim, HAML, ERB or any other template language 
> supported by Tilt (none of which I know yet). There are some samples 
> available via the Asciidoctor project which we can borrow, but it's otherwise 
> unknown as of yet what parts of the output are causing the worst of the 
> problems. This issue is to explore how to fix it to improve this part of the 
> HTML reading experience.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12878) FacetFieldProcessorByHashDV is reconstructing FieldInfos on every instantiation

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev resolved SOLR-12878.
-
   Resolution: Fixed
 Assignee: Mikhail Khludnev
Fix Version/s: master (8.0)
   7.6

> FacetFieldProcessorByHashDV is reconstructing FieldInfos on every 
> instantiation
> ---
>
> Key: SOLR-12878
> URL: https://issues.apache.org/jira/browse/SOLR-12878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Assignee: Mikhail Khludnev
>Priority: Major
>  Labels: performance
> Fix For: 7.6, master (8.0)
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV constructor is currently calling:
> {noformat}
> FieldInfo fieldInfo = 
> fcontext.searcher.getSlowAtomicReader().getFieldInfos().fieldInfo(sf.getName());
> {noformat}
> Which is reconstructing FieldInfos each time.  Simply switching it to:
> {noformat}
> FieldInfo fieldInfo = 
> fcontext.searcher.getFieldInfos().fieldInfo(sf.getName());
> {noformat}
>  
> causes it to use the cached version of FieldInfos in the SolrIndexSearcher.
> On my index the FacetFieldProcessorByHashDV is 2-3 times slower than the 
> legacy facets without this fix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (SOLR-12878) FacetFieldProcessorByHashDV is reconstructing FieldInfos on every instantiation

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev reopened SOLR-12878:
-

> FacetFieldProcessorByHashDV is reconstructing FieldInfos on every 
> instantiation
> ---
>
> Key: SOLR-12878
> URL: https://issues.apache.org/jira/browse/SOLR-12878
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Assignee: Mikhail Khludnev
>Priority: Major
>  Labels: performance
> Fix For: 7.6, master (8.0)
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV constructor is currently calling:
> {noformat}
> FieldInfo fieldInfo = 
> fcontext.searcher.getSlowAtomicReader().getFieldInfos().fieldInfo(sf.getName());
> {noformat}
> Which is reconstructing FieldInfos each time.  Simply switching it to:
> {noformat}
> FieldInfo fieldInfo = 
> fcontext.searcher.getFieldInfos().fieldInfo(sf.getName());
> {noformat}
>  
> causes it to use the cached version of FieldInfos in the SolrIndexSearcher.
> On my index the FacetFieldProcessorByHashDV is 2-3 times slower than the 
> legacy facets without this fix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-12875) ArrayIndexOutOfBoundsException when using uniqueBlock(_root_) in JSON Facets

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev reassigned SOLR-12875:
---

Assignee: Mikhail Khludnev

> ArrayIndexOutOfBoundsException when using uniqueBlock(_root_) in JSON Facets
> 
>
> Key: SOLR-12875
> URL: https://issues.apache.org/jira/browse/SOLR-12875
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Assignee: Mikhail Khludnev
>Priority: Major
> Attachments: SOLR-12875.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I'm seeing java.lang.ArrayIndexOutOfBoundsException exceptions for some 
> requests when trying to make use of
> {noformat}
> uniqueBlock(_root_){noformat}
> within JSON Facets.
> Here are some example Stack Traces:
> {noformat}
> 2018-10-12 14:08:50.587 ERROR (qtp215078753-3353) [   x:my_core] 
> o.a.s.s.HttpSolrCall null:java.lang.ArrayIndexOutOfBoundsException: Index 13 
> out of bounds for length 8
> at 
> org.apache.solr.search.facet.UniqueBlockAgg$UniqueBlockSlotAcc.collectOrdToSlot(UniqueBlockAgg.java:40)
> at 
> org.apache.solr.search.facet.UniqueSinglevaluedSlotAcc.collect(UniqueSinglevaluedSlotAcc.java:85)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.collectFirstPhase(FacetFieldProcessor.java:243)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.collectValFirstPhase(FacetFieldProcessorByHashDV.java:432)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.access$100(FacetFieldProcessorByHashDV.java:50)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV$5.collect(FacetFieldProcessorByHashDV.java:395)
> at 
> org.apache.solr.search.DocSetUtil.collectSortedDocSet(DocSetUtil.java:284)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.collectDocs(FacetFieldProcessorByHashDV.java:376)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.calcFacets(FacetFieldProcessorByHashDV.java:247)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.process(FacetFieldProcessorByHashDV.java:214)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:472)
> at 
> org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:429)
> at 
> org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetModule.process(FacetModule.java:139)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> {noformat}
>  
> Here is another one at a different location in UniqueBlockAgg:
>   
> {noformat}
> 2018-10-12 21:37:57.322 ERROR (qtp215078753-4072) [   x:my_core] 
> o.a.s.h.RequestHandlerBase java.lang.ArrayIndexOutOfBoundsException: Index 23 
> out of bounds for length 16
> at 
> org.apache.solr.search.facet.UniqueBlockAgg$UniqueBlockSlotAcc.getValue(UniqueBlockAgg.java:59)
> at org.apache.solr.search.facet.SlotAcc.setValues(SlotAcc.java:146)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.fillBucket(FacetFieldProcessor.java:431)
> at 
> org.apache.solr.search.facet.FacetFieldProcessor.findTopSlots(FacetFieldProcessor.java:381)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.calcFacets(FacetFieldProcessorByHashDV.java:249)
> at 
> org.apache.solr.search.facet.FacetFieldProcessorByHashDV.process(FacetFieldProcessorByHashDV.java:214)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetProcessor.processSubs(FacetProcessor.java:472)
> at 
> org.apache.solr.search.facet.FacetProcessor.fillBucket(FacetProcessor.java:429)
> at 
> org.apache.solr.search.facet.FacetQueryProcessor.process(FacetQuery.java:64)
> at 
> org.apache.solr.search.facet.FacetRequest.process(FacetRequest.java:368)
> at 
> org.apache.solr.search.facet.FacetModule.process(FacetModule.java:139)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:298)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
> {noformat}
>  
>  
>  
>  



--
This message was sent by Atlassian 

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-12-ea+12) - Build # 3005 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3005/
Java: 64bit/jdk-12-ea+12 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

30 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.schema.SchemaApiFailureTest

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at __randomizedtesting.SeedInfo.seed([BB67199FA2C1533F]:0)
at 
org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1474)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.waitForState(CloudSolrClient.java:450)
at 
org.apache.solr.schema.SchemaApiFailureTest.setupCluster(SchemaApiFailureTest.java:43)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:835)


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.SchemaApiFailureTest

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at __randomizedtesting.SeedInfo.seed([BB67199FA2C1533F]:0)
at 
org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1474)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.waitForState(CloudSolrClient.java:450)
at 
org.apache.solr.schema.SchemaApiFailureTest.setupCluster(SchemaApiFailureTest.java:43)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)

[JENKINS] Lucene-Solr-Tests-master - Build # 2916 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2916/

1 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica

Error Message:
Timeout waiting for collection to become active Live Nodes: 
[127.0.0.1:10017_solr, 127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 
127.0.0.1:10020_solr, 127.0.0.1:10019_solr] Last available state: 
DocCollection(testCreateCollectionAddReplica//clusterstate.json/26)={   
"replicationFactor":"1",   "pullReplicas":"0",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false",   "nrtReplicas":"1",   "tlogReplicas":"0",   
"autoCreated":"true",   "policy":"c1",   "shards":{"shard1":{   
"replicas":{"core_node1":{   
"core":"testCreateCollectionAddReplica_shard1_replica_n1",   
"SEARCHER.searcher.maxDoc":0,   "SEARCHER.searcher.deletedDocs":0,  
 "INDEX.sizeInBytes":10240,   "node_name":"127.0.0.1:10016_solr",   
"state":"active",   "type":"NRT",   
"INDEX.sizeInGB":9.5367431640625E-6,   "SEARCHER.searcher.numDocs":0}}, 
  "range":"8000-7fff",   "state":"active"}}}

Stack Trace:
java.lang.AssertionError: Timeout waiting for collection to become active
Live Nodes: [127.0.0.1:10017_solr, 127.0.0.1:10018_solr, 127.0.0.1:10016_solr, 
127.0.0.1:10020_solr, 127.0.0.1:10019_solr]
Last available state: 
DocCollection(testCreateCollectionAddReplica//clusterstate.json/26)={
  "replicationFactor":"1",
  "pullReplicas":"0",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false",
  "nrtReplicas":"1",
  "tlogReplicas":"0",
  "autoCreated":"true",
  "policy":"c1",
  "shards":{"shard1":{
  "replicas":{"core_node1":{
  "core":"testCreateCollectionAddReplica_shard1_replica_n1",
  "SEARCHER.searcher.maxDoc":0,
  "SEARCHER.searcher.deletedDocs":0,
  "INDEX.sizeInBytes":10240,
  "node_name":"127.0.0.1:10016_solr",
  "state":"active",
  "type":"NRT",
  "INDEX.sizeInGB":9.5367431640625E-6,
  "SEARCHER.searcher.numDocs":0}},
  "range":"8000-7fff",
  "state":"active"}}}
at 
__randomizedtesting.SeedInfo.seed([3DA5C9506AFBAA38:BD85AC7E7BB8429E]:0)
at 
org.apache.solr.cloud.CloudTestUtils.waitForState(CloudTestUtils.java:70)
at 
org.apache.solr.cloud.autoscaling.sim.TestSimPolicyCloud.testCreateCollectionAddReplica(TestSimPolicyCloud.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms

[Michael Gibney (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669095#comment-16669095
 ] 

Michael Gibney commented on SOLR-12243:
---

[~thetaphi], that makes sense to me.

"The fix would be to improve SpanNear to produce the same query like 
PhraseQuery for certain slop values," see LUCENE-8544.

> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, 
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt, 
> schema.xml, solrconfig.xml
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
> 
>  
> 
>  edismax
>   0.4
>  title^100
>  title~20^5000
>  title~11
>  title~22^1000
>  text
>  
>  3-1 6-3 930%
>  *:*
>  25
> 
> 
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin"  against the 
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not 
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction 
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin 
> dose" or pf3:"aspirin dose ?"
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12940) Optimize and expunge deletes should execute only on the leader instead of all replicas of the collection

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669028#comment-16669028
 ] 

Erick Erickson commented on SOLR-12940:
---

We should add the operation at 12259 to the operations that should be performed 
on the leader only then replicated.

> Optimize and expunge deletes should execute only on the leader instead of all 
> replicas of the collection
> 
>
> Key: SOLR-12940
> URL: https://issues.apache.org/jira/browse/SOLR-12940
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.6, master (8.0)
>
>
> Optimize and expunge deletes are broadcasted to all replicas of the 
> collection (even to replicas of inactive shards!) but they don't need to be. 
> We can optimize by only executing such commands on the leader and have the 
> replicas pull the index over the network when ready.
> Synchronizing replication recovery to happen after optimize completes was 
> trickier in the past when all we had was HTTP requests but now we have the 
> terms mechanism which goes via ZK and can be relied on.
> This gives us a nice way to index/update fully and then call optimize/expunge 
> deletes only on the leader while replicas continue to serve query traffic 
> without disruption. This use-case will also need the ability to route queries 
> only to replicas and not to the leader.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12940) Optimize and expunge deletes should execute only on the leader instead of all replicas of the collection

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16669027#comment-16669027
 ] 

Erick Erickson commented on SOLR-12940:
---

I want to raise one concern here. This will essentially do a "full sync" of the 
index on the leader on all replicas in a collection at once, correct? That can 
move massive amounts of data around the network, and I've seen cascading 
failures result from that, endless recovery loops and the like.

Is there a way to throttle this?

I happen to be working on SOLR-12259 currently, which will add another 
operation to the mix so I'll link that in. The thrust of that JIRA is to 
rewrite all the segments to "do something", one case is to add docValues 
without re-indexing. Seems to me that the mechanism in this JIRA is applicable 
to SOLR-12259 too.

> Optimize and expunge deletes should execute only on the leader instead of all 
> replicas of the collection
> 
>
> Key: SOLR-12940
> URL: https://issues.apache.org/jira/browse/SOLR-12940
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Priority: Major
> Fix For: 7.6, master (8.0)
>
>
> Optimize and expunge deletes are broadcasted to all replicas of the 
> collection (even to replicas of inactive shards!) but they don't need to be. 
> We can optimize by only executing such commands on the leader and have the 
> replicas pull the index over the network when ready.
> Synchronizing replication recovery to happen after optimize completes was 
> trickier in the past when all we had was HTTP requests but now we have the 
> terms mechanism which goes via ZK and can be relied on.
> This gives us a nice way to index/update fully and then call optimize/expunge 
> deletes only on the leader while replicas continue to serve query traffic 
> without disruption. This use-case will also need the ability to route queries 
> only to replicas and not to the leader.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12943) FacetStream should not always calculate branch level aggregations.

[Joel Bernstein (JIRA)]

Joel Bernstein created SOLR-12943:
-

 Summary: FacetStream should not always calculate branch level 
aggregations.
 Key: SOLR-12943
 URL: https://issues.apache.org/jira/browse/SOLR-12943
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Joel Bernstein


Since the FacetStream only returns leaf level tuples we only need to calculate 
branch level aggregations to support branch level sorting. If the aggregation 
is not included in the branch level sort then it should not be calculated at 
the branch level.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-12911) Include "refine" param (refinement SOLR-9432) to the FacetStream

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein reassigned SOLR-12911:
-

Assignee: Joel Bernstein

> Include "refine" param (refinement SOLR-9432) to the FacetStream 
> -
>
> Key: SOLR-12911
> URL: https://issues.apache.org/jira/browse/SOLR-12911
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Amrit Sarkar
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: SOLR-12911.patch, SOLR-12911.patch
>
>
> SOLR-9432 introduces refinement in JSON Faceting which makes sure all the 
> respective buckets are fetched from each shard of the collection. This 
> ensures authentic mathematical bucket counts. 
> FacetStream should have refinement as an optional parameter like below, with 
> default being "false":
> {code}
> facet(collection1,
>   q="*:*",
>   refine="true",
>   buckets="a_s",
>   bucketSorts="sum(a_i) desc",
>   bucketSizeLimit=100,
>   sum(a_i),
>   count(*))
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-12795) Introduce 'offset' and 'limit' parameter in FacetStream.

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein reassigned SOLR-12795:
-

Assignee: Joel Bernstein

> Introduce 'offset' and 'limit' parameter in FacetStream.
> 
>
> Key: SOLR-12795
> URL: https://issues.apache.org/jira/browse/SOLR-12795
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Amrit Sarkar
>Assignee: Joel Bernstein
>Priority: Major
> Attachments: SOLR-12795.patch, SOLR-12795.patch, SOLR-12795.patch
>
>
> Today facetStream takes a "bucketSizeLimit" parameter. Here is what the doc 
> says about this parameter -  The number of buckets to include. This value is 
> applied to each dimension.
> Now let's say we create a facet stream with 3 nested facets. For example 
> "year_i,month_i,day_i" and provide 10 as the bucketSizeLimit. 
> FacetStream would return 10 results to us for this facet expression while the 
> total number of unqiue values are 1000 (10*10*10 )
> The API should have a separate parameter "limit" which limits the number of 
> tuples (say 500) while bucketSizeLimit should be used to specify the size of 
> each bucket in the JSON Facet API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12795) Introduce 'offset' and 'limit' parameter in FacetStream.

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-12795:
--
Summary: Introduce 'offset' and 'limit' parameter in FacetStream.  (was: 
Introduce 'limit' parameter in FacetStream.)

> Introduce 'offset' and 'limit' parameter in FacetStream.
> 
>
> Key: SOLR-12795
> URL: https://issues.apache.org/jira/browse/SOLR-12795
> Project: Solr
>  Issue Type: Sub-task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Amrit Sarkar
>Priority: Major
> Attachments: SOLR-12795.patch, SOLR-12795.patch, SOLR-12795.patch
>
>
> Today facetStream takes a "bucketSizeLimit" parameter. Here is what the doc 
> says about this parameter -  The number of buckets to include. This value is 
> applied to each dimension.
> Now let's say we create a facet stream with 3 nested facets. For example 
> "year_i,month_i,day_i" and provide 10 as the bucketSizeLimit. 
> FacetStream would return 10 results to us for this facet expression while the 
> total number of unqiue values are 1000 (10*10*10 )
> The API should have a separate parameter "limit" which limits the number of 
> tuples (say 500) while bucketSizeLimit should be used to specify the size of 
> each bucket in the JSON Facet API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668952#comment-16668952
 ] 

Uwe Schindler commented on SOLR-12243:
--

I agree with both of you that it's an issue for many synonyms. This was already 
discussed in the Lucene issue. As long as span queries don't support the same 
type of near matches (in order, out of order, with gaps) it's hard to apply the 
optimization correctly. The fix would be to improve SpanNear to produce the 
same query like PhraseQuery for certain slop values.

For now, Steve is right, the max clause count will protect users from 
combinatoric explosion. I was suggesting to improve EDisMax to allow limiting 
the number of expansions when producing bi and trigrams. As the query is very 
fuzzy anyways, it might be OK to restrict the combinations (e.g., if trigrams 
explode, just disable trigrams for the huge query and onyl create bigrams).

> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, 
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt, 
> schema.xml, solrconfig.xml
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
> 
>  
> 
>  edismax
>   0.4
>  title^100
>  title~20^5000
>  title~11
>  title~22^1000
>  text
>  
>  3-1 6-3 930%
>  *:*
>  25
> 
> 
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin"  against the 
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not 
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction 
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin 
> dose" or pf3:"aspirin dose ?"
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-10.0.1) - Build # 23122 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23122/
Java: 64bit/jdk-10.0.1 -XX:+UseCompressedOops -XX:+UseG1GC

6 tests failed.
FAILED:  
org.apache.solr.client.solrj.impl.CloudSolrClientTest.testParallelUpdateQTime

Error Message:
Error from server at 
https://127.0.0.1:38827/solr/collection1_shard2_replica_n2: Expected mime type 
application/octet-stream but got text/html.Error 404 
Can not find: /solr/collection1_shard2_replica_n2/update  
HTTP ERROR 404 Problem accessing 
/solr/collection1_shard2_replica_n2/update. Reason: Can not find: 
/solr/collection1_shard2_replica_n2/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605  
  

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at https://127.0.0.1:38827/solr/collection1_shard2_replica_n2: Expected 
mime type application/octet-stream but got text/html. 


Error 404 Can not find: 
/solr/collection1_shard2_replica_n2/update

HTTP ERROR 404
Problem accessing /solr/collection1_shard2_replica_n2/update. Reason:
Can not find: 
/solr/collection1_shard2_replica_n2/updatehttp://eclipse.org/jetty;>Powered by Jetty:// 9.4.11.v20180605




at 
__randomizedtesting.SeedInfo.seed([3840CD3CBB5C0402:D698B6ADF523D191]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.client.solrj.impl.CloudSolrClientTest.testParallelUpdateQTime(CloudSolrClientTest.java:146)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 

Re: Update on the test fixing effort.

[Erick Erickson]

Sounds great, I'm anxious to see it in action.

Let me know if I can help with beasting. I have two machines I can use.

Of course having volunteered just about when things are getting under
way I'll be gone Friday->Monday. Siii.

Erick
On Tue, Oct 30, 2018 at 7:34 AM Mark Miller  wrote:
>
> We are off to the races ...
>
> With any luck I'll put my first patch up on SOLR-12801 today. It works 
> towards addressing 3-4 of the sub issues. Most importantly to me, it gives 
> ant test a chance to pass.
>
> This was about 40 hours of almost continuous work and it is really just the 
> start. I've addressed a ton of test stuff, but we are not even close to done. 
> All of these tests need to be beasted thoroughly and defended for a start. 
> That is the effort I'll be taking on by package.
>
> I'm working on making beasting super simple and adding doc for it. I'm also 
> working on making precommit do a light beasting to any new or modified tests. 
> I'm also working on some other things.
>
> ant test was really, really bad. I'm going to need a lot of help to make all 
> this effort not turn into nothing. Without some real effort and change, I 
> promise these improvements will melt away faster than they are felt.
>
> - Mark
> --
> - Mark
> about.me/markrmiller

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12746) Ref Guide HTML output should adhere to more standard HTML5

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668938#comment-16668938
 ] 

Alexandre Rafalovitch commented on SOLR-12746:
--

I can build on master yes. And it now builds on the branch with those two 
dependencies commented out.

And the HTML looks so much cleaner now! And smaller.

There are still build errors for PDF and a deprecation warning for HTML stages, 
but both of these are on trunk as well, so not an issue here.

+1

> Ref Guide HTML output should adhere to more standard HTML5
> --
>
> Key: SOLR-12746
> URL: https://issues.apache.org/jira/browse/SOLR-12746
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
>
> The default HTML produced by Jekyll/Asciidoctor adds a lot of extra {{}} 
> tags to the content which break up our content into very small chunks. This 
> is acceptable to a casual website reader as far as it goes, but any Reader 
> view in a browser or another type of content extraction system that uses a 
> similar "readability" scoring algorithm is going to either miss a lot of 
> content or fail to display the page entirely.
> To see what I mean, take a page like 
> https://lucene.apache.org/solr/guide/7_4/language-analysis.html and enable 
> Reader View in your browser (I used Firefox; Steve Rowe told me offline 
> Safari would not even offer the option on the page for him). You will notice 
> a lot of missing content. It's almost like someone selected sentences at 
> random.
> Asciidoctor has a long-standing issue to provide a better more 
> semantic-oriented HTML5 output, but it has not been resolved yet: 
> https://github.com/asciidoctor/asciidoctor/issues/242
> Asciidoctor does provide a way to override the default output templates by 
> providing your own in Slim, HAML, ERB or any other template language 
> supported by Tilt (none of which I know yet). There are some samples 
> available via the Asciidoctor project which we can borrow, but it's otherwise 
> unknown as of yet what parts of the output are causing the worst of the 
> problems. This issue is to explore how to fix it to improve this part of the 
> HTML reading experience.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-8550) Tessellator fails when filtering coplanar points when creating linked list

[Ignacio Vera (JIRA)]

Ignacio Vera created LUCENE-8550:


 Summary: Tessellator fails when filtering coplanar points when 
creating linked list 
 Key: LUCENE-8550
 URL: https://issues.apache.org/jira/browse/LUCENE-8550
 Project: Lucene - Core
  Issue Type: Bug
  Components: modules/sandbox
Reporter: Ignacio Vera


Currently when creating the linked list on the tessellator, coplanar points are 
filtered. The problem is the following: 

if we have three coplanar points, the code is actually removing the last point, 
instead it should remove the middle one.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms

[Michael Gibney (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668907#comment-16668907
 ] 

Michael Gibney commented on SOLR-12243:
---

I share [~ehaubert]'s intuition that "adding top-level clauses comes up as more 
expensive than the embedded spans", including the caveat: "but suppose would 
need to add a test to say that for sure".

I would characterize the permutation problem as substantially _different_ as a 
result of the recent Lucene fixes. The fixes (LUCENE-8531) essentially reverted 
LUCENE-7638, an optimization that explicitly set out to address the potential 
for combinatoric explosion. A {{SpanQuery}}-based implementation still has to 
deal with the possibility of combinatoric _matching_ in documents, but the 
{{SpanQuery}} implementation handles graph "branching" natively (as nested 
Spans), as opposed to running a full, separate, top-level, {{PhraseQuery}} for 
each possible combination of terms.

> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, 
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt, 
> schema.xml, solrconfig.xml
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
> 
>  
> 
>  edismax
>   0.4
>  title^100
>  title~20^5000
>  title~11
>  title~22^1000
>  text
>  
>  3-1 6-3 930%
>  *:*
>  25
> 
> 
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin"  against the 
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not 
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction 
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin 
> dose" or pf3:"aspirin dose ?"
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12942) Update IndexSizeTrigger to allow selecting the split method

[Andrzej Bialecki (JIRA)]

Andrzej Bialecki  created SOLR-12942:


 Summary: Update IndexSizeTrigger to allow selecting the split 
method
 Key: SOLR-12942
 URL: https://issues.apache.org/jira/browse/SOLR-12942
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Andrzej Bialecki 
Assignee: Andrzej Bialecki 


SOLR-12509 added a new method of shard splitting (\{{splitmethod-link}}). 
IndexSizeTrigger should be able to select this method as a configurable option, 
when requesting SPLITSHARD operations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-4919) Allow setting ResponseParser and RequestWriter on LBHttpSolrServer

[Jason Gerlowski (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gerlowski resolved SOLR-4919.
---
Resolution: Not A Problem

> Allow setting ResponseParser and RequestWriter on LBHttpSolrServer
> --
>
> Key: SOLR-4919
> URL: https://issues.apache.org/jira/browse/SOLR-4919
> Project: Solr
>  Issue Type: Improvement
>  Components: clients - java
>Affects Versions: 4.3
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Fix For: 6.0, 4.9
>
> Attachments: SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, 
> SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, 
> SOLR-4919.patch, SOLR-4919.patch, SOLR-4919.patch, 
> SolrExampleJettyTest-testfail.txt, 
> SolrExampleStreamingTest-failure-linux.txt, 
> TestReplicationHandler-testfail.txt
>
>
> Patch to allow setting parser/writer on LBHttpSolrServer.  Will only work if 
> no server objects exist within.  Part of larger issue SOLR-4715.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 1816 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/1816/

[...truncated 28 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/362/consoleText

[repro] Revision: 46fd0873f3790a1567334e7e71df747915381093

[repro] Ant options: -Dtests.multiplier=2 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
[repro] Repro line:  ant test  -Dtestcase=HdfsRestartWhileUpdatingTest 
-Dtests.method=test -Dtests.seed=949909B0C215D49 -Dtests.multiplier=2 
-Dtests.nightly=true -Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=ja -Dtests.timezone=Europe/Luxembourg -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[repro] Repro line:  ant test  -Dtestcase=HdfsRestartWhileUpdatingTest 
-Dtests.seed=949909B0C215D49 -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=ja -Dtests.timezone=Europe/Luxembourg -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
5aa8ad5b14ab7f6f8f3191cba39285c3a0bf9112
[repro] git fetch
[repro] git checkout 46fd0873f3790a1567334e7e71df747915381093

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   HdfsRestartWhileUpdatingTest
[repro] ant compile-test

[...truncated 3580 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.HdfsRestartWhileUpdatingTest" -Dtests.showOutput=onerror 
-Dtests.multiplier=2 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
 -Dtests.seed=949909B0C215D49 -Dtests.multiplier=2 -Dtests.nightly=true 
-Dtests.slow=true 
-Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt
 -Dtests.locale=ja -Dtests.timezone=Europe/Luxembourg -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1

[...truncated 143203 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   3/5 failed: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
[repro] git checkout 5aa8ad5b14ab7f6f8f3191cba39285c3a0bf9112

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12879) Query Parser for MinHash/LSH

[Andy Hind (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668841#comment-16668841
 ] 

Andy Hind commented on SOLR-12879:
--

Should I raise separate issues for the documentation?

> Query Parser for MinHash/LSH
> 
>
> Key: SOLR-12879
> URL: https://issues.apache.org/jira/browse/SOLR-12879
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: master (8.0)
>Reporter: Andy Hind
>Assignee: Tommaso Teofili
>Priority: Major
> Fix For: master (8.0)
>
> Attachments: minhash.filter.adoc.fragment, minhash.patch
>
>
> Following on from https://issues.apache.org/jira/browse/LUCENE-6968, provide 
> a query parser that builds queries that provide a measure of Jaccard 
> similarity. The initial patch includes banded queries that were also proposed 
> on the original issue.
>  
> I have one outstanding questions:
>  * Should the score from the overall query be normalised?
> Note, that the band count is currently approximate and may be one less than 
> in practise.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12313) TestInjection#waitForInSyncWithLeader needs improvement.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668831#comment-16668831
 ] 

Mark Miller commented on SOLR-12313:


When this is fixed we need to turn back on tlog testing in TestCloudRecovery 
and the AbstractDistribZkTestBase tests.

> TestInjection#waitForInSyncWithLeader needs improvement.
> 
>
> Key: SOLR-12313
> URL: https://issues.apache.org/jira/browse/SOLR-12313
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Priority: Major
>
> This really should have some doc for why it would be used.
> I also think it causes BasicDistributedZkTest to take forever for sometimes 
> and perhaps other tests?
> I think checking for uncommitted data is probably a race condition and should 
> be removed.
> Checking index versions should follow the rules that replication does - if 
> the slave is higher than the leader, it's in sync, being equal is not 
> required. If it's expected for a test it should be a specific test that 
> fails. This just introduces massive delays.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Update on the test fixing effort.

[Mark Miller]

We are off to the races ...

With any luck I'll put my first patch up on SOLR-12801 today. It works
towards addressing 3-4 of the sub issues. Most importantly to me, it gives
ant test a chance to pass.

This was about 40 hours of almost continuous work and it is really just the
start. I've addressed a ton of test stuff, but we are not even close to
done. All of these tests need to be beasted thoroughly and defended for a
start. That is the effort I'll be taking on by package.

I'm working on making beasting super simple and adding doc for it. I'm also
working on making precommit do a light beasting to any new or modified
tests. I'm also working on some other things.

ant test was really, really bad. I'm going to need a lot of help to make
all this effort not turn into nothing. Without some real effort and change,
I promise these improvements will melt away faster than they are felt.

- Mark
-- 
- Mark
about.me/markrmiller

[jira] [Commented] (SOLR-12243) Edismax missing phrase queries when phrases contain multiterm synonyms

[Steve Rowe (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668790#comment-16668790
 ] 

Steve Rowe commented on SOLR-12243:
---

{quote}I think the permutation problem is not new with the recent Lucene fixes. 
This problem should also have happened with Span expansions, right? Maybe we 
should add an option to limit the number of phrase expansions (as a safety 
feature). If those limits are reached, the phrase expansion should be stopped 
(maybe then only bigrams and no trigrams).
{quote}
I don't know how Span queries are rewritten, or how the search time complexity 
would work out, but AFAIK the Lucene fixes didn't change the permutation 
problem, just recast it as explicit clauses.

As far as safety is concerned, doesn't the Boolean clause limit already apply 
in this case, since the generated query is a BooleanQuery of PhraseQuery-s?

Elizabeth, I like the idea of adding something along the lines of the text you 
suggested to the ref guide. I made a few tweaks (described below) - please let 
me know if you think this is okay:
{quote}
h3. Synonyms expansion in phrase queries with slop

When a phrase query with slop (e.g. {{pf}} with {{ps}}) triggers synonym 
expansions, a separate clause will be generated for each combination of 
synonyms. For example, with configured synonyms {{dog,canine}} and 
{{cat,feline}}, the query {{"dog chased cat"}} will generate the following 
phrase query clauses:
 * {{"dog chased cat"}}
 * {{"canine chased cat"}}
 * {{"dog chased feline"}}
 * {{"canine chased feline"}}{quote}
My changes: this situation happens with all synonyms, not just multi-term 
synonyms; user-specified phrase queries (in {{q}} param) trigger this situation 
when {{qs}} is specified, so I generalized it a bit to refer to all phrase+slop 
contexts; and I think "combination" is better than "permutation" here.

> Edismax missing phrase queries when phrases contain multiterm synonyms
> --
>
> Key: SOLR-12243
> URL: https://issues.apache.org/jira/browse/SOLR-12243
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: query parsers
>Affects Versions: 7.1
> Environment: RHEL, MacOS X
> Do not believe this is environment-specific.
>Reporter: Elizabeth Haubert
>Assignee: Uwe Schindler
>Priority: Major
> Attachments: SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, 
> SOLR-12243.patch, SOLR-12243.patch, SOLR-12243.patch, multiword-synonyms.txt, 
> schema.xml, solrconfig.xml
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> synonyms.txt:
> {code}
> allergic, hypersensitive
> aspirin, acetylsalicylic acid
> dog, canine, canis familiris, k 9
> rat, rattus
> {code}
> request handler:
> {code:xml}
> 
>  
> 
>  edismax
>   0.4
>  title^100
>  title~20^5000
>  title~11
>  title~22^1000
>  text
>  
>  3-1 6-3 930%
>  *:*
>  25
> 
> 
> {code}
> Phrase queries (pf, pf2, pf3) containing "dog" or "aspirin"  against the 
> above list will not be generated.
> "allergic reaction dog" will generate pf2: "allergic reaction", but not 
> pf:"allergic reaction dog", pf2: "reaction dog", or pf3: "allergic reaction 
> dog"
> "aspirin dose in rats" will generate pf3: "dose ? rats" but not pf2: "aspirin 
> dose" or pf3:"aspirin dose ?"
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12746) Ref Guide HTML output should adhere to more standard HTML5

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668789#comment-16668789
 ] 

Cassandra Targett commented on SOLR-12746:
--

I figured out that we can remove the dependency on the {{asciidoctor-html5s}} 
gem by removing the 2 lines at the start of {{_templates/helpers.rb}} that 
require it. This seems to have no impact on the templates (but we can't remove 
the file entirely; the templates rely on some of what it does).

[~arafalov], you're able to build the Ref Guide HTML fine on master, right?

> Ref Guide HTML output should adhere to more standard HTML5
> --
>
> Key: SOLR-12746
> URL: https://issues.apache.org/jira/browse/SOLR-12746
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
>
> The default HTML produced by Jekyll/Asciidoctor adds a lot of extra {{}} 
> tags to the content which break up our content into very small chunks. This 
> is acceptable to a casual website reader as far as it goes, but any Reader 
> view in a browser or another type of content extraction system that uses a 
> similar "readability" scoring algorithm is going to either miss a lot of 
> content or fail to display the page entirely.
> To see what I mean, take a page like 
> https://lucene.apache.org/solr/guide/7_4/language-analysis.html and enable 
> Reader View in your browser (I used Firefox; Steve Rowe told me offline 
> Safari would not even offer the option on the page for him). You will notice 
> a lot of missing content. It's almost like someone selected sentences at 
> random.
> Asciidoctor has a long-standing issue to provide a better more 
> semantic-oriented HTML5 output, but it has not been resolved yet: 
> https://github.com/asciidoctor/asciidoctor/issues/242
> Asciidoctor does provide a way to override the default output templates by 
> providing your own in Slim, HAML, ERB or any other template language 
> supported by Tilt (none of which I know yet). There are some samples 
> available via the Asciidoctor project which we can borrow, but it's otherwise 
> unknown as of yet what parts of the output are causing the worst of the 
> problems. This issue is to explore how to fix it to improve this part of the 
> HTML reading experience.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12801) Fix the tests, remove BadApples and AwaitsFix annotations, improve env for test development.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668780#comment-16668780
 ] 

Mark Miller commented on SOLR-12801:


Still coming.

FYI, I'm turning off most tlog replica testing until SOLR-12313 is fixed.

> Fix the tests, remove BadApples and AwaitsFix annotations, improve env for 
> test development.
> 
>
> Key: SOLR-12801
> URL: https://issues.apache.org/jira/browse/SOLR-12801
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Critical
>
> A single issue to counteract the single issue adding tons of annotations, the 
> continued addition of new flakey tests, and the continued addition of 
> flakiness to existing tests.
> Lots more to come.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8374) Reduce reads for sparse DocValues

[Toke Eskildsen (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668773#comment-16668773
 ] 

Toke Eskildsen commented on LUCENE-8374:


[~tpunder] it might be that it is because of sampling that the 
{{advanceExactWithinBlock}} does not show up. Don't fret about it - I'll try 
some profiling experiments myself to see what's up with that.

Your logs are very informative, thanks:
{{
Segments: 41
Fields: Total=41, vBPV(long)=30 (blocks: 128095)
DISI-blocks: ALL=41, DENSE=76793, SPARSE=123, EMPTY=0
Overhead: 20.98 MB, 412 ms startup
}}

The patch introduces jumps for DISI-blocks, DENSE and vBPV. As most of your 
fields has this exact combination, your index is very receptive to the 
optimizations. Good for you and a caveat that this level of speed-up is not to 
be expected generally.

There is a lot of numerics to a front-end that uses string-facets. I guess you 
represent the facet entries as IDs and look up the string representations from 
another source? Is this because you found string faceting to be too slow? I 
tried looking at DocValues string resolving but there did not seem to be any 
easy gains like the one for numerics.

> Reduce reads for sparse DocValues
> -
>
> Key: LUCENE-8374
> URL: https://issues.apache.org/jira/browse/LUCENE-8374
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/codecs
>Affects Versions: 7.5, master (8.0)
>Reporter: Toke Eskildsen
>Priority: Major
>  Labels: performance
> Fix For: 7.6
>
> Attachments: LUCENE-8374.patch, LUCENE-8374.patch, LUCENE-8374.patch, 
> LUCENE-8374.patch, LUCENE-8374.patch, LUCENE-8374_branch_7_3.patch, 
> LUCENE-8374_branch_7_3.patch.20181005, LUCENE-8374_branch_7_4.patch, 
> LUCENE-8374_branch_7_5.patch, entire_index_logs.txt, 
> image-2018-10-24-07-30-06-663.png, image-2018-10-24-07-30-56-962.png, 
> single_vehicle_logs.txt, 
> start-2018-10-24-1_snapshot___Users_tim_Snapshots__-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png,
>  
> start-2018-10-24_snapshot___Users_tim_Snapshots__-_YourKit_Java_Profiler_2017_02-b75_-_64-bit.png
>
>
> The {{Lucene70DocValuesProducer}} has the internal classes 
> {{SparseNumericDocValues}} and {{BaseSortedSetDocValues}} (sparse code path), 
> which again uses {{IndexedDISI}} to handle the docID -> value-ordinal lookup. 
> The value-ordinal is the index of the docID assuming an abstract tightly 
> packed monotonically increasing list of docIDs: If the docIDs with 
> corresponding values are {{[0, 4, 1432]}}, their value-ordinals will be {{[0, 
> 1, 2]}}.
> h2. Outer blocks
> The lookup structure of {{IndexedDISI}} consists of blocks of 2^16 values 
> (65536), where each block can be either {{ALL}}, {{DENSE}} (2^12 to 2^16 
> values) or {{SPARSE}} (< 2^12 values ~= 6%). Consequently blocks vary quite a 
> lot in size and ordinal resolving strategy.
> When a sparse Numeric DocValue is needed, the code first locates the block 
> containing the wanted docID flag. It does so by iterating blocks one-by-one 
> until it reaches the needed one, where each iteration requires a lookup in 
> the underlying {{IndexSlice}}. For a common memory mapped index, this 
> translates to either a cached request or a read operation. If a segment has 
> 6M documents, worst-case is 91 lookups. In our web archive, our segments has 
> ~300M values: A worst-case of 4577 lookups!
> One obvious solution is to use a lookup-table for blocks: A long[]-array with 
> an entry for each block. For 6M documents, that is < 1KB and would allow for 
> direct jumping (a single lookup) in all instances. Unfortunately this 
> lookup-table cannot be generated upfront when the writing of values is purely 
> streaming. It can be appended to the end of the stream before it is closed, 
> but without knowing the position of the lookup-table the reader cannot seek 
> to it.
> One strategy for creating such a lookup-table would be to generate it during 
> reads and cache it for next lookup. This does not fit directly into how 
> {{IndexedDISI}} currently works (it is created anew for each invocation), but 
> could probably be added with a little work. An advantage to this is that this 
> does not change the underlying format and thus could be used with existing 
> indexes.
> h2. The lookup structure inside each block
> If {{ALL}} of the 2^16 values are defined, the structure is empty and the 
> ordinal is simply the requested docID with some modulo and multiply math. 
> Nothing to improve there.
> If the block is {{DENSE}} (2^12 to 2^16 values are defined), a bitmap is used 
> and the number of set bits up to the wanted index (the docID modulo the block 
> origo) are counted. That bitmap is a long[1024], meaning that worst case is 
> to lookup and count all set bits for 1024 longs!
> One known 

[jira] [Commented] (SOLR-11572) Add recip Stream Evaluator to support reciprocal transformations

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668740#comment-16668740
 ] 

ASF subversion and git services commented on SOLR-11572:


Commit e8503da7c43b8ff3eb968c631ca2ac718a2c736b in lucene-solr's branch 
refs/heads/branch_7x from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e8503da ]

SOLR-11572: Add recip Stream Evaluator to support reciprocal transformations


> Add recip Stream Evaluator to support reciprocal transformations
> 
>
> Key: SOLR-11572
> URL: https://issues.apache.org/jira/browse/SOLR-11572
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-11572.patch
>
>
> This ticket adds the recip Stream Evaluator to support inverse/reciprocal 
> scalar and vector transformations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11572) Add recip Stream Evaluator to support reciprocal transformations

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668732#comment-16668732
 ] 

ASF subversion and git services commented on SOLR-11572:


Commit 856e28d8cf07cc34bc1361784bf00e7aceb3af97 in lucene-solr's branch 
refs/heads/master from [~joel.bernstein]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=856e28d ]

SOLR-11572: Add recip Stream Evaluator to support reciprocal transformations


> Add recip Stream Evaluator to support reciprocal transformations
> 
>
> Key: SOLR-11572
> URL: https://issues.apache.org/jira/browse/SOLR-11572
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-11572.patch
>
>
> This ticket adds the recip Stream Evaluator to support inverse/reciprocal 
> scalar and vector transformations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-12-ea+12) - Build # 3004 - Still unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/3004/
Java: 64bit/jdk-12-ea+12 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

33 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DocValuesNotIndexedTest

Error Message:
Collection not found: dv_coll

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: dv_coll
at __randomizedtesting.SeedInfo.seed([40858C4055CA7810]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.createCluster(DocValuesNotIndexedTest.java:154)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:835)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DocValuesNotIndexedTest

Error Message:
Collection not found: dv_coll

Stack Trace:
org.apache.solr.common.SolrException: Collection not found: dv_coll
at __randomizedtesting.SeedInfo.seed([40858C4055CA7810]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at 
org.apache.solr.cloud.DocValuesNotIndexedTest.createCluster(DocValuesNotIndexedTest.java:154)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 

[jira] [Commented] (SOLR-12746) Ref Guide HTML output should adhere to more standard HTML5

[Cassandra Targett (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668682#comment-16668682
 ] 

Cassandra Targett commented on SOLR-12746:
--

I wrote up in the README in the branch that all you should need is the {{slim}} 
gem. You're running into a similar problem I had when I tried to integrate 
{{asciidoctor-html5s}} directly, so I did not integrate that project directly; 
I only copied the templates themselves.

However, I listed my own gems and realized that I've been running my tests with 
that gem installed; removing it causes the errors you probably saw about it 
being missing. I'll add it back in and take a look at what's happening and fix 
the README accordingly. Thanks for trying it out to find this.

> Ref Guide HTML output should adhere to more standard HTML5
> --
>
> Key: SOLR-12746
> URL: https://issues.apache.org/jira/browse/SOLR-12746
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Reporter: Cassandra Targett
>Assignee: Cassandra Targett
>Priority: Major
>
> The default HTML produced by Jekyll/Asciidoctor adds a lot of extra {{}} 
> tags to the content which break up our content into very small chunks. This 
> is acceptable to a casual website reader as far as it goes, but any Reader 
> view in a browser or another type of content extraction system that uses a 
> similar "readability" scoring algorithm is going to either miss a lot of 
> content or fail to display the page entirely.
> To see what I mean, take a page like 
> https://lucene.apache.org/solr/guide/7_4/language-analysis.html and enable 
> Reader View in your browser (I used Firefox; Steve Rowe told me offline 
> Safari would not even offer the option on the page for him). You will notice 
> a lot of missing content. It's almost like someone selected sentences at 
> random.
> Asciidoctor has a long-standing issue to provide a better more 
> semantic-oriented HTML5 output, but it has not been resolved yet: 
> https://github.com/asciidoctor/asciidoctor/issues/242
> Asciidoctor does provide a way to override the default output templates by 
> providing your own in Slim, HAML, ERB or any other template language 
> supported by Tilt (none of which I know yet). There are some samples 
> available via the Asciidoctor project which we can borrow, but it's otherwise 
> unknown as of yet what parts of the output are causing the worst of the 
> problems. This issue is to explore how to fix it to improve this part of the 
> HTML reading experience.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12941) IndexSizeTrigger and splitMethod=link problems

[Andrzej Bialecki (JIRA)]

Andrzej Bialecki  created SOLR-12941:


 Summary: IndexSizeTrigger and splitMethod=link problems
 Key: SOLR-12941
 URL: https://issues.apache.org/jira/browse/SOLR-12941
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 7.6, master (8.0)
Reporter: Andrzej Bialecki 
Assignee: Andrzej Bialecki 


{{IndexSizeTrigger}} can be configured to use {{splitMethod=link}} 
(SOLR-12730), which uses hard-linking for creating sub-shards.

However, if the trigger uses {{aboveBytes}} condition the resulting sub-shards 
will not immediately decrease in size, until all of the deleted documents will 
be expunged (either by gradual merges or by explicit and costly expungeDeletes 
command). As a result the new sub-shards will still exceed the {{aboveBytes}} 
threshold, which will cause the trigger to keep generating new split requests.

I see two options how to solve this:
 * disallow using {{aboveBytes}} with {{splitMethod=link}}. This unfortunately 
is a very desirable combination because it monitors the actual index size and 
uses the fast splitting method.
 * calculate an internal estimate of "eventual index size" for an index with 
deletions, and use this estimate when checking with {{aboveBytes}} instead of 
the real index size. This of course introduces a potentially significant 
estimation error but allows to properly treat hard-linked sub-shards with 
deletions as (eventually) significantly smaller than the parent shard.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12882) Eliminate excessive lambda allocation in FacetFieldProcessorByHashDV.collectValFirstPhase

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668664#comment-16668664
 ] 

David Smiley commented on SOLR-12882:
-

+1

I wonder how much this helps?  Did you do benchmarking?

> Eliminate excessive lambda allocation in 
> FacetFieldProcessorByHashDV.collectValFirstPhase
> -
>
> Key: SOLR-12882
> URL: https://issues.apache.org/jira/browse/SOLR-12882
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module
>Affects Versions: 7.5
>Reporter: Tim Underwood
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> The FacetFieldProcessorByHashDV.collectValFirstPhase method looks like this:
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>  int slot = table.add(val); // this can trigger a rehash
>  // Our countAcc is virtual, so this is not needed:
>  // countAcc.incrementCount(slot, 1);
> super.collectFirstPhase(segDoc, slot, slotNum ->
> { Comparable value = calc.bitsToValue(val); return new 
> SlotContext(sf.getType().getFieldQuery(null, sf, calc.formatValue(value))); }
> );
> }
> {noformat}
>  
> For each value that is being iterated over there is a lambda allocation that 
> is passed as the slotContext argument to the super.collectFirstPhase method. 
> The lambda can be factored out such that there is only a single instance per 
> FacetFieldProcessorByHashDV instance. 
> The only tradeoff being that the value needs to be looked up from the table 
> in the lambda.  However looking the value up in the table is going to be less 
> expensive than a memory allocation and also the slotContext lambda is only 
> used in RelatednessAgg and not for any of the field faceting or metrics.
>  
> {noformat}
> private void collectValFirstPhase(int segDoc, long val) throws IOException {
>   int slot = table.add(val); // this can trigger a rehash
>   // Our countAcc is virtual, so this is not needed:
>   // countAcc.incrementCount(slot, 1);
>   super.collectFirstPhase(segDoc, slot, slotContext);
> }
> /**
>  * SlotContext to use during all {@link SlotAcc} collection.
>  *
>  * This avoids a memory allocation for each invocation of 
> collectValFirstPhase.
>  */
> private IntFunction slotContext = (slotNum) -> {
>   long val = table.vals[slotNum];
>   Comparable value = calc.bitsToValue(val);
>   return new SlotContext(sf.getType().getFieldQuery(null, sf, 
> calc.formatValue(value)));
> };
> {noformat}
>  
> FacetFieldProcessorByArray already follows this same pattern



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-9.0.4) - Build # 23121 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23121/
Java: 64bit/jdk-9.0.4 -XX:+UseCompressedOops -XX:+UseParallelGC

4 tests failed.
FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:43709/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:43709/solr
at 
__randomizedtesting.SeedInfo.seed([A3F8F44A07B7A7D2:62088DE62AE76D75]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
at 
org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:33533/solr

Stack Trace:

[jira] [Commented] (LUCENE-8497) Rethink multi-term analysis handling

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668544#comment-16668544
 ] 

Alan Woodward commented on LUCENE-8497:
---

New patch, this time actually removing the outdated test...

> Rethink multi-term analysis handling
> 
>
> Key: LUCENE-8497
> URL: https://issues.apache.org/jira/browse/LUCENE-8497
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8497.patch, LUCENE-8497.patch, LUCENE-8497.patch, 
> LUCENE-8497.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> The current framework for handling term normalisation works via instanceof 
> checks for MultiTermAwareComponent and casts.  MultiTermAwareComponent itself 
> deals in AbstractAnalysisComponents, and so callers need to cast to the 
> correct component type before use, which is ripe for misuse.
> We should re-organise all this to be type-safe and usable without casts.  One 
> possibility is to add `normalize` methods to CharFilterFactory and 
> TokenFilterFactory that mirror their existing `create` methods.  The default 
> implementation would return the input unchanged, while filters that should 
> apply at normalization time can delegate to `create`.
> Related to this, we should deprecate and remove LowerCaseTokenizer, which 
> combines tokenization and normalization in a way that will break this API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8497) Rethink multi-term analysis handling

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-8497:
--
Attachment: LUCENE-8497.patch

> Rethink multi-term analysis handling
> 
>
> Key: LUCENE-8497
> URL: https://issues.apache.org/jira/browse/LUCENE-8497
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Alan Woodward
>Priority: Major
> Attachments: LUCENE-8497.patch, LUCENE-8497.patch, LUCENE-8497.patch, 
> LUCENE-8497.patch
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> The current framework for handling term normalisation works via instanceof 
> checks for MultiTermAwareComponent and casts.  MultiTermAwareComponent itself 
> deals in AbstractAnalysisComponents, and so callers need to cast to the 
> correct component type before use, which is ripe for misuse.
> We should re-organise all this to be type-safe and usable without casts.  One 
> possibility is to add `normalize` methods to CharFilterFactory and 
> TokenFilterFactory that mirror their existing `create` methods.  The default 
> implementation would return the input unchanged, while filters that should 
> apply at normalization time can delegate to `create`.
> Related to this, we should deprecate and remove LowerCaseTokenizer, which 
> combines tokenization and normalization in a way that will break this API.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk-9.0.4) - Build # 861 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/861/
Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseParallelGC

9 tests failed.
FAILED:  org.apache.solr.cloud.TestCloudPivotFacet.test

Error Message:
Error from server at http://127.0.0.1:50043/collection1_shard1_replica_n43: 
ERROR: [doc=1] unknown field 'pivot_b1'

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:50043/collection1_shard1_replica_n43: ERROR: 
[doc=1] unknown field 'pivot_b1'
at 
__randomizedtesting.SeedInfo.seed([C5E68E6BBCCA7F16:4DB2B1B1123612EE]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:561)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:156)
at 
org.apache.solr.cloud.TestCloudPivotFacet.test(TestCloudPivotFacet.java:134)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1010)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:985)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-12023) Autoscaling policy engine shuffles replicas needlessly and can also suggest nonexistent replicas to be moved

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668505#comment-16668505
 ] 

Shalin Shekhar Mangar commented on SOLR-12023:
--

[~noble.paul] - I see that the commit only fixes suggesting non-existent cores 
problem. What about the needlessly shuffling replicas -- is that already fixed? 
Also, where is the corresponding commit to master?

> Autoscaling policy engine shuffles replicas needlessly and can also suggest 
> nonexistent replicas to be moved
> 
>
> Key: SOLR-12023
> URL: https://issues.apache.org/jira/browse/SOLR-12023
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-11066-failing.patch, SOLR-12023.patch
>
>
> A test that I wrote in SOLR-11066 found the following problem:
> Cluster: 2 nodes
> Collection: 1 shard, 3 replicas, maxShardsPerNode=5
> No autoscaling policy or preference applied
> When the trigger runs, the computed plan needlessly shuffles all three 
> replicas and then proceeds to return suggestions with only numbers as core 
> names. These cores do not exist. I found that these numbers are generated 
> internally by the framework as placeholders for moved cores for further 
> calculations. They should never ever be suggested to the users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12023) Autoscaling policy engine shuffles replicas needlessly and can also suggest nonexistent replicas to be moved

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul resolved SOLR-12023.
---
Resolution: Fixed  (was: Duplicate)

> Autoscaling policy engine shuffles replicas needlessly and can also suggest 
> nonexistent replicas to be moved
> 
>
> Key: SOLR-12023
> URL: https://issues.apache.org/jira/browse/SOLR-12023
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling, SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
>Priority: Major
> Fix For: 7.6, master (8.0)
>
> Attachments: SOLR-11066-failing.patch, SOLR-12023.patch
>
>
> A test that I wrote in SOLR-11066 found the following problem:
> Cluster: 2 nodes
> Collection: 1 shard, 3 replicas, maxShardsPerNode=5
> No autoscaling policy or preference applied
> When the trigger runs, the computed plan needlessly shuffles all three 
> replicas and then proceeds to return suggestions with only numbers as core 
> names. These cores do not exist. I found that these numbers are generated 
> internally by the framework as placeholders for moved cores for further 
> calculations. They should never ever be suggested to the users.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org