[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393271#comment-15393271
 ] 

David Smiley commented on SOLR-9269:


Aside from my last comment RE one more synchronized to remove, I think I'm keen 
on getting this committed.  This is of course very new with tricky interactions 
with the rest of the code base so should be considered "experimental".

> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level. 
> Please refer to parent JIRA for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9269) Ability to create/delete/list snapshots for a solr core

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393268#comment-15393268
 ] 

ASF GitHub Bot commented on SOLR-9269:
--

Github user dsmiley commented on the issue:

https://github.com/apache/lucene-solr/pull/52
  
The last SolrSnapshotManager.deleteIndexFiles is still synchronized; I 
think you forgot this one?


> Ability to create/delete/list snapshots for a solr core
> ---
>
> Key: SOLR-9269
> URL: https://issues.apache.org/jira/browse/SOLR-9269
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Hrishikesh Gadre
>Assignee: David Smiley
> Attachments: SOLR-9269.patch
>
>
> Support snapshot create/delete/list functionality @ the Solr core level. 
> Please refer to parent JIRA for more details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr issue #52: [SOLR-9269] Ability to create/delete/list snapshots f...

[dsmiley]

Github user dsmiley commented on the issue:

https://github.com/apache/lucene-solr/pull/52
  
The last SolrSnapshotManager.deleteIndexFiles is still synchronized; I 
think you forgot this one?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5146) Figure out what it would take for lazily-loaded cores to play nice with SolrCloud

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393266#comment-15393266
 ] 

Erick Erickson commented on SOLR-5146:
--

In a word, "no". This is one of those special-purpose cases that I don't 
believe generalizes well to SolrCloud without some significant design work.

> Figure out what it would take for lazily-loaded cores to play nice with 
> SolrCloud
> -
>
> Key: SOLR-5146
> URL: https://issues.apache.org/jira/browse/SOLR-5146
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 4.5, 6.0
>Reporter: Erick Erickson
>Assignee: Shalin Shekhar Mangar
> Fix For: 6.0
>
>
> The whole lazy-load core thing was implemented with non-SolrCloud use-cases 
> in mind. There are several user-list threads that ask about using lazy cores 
> with SolrCloud, especially in multi-tenant use-cases.
> This is a marker JIRA to investigate what it would take to make lazy-load 
> cores play nice with SolrCloud. It's especially interesting how this all 
> works with shards, replicas, leader election, recovery, etc.
> NOTE: This is pretty much totally unexplored territory. It may be that a few 
> trivial modifications are all that's needed. OTOH, It may be that we'd have 
> to rip apart SolrCloud to handle this case. Until someone dives into the 
> code, we don't know.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 291 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/291/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:45897/c8n_1x3_lf_shard1_replica3]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:45897/c8n_1x3_lf_shard1_replica3]
at 
__randomizedtesting.SeedInfo.seed([9ACB1D22D6F00A12:129F22F8780C67EA]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:774)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:592)
at 
org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:578)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:174)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-3232) Admin UI: query form should have a menu to pick a request handler

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393202#comment-15393202
 ] 

David Smiley commented on SOLR-3232:


List everything that extends SearchHandler for sure.  Perhaps others could be 
entered manually depending on the details of the UI control type... or just not 
bother trying.

> Admin UI: query form should have a menu to pick a request handler
> -
>
> Key: SOLR-3232
> URL: https://issues.apache.org/jira/browse/SOLR-3232
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Reporter: David Smiley
> Attachments: SOLR-3232.patch
>
>
> The query form in the admin UI could use an improvement regarding how the 
> request handler is chosen; presently all there is is a text input for 'qt'.  
> The first choice to make in the form above the query should really be the 
> request handler since it actually handles the request before any other 
> parameters do anything.  It'd be great if it was a dynamically driven menu 
> defaulting to "/select".  Similar to how the DIH page finds DIH request 
> handlers, this page could find the request handlers with a class of 
> "SearchHandler".  Their names would be added to a list, and if the name 
> didn't start with a '/' then it would be prefixed with '/select?qt='.
> I did something similar (without the menu) to the old 3x UI in a patch to 
> SOLR-3161 which will hopefully get committed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9338) Solr cloud setup for 2 node cluster using embedded zookeepr

[nirav patel (JIRA)]

nirav patel created SOLR-9338:
-

 Summary: Solr cloud setup for 2 node cluster using embedded 
zookeepr
 Key: SOLR-9338
 URL: https://issues.apache.org/jira/browse/SOLR-9338
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: documentation
Affects Versions: 6.1
Reporter: nirav patel


I recently started using solr. I explored quite a bit into solr core over 
single machine. But I struggled a lot when I wanted to start 2 node solr server 
with embedded zookeeper. Well cause there is no documentation for it. I think 
anyone who want to run solr as a cloud would like to set it up on more then one 
cluster and play around with collections, admin UI to get realistic learning of 
a solr cloud. 
Here are simple steps to achieve this:
Run following on node where you want to start embedded zookeeper
./bin/solr start -cloud
All other nodes:
./bin/solr start -cloud -m 4g -z :

Also all command line utility like solr and post can use a decent 
documentation. Lot of options and commands are not even visible through call to 
`solr -help` . e.g. I didn't know till now that -healthcheck is a command!

I know there's a rich solr API which is invaluable for development and 
production use but anyone who just picking up and wants to play around CLI is a 
great way. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 740 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/740/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Expected to find shardAddress in the up shard info

Stack Trace:
java.lang.AssertionError: Expected to find shardAddress in the up shard info
at 
__randomizedtesting.SeedInfo.seed([E1F6BB08BFF248B8:69A284D2110E2540]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.TestDistributedSearch.comparePartialResponses(TestDistributedSearch.java:1172)
at 
org.apache.solr.TestDistributedSearch.queryPartialResults(TestDistributedSearch.java:1113)
at 
org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:973)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1011)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_92) - Build # 17368 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17368/
Java: 32bit/jdk1.8.0_92 -server -XX:+UseG1GC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.client.solrj.TestLBHttpSolrClient

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.client.solrj.TestLBHttpSolrClient: 1) Thread[id=1112, 
name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient]  
   at java.lang.Thread.sleep(Native Method) at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
 at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.client.solrj.TestLBHttpSolrClient: 
   1) Thread[id=1112, name=Connection evictor, state=TIMED_WAITING, 
group=TGRP-TestLBHttpSolrClient]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([4210D5C7FBC0BD90]:0)


FAILED:  org.apache.solr.client.solrj.TestLBHttpSolrClient.testReliability

Error Message:
Java heap space

Stack Trace:
java.lang.OutOfMemoryError: Java heap space




Build Log:
[...truncated 13253 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.TestLBHttpSolrClient
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.TestLBHttpSolrClient_4210D5C7FBC0BD90-001/init-core-data-001
   [junit4]   2> 75810 INFO  
(SUITE-TestLBHttpSolrClient-seed#[4210D5C7FBC0BD90]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 75812 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.SolrTestCaseJ4 ###Starting testTwoServers
   [junit4]   2> 75822 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 75822 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@109e054{/solr,null,AVAILABLE}
   [junit4]   2> 75824 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.e.j.s.ServerConnector Started ServerConnector@13b45c0{SSL,[ssl, 
http/1.1]}{127.0.0.1:46025}
   [junit4]   2> 75824 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.e.j.s.Server Started @77801ms
   [junit4]   2> 75824 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: 
{solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.TestLBHttpSolrClient_4210D5C7FBC0BD90-001/instance-0-001/collection1/data,
 solrconfig=bad_solrconfig.xml, hostContext=/solr, hostPort=46025}
   [junit4]   2> 75824 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): 
sun.misc.Launcher$AppClassLoader@1d16e93
   [junit4]   2> 75824 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: 
'/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.TestLBHttpSolrClient_4210D5C7FBC0BD90-001/instance-0-001'
   [junit4]   2> 75825 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 75825 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find 
system property or JNDI)
   [junit4]   2> 75825 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.c.SolrXmlConfig Loading container configuration from 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.TestLBHttpSolrClient_4210D5C7FBC0BD90-001/instance-0-001/solr.xml
   [junit4]   2> 75828 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.c.CorePropertiesLocator Config-defined core root directory: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.TestLBHttpSolrClient_4210D5C7FBC0BD90-001/instance-0-001/.
   [junit4]   2> 75828 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.c.CoreContainer New CoreContainer 31632579
   [junit4]   2> 75828 INFO  
(TEST-TestLBHttpSolrClient.testTwoServers-seed#[4210D5C7FBC0BD90]) [] 
o.a.s.c.CoreContainer Loading cores 

[jira] [Commented] (SOLR-6399) Implement unloadCollection in the Collections API

[Jerome Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393152#comment-15393152
 ] 

Jerome Yang commented on SOLR-6399:
---

Backup/Restore is much heavier than just unload/load a collection.

> Implement unloadCollection in the Collections API
> -
>
> Key: SOLR-6399
> URL: https://issues.apache.org/jira/browse/SOLR-6399
> Project: Solr
>  Issue Type: New Feature
>Reporter: dfdeshom
>Assignee: Shalin Shekhar Mangar
> Fix For: 6.0
>
>
> There is currently no way to unload a collection without deleting its 
> contents. There should be a way in the collections API to unload a collection 
> and reload it later, as needed.
> A use case for this is the following: you store logs by day, with each day 
> having its own collection. You are required to store up to 2 years of data, 
> which adds up to 730 collections.  Most of the time, you'll want to have 3 
> days of data loaded for search. Having just 3 collections loaded into memory, 
> instead of 730 will make managing Solr easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5146) Figure out what it would take for lazily-loaded cores to play nice with SolrCloud

[Jerome Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393148#comment-15393148
 ] 

Jerome Yang commented on SOLR-5146:
---

Any news on this issue?

> Figure out what it would take for lazily-loaded cores to play nice with 
> SolrCloud
> -
>
> Key: SOLR-5146
> URL: https://issues.apache.org/jira/browse/SOLR-5146
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Affects Versions: 4.5, 6.0
>Reporter: Erick Erickson
>Assignee: Shalin Shekhar Mangar
> Fix For: 6.0
>
>
> The whole lazy-load core thing was implemented with non-SolrCloud use-cases 
> in mind. There are several user-list threads that ask about using lazy cores 
> with SolrCloud, especially in multi-tenant use-cases.
> This is a marker JIRA to investigate what it would take to make lazy-load 
> cores play nice with SolrCloud. It's especially interesting how this all 
> works with shards, replicas, leader election, recovery, etc.
> NOTE: This is pretty much totally unexplored territory. It may be that a few 
> trivial modifications are all that's needed. OTOH, It may be that we'd have 
> to rip apart SolrCloud to handle this case. Until someone dives into the 
> code, we don't know.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (64bit/jdk-9-ea+127) - Build # 1273 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1273/
Java: 64bit/jdk-9-ea+127 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:41331/forceleader_test_collection_shard1_replica1]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:41331/forceleader_test_collection_shard1_replica1]
at 
__randomizedtesting.SeedInfo.seed([841611BA10B4994D:6281257A2936602C]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

Lucene/Solr Nightly Tests 6.x Build #126 stuck for 3 days?

[Kevin Risden]

https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/126/

Looks like there are lots of lines like:

[junit4] HEARTBEAT J0 PID(530@lucene1-us-west): 2016-07-26T02:10:40,
stalled for 272592s at: HdfsCollectionsAPIDistributedZkTest.test


Kevin Risden

[jira] [Commented] (LUCENE-7394) Make MemoryIndex immutable

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393061#comment-15393061
 ] 

David Smiley commented on LUCENE-7394:
--

I have to wonder if the design elegance goal you're after here is worth the 
price.  MemoryIndex is used in cases where it needs to be cheap to use/re-use 
for high-throughput.  Just keep this in mind please.

> Make MemoryIndex immutable
> --
>
> Key: LUCENE-7394
> URL: https://issues.apache.org/jira/browse/LUCENE-7394
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Martijn van Groningen
>
> The MemoryIndex itself should just be a builder that constructs an 
> IndexReader instance. The whole notion of freezing a memory index should be 
> removed.
> While we change this we should also clean this class up. There are many 
> methods to add a field, we should just have a single method that accepts a 
> `IndexableField`.
> The `keywordTokenStream(...)` method is unused and untested and should be 
> removed and it doesn't belong with the memory index.
> The `setSimilarity(...)`, `createSearcher(...)` and `search(...)` methods 
> should be removed, because the MemoryIndex should just be responsible for 
> creating an IndexReader instance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3232) Admin UI: query form should have a menu to pick a request handler

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393044#comment-15393044
 ] 

Alexandre Rafalovitch commented on SOLR-3232:
-

So, how would one mark /select from /export or /browse that make no sense to 
expose in Admin UI? Or do we just expose everything and let user sort it out?

> Admin UI: query form should have a menu to pick a request handler
> -
>
> Key: SOLR-3232
> URL: https://issues.apache.org/jira/browse/SOLR-3232
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Reporter: David Smiley
> Attachments: SOLR-3232.patch
>
>
> The query form in the admin UI could use an improvement regarding how the 
> request handler is chosen; presently all there is is a text input for 'qt'.  
> The first choice to make in the form above the query should really be the 
> request handler since it actually handles the request before any other 
> parameters do anything.  It'd be great if it was a dynamically driven menu 
> defaulting to "/select".  Similar to how the DIH page finds DIH request 
> handlers, this page could find the request handlers with a class of 
> "SearchHandler".  Their names would be added to a list, and if the name 
> didn't start with a '/' then it would be prefixed with '/select?qt='.
> I did something similar (without the menu) to the old 3x UI in a patch to 
> SOLR-3161 which will hopefully get committed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3232) Admin UI: query form should have a menu to pick a request handler

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15393037#comment-15393037
 ] 

David Smiley commented on SOLR-3232:


I definitely think it would be useful.  It's annoying and error-prone to 
remember what the request handler is.

> Admin UI: query form should have a menu to pick a request handler
> -
>
> Key: SOLR-3232
> URL: https://issues.apache.org/jira/browse/SOLR-3232
> Project: Solr
>  Issue Type: Improvement
>  Components: web gui
>Reporter: David Smiley
> Attachments: SOLR-3232.patch
>
>
> The query form in the admin UI could use an improvement regarding how the 
> request handler is chosen; presently all there is is a text input for 'qt'.  
> The first choice to make in the form above the query should really be the 
> request handler since it actually handles the request before any other 
> parameters do anything.  It'd be great if it was a dynamically driven menu 
> defaulting to "/select".  Similar to how the DIH page finds DIH request 
> handlers, this page could find the request handlers with a class of 
> "SearchHandler".  Their names would be added to a list, and if the name 
> didn't start with a '/' then it would be prefixed with '/select?qt='.
> I did something similar (without the menu) to the old 3x UI in a patch to 
> SOLR-3161 which will hopefully get committed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+127) - Build # 17367 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17367/
Java: 32bit/jdk-9-ea+127 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this 
request:[http://127.0.0.1:33932/upgo/forceleader_test_collection_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live 
SolrServers available to handle this 
request:[http://127.0.0.1:33932/upgo/forceleader_test_collection_shard1_replica2]
at 
__randomizedtesting.SeedInfo.seed([4DE02072471D17CD:AB7714B27E9FEEAC]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Updated] (SOLR-9337) Add fetch Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Summary: Add fetch Streaming Expression  (was: Add fetch() Streaming 
Expression)

> Add fetch Streaming Expression
> --
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
>update(collectionC, batchSize="100"
>   fetch(collectionB, 
> topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
> rows="50"),
> fl="j,m,z",
> on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Description: 
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
   update(collectionC, batchSize="100"
  fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}

  was:
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
   update(collectionC, batchSize="100"
  fetch(collectionB, 
   topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
rows="50"),
   fl="j,m,z",
   on="a=j")))
   
{code}


> Add fetch() Streaming Expression
> 
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
>update(collectionC, batchSize="100"
>   fetch(collectionB, 
> topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
> rows="50"),
> fl="j,m,z",
> on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Description: 
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
   update(collectionC, batchSize="100"
  fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}

  was:
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
   update(collectionC, batchSize="100"
fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}


> Add fetch() Streaming Expression
> 
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
>update(collectionC, batchSize="100"
>   fetch(collectionB, 
> topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
> rows="50"),
> fl="j,m,z",
> on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Description: 
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
   update(collectionC, batchSize="100"
  fetch(collectionB, 
   topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
rows="50"),
   fl="j,m,z",
   on="a=j")))
   
{code}

  was:
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
   update(collectionC, batchSize="100"
  fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}


> Add fetch() Streaming Expression
> 
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
>update(collectionC, batchSize="100"
>   fetch(collectionB, 
>topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
> rows="50"),
>fl="j,m,z",
>on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Description: 
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
   update(collectionC, batchSize="100"
fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}

  was:
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
  update(collectionC, batchSize="100"
fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}


> Add fetch() Streaming Expression
> 
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
>update(collectionC, batchSize="100"
> fetch(collectionB, 
> topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
> rows="50"),
> fl="j,m,z",
> on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Description: 
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
update(collectionC, batchSize="100"
fetch(collectionB, 
 topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
 fl="j,m,z",
 on="a=j")))
   
{code}

  was:
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
 update(collectionC, batchSize="100"
 fetch(collectionB, 
  topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
rows="50"),
  fl="j,m,z",
  on="a=j")))
   
{code}


> Add fetch() Streaming Expression
> 
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
> update(collectionC, batchSize="100"
> fetch(collectionB, 
>  topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
>  fl="j,m,z",
>  on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Description: 
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
  update(collectionC, batchSize="100"
fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}

  was:
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
update(collectionC, batchSize="100"
fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}


> Add fetch() Streaming Expression
> 
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
>   update(collectionC, batchSize="100"
> fetch(collectionB, 
> topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
> rows="50"),
> fl="j,m,z",
> on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Description: 
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
update(collectionC, batchSize="100"
fetch(collectionB, 
topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
fl="j,m,z",
on="a=j")))
   
{code}

  was:
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
update(collectionC, batchSize="100"
fetch(collectionB, 
 topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
 fl="j,m,z",
 on="a=j")))
   
{code}


> Add fetch() Streaming Expression
> 
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
> update(collectionC, batchSize="100"
> fetch(collectionB, 
> topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
> rows="50"),
> fl="j,m,z",
> on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-9337:
-
Description: 
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
 update(collectionC, batchSize="100"
 fetch(collectionB, 
  topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
rows="50"),
  fl="j,m,z",
  on="a=j")))
   
{code}

  was:
The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
 fetch(collectionB, 
 topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
 fl="j,m,z",
 on="a=j"))


{code}


> Add fetch() Streaming Expression
> 
>
> Key: SOLR-9337
> URL: https://issues.apache.org/jira/browse/SOLR-9337
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>
> The fetch() Streaming Expression wraps another expression and fetches 
> additional fields for documents in batches. The fetch() expression will 
> stream out the Tuples after the data has been fetched. Fields can be fetched 
> from any SolrCloud collection. 
> Sample syntax:
> {code}
> daemon(
>  update(collectionC, batchSize="100"
>  fetch(collectionB, 
>   topic(checkpoints, collectionA, q="*:*", fl="a,b,c", 
> rows="50"),
>   fl="j,m,z",
>   on="a=j")))
>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9337) Add fetch() Streaming Expression

[Joel Bernstein (JIRA)]

Joel Bernstein created SOLR-9337:


 Summary: Add fetch() Streaming Expression
 Key: SOLR-9337
 URL: https://issues.apache.org/jira/browse/SOLR-9337
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Joel Bernstein


The fetch() Streaming Expression wraps another expression and fetches 
additional fields for documents in batches. The fetch() expression will stream 
out the Tuples after the data has been fetched. Fields can be fetched from any 
SolrCloud collection. 

Sample syntax:

{code}
daemon(
 fetch(collectionB, 
 topic(checkpoints, collectionA, q="*:*", fl="a,b,c", rows="50"),
 fl="j,m,z",
 on="a=j"))


{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 342 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/342/
Java: 32bit/jdk1.8.0_92 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
expected: but was:

Stack Trace:
java.lang.AssertionError: expected: but was:
at 
__randomizedtesting.SeedInfo.seed([77C647B5284C93B5:FF92786F86B0FE4D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:208)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10808 lines...]
   [junit4] 

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_92) - Build # 6005 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6005/
Java: 32bit/jdk1.8.0_92 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
Error from server at http://127.0.0.1:62935/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J0/temp/solr.cloud.TestLocalFSCloudBackupRestore_1051272C8DDF54CA-001/tempDir-002/mytestbackup/

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:62935/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J0/temp/solr.cloud.TestLocalFSCloudBackupRestore_1051272C8DDF54CA-001/tempDir-002/mytestbackup/
at 
__randomizedtesting.SeedInfo.seed([1051272C8DDF54CA:980518F623233932]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:366)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:206)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+127) - Build # 17365 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17365/
Java: 64bit/jdk-9-ea+127 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:40940/fydph/c8n_1x3_lf_shard1_replica3]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://127.0.0.1:40940/fydph/c8n_1x3_lf_shard1_replica3]
at 
__randomizedtesting.SeedInfo.seed([CA6DC8B07B9E28D:84F2E351A9458F75]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:774)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (LUCENE-7395) Query Norm and coordination factor not calculated when PerFieldSimilarityWrapper is used

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392536#comment-15392536
 ] 

Uwe Schindler commented on LUCENE-7395:
---

I think a required ctor param is also fine. The abstract method is not needed 
as it should always be a constant. Like the other methods, I will make the 
impls final. Thanks Adrien!

As this changes API we can do this in 6.2, not earlier.

> Query Norm and coordination factor not calculated when 
> PerFieldSimilarityWrapper is used 
> -
>
> Key: LUCENE-7395
> URL: https://issues.apache.org/jira/browse/LUCENE-7395
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.3.1, 5.4.1
>Reporter: Sascha Markus
>
> If any kind of similarity is defined and therefore the 
> SchemaSimilarityFactory is defined as global similarity the queryNorm is 
> always 1.0
> The PerFieldSimilarityWrapper delegates some of the methods to the desired 
> Similarity but misses to delegate public float queryNorm(float 
> valueForNormalization)
> Instead the IndexReader calls this method on the base class Similarity.
> The result is that all scores are much higher.
> I created a custom similarity which extends ClassicSimilarity.
> To have the calculation fixed I did a local "hotfix"  which always uses the 
> default similarity. Also wrong for some cases but fine in my scenario.
>   @Override
>   public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization); // use default 
> similarity to calculate
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 290 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/290/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val modified' for path 'response/params/y/c' 
full output: {   "responseHeader":{ "status":0, "QTime":0},   
"response":{ "znodeVersion":0, "params":{"x":{ "a":"A val", 
"b":"B val", "":{"v":0},  from server:  
http://127.0.0.1:41650/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val modified' for 
path 'response/params/y/c' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "response":{
"znodeVersion":0,
"params":{"x":{
"a":"A val",
"b":"B val",
"":{"v":0},  from server:  http://127.0.0.1:41650/collection1
at 
__randomizedtesting.SeedInfo.seed([291A4B9ED2BD7993:A14E74447C41146B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:194)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[GitHub] lucene-solr issue #49: SOLR-9279 Adds comparison function queries

[softwaredoug]

Github user softwaredoug commented on the issue:

https://github.com/apache/lucene-solr/pull/49
  
Ok this needs another look at @dsmiley as I've addressed a bug where Long 
comparisons failed 

- Adds test that demonstrates bug comparing Longs
- Changed compare to take two Comparables instead of doubles
- Perform an integer comparison when it appears the two input values are 
effectively integers
- Otherwise does a floating point comparison

Unfortunately, I think tihs interface can't take arbitrary value sources. I 
feel like I need to enforce the safest comparison (integer vs floating pt) to 
avoid bugs.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9279) Add greater than, less than, etc in Solr function queries

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392520#comment-15392520
 ] 

ASF GitHub Bot commented on SOLR-9279:
--

Github user softwaredoug commented on the issue:

https://github.com/apache/lucene-solr/pull/49
  
Ok this needs another look at @dsmiley as I've addressed a bug where Long 
comparisons failed 

- Adds test that demonstrates bug comparing Longs
- Changed compare to take two Comparables instead of doubles
- Perform an integer comparison when it appears the two input values are 
effectively integers
- Otherwise does a floating point comparison

Unfortunately, I think tihs interface can't take arbitrary value sources. I 
feel like I need to enforce the safest comparison (integer vs floating pt) to 
avoid bugs.


> Add greater than, less than, etc in Solr function queries
> -
>
> Key: SOLR-9279
> URL: https://issues.apache.org/jira/browse/SOLR-9279
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Reporter: Doug Turnbull
> Fix For: master (7.0)
>
>
> If you use the "if" function query, you'll often expect to be able to use 
> greater than/less than functions. For example, you might want to boost books 
> written in the past 7 years. Unfortunately, there's no "greater than" 
> function query that will return non-zero when the lhs > rhs. Instead to get 
> this, you need to create really awkward function queries like I do here 
> (http://opensourceconnections.com/blog/2014/11/26/stepwise-date-boosting-in-solr/):
> if(min(0,sub(ms(mydatefield),sub(ms(NOW),315569259747))),0.8,1)
> The pull request attached to this Jira adds the following function queries
> (https://github.com/apache/lucene-solr/pull/49)
> -gt(lhs, rhs) (returns 1 if lhs > rhs, 0 otherwise)
> -lt(lhs, rhs) (returns 1 if lhs < rhs, 0 otherwise)
> -gte
> -lte
> -eq
> So instead of 
> if(min(0,sub(ms(mydatefield),sub(ms(NOW),315569259747))),0.8,1)
> one could now write
> if(lt(ms(mydatefield),315569259747,0.8,1)
> (if mydatefield < 315569259747 then 0.8 else 1)
> A bit more readable and less puzzling



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[Jeff Wartes (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392504#comment-15392504
 ] 

Jeff Wartes commented on SOLR-6581:
---

For what it's worth, I recall having a bad experience with that hint in a Solr 
5.4 cluster late last year. I never did dig into why though.
I had a similar case where I was collapsing on a highly distinct field, and as 
Joel indicates, the memory allocation rate was bad enough I had to give up on 
the whole thing. Joel and I discussed this a little in SOLR-9125 if you're 
curious.

> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7396) Speed up flush of 1-dimension points

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392481#comment-15392481
 ] 

Michael McCandless commented on LUCENE-7396:


+1, cool!

> Speed up flush of 1-dimension points
> 
>
> Key: LUCENE-7396
> URL: https://issues.apache.org/jira/browse/LUCENE-7396
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7396.patch
>
>
> 1D points already have an optimized merge implementation which works when 
> points come in order. So maybe we could make IndexWriter's PointValuesWriter 
> sort before feeding the PointsFormat and somehow propagate the information to 
> the PointsFormat?
> The benefit is that flushing could directly stream points to disk with little 
> memory usage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7396) Speed up flush of 1-dimension points

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-7396:
-
Attachment: LUCENE-7396.patch

Here is a patch that demonstrates the idea. PointValuesWriter sorts before 
feeding the PointsFormat's writer and Lucene60PointsWriter.addField consumes 
the PointsReader twice in the one dimension case: once to detect whether it is 
sorted, and once to write values similarly to the way merging works. In spite 
of the fact that the reader is consumed twice, I saw better indexing 
performance of IndexAndSearchOpenStreetMaps1D: 3 runs on master gave me 74, 78 
and 75 seconds while 3 runs with this patch gave me 65, 67 and 67 seconds.

It would be nice if somehow we could propagate the information that the reader 
is sorted rather than having to iterate it twice?

> Speed up flush of 1-dimension points
> 
>
> Key: LUCENE-7396
> URL: https://issues.apache.org/jira/browse/LUCENE-7396
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-7396.patch
>
>
> 1D points already have an optimized merge implementation which works when 
> points come in order. So maybe we could make IndexWriter's PointValuesWriter 
> sort before feeding the PointsFormat and somehow propagate the information to 
> the PointsFormat?
> The benefit is that flushing could directly stream points to disk with little 
> memory usage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392478#comment-15392478
 ] 

David Smiley commented on SOLR-6581:


This is just one shard; the actual total collection is sharded beyond the 
levels most people shard.

In any case, maybe some time I'll peek into profiling what's going on to see if 
I can find any insights.  But that's low priority as our Solr 6 upgrade without 
the hint=top_fc is already a huge improvement overall.

> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7396) Speed up flush of 1-dimension points

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-7396:


 Summary: Speed up flush of 1-dimension points
 Key: LUCENE-7396
 URL: https://issues.apache.org/jira/browse/LUCENE-7396
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Priority: Minor


1D points already have an optimized merge implementation which works when 
points come in order. So maybe we could make IndexWriter's PointValuesWriter 
sort before feeding the PointsFormat and somehow propagate the information to 
the PointsFormat?

The benefit is that flushing could directly stream points to disk with little 
memory usage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392469#comment-15392469
 ] 

Joel Bernstein commented on SOLR-6581:
--

Wow that's a hard case.

The string collapse is done into a fixed array the size of the unique values in 
the field. Similar to faceting on a string field. So we're talking about a huge 
amount of memory for a single query. Still not sure why MultiDocValues would 
outperform the top level field cache in this scenario. 

In this scenario sharding would be very useful, but you would have to shard on 
the collapse field.



> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_92) - Build # 17364 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17364/
Java: 64bit/jdk1.8.0_92 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.security.BasicAuthIntegrationTest.testBasics

Error Message:
IOException occured when talking to server at: 
http://127.0.0.1:41211/solr/testSolrCloudCollection_shard1_replica2

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: IOException 
occured when talking to server at: 
http://127.0.0.1:41211/solr/testSolrCloudCollection_shard1_replica2
at 
__randomizedtesting.SeedInfo.seed([95CA21ACC0A870C8:A8128F80F8462EB8]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.security.BasicAuthIntegrationTest.doExtraTests(BasicAuthIntegrationTest.java:193)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testCollectionCreateSearchDelete(TestMiniSolrCloudClusterBase.java:196)
at 
org.apache.solr.cloud.TestMiniSolrCloudClusterBase.testBasics(TestMiniSolrCloudClusterBase.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392451#comment-15392451
 ] 

Joel Bernstein commented on SOLR-6581:
--

Ok, false alarm. my initial tests were faulty. I thought a had loaded 5,000,000 
docs and actually a had set the job to load 50,000,000 docs. So the test was 
running with the indexing running.

After running a proper test I found that things are as expected. I'm seeing 
to_fc queries running almost 3 times faster with the top_fc hint.

I was running with these simple queries:

{code}
{!collapse field=test_s hint=top_fc}

and

{!collapse field=test_s}
{code}

I had an index of 5 million docs and the test_s field had 1.8 million unique 
values.

With the top_fc hint the query was taking around 160 millis.
Without the top_fc hint the query was taking around 440 millis.

> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9088) solr.schema.TestManagedSchemaAPI.test failures ([doc=2] unknown field 'myNewField1')

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-9088:

Attachment: 1280-fail.log

Attaching the entire console log from the failure Steve mentioned.

This test used to fail daily before the commit , but since the commit ( 20 days 
back ) there's been just this one failure.

I'll have to look at it more closely , but couldn't tell based on an initial 
glance what the problem is 

> solr.schema.TestManagedSchemaAPI.test failures ([doc=2] unknown field 
> 'myNewField1')
> 
>
> Key: SOLR-9088
> URL: https://issues.apache.org/jira/browse/SOLR-9088
> Project: Solr
>  Issue Type: Test
>Reporter: Christine Poerschke
>Assignee: Varun Thacker
>Priority: Minor
> Fix For: 6.2
>
> Attachments: 1280-fail.log, SOLR-9088.patch, SOLR-9088.patch
>
>
> e.g.
> http://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3256/
> http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/588/
> http://jenkins.thetaphi.de/job/Lucene-Solr-5.5-Linux/266/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Pushkar Raste (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392412#comment-15392412
 ] 

Pushkar Raste commented on SOLR-9310:
-

Another question. 

Why update commands with flag {{REPLAY}} are not buffered in 
{{DistributedUpdateProcessor.versionAdd()}} but update commands with flag 
{{PEER_SYNC}} are. 

Also even you ask for IndexFingerPrint of a specific version, you will end up 
sending {{Long.MAX_VALUE}} while asking for versions to 
{{RealTimeGetComponent}}. 

Given all different race conditions, I am not sure how much we really gain out 
of fingerprint check

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392402#comment-15392402
 ] 

David Smiley commented on SOLR-6581:


[~joel.bernstein] It's a single index (shard) of 20.8M docs spanning 180 
segments (kinda a lot I know) containing a string parent-document-id field with 
docValues.  The field is always populated and has almost as many distinct 
values as there are documents -- 18.9M. The collapse usage has 
{{max=(aBooleanField)}} and no sort.  I do see a field cache (UninvertedReader) 
entry on this field via looking at the admin screen (says it takes up ~403MB).

> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392392#comment-15392392
 ] 

Joel Bernstein commented on SOLR-6581:
--

I just ran a test with collapse on trunk and validated [~dsmiley] findings.  
Collapse at this point is so slow both with and without top_fc that it's really 
not fit for purpose. 

I'll dig a little to see if I can see what's happening.


> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 739 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/739/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateChecksCurrentState

Error Message:


Stack Trace:
java.util.concurrent.TimeoutException
at 
__randomizedtesting.SeedInfo.seed([195B4ACEF7A06A4:D779DB74BF237BF1]:0)
at 
org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1233)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.waitForState(CloudSolrClient.java:651)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateChecksCurrentState(TestCollectionStateWatchers.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13026 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating dataDir: 

[jira] [Commented] (LUCENE-7395) Query Norm and coordination factor not calculated when PerFieldSimilarityWrapper is used

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392331#comment-15392331
 ] 

Adrien Grand commented on LUCENE-7395:
--

I agree the default impl of coord and queryNorm on PerFieldSimilarity is trappy 
in 6.x. +1 to adding an abstract method or a required constructor argument.

> Query Norm and coordination factor not calculated when 
> PerFieldSimilarityWrapper is used 
> -
>
> Key: LUCENE-7395
> URL: https://issues.apache.org/jira/browse/LUCENE-7395
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.3.1, 5.4.1
>Reporter: Sascha Markus
>
> If any kind of similarity is defined and therefore the 
> SchemaSimilarityFactory is defined as global similarity the queryNorm is 
> always 1.0
> The PerFieldSimilarityWrapper delegates some of the methods to the desired 
> Similarity but misses to delegate public float queryNorm(float 
> valueForNormalization)
> Instead the IndexReader calls this method on the base class Similarity.
> The result is that all scores are much higher.
> I created a custom similarity which extends ClassicSimilarity.
> To have the calculation fixed I did a local "hotfix"  which always uses the 
> default similarity. Also wrong for some cases but fine in my scenario.
>   @Override
>   public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization); // use default 
> similarity to calculate
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392314#comment-15392314
 ] 

Yonik Seeley commented on SOLR-9310:


bq. The variable name is confusing. maxInHash is not a hash 

max-in-hash... t's the maximum value included in the hash.  That name does not 
seem to imply that it *is* a hash.


> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392310#comment-15392310
 ] 

Joel Bernstein commented on SOLR-6581:
--

[~dsmiley], can you post the query you were using. And also some specs on the 
data set and result set. Things like how many unique values in the collapse 
field, how documents in the result set pre-collapse. Thanks!

> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392301#comment-15392301
 ] 

Noble Paul commented on SOLR-9310:
--

Well, There is a way to compute a fingerprint upto a point . So, if replica has 
updates  till version 'x' and if  fingerprint of the leader matches till 'x' 
wouldn't it be good enough? 

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9335) Move solr stats collections to use LongAdder

[Jeff Wartes (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392297#comment-15392297
 ] 

Jeff Wartes commented on SOLR-9335:
---

fwiw, SOLR-8241 involves cache implementations that (among other improvements) 
uses LongAddr, and the author has been having trouble getting committer 
attention.

> Move solr stats collections to use LongAdder
> 
>
> Key: SOLR-9335
> URL: https://issues.apache.org/jira/browse/SOLR-9335
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Minor
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9335.patch, SOLR-9335.patch
>
>
> With Java 8 we can use LongAdder which has more throughput under high 
> contention .
> These classes of Solr should benefit from LongAdder
> - Caches ( ConcurentLRUCache / LRUCache )
> - Searches ( RequestHandlerBase )
> - Updates ( DirectUpdateHandler2 )



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-9315) SchemaSimilarityFactory should delegate queryNorm and coord to the default similarity

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand closed SOLR-9315.
--
Resolution: Duplicate

> SchemaSimilarityFactory should delegate queryNorm and coord to the default 
> similarity
> -
>
> Key: SOLR-9315
> URL: https://issues.apache.org/jira/browse/SOLR-9315
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: SOLR-9315.patch
>
>
> This is a follow-up to the discussion with [~upayavira] on LUCENE-6590: 
> SchemaSimilarityFactory can easily build similarities that apply the idf 
> twice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9315) SchemaSimilarityFactory should delegate queryNorm and coord to the default similarity

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392295#comment-15392295
 ] 

Adrien Grand commented on SOLR-9315:


Yes Uwe, both queryNorm and coords are gone in master.

> SchemaSimilarityFactory should delegate queryNorm and coord to the default 
> similarity
> -
>
> Key: SOLR-9315
> URL: https://issues.apache.org/jira/browse/SOLR-9315
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: SOLR-9315.patch
>
>
> This is a follow-up to the discussion with [~upayavira] on LUCENE-6590: 
> SchemaSimilarityFactory can easily build similarities that apply the idf 
> twice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392287#comment-15392287
 ] 

Joel Bernstein commented on SOLR-6581:
--

Were you using the sort param or max/min param? All the top_fc performance 
testing was done with max/min param. The sort param came later and I'm not sure 
how it performs with top_fc.

The top_fc param should always be the fastest approach when using min/max. If 
it's not then something odd is going on.



> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9279) Add greater than, less than, etc in Solr function queries

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392283#comment-15392283
 ] 

ASF GitHub Bot commented on SOLR-9279:
--

Github user softwaredoug commented on the issue:

https://github.com/apache/lucene-solr/pull/49
  
Actually yes, I would see that as a bug right now. I bet comparing two
longs one off would fail a > or < comparison due to loss of precision as a
double. I'm going to try to create a test that recreates that.

On Fri, Jul 8, 2016 at 2:37 PM David Smiley 
wrote:

> Just one thing -- have compare() take the FunctionValue so that a compare
> impl can choose to call doubleVal vs longVal or whatever else. And the
> impls you add to Solr can call doubleVal. Someone truly might want to
> extend this to call something other than doubleVal; the set of values of
> doubleVal is disjoint from longVal. Or maybe someone has got the data in
> objectVal for some reason.
>
> After that please post a .patch file to JIRA.
> https://wiki.apache.org/lucene-java/HowToContribute#Creating_a_patch
> though those instructions should be modified to indicate how to generate a
> diff from the point the current branch diverged from master.
>
> —
> You are receiving this because you commented.
>
>
> Reply to this email directly, view it on GitHub
> ,
> or mute the thread
> 

> .
>



> Add greater than, less than, etc in Solr function queries
> -
>
> Key: SOLR-9279
> URL: https://issues.apache.org/jira/browse/SOLR-9279
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Reporter: Doug Turnbull
> Fix For: master (7.0)
>
>
> If you use the "if" function query, you'll often expect to be able to use 
> greater than/less than functions. For example, you might want to boost books 
> written in the past 7 years. Unfortunately, there's no "greater than" 
> function query that will return non-zero when the lhs > rhs. Instead to get 
> this, you need to create really awkward function queries like I do here 
> (http://opensourceconnections.com/blog/2014/11/26/stepwise-date-boosting-in-solr/):
> if(min(0,sub(ms(mydatefield),sub(ms(NOW),315569259747))),0.8,1)
> The pull request attached to this Jira adds the following function queries
> (https://github.com/apache/lucene-solr/pull/49)
> -gt(lhs, rhs) (returns 1 if lhs > rhs, 0 otherwise)
> -lt(lhs, rhs) (returns 1 if lhs < rhs, 0 otherwise)
> -gte
> -lte
> -eq
> So instead of 
> if(min(0,sub(ms(mydatefield),sub(ms(NOW),315569259747))),0.8,1)
> one could now write
> if(lt(ms(mydatefield),315569259747,0.8,1)
> (if mydatefield < 315569259747 then 0.8 else 1)
> A bit more readable and less puzzling



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr issue #49: SOLR-9279 Adds comparison function queries

[softwaredoug]

Github user softwaredoug commented on the issue:

https://github.com/apache/lucene-solr/pull/49
  
Actually yes, I would see that as a bug right now. I bet comparing two
longs one off would fail a > or < comparison due to loss of precision as a
double. I'm going to try to create a test that recreates that.

On Fri, Jul 8, 2016 at 2:37 PM David Smiley 
wrote:

> Just one thing -- have compare() take the FunctionValue so that a compare
> impl can choose to call doubleVal vs longVal or whatever else. And the
> impls you add to Solr can call doubleVal. Someone truly might want to
> extend this to call something other than doubleVal; the set of values of
> doubleVal is disjoint from longVal. Or maybe someone has got the data in
> objectVal for some reason.
>
> After that please post a .patch file to JIRA.
> https://wiki.apache.org/lucene-java/HowToContribute#Creating_a_patch
> though those instructions should be modified to indicate how to generate a
> diff from the point the current branch diverged from master.
>
> —
> You are receiving this because you commented.
>
>
> Reply to this email directly, view it on GitHub
> ,
> or mute the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392264#comment-15392264
 ] 

Joel Bernstein edited comment on SOLR-9252 at 7/25/16 4:50 PM:
---

Ok, I reviewed the TextLogisticRegressionCollector. I think we're going to have 
to change this implementation. I thought I saw an older version of this that 
was using the finish() method to perform the logit. But in the current version 
it's doing this in the collect method. This is going to have trouble scaling 
and it also requires term vectors which we don't want to have to use.

I think the approach to take is to collect the matching bitset in the collect() 
method.

Then in the finish() method the logic is:

1) Get the top level TermsEnum
2) Iterate the features and seek into the terms enum
3) For each feature iterate the DocsEnum and compare to the matching docs 
bitset to build the doc vectors.

With this approach the DocVectors are a multi-dimension array:

doc0-> [featureValue, featureValue, featureValue]
doc1-> [featureValue, featureValue, featureValue]
...

With this approach we'll have to hold all the docVectors in memory at once. So 
if you have hundreds of features and millions of records in the training set 
then you'll have a large cluster to do the work. 

We can also add a randomized approach to this so that not every doc vector is 
calculated on each iteration.






was (Author: joel.bernstein):
Ok, I reviewed the TextLogisticRegressionCollector. I think we're going to have 
to change this implementation. I thought I saw an older version of this that 
was using the finish() method to perform the logit. But in the current version 
it's doing this in the collect method. This is going to have trouble scaling 
and it also requires term vectors which we don't want to have to use.

I think the approach to take is to collect the matching bitset in the collect() 
method.

Then in the finish() method the logic is:

1) Get the top level TermEnum
2) Iterate the features and seek into the terms enum
3) For each feature iterate the DocsEnum and compare to the matching docs 
bitset to build the doc vectors.

With this approach the DocVectors are a multi-dimension array:

doc0-> [featureValue, featureValue, featureValue]
doc1-> [featureValue, featureValue, featureValue]
...

With this approach we'll have to hold all the docVectors in memory at once. So 
if you have hundreds of features and millions of records in the training set 
then you'll have a large cluster to do the work. 

We can also add a randomized approach to this so that not every doc vector is 
calculated on each iteration.





> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, 

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392264#comment-15392264
 ] 

Joel Bernstein commented on SOLR-9252:
--

Ok, I reviewed the TextLogisticRegressionCollector. I think we're going to have 
to change this implementation. I thought I saw an older version of this that 
was using the finish() method to perform the logit. But in the current version 
it's doing this in the collect method. This is going to have trouble scaling 
and it also requires term vectors which we don't want to have to use.

I think the approach to take is to collect the matching bitset in the collect() 
method.

Then in the finish() method the logic is:

1) Get the top level TermEnum
2) Iterate the features and seek into the terms enum
3) For each feature iterate the DocsEnum and compare to the matching docs 
bitset to build the doc vectors.

With this approach the DocVectors are a multi-dimension array:

doc0-> [featureValue, featureValue, featureValue]
doc1-> [featureValue, featureValue, featureValue]
...

With this approach we'll have to hold all the docVectors in memory at once. So 
if you have hundreds of features and millions of records in the training set 
then you'll have a large cluster to do the work. 

We can also add a randomized approach to this so that not every doc vector is 
calculated on each iteration.





> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Pushkar Raste (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392262#comment-15392262
 ] 

Pushkar Raste commented on SOLR-9310:
-

That won't work as leader may have already diverged when compared to node that 
is just coming up

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392232#comment-15392232
 ] 

Noble Paul commented on SOLR-9310:
--

I guess we should compare the fingerprint before the updates are applied. I'm 
testing another approach. Will report back

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9310:
-
Comment: was deleted

(was: Commit 7d388c02b1f71f7af0879925a7496050a4644e25 in lucene-solr's branch 
refs/heads/branch_6x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7d388c0 ]

SOLR-9310: reverting the variable name change becaause it can break back-compat
)

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9310:
-
Comment: was deleted

(was: Commit 830228602df1da2ba5c7e4c2688434108414e221 in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8302286 ]

SOLR-9310: The variable name is confusing. maxInHash is not a hash
)

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9310:
-
Comment: was deleted

(was: Commit 4c61d5aaccb420a359501612003913fb619e495c in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4c61d5a ]

SOLR-9310: reverting the variable name change becaause it can break back-compat
)

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9335) Move solr stats collections to use LongAdder

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-9335:

Attachment: SOLR-9335.patch

Thanks David for the review!

Updated patch which fixes the test case which was broken by this. I'll run 
tests and if everything goes fine commit it tomorrow

> Move solr stats collections to use LongAdder
> 
>
> Key: SOLR-9335
> URL: https://issues.apache.org/jira/browse/SOLR-9335
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Minor
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9335.patch, SOLR-9335.patch
>
>
> With Java 8 we can use LongAdder which has more throughput under high 
> contention .
> These classes of Solr should benefit from LongAdder
> - Caches ( ConcurentLRUCache / LRUCache )
> - Searches ( RequestHandlerBase )
> - Updates ( DirectUpdateHandler2 )



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+127) - Build # 17363 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17363/
Java: 32bit/jdk-9-ea+127 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'params/c' full output: {   
"responseHeader":{ "status":0, "QTime":0},   "params":{ "a":"A 
val", "b":"B val", "wt":"json", "useParams":""},   "context":{ 
"webapp":"/ou/nx", "path":"/dump1", "httpMethod":"GET"}},  from server: 
 http://127.0.0.1:36422/ou/nx/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 
'params/c' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "params":{
"a":"A val",
"b":"B val",
"wt":"json",
"useParams":""},
  "context":{
"webapp":"/ou/nx",
"path":"/dump1",
"httpMethod":"GET"}},  from server:  
http://127.0.0.1:36422/ou/nx/collection1
at 
__randomizedtesting.SeedInfo.seed([6AF0ABDDC18F009E:E2A494076F736D66]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:171)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Issue Comment Deleted] (SOLR-9310) PeerSync fails on a node restart due to IndexFingerPrint mismatch

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9310:
-
Comment: was deleted

(was: Commit 3740156adfd8c85ce59af1f257df8c9b53c688ca in lucene-solr's branch 
refs/heads/branch_6x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3740156 ]

SOLR-9310: The variable name is confusing. maxInHash is not a hash
)

> PeerSync fails on a node restart due to IndexFingerPrint mismatch
> -
>
> Key: SOLR-9310
> URL: https://issues.apache.org/jira/browse/SOLR-9310
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Assignee: Noble Paul
> Attachments: SOLR-9310.patch, SOLR-9310.patch
>
>
> I found that Peer Sync fails if a node restarts and documents were indexed 
> while node was down. IndexFingerPrint check fails after recovering node 
> applies updates. 
> This happens only when node restarts and not if node just misses updates due 
> reason other than it being down.
> Please check attached patch for the test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9315) SchemaSimilarityFactory should delegate queryNorm and coord to the default similarity

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392192#comment-15392192
 ] 

Uwe Schindler commented on SOLR-9315:
-

I think this should be done in Lucene, not Solr. The problem exists there, too.

I agree, for the coord factor its no longer needed in master. Is queryNorm also 
gone in master?

> SchemaSimilarityFactory should delegate queryNorm and coord to the default 
> similarity
> -
>
> Key: SOLR-9315
> URL: https://issues.apache.org/jira/browse/SOLR-9315
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: SOLR-9315.patch
>
>
> This is a follow-up to the discussion with [~upayavira] on LUCENE-6590: 
> SchemaSimilarityFactory can easily build similarities that apply the idf 
> twice.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392187#comment-15392187
 ] 

Joel Bernstein commented on SOLR-9252:
--

Ok, I made the change with the sort locally and now the terms are not dropping 
off as the numTerms changes. I'll keep testing.

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7395) Query Norm and coordination factor not calculated when PerFieldSimilarityWrapper is used

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392181#comment-15392181
 ] 

Uwe Schindler commented on LUCENE-7395:
---

My proposal is to add a second abstarct method like the protected {{get()}}: 
[https://lucene.apache.org/core/6_1_0/core/org/apache/lucene/search/similarities/PerFieldSimilarityWrapper.html#get-java.lang.String-]

In that case it could be named {{abstract Similarity default()}}.

> Query Norm and coordination factor not calculated when 
> PerFieldSimilarityWrapper is used 
> -
>
> Key: LUCENE-7395
> URL: https://issues.apache.org/jira/browse/LUCENE-7395
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.3.1, 5.4.1
>Reporter: Sascha Markus
>
> If any kind of similarity is defined and therefore the 
> SchemaSimilarityFactory is defined as global similarity the queryNorm is 
> always 1.0
> The PerFieldSimilarityWrapper delegates some of the methods to the desired 
> Similarity but misses to delegate public float queryNorm(float 
> valueForNormalization)
> Instead the IndexReader calls this method on the base class Similarity.
> The result is that all scores are much higher.
> I created a custom similarity which extends ClassicSimilarity.
> To have the calculation fixed I did a local "hotfix"  which always uses the 
> default similarity. Also wrong for some cases but fine in my scenario.
>   @Override
>   public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization); // use default 
> similarity to calculate
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7395) Query Norm and coordination factor not calculated when PerFieldSimilarityWrapper is used

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392178#comment-15392178
 ] 

Uwe Schindler commented on LUCENE-7395:
---

I think the Solr one is a duplicate of this one. The problem already exists in 
Lucene, because it returns a static constant of 1 for those methods.

I have the feeling that PerFieldSimilarityWrapper should provide some 
"fallback" similraity (defaultSimilarity in Solr), too. Same as 
PerFieldAnalyzerWrapper. It is currently abstract and you have to implement the 
"per field" method. But for all those which do not use fields, it should also 
provide some abstract method to return a default for all methods in similarity 
that are not per-field.

> Query Norm and coordination factor not calculated when 
> PerFieldSimilarityWrapper is used 
> -
>
> Key: LUCENE-7395
> URL: https://issues.apache.org/jira/browse/LUCENE-7395
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.3.1, 5.4.1
>Reporter: Sascha Markus
>
> If any kind of similarity is defined and therefore the 
> SchemaSimilarityFactory is defined as global similarity the queryNorm is 
> always 1.0
> The PerFieldSimilarityWrapper delegates some of the methods to the desired 
> Similarity but misses to delegate public float queryNorm(float 
> valueForNormalization)
> Instead the IndexReader calls this method on the base class Similarity.
> The result is that all scores are much higher.
> I created a custom similarity which extends ClassicSimilarity.
> To have the calculation fixed I did a local "hotfix"  which always uses the 
> default similarity. Also wrong for some cases but fine in my scenario.
>   @Override
>   public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization); // use default 
> similarity to calculate
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6581) Efficient DocValues support and numeric collapse field implementations for Collapse and Expand

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392175#comment-15392175
 ] 

David Smiley commented on SOLR-6581:


Dallan:
bq. Any ideas how much slower numeric collapse/expand implementation is than 
string collapse/expand with the top_fc hint?

I would guess a numeric impl would beat a string impl every time.

Hey FYI everyone... this is weird but while I just finished doing a major Solr 
4.10 -> Solr 6.1.0 upgrade, I found that the top_fc for the collapse had quite 
the opposite effect in a test environment _with no indexing/commits_.  top_fc 
took twice as long as without.  Shrug; no clue.  So folks don't go setting this 
blindly without actually measuirng before & after in your own environment.

> Efficient DocValues support and numeric collapse field implementations for 
> Collapse and Expand
> --
>
> Key: SOLR-6581
> URL: https://issues.apache.org/jira/browse/SOLR-6581
> Project: Solr
>  Issue Type: Bug
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Minor
> Fix For: 5.0, 6.0
>
> Attachments: SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, SOLR-6581.patch, 
> renames.diff
>
>
> The 4x implementation of the CollapsingQParserPlugin and the ExpandComponent 
> are optimized to work with a top level FieldCache. Top level FieldCaches have 
> a very fast docID to top-level ordinal lookup. Fast access to the top-level 
> ordinals allows for very high performance field collapsing on high 
> cardinality fields. 
> LUCENE-5666 unified the DocValues and FieldCache api's so that the top level 
> FieldCache is no longer in regular use. Instead all top level caches are 
> accessed through MultiDocValues. 
> This ticket does the following:
> 1) Optimizes Collapse and Expand to use MultiDocValues and makes this the 
> default approach when collapsing on String fields
> 2) Provides an option to use a top level FieldCache if the performance of 
> MultiDocValues is a blocker. The mechanism for switching to the FieldCache is 
> a new "hint" parameter. If the hint parameter is set to "top_fc" then the 
> top-level FieldCache would be used for both Collapse and Expand.
> Example syntax:
> {code}
> fq={!collapse field=x hint=TOP_FC}
> {code}
> 3)  Adds numeric collapse field implementations.
> 4) Resolves issue SOLR-6066
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9252) Feature selection and logistic regression on text

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392141#comment-15392141
 ] 

Cao Manh Dat edited comment on SOLR-9252 at 7/25/16 3:59 PM:
-

I'm thinking about change *tlogit* to *train* function. Because different 
algorithms have different set of parameters. For example : *tlogit* vs *logit* 
have totally different parameters. I think we should change *featuresSelection* 
to features but keep *tlogit* as it is.

[~joel.bernstein] +1 for sum up the igain score from all shards. So we can get 
best terms from all shards. But this is not yet proven because it based on a 
lot of assumption about how documents, classes, terms is distributed. also, I 
think it will be good enough for most cases. If you dont have any comments, I 
will submit a fixed patch soon.


was (Author: caomanhdat):
I'm thinking about change *tlogit* to *train* function. Because different 
algorithms have different set of parameters. For example : *tlogit* vs *logit* 
have totally different parameters. I think we should change *featuresSelection* 
to features but keep *tlogit* as it is.

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9335) Move solr stats collections to use LongAdder

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392143#comment-15392143
 ] 

David Smiley commented on SOLR-9335:


+1 LGTM

> Move solr stats collections to use LongAdder
> 
>
> Key: SOLR-9335
> URL: https://issues.apache.org/jira/browse/SOLR-9335
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Minor
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9335.patch
>
>
> With Java 8 we can use LongAdder which has more throughput under high 
> contention .
> These classes of Solr should benefit from LongAdder
> - Caches ( ConcurentLRUCache / LRUCache )
> - Searches ( RequestHandlerBase )
> - Updates ( DirectUpdateHandler2 )



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9252) Feature selection and logistic regression on text

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392141#comment-15392141
 ] 

Cao Manh Dat commented on SOLR-9252:


I'm thinking about change *tlogit* to *train* function. Because different 
algorithms have different set of parameters. For example : *tlogit* vs *logit* 
have totally different parameters. I think we should change *featuresSelection* 
to features but keep *tlogit* as it is.

> Feature selection and logistic regression on text
> -
>
> Key: SOLR-9252
> URL: https://issues.apache.org/jira/browse/SOLR-9252
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Joel Bernstein
> Attachments: SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, 
> SOLR-9252.patch, SOLR-9252.patch, SOLR-9252.patch, enron1.zip
>
>
> SOLR-9186 come up with a challenges that for each iterative we have to 
> rebuild the tf-idf vector for each documents. It is costly computation if we 
> represent doc by a lot of terms. Features selection can help reducing the 
> computation.
> Due to its computational efficiency and simple interpretation, information 
> gain is one of the most popular feature selection methods. It is used to 
> measure the dependence between features and labels and calculates the 
> information gain between the i-th feature and the class labels 
> (http://www.jiliang.xyz/publication/feature_selection_for_classification.pdf).
> I confirmed that by running logistics regressions on enron mail dataset (in 
> which each email is represented by top 100 terms that have highest 
> information gain) and got the accuracy by 92% and precision by 82%.
> This ticket will create two new streaming expression. Both of them use the 
> same *parallel iterative framework* as SOLR-8492.
> {code}
> featuresSelection(collection1, q="*:*",  field="tv_text", outcome="out_i", 
> positiveLabel=1, numTerms=100)
> {code}
> featuresSelection will emit top terms that have highest information gain 
> scores. It can be combined with new tlogit stream.
> {code}
> tlogit(collection1, q="*:*",
>  featuresSelection(collection1, 
>   q="*:*",  
>   field="tv_text", 
>   outcome="out_i", 
>   positiveLabel=1, 
>   numTerms=100),
>  field="tv_text",
>  outcome="out_i",
>  maxIterations=100)
> {code}
> In the iteration n, the text logistics regression will emit nth model, and 
> compute the error of (n-1)th model. Because the error will be wrong if we 
> compute the error dynamically in each iteration. 
> In each iteration tlogit will change learning rate based on error of previous 
> iteration. It will increase the learning rate by 5% if error is going down 
> and It will decrease the learning rate by 50% if error is going up.
> This will support use cases such as building models for spam detection, 
> sentiment analysis and threat detection. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9334) CloudSolrClient.collectionStateCache is unbounded

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392131#comment-15392131
 ] 

ASF subversion and git services commented on SOLR-9334:
---

Commit a30084ec820c483d6109e191f1aab7e93668420b in lucene-solr's branch 
refs/heads/branch_6x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a30084e ]

SOLR-9334: CloudSolrClient.collectionStateCache is unbounded


> CloudSolrClient.collectionStateCache is unbounded
> -
>
> Key: SOLR-9334
> URL: https://issues.apache.org/jira/browse/SOLR-9334
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.2
>
> Attachments: SOLR-9334.patch
>
>
> This could lead to memory leak if new collections are created and old 
> collections are dropped continously 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9334) CloudSolrClient.collectionStateCache is unbounded

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul resolved SOLR-9334.
--
   Resolution: Fixed
Fix Version/s: 6.2

> CloudSolrClient.collectionStateCache is unbounded
> -
>
> Key: SOLR-9334
> URL: https://issues.apache.org/jira/browse/SOLR-9334
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.2
>
> Attachments: SOLR-9334.patch
>
>
> This could lead to memory leak if new collections are created and old 
> collections are dropped continously 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7280) Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts

[Erick Erickson (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392129#comment-15392129
 ] 

Erick Erickson commented on SOLR-7280:
--

Do note that all of my (non jUnit) tests were with 4 nodes. Of course having 
those built into junit tests is A Good Thing, just FYI.

> Load cores in sorted order and tweak coreLoadThread counts to improve cluster 
> stability on restarts
> ---
>
> Key: SOLR-7280
> URL: https://issues.apache.org/jira/browse/SOLR-7280
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
> Fix For: 6.2, 5.5.3
>
> Attachments: SOLR-7280-5x.patch, SOLR-7280-5x.patch, 
> SOLR-7280-5x.patch, SOLR-7280-test.patch, SOLR-7280.patch, SOLR-7280.patch
>
>
> In SOLR-7191, Damien mentioned that by loading solr cores in a sorted order 
> and tweaking some of the coreLoadThread counts, he was able to improve the 
> stability of a cluster with thousands of collections. We should explore some 
> of these changes and fold them into Solr.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7390) Let BKDWriter use temp heap for sorting points in proportion to IndexWriter's indexing buffer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392125#comment-15392125
 ] 

ASF subversion and git services commented on LUCENE-7390:
-

Commit 686f67d9a3afff79dbe8b3a47d47e73cfad2dec5 in lucene-solr's branch 
refs/heads/branch_6x from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=686f67d ]

LUCENE-7390: improve points indexing performance by letting the codec use 
transient heap in proportion to IndexWriter's indexing buffer, by default


> Let BKDWriter use temp heap for sorting points in proportion to IndexWriter's 
> indexing buffer
> -
>
> Key: LUCENE-7390
> URL: https://issues.apache.org/jira/browse/LUCENE-7390
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7390.patch, LUCENE-7390.patch
>
>
> With Lucene's default codec, when writing dimensional points, we only give 
> {{BKDWriter}} 16 MB heap to use for sorting, regardless of how large IW's 
> indexing buffer is.  A custom codec can change this but that's a little steep.
> I've been testing indexing performance on a points-heavy dataset, 1.2 billion 
> taxi rides from http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml 
> , indexing with a 1 GB IW buffer, and the small 16 MB heap limit causes clear 
> performance problems because flushing the large segments forces {{BKDwriter}} 
> to switch to offline sorting which causes the DWPTs take too long to flush.  
> They then fall behind, and Lucene does a hard stall on incoming indexing 
> threads until they catch up.
> [~rcmuir] had a simple idea to let IW pass the allowed temp heap usage to 
> {{PointsWriter.writeField}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7390) Let BKDWriter use temp heap for sorting points in proportion to IndexWriter's indexing buffer

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-7390.

Resolution: Fixed

> Let BKDWriter use temp heap for sorting points in proportion to IndexWriter's 
> indexing buffer
> -
>
> Key: LUCENE-7390
> URL: https://issues.apache.org/jira/browse/LUCENE-7390
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7390.patch, LUCENE-7390.patch
>
>
> With Lucene's default codec, when writing dimensional points, we only give 
> {{BKDWriter}} 16 MB heap to use for sorting, regardless of how large IW's 
> indexing buffer is.  A custom codec can change this but that's a little steep.
> I've been testing indexing performance on a points-heavy dataset, 1.2 billion 
> taxi rides from http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml 
> , indexing with a 1 GB IW buffer, and the small 16 MB heap limit causes clear 
> performance problems because flushing the large segments forces {{BKDwriter}} 
> to switch to offline sorting which causes the DWPTs take too long to flush.  
> They then fall behind, and Lucene does a hard stall on incoming indexing 
> threads until they catch up.
> [~rcmuir] had a simple idea to let IW pass the allowed temp heap usage to 
> {{PointsWriter.writeField}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7394) Make MemoryIndex immutable

[Martijn van Groningen (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392121#comment-15392121
 ] 

Martijn van Groningen commented on LUCENE-7394:
---

bq. We'd still need a way to set a Similarity so that we can encode norms, I 
think?

Yes, but that should be set before adding fields as a constructor parameter.

bq. The tricky part here is going to be untangling the various shared block 
pools. We need to make sure that calling .addField() doesn't change the data 
referenced by a previously created IndexReader, which is where I got stuck last 
time I tried playing around with this idea.

I think we should avoid sharing shared block pools between IndexReader 
instances, this makes it hard (impossible?) to make MemoryIndex immutable and 
cleanup this class. To be clear about this we should from an usage / api 
perspective MemoryIndex should be renamed to MemoryIndexBuilder (that has 
constructor that accepts a Similarity) and has two methods (addField(...) and 
build(), after build has been invoked calling addField(...) will fail).

Later on we can investigate some kind of a reuse by adding an extra constructor 
to MemoryIndexBuilder that accepts an IndexReader. This would make copy of the 
previous create MemoryIndex and where possible shallow copies / clones of the 
previous created data structures.

> Make MemoryIndex immutable
> --
>
> Key: LUCENE-7394
> URL: https://issues.apache.org/jira/browse/LUCENE-7394
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Martijn van Groningen
>
> The MemoryIndex itself should just be a builder that constructs an 
> IndexReader instance. The whole notion of freezing a memory index should be 
> removed.
> While we change this we should also clean this class up. There are many 
> methods to add a field, we should just have a single method that accepts a 
> `IndexableField`.
> The `keywordTokenStream(...)` method is unused and untested and should be 
> removed and it doesn't belong with the memory index.
> The `setSimilarity(...)`, `createSearcher(...)` and `search(...)` methods 
> should be removed, because the MemoryIndex should just be responsible for 
> creating an IndexReader instance.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7390) Let BKDWriter use temp heap for sorting points in proportion to IndexWriter's indexing buffer

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392112#comment-15392112
 ] 

ASF subversion and git services commented on LUCENE-7390:
-

Commit 3a0a9fd2c25965db4bde20975d9e6df4b540588a in lucene-solr's branch 
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3a0a9fd ]

LUCENE-7390: improve points indexing performance by letting the codec use 
transient heap in proportion to IndexWriter's indexing buffer, by default


> Let BKDWriter use temp heap for sorting points in proportion to IndexWriter's 
> indexing buffer
> -
>
> Key: LUCENE-7390
> URL: https://issues.apache.org/jira/browse/LUCENE-7390
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7390.patch, LUCENE-7390.patch
>
>
> With Lucene's default codec, when writing dimensional points, we only give 
> {{BKDWriter}} 16 MB heap to use for sorting, regardless of how large IW's 
> indexing buffer is.  A custom codec can change this but that's a little steep.
> I've been testing indexing performance on a points-heavy dataset, 1.2 billion 
> taxi rides from http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml 
> , indexing with a 1 GB IW buffer, and the small 16 MB heap limit causes clear 
> performance problems because flushing the large segments forces {{BKDwriter}} 
> to switch to offline sorting which causes the DWPTs take too long to flush.  
> They then fall behind, and Lucene does a hard stall on incoming indexing 
> threads until they catch up.
> [~rcmuir] had a simple idea to let IW pass the allowed temp heap usage to 
> {{PointsWriter.writeField}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9334) CloudSolrClient.collectionStateCache is unbounded

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392076#comment-15392076
 ] 

ASF subversion and git services commented on SOLR-9334:
---

Commit 4ed68bc80e7990f4acd1b73dce3b5b8cd16d9fe5 in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4ed68bc ]

SOLR-9334: CloudSolrClient.collectionStateCache is unbounded


> CloudSolrClient.collectionStateCache is unbounded
> -
>
> Key: SOLR-9334
> URL: https://issues.apache.org/jira/browse/SOLR-9334
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Attachments: SOLR-9334.patch
>
>
> This could lead to memory leak if new collections are created and old 
> collections are dropped continously 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+127) - Build # 1268 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1268/
Java: 32bit/jdk-9-ea+127 -client -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'P val' for path 'response/params/y/p' full 
output: {   "responseHeader":{ "status":0, "QTime":0},   "response":{   
  "znodeVersion":2, "params":{   "x":{ "a":"A val", 
"b":"B val", "":{"v":0}},   "y":{ "c":"CY val modified",
 "b":"BY val", "i":20, "d":[   "val 1",   
"val 2"], "e":"EY val", "":{"v":1},  from server:  
http://127.0.0.1:45762/ox/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'P val' for path 
'response/params/y/p' full output: {
  "responseHeader":{
"status":0,
"QTime":0},
  "response":{
"znodeVersion":2,
"params":{
  "x":{
"a":"A val",
"b":"B val",
"":{"v":0}},
  "y":{
"c":"CY val modified",
"b":"BY val",
"i":20,
"d":[
  "val 1",
  "val 2"],
"e":"EY val",
"":{"v":1},  from server:  http://127.0.0.1:45762/ox/collection1
at 
__randomizedtesting.SeedInfo.seed([AE558FD5A3608B11:2601B00F0D9CE6E9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
at 
org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:215)
at 
org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[jira] [Commented] (LUCENE-7395) Query Norm and coordination factor not calculated when PerFieldSimilarityWrapper is used

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392039#comment-15392039
 ] 

David Smiley commented on LUCENE-7395:
--

Is this a duplicate of SOLR-9315?

> Query Norm and coordination factor not calculated when 
> PerFieldSimilarityWrapper is used 
> -
>
> Key: LUCENE-7395
> URL: https://issues.apache.org/jira/browse/LUCENE-7395
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.3.1, 5.4.1
>Reporter: Sascha Markus
>
> If any kind of similarity is defined and therefore the 
> SchemaSimilarityFactory is defined as global similarity the queryNorm is 
> always 1.0
> The PerFieldSimilarityWrapper delegates some of the methods to the desired 
> Similarity but misses to delegate public float queryNorm(float 
> valueForNormalization)
> Instead the IndexReader calls this method on the base class Similarity.
> The result is that all scores are much higher.
> I created a custom similarity which extends ClassicSimilarity.
> To have the calculation fixed I did a local "hotfix"  which always uses the 
> default similarity. Also wrong for some cases but fine in my scenario.
>   @Override
>   public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization); // use default 
> similarity to calculate
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7390) Let BKDWriter use temp heap for sorting points in proportion to IndexWriter's indexing buffer

[Robert Muir (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15392023#comment-15392023
 ] 

Robert Muir commented on LUCENE-7390:
-

+1

> Let BKDWriter use temp heap for sorting points in proportion to IndexWriter's 
> indexing buffer
> -
>
> Key: LUCENE-7390
> URL: https://issues.apache.org/jira/browse/LUCENE-7390
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
> Fix For: master (7.0), 6.2
>
> Attachments: LUCENE-7390.patch, LUCENE-7390.patch
>
>
> With Lucene's default codec, when writing dimensional points, we only give 
> {{BKDWriter}} 16 MB heap to use for sorting, regardless of how large IW's 
> indexing buffer is.  A custom codec can change this but that's a little steep.
> I've been testing indexing performance on a points-heavy dataset, 1.2 billion 
> taxi rides from http://www.nyc.gov/html/tlc/html/about/trip_record_data.shtml 
> , indexing with a 1 GB IW buffer, and the small 16 MB heap limit causes clear 
> performance problems because flushing the large segments forces {{BKDwriter}} 
> to switch to offline sorting which causes the DWPTs take too long to flush.  
> They then fall behind, and Lucene does a hard stall on incoming indexing 
> threads until they catch up.
> [~rcmuir] had a simple idea to let IW pass the allowed temp heap usage to 
> {{PointsWriter.writeField}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9324) Support Secure Impersonation / Proxy User for solr authentication

[Gregory Chanan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan reassigned SOLR-9324:


Assignee: Gregory Chanan

> Support Secure Impersonation / Proxy User for solr authentication
> -
>
> Key: SOLR-9324
> URL: https://issues.apache.org/jira/browse/SOLR-9324
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>
> Solr should support Proxy User / Secure Impersonation for authentication, as 
> supported by hadoop 
> (http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Superusers.html)
>  and supported by the hadoop AuthenticationFilter (which we use for the 
> KerberosPlugin).
> There are a number of use cases, but a common one is this:
> There is a front end for searches (say, Hue http://gethue.com/) that supports 
> its own login mechanisms.  If the cluster uses kerberos for authentication, 
> hue must have kerberos credentials for each user, which is a pain to manage.  
> Instead, hue can be allowed to impersonate known users from known machines so 
> it only needs its own kerberos credentials.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Closed] (SOLR-752) Allow better Field Compression options

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley closed SOLR-752.
-
Resolution: Won't Fix

Basically already addressed by SOLR-8477.  Good enough.

> Allow better Field Compression options
> --
>
> Key: SOLR-752
> URL: https://issues.apache.org/jira/browse/SOLR-752
> Project: Solr
>  Issue Type: Improvement
>Reporter: Grant Ingersoll
>Priority: Minor
> Attachments: compressed_field.patch, compressedtextfield.patch
>
>
> See http://lucene.markmail.org/message/sd4mgwud6caevb35?q=compression
> It would be good if Solr handled field compression outside of Lucene's 
> Field.COMPRESS capabilities, since those capabilities are less than ideal 
> when it comes to control over compression.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7391) MemoryIndexReader.fields() performance regression

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved LUCENE-7391.
--
   Resolution: Fixed
Fix Version/s: 6.2

Thanks for the patch Steve!  Committed and closing now.

> MemoryIndexReader.fields() performance regression
> -
>
> Key: LUCENE-7391
> URL: https://issues.apache.org/jira/browse/LUCENE-7391
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.0
>Reporter: Steve Mason
>Assignee: David Smiley
> Fix For: 6.2
>
> Attachments: LUCENE-7391-test.patch, LUCENE-7391.patch, 
> LUCENE-7391.patch
>
>
> While upgrading our codebase from Lucene 4 to Lucene 6 we found a significant 
> performance regression - a 5x slowdown
> On profiling the code, the method MemoryIndexReader.fields() shows up as one 
> of the hottest methods
> Looking at the method, it just creates a copy of the inner {{fields}} Map 
> before passing it to {{MemoryFields}}. It does this so that it can filter out 
> fields with {{numTokens <= 0}}.
> The simplest "fix" would be to just remove the copying of the map completely, 
> and pass {{fields}} directly to {{MemoryFields}}.  It's simple and removes 
> any slowdown caused by this method.  It does potentially change behaviour 
> though, but none of the unit tests seem to test that behaviour so I wonder 
> whether it's necessary (I looked at the original ticket LUCENE-7091 that 
> introduced this code, I can't find much in way of an explanation). I'm going 
> to attach a patch to this effect anyway and we can take things from there



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_92) - Build # 17362 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17362/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandler.doTestStressReplication

Error Message:
timed out waiting for collection1 startAt time to exceed: Mon Jul 25 20:35:32 
ICT 2016

Stack Trace:
java.lang.AssertionError: timed out waiting for collection1 startAt time to 
exceed: Mon Jul 25 20:35:32 ICT 2016
at 
__randomizedtesting.SeedInfo.seed([9BA6A27AEE0BBE14:400DA2BCEB23D7A7]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.handler.TestReplicationHandler.watchCoreStartAt(TestReplicationHandler.java:1506)
at 
org.apache.solr.handler.TestReplicationHandler.doTestStressReplication(TestReplicationHandler.java:856)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11548 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   

[jira] [Updated] (LUCENE-7391) MemoryIndexReader.fields() performance regression

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-7391:
-
Affects Version/s: 6.0

> MemoryIndexReader.fields() performance regression
> -
>
> Key: LUCENE-7391
> URL: https://issues.apache.org/jira/browse/LUCENE-7391
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.0
>Reporter: Steve Mason
>Assignee: David Smiley
> Attachments: LUCENE-7391-test.patch, LUCENE-7391.patch, 
> LUCENE-7391.patch
>
>
> While upgrading our codebase from Lucene 4 to Lucene 6 we found a significant 
> performance regression - a 5x slowdown
> On profiling the code, the method MemoryIndexReader.fields() shows up as one 
> of the hottest methods
> Looking at the method, it just creates a copy of the inner {{fields}} Map 
> before passing it to {{MemoryFields}}. It does this so that it can filter out 
> fields with {{numTokens <= 0}}.
> The simplest "fix" would be to just remove the copying of the map completely, 
> and pass {{fields}} directly to {{MemoryFields}}.  It's simple and removes 
> any slowdown caused by this method.  It does potentially change behaviour 
> though, but none of the unit tests seem to test that behaviour so I wonder 
> whether it's necessary (I looked at the original ticket LUCENE-7091 that 
> introduced this code, I can't find much in way of an explanation). I'm going 
> to attach a patch to this effect anyway and we can take things from there



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7391) MemoryIndexReader.fields() performance regression

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391936#comment-15391936
 ] 

ASF subversion and git services commented on LUCENE-7391:
-

Commit d85a9718564abf3dd33f66d057f8cfc835a56715 in lucene-solr's branch 
refs/heads/branch_6x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d85a971 ]

LUCENE-7391: Fix MemoryIndex fields() perf regression
(cherry picked from commit dcc9a4b)


> MemoryIndexReader.fields() performance regression
> -
>
> Key: LUCENE-7391
> URL: https://issues.apache.org/jira/browse/LUCENE-7391
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 6.0
>Reporter: Steve Mason
>Assignee: David Smiley
> Attachments: LUCENE-7391-test.patch, LUCENE-7391.patch, 
> LUCENE-7391.patch
>
>
> While upgrading our codebase from Lucene 4 to Lucene 6 we found a significant 
> performance regression - a 5x slowdown
> On profiling the code, the method MemoryIndexReader.fields() shows up as one 
> of the hottest methods
> Looking at the method, it just creates a copy of the inner {{fields}} Map 
> before passing it to {{MemoryFields}}. It does this so that it can filter out 
> fields with {{numTokens <= 0}}.
> The simplest "fix" would be to just remove the copying of the map completely, 
> and pass {{fields}} directly to {{MemoryFields}}.  It's simple and removes 
> any slowdown caused by this method.  It does potentially change behaviour 
> though, but none of the unit tests seem to test that behaviour so I wonder 
> whether it's necessary (I looked at the original ticket LUCENE-7091 that 
> introduced this code, I can't find much in way of an explanation). I'm going 
> to attach a patch to this effect anyway and we can take things from there



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7391) MemoryIndexReader.fields() performance regression

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391932#comment-15391932
 ] 

ASF subversion and git services commented on LUCENE-7391:
-

Commit dcc9a4b9a411a0436e5cb21e7d6251691640f3db in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dcc9a4b ]

LUCENE-7391: Fix MemoryIndex fields() perf regression


> MemoryIndexReader.fields() performance regression
> -
>
> Key: LUCENE-7391
> URL: https://issues.apache.org/jira/browse/LUCENE-7391
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Steve Mason
>Assignee: David Smiley
> Attachments: LUCENE-7391-test.patch, LUCENE-7391.patch, 
> LUCENE-7391.patch
>
>
> While upgrading our codebase from Lucene 4 to Lucene 6 we found a significant 
> performance regression - a 5x slowdown
> On profiling the code, the method MemoryIndexReader.fields() shows up as one 
> of the hottest methods
> Looking at the method, it just creates a copy of the inner {{fields}} Map 
> before passing it to {{MemoryFields}}. It does this so that it can filter out 
> fields with {{numTokens <= 0}}.
> The simplest "fix" would be to just remove the copying of the map completely, 
> and pass {{fields}} directly to {{MemoryFields}}.  It's simple and removes 
> any slowdown caused by this method.  It does potentially change behaviour 
> though, but none of the unit tests seem to test that behaviour so I wonder 
> whether it's necessary (I looked at the original ticket LUCENE-7091 that 
> introduced this code, I can't find much in way of an explanation). I'm going 
> to attach a patch to this effect anyway and we can take things from there



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9336) ReRankQuery.(hashCode|equalsTo) to consider length

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391898#comment-15391898
 ] 

Joel Bernstein commented on SOLR-9336:
--

This going to take some careful testing. Let's review how the QueryResultCache 
is handling length and see what would happen if we start including the length 
in the hashCode|equals.

> ReRankQuery.(hashCode|equalsTo) to consider length
> --
>
> Key: SOLR-9336
> URL: https://issues.apache.org/jira/browse/SOLR-9336
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9336.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7395) Query Norm and coordination factor not calculated when PerFieldSimilarityWrapper is used

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-7395:
--
Summary: Query Norm and coordination factor not calculated when 
PerFieldSimilarityWrapper is used   (was: Query Norm not calculated when 
SchemaSimilarityFactory is used )

> Query Norm and coordination factor not calculated when 
> PerFieldSimilarityWrapper is used 
> -
>
> Key: LUCENE-7395
> URL: https://issues.apache.org/jira/browse/LUCENE-7395
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.3.1, 5.4.1
>Reporter: Sascha Markus
>
> If any kind of similarity is defined and therefore the 
> SchemaSimilarityFactory is defined as global similarity the queryNorm is 
> always 1.0
> The PerFieldSimilarityWrapper delegates some of the methods to the desired 
> Similarity but misses to delegate public float queryNorm(float 
> valueForNormalization)
> Instead the IndexReader calls this method on the base class Similarity.
> The result is that all scores are much higher.
> I created a custom similarity which extends ClassicSimilarity.
> To have the calculation fixed I did a local "hotfix"  which always uses the 
> default similarity. Also wrong for some cases but fine in my scenario.
>   @Override
>   public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization); // use default 
> similarity to calculate
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7395) Query Norm not calculated when SchemaSimilarityFactory is used

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15391854#comment-15391854
 ] 

Uwe Schindler commented on LUCENE-7395:
---

This affects also Lucene, so I moved it.

> Query Norm not calculated when SchemaSimilarityFactory is used 
> ---
>
> Key: LUCENE-7395
> URL: https://issues.apache.org/jira/browse/LUCENE-7395
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.3.1, 5.4.1
>Reporter: Sascha Markus
>
> If any kind of similarity is defined and therefore the 
> SchemaSimilarityFactory is defined as global similarity the queryNorm is 
> always 1.0
> The PerFieldSimilarityWrapper delegates some of the methods to the desired 
> Similarity but misses to delegate public float queryNorm(float 
> valueForNormalization)
> Instead the IndexReader calls this method on the base class Similarity.
> The result is that all scores are much higher.
> I created a custom similarity which extends ClassicSimilarity.
> To have the calculation fixed I did a local "hotfix"  which always uses the 
> default similarity. Also wrong for some cases but fine in my scenario.
>   @Override
>   public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization); // use default 
> similarity to calculate
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Moved] (LUCENE-7395) Query Norm not calculated when SchemaSimilarityFactory is used

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler moved SOLR-8739 to LUCENE-7395:
-

Affects Version/s: (was: 5.4.1)
   (was: 5.3.1)
   5.3.1
   5.4.1
  Component/s: (was: search)
Lucene Fields: New
  Key: LUCENE-7395  (was: SOLR-8739)
  Project: Lucene - Core  (was: Solr)

> Query Norm not calculated when SchemaSimilarityFactory is used 
> ---
>
> Key: LUCENE-7395
> URL: https://issues.apache.org/jira/browse/LUCENE-7395
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 5.4.1, 5.3.1
>Reporter: Sascha Markus
>
> If any kind of similarity is defined and therefore the 
> SchemaSimilarityFactory is defined as global similarity the queryNorm is 
> always 1.0
> The PerFieldSimilarityWrapper delegates some of the methods to the desired 
> Similarity but misses to delegate public float queryNorm(float 
> valueForNormalization)
> Instead the IndexReader calls this method on the base class Similarity.
> The result is that all scores are much higher.
> I created a custom similarity which extends ClassicSimilarity.
> To have the calculation fixed I did a local "hotfix"  which always uses the 
> default similarity. Also wrong for some cases but fine in my scenario.
>   @Override
>   public float queryNorm(float valueForNormalization) {
> return get("").queryNorm(valueForNormalization); // use default 
> similarity to calculate
>   }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Linux (32bit/jdk1.8.0_92) - Build # 1267 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1267/
Java: 32bit/jdk1.8.0_92 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[http://127.0.0.1:41100/forceleader_test_collection_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this 
request:[http://127.0.0.1:41100/forceleader_test_collection_shard1_replica2]
at 
__randomizedtesting.SeedInfo.seed([EB793AC4691430F9:DEE0E045096C998]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:753)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
at 
org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
at 
org.apache.solr.cloud.ForceLeaderTest.assertSendDocFails(ForceLeaderTest.java:315)
at 
org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_92) - Build # 341 - Still Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/341/
Java: 32bit/jdk1.8.0_92 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLocalFSCloudBackupRestore.test

Error Message:
Error from server at https://127.0.0.1:57070/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-6.x-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_8EFBEB4D39DC41DF-001/tempDir-002/mytestbackup/

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:57070/solr: The backup directory already 
exists: 
file:///C:/Users/jenkins/workspace/Lucene-Solr-6.x-Windows/solr/build/solr-core/test/J1/temp/solr.cloud.TestLocalFSCloudBackupRestore_8EFBEB4D39DC41DF-001/tempDir-002/mytestbackup/
at 
__randomizedtesting.SeedInfo.seed([8EFBEB4D39DC41DF:6AFD49797202C27]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1270)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:206)
at 
org.apache.solr.cloud.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+127) - Build # 17361 - Unstable!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17361/
Java: 32bit/jdk-9-ea+127 -client -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.update.HardAutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([70A68C3368A4C278:CA74E34BEB8A2C6D]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:782)
at 
org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:99)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:533)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(java.base@9-ea/Thread.java:843)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound=1]
xml response was: 

00


request was:q=id:529=standard=0=20=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:775)