[jira] [Commented] (LUCENE-6276) Add matchCost() api to TwoPhaseDocIdSetIterator

[Paul Elschot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002831#comment-15002831
 ] 

Paul Elschot commented on LUCENE-6276:
--

This failure disappeared after adding asTwoPhaseIterator() to 
ScoringWrapperSpans. I'll post a new patch later.

> Add matchCost() api to TwoPhaseDocIdSetIterator
> ---
>
> Key: LUCENE-6276
> URL: https://issues.apache.org/jira/browse/LUCENE-6276
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
> Attachments: LUCENE-6276-ExactPhraseOnly.patch, 
> LUCENE-6276-NoSpans.patch, LUCENE-6276-NoSpans2.patch, LUCENE-6276.patch, 
> LUCENE-6276.patch, LUCENE-6276.patch, LUCENE-6276.patch, LUCENE-6276.patch, 
> LUCENE-6276.patch, LUCENE-6276.patch
>
>
> We could add a method like TwoPhaseDISI.matchCost() defined as something like 
> estimate of nanoseconds or similar. 
> ConjunctionScorer could use this method to sort its 'twoPhaseIterators' array 
> so that cheaper ones are called first. Today it has no idea if one scorer is 
> a simple phrase scorer on a short field vs another that might do some geo 
> calculation or more expensive stuff.
> PhraseScorers could implement this based on index statistics (e.g. 
> totalTermFreq/maxDoc)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4905) Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded collection that has a replica on all nodes

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002865#comment-15002865
 ] 

Mikhail Khludnev commented on SOLR-4905:


[~p...@search-solutions.net] would you mind to raise a separate jira for this? 

> Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded 
> collection that has a replica on all nodes
> --
>
> Key: SOLR-4905
> URL: https://issues.apache.org/jira/browse/SOLR-4905
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Philip K. Warren
>Assignee: Timothy Potter
> Fix For: 5.1, Trunk
>
> Attachments: SOLR-4905.patch, SOLR-4905.patch, patch.txt
>
>
> Using a non-SolrCloud setup, it is possible to perform cross core joins 
> (http://wiki.apache.org/solr/Join). When testing with SolrCloud, however, 
> neither the collection name, alias name (we have created aliases to SolrCloud 
> collections), or the automatically generated core name (i.e. 
> _shard1_replica1) work as the fromIndex parameter for a 
> cross-core join.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002905#comment-15002905
 ] 

Mark Miller commented on SOLR-8075:
---

I'll add a little comment above checkLIR: we must check LIR before registering 
as leader

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3191) field exclusion from fl

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003064#comment-15003064
 ] 

Yonik Seeley commented on SOLR-3191:


I think new features should be free to ignore funky field name... no back 
compat requirements.
Changing existing APIs... I don't know.  I guess it depends on how many cases 
there are out there of funky names, how many of them will upgrade, and what the 
cost is to us to support.

bq. That's a good question about field aliases - a number literal, for example, 
wouldn't be a valid Solr field name by these rules. So I think field aliases 
are fair game to be funky.

My guess is that Scott was referring to just the alias name... as in 
 : , where alias_value can be a function query, 
literal, or transformer, as well as another field.



> field exclusion from fl
> ---
>
> Key: SOLR-3191
> URL: https://issues.apache.org/jira/browse/SOLR-3191
> Project: Solr
>  Issue Type: Improvement
>Reporter: Luca Cavanna
>Priority: Minor
> Attachments: SOLR-3191.patch, SOLR-3191.patch, SOLR-3191.patch, 
> SOLR-3191.patch
>
>
> I think it would be useful to add a way to exclude field from the Solr 
> response. If I have for example 100 stored fields and I want to return all of 
> them but one, it would be handy to list just the field I want to exclude 
> instead of the 99 fields for inclusion through fl.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002875#comment-15002875
 ] 

Mike Drob commented on SOLR-8075:
-

Update looks good to me. I know you expanded the comments in {{checkLIR}}, but 
can you add a one-line note before the method is called with something to the 
effect of "needs to happen synchronously before setLeader(true) to avoid 
rejection?"

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-8284) JSON Facet API sort:index causes NPE

[Yonik Seeley (JIRA)]

Yonik Seeley created SOLR-8284:
--

 Summary: JSON Facet API sort:index causes NPE
 Key: SOLR-8284
 URL: https://issues.apache.org/jira/browse/SOLR-8284
 Project: Solr
  Issue Type: Bug
  Components: Facet Module
Reporter: Yonik Seeley
Priority: Minor


sort:index was meant to be a shortcut for sort:"index asc", but this currently 
causes a NPE. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002892#comment-15002892
 ] 

Mark Miller commented on SOLR-8075:
---

It's not that it needs to happen before setLeader. The pertinent move it 
putting it above the super. call.

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002901#comment-15002901
 ] 

Mark Miller commented on SOLR-8075:
---

Also, the logic here is changed. We know do LIR prevention checks in checkLIR 
as well. I think it clearly has to come before we declare being the leader. 
Previously, it was not a checkLIR call - it was logic to just possible turn LIR 
off for that replica.

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3191) field exclusion from fl

[Erik Hatcher (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003020#comment-15003020
 ] 

Erik Hatcher commented on SOLR-3191:


That's a good question about field aliases - a number literal, for example, 
wouldn't be a valid Solr field name by these rules.   So I think field aliases 
are fair game to be funky.

I don't necessarily propose we tackle all of these field name rules or maybe 
none of them at all, as part of the main gist of this issue though.  Maybe we 
just get the fl field exclusion aspect of this committed, and deal with field 
name rules and parsing in other contexts to a separate effort?

> field exclusion from fl
> ---
>
> Key: SOLR-3191
> URL: https://issues.apache.org/jira/browse/SOLR-3191
> Project: Solr
>  Issue Type: Improvement
>Reporter: Luca Cavanna
>Priority: Minor
> Attachments: SOLR-3191.patch, SOLR-3191.patch, SOLR-3191.patch, 
> SOLR-3191.patch
>
>
> I think it would be useful to add a way to exclude field from the Solr 
> response. If I have for example 100 stored fields and I want to return all of 
> them but one, it would be handy to list just the field I want to exclude 
> instead of the 99 fields for inclusion through fl.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002901#comment-15002901
 ] 

Mark Miller edited comment on SOLR-8075 at 11/12/15 8:50 PM:
-

Also, the logic here is changed. We now do LIR prevention checks in checkLIR as 
well. I think it clearly has to come before we declare being the leader. 
Previously, it was not a checkLIR call - it was logic to just possibly turn LIR 
off for that replica.


was (Author: markrmil...@gmail.com):
Also, the logic here is changed. We know do LIR prevention checks in checkLIR 
as well. I think it clearly has to come before we declare being the leader. 
Previously, it was not a checkLIR call - it was logic to just possible turn LIR 
off for that replica.

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6894) Improve DISI.cost() by assuming independence for match probabilities

[Paul Elschot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003039#comment-15003039
 ] 

Paul Elschot commented on LUCENE-6894:
--

Another reason why I started this is that the result of cost() is also used as 
weights for matchCost() at LUCENE-6276, and I'd prefer those weights to be as 
accurate as reasonably possible.

I think we can keep this alternative (assuming independence for conjunctions 
and disjunctions) as a possible alternative until the current implementation 
gives a bad result.

For the proximity queries (Phrases, Spans) this reduces the conjunction cost() 
using the allowed slop.
Would it be worthwhile to open a separate issue for that?


> Improve DISI.cost() by assuming independence for match probabilities
> 
>
> Key: LUCENE-6894
> URL: https://issues.apache.org/jira/browse/LUCENE-6894
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Paul Elschot
>Priority: Minor
> Attachments: LUCENE-6894.patch
>
>
> The DocIdSetIterator.cost() method returns an estimation of the number of 
> matching docs. Currently conjunctions use the minimum cost, and disjunctions 
> use the sum of the costs, and both are too high.
> The probability of a match is estimated by dividing available cost() by the 
> number of docs in a segment.
> The conjunction probability is then the product of the inputs, and the 
> disjunction probability follows from De Morgan's rule:
> "not (A and B)" is the same as "(not A) or (not B)"
> with the probability for "not" computed as 1 minus the input probability.
> The independence that is assumed is normally not there. However, the cost() 
> results are only used to order the input DISIs/Scorers for optimization, and 
> for that I expect this assumption to work nicely.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-3191) field exclusion from fl

[Scott Stults (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002827#comment-15002827
 ] 

Scott Stults commented on SOLR-3191:


I'd definitely be up for trimming out funky field name checks in the tests. 
Applying field name restrictions just in fl might lead to the situation where 
data is searchable but not returnable, so should the enforcement portion go to 
a more central location? Also, should field aliases be restricted in exactly 
the same way as real field names?

> field exclusion from fl
> ---
>
> Key: SOLR-3191
> URL: https://issues.apache.org/jira/browse/SOLR-3191
> Project: Solr
>  Issue Type: Improvement
>Reporter: Luca Cavanna
>Priority: Minor
> Attachments: SOLR-3191.patch, SOLR-3191.patch, SOLR-3191.patch, 
> SOLR-3191.patch
>
>
> I think it would be useful to add a way to exclude field from the Solr 
> response. If I have for example 100 stored fields and I want to return all of 
> them but one, it would be handy to list just the field I want to exclude 
> instead of the 99 fields for inclusion through fl.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-trunk-Java8 - Build # 614 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-Java8/614/

1 tests failed.
FAILED:  
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior

Error Message:
Illegal state, was: down expected:active clusterState:live 
nodes:[]collections:{c1=DocCollection(c1)={   "shards":{"shard1":{   
"parent":null,   "range":null,   "state":"active",   
"replicas":{"core_node1":{   "base_url":"http://127.0.0.1/solr;,
   "node_name":"node1",   "core":"core1",   "roles":"", 
  "state":"down",   "router":{"name":"implicit"}}, 
test=LazyCollectionRef(test)}

Stack Trace:
java.lang.AssertionError: Illegal state, was: down expected:active 
clusterState:live nodes:[]collections:{c1=DocCollection(c1)={
  "shards":{"shard1":{
  "parent":null,
  "range":null,
  "state":"active",
  "replicas":{"core_node1":{
  "base_url":"http://127.0.0.1/solr;,
  "node_name":"node1",
  "core":"core1",
  "roles":"",
  "state":"down",
  "router":{"name":"implicit"}}, test=LazyCollectionRef(test)}
at 
__randomizedtesting.SeedInfo.seed([F2841DF9912C36C4:9A9A1E1573BC6C8A]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.verifyReplicaStatus(AbstractDistribZkTestBase.java:229)
at 
org.apache.solr.cloud.OverseerTest.testExternalClusterStateChangeBehavior(OverseerTest.java:1247)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 

[jira] [Created] (SOLR-8285) Insure that /export handles documents that have no value for the field gracefully.

[Erick Erickson (JIRA)]

Erick Erickson created SOLR-8285:


 Summary: Insure that /export handles documents that have no value 
for the field gracefully.
 Key: SOLR-8285
 URL: https://issues.apache.org/jira/browse/SOLR-8285
 Project: Solr
  Issue Type: Bug
Reporter: Erick Erickson






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8275) Unclear error message during recovery

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002907#comment-15002907
 ] 

Mark Miller commented on SOLR-8275:
---

bq. then you're on your own to figure out the failing condition.

Yeah, but that is easy enough to deduce.

bq. What error code do you think is more appropriate? SERVER_ERROR?

Perhaps. It's closer than malformed request.

> Unclear error message during recovery
> -
>
> Key: SOLR-8275
> URL: https://issues.apache.org/jira/browse/SOLR-8275
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 4.10.3
>Reporter: Mike Drob
> Attachments: SOLR-8275.patch, SOLR-8275.patch
>
>
> A SolrCloud install got into a bad state (mostly around LeaderElection, I 
> think) and during recovery one of the nodes was giving me this message:
> {noformat}
> 2015-11-09 13:00:56,158 ERROR org.apache.solr.cloud.RecoveryStrategy: Error 
> while trying to recover. 
> core=c1_shard1_replica4:java.util.concurrent.ExecutionException: 
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: I was 
> asked to wait on state recovering for shard1 in c1 on node2:8983_solr but I 
> still do not see the requested state. I see state: recovering live:true 
> leader from ZK: http://node1:8983/solr/c1_shard1_replica2/
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:599)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:370)
>   at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:236)
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: I was 
> asked to wait on state recovering for shard1 in c1 on node2:8983_solr but I 
> still do not see the requested state. I see state: recovering live:true 
> leader from ZK: http://node1:8983/solr/c1_shard1_replica2/
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:621)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:292)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:288)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The crux of this message: "I was asked to wait on state recovering for shard1 
> in c1 on node2:8983_solr but I still do not see the requested state. I see 
> state: recovering" seems contradictory. At a minimum, we should improve this 
> error, but there might also be some erroneous logic going on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8273) deprecate implicitly uninverted fields, force people to either use docValues, or be explicit that they want query time uninversion

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003028#comment-15003028
 ] 

Hoss Man commented on SOLR-8273:


yeah, there's no reason we can't do both...

* in IndexSchema/FieldType initialization:
** use schema.version=X, log an INFO for any uninverted field

> deprecate implicitly uninverted fields, force people to either use docValues, 
> or be explicit that they want query time uninversion
> --
>
> Key: SOLR-8273
> URL: https://issues.apache.org/jira/browse/SOLR-8273
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>
> once upon a time, there was nothing we could do to *stop* people from using 
> the FieldCache - even if they didn't realize they were using it.
> Then DocValues was added - and now people have a choice: they can set 
> {{docValues=true}} on a field/fieldtype and know that when they do 
> functions/sorting/faceting on that field, it won't require a big hunk of ram 
> and a big stall everytime a reader was reopened.  But it's easy to overlook 
> when clients might be doing something that required the FieldCache w/o 
> realizing it -- and there is no way to stop them, because Solr automatically 
> uses UninvertingReader under the covers and automatically allows every field 
> to be uninverted in this way.
> we should change that.
> 
> Straw man proposal...
> * introduce a new boolean fieldType/field property {{uninvertable}}
> * all existing FieldType classes should default to {{uninvertable==false}}
> * a field or fieldType that contains {{indexed="false" uninvertable="true"}} 
> should be an error.
> * the Schema {{version}} value should be incremented, such that any Schema 
> with an older version is treated as if every field with {{docValues==false}} 
> has an implict {{uninvertable="true"}} on it.
> * the Map passed to UninvertedReader should now only list items that have an 
> effective value of {{uninvertable==true}}
> * sample schemas should be updated to use docValues on any field where the 
> examples using those schemas suggest using those fields in that way (ie: 
> sorting, faceting, etc...)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8275) Unclear error message during recovery

[Mike Drob (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Drob updated SOLR-8275:

Attachment: SOLR-8275.patch

New patch attached.

> Unclear error message during recovery
> -
>
> Key: SOLR-8275
> URL: https://issues.apache.org/jira/browse/SOLR-8275
> Project: Solr
>  Issue Type: Bug
>  Components: SolrCloud
>Affects Versions: 4.10.3
>Reporter: Mike Drob
> Attachments: SOLR-8275.patch, SOLR-8275.patch
>
>
> A SolrCloud install got into a bad state (mostly around LeaderElection, I 
> think) and during recovery one of the nodes was giving me this message:
> {noformat}
> 2015-11-09 13:00:56,158 ERROR org.apache.solr.cloud.RecoveryStrategy: Error 
> while trying to recover. 
> core=c1_shard1_replica4:java.util.concurrent.ExecutionException: 
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: I was 
> asked to wait on state recovering for shard1 in c1 on node2:8983_solr but I 
> still do not see the requested state. I see state: recovering live:true 
> leader from ZK: http://node1:8983/solr/c1_shard1_replica2/
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.sendPrepRecoveryCmd(RecoveryStrategy.java:599)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:370)
>   at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:236)
> Caused by: 
> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: I was 
> asked to wait on state recovering for shard1 in c1 on node2:8983_solr but I 
> still do not see the requested state. I see state: recovering live:true 
> leader from ZK: http://node1:8983/solr/c1_shard1_replica2/
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:621)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:292)
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrServer$1.call(HttpSolrServer.java:288)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The crux of this message: "I was asked to wait on state recovering for shard1 
> in c1 on node2:8983_solr but I still do not see the requested state. I see 
> state: recovering" seems contradictory. At a minimum, we should improve this 
> error, but there might also be some erroneous logic going on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Welcome Areek Zillur to the Lucene / Solr PMC

[Dawid Weiss]

Congratulations and welcome, Areek!
Dawid

On Thu, Nov 12, 2015 at 8:46 AM, Shalin Shekhar Mangar
 wrote:
> Welcome Areek!
>
> On Thu, Nov 12, 2015 at 2:18 AM, Simon Willnauer
>  wrote:
>> I'm pleased to announce that Areek has accepted the PMC's invitation to
>> join.
>>
>> Welcome Areek!
>>
>> Simon
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7912) Add support for boost and exclude the queried document id in MoreLikeThis QParser

[Jens Wille (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001869#comment-15001869
 ] 

Jens Wille commented on SOLR-7912:
--

Uwe, can you expand on your suggestion? Was {{realMLTQuery}} the one you had in 
mind? Does it only apply to {{CloudMLTQParser}} and {{SimpleMLTQParser}} or 
also to {{MoreLikeThisHandler}}? If the former, why? And how should this change 
be reflected in the tests?

> Add support for boost and exclude the queried document id in MoreLikeThis 
> QParser
> -
>
> Key: SOLR-7912
> URL: https://issues.apache.org/jira/browse/SOLR-7912
> Project: Solr
>  Issue Type: Improvement
>Reporter: Anshum Gupta
>Assignee: Anshum Gupta
> Attachments: SOLR-7912.patch, SOLR-7912.patch, SOLR-7912.patch, 
> SOLR-7912.patch
>
>
> Continuing from SOLR-7639. We need to support boost, and also exclude input 
> document from returned doc list.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Solaris (64bit/jdk1.8.0) - Build # 183 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Solaris/183/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.update.AutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([67D76409C926EF28:DD050B714A08013D]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:766)
at 
org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:321)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound=1]
xml response was: 

00


request was:q=id:529=standard=0=20=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:759)
... 40 more




Build Log:
[...truncated 9602 lines...]
   [junit4] Suite: 

Re: Sharing a class across cores

[Erik Hatcher]

Or in a separate Solr core/collection.  :)

> On Nov 11, 2015, at 19:05, Walter Underwood  wrote:
> 
> Depending on how fast the access needs to be, you could put that big map in 
> memcache.
> 
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
> 
> 
>> On Nov 11, 2015, at 4:04 PM, Gus Heck  wrote:
>> 
>> P.S. I posted the original message concurrently with the chat session's 
>> occurance I beleive, certainly before I had read it, so no I haven't 
>> actually tried what you suggest yet.
>> 
>>> On Wed, Nov 11, 2015 at 7:02 PM, Gus Heck  wrote:
>>> Yes asked by a colleague :). The chat session is now in our jira ticket :). 
>>> 
>>> However, my take on it is that this seems like a pretty broad brush to 
>>> paint with to move *all* our classes up and out of the normal core loading 
>>> process. I assume there are good reasons for segregating this stuff into 
>>> separate class loaders to begin with. It would also be fairly burdensom to 
>>> make a separate jar file to break out this one component...
>>> 
>>> I really just want a way to stash the map in a place where other cores can 
>>> see it (and thus I can appropriately synchronize things so that the loading 
>>> only happens once). I'm asking because it seems like surely this must be a 
>>> solved problem... if not, it might be easiest to just solve it by adding 
>>> some sort of shared resources facility to CoreContainer?
>>> 
>>> -Gus
>>> 
 On Wed, Nov 11, 2015 at 6:54 PM, Shawn Heisey  wrote:
 On 11/11/2015 4:11 PM, Gus Heck wrote:
 > I have a case where a component loads up a large CSV file (2.5 million
 > lines) to build a map. This worked ok in a case where we had a single
 > core, but it isn't working so well with 40 cores because each core loads
 > a new copy of the component in a new classloader and I get 40 new
 > versions of the same class each holding it's own private static final
 > map (one for each core). Each line is small, but a billion of anything
 > gets kinda heavy. Is this the intended class loading behavior?
 >
 > Is there some where that one can cause a class to be loaded in a parent
 > classloader above the core so that it's loaded just once? I want to load
 > it in some way that leverages standard solr resource loading, so that
 > I'm not hard coding or setting sysprops just to be able to find it.
 >
 > This is in a copy of trunk from about a month ago... so 6.x stuff is
 > mostly available.
 
 This sounds like a question that I just recently answered on IRC.
 
 If you remove all  elements from your solrconfig.xml files and
 place all extra jars for Solr into ${solr.solr.home}/lib ... Solr will
 load those jars before any cores are created and they will be available
 to all cores.
 
 There is a minor bug with this that will be fixed in Solr 5.4.0.  It is
 unlikely that this will affect third-party components, but be aware that
 until 5.4, jars in that lib directory will be loaded twice by older 5.x
 versions.
 
 https://issues.apache.org/jira/browse/SOLR-6188
 
 Thanks,
 Shawn
 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org
>>> 
>>> 
>>> 
>>> -- 
>>> http://www.the111shift.com
>> 
>> 
>> 
>> -- 
>> http://www.the111shift.com
> 

[jira] [Updated] (LUCENE-6874) WhitespaceTokenizer should tokenize on NBSP

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6874:
--
Attachment: unicode-ws-tokenizer.patch

Cleanup.

In trunk you now can create the UnicodeWhitespaceTokenizer with the following 
code:

{code:java}
CharTokenizer.fromSeparatorCharPredicate(UnicodeProps.WHITESPACE::get);
{code}

So the extra class is theoretically obsolete, but provided for convenience (at 
least required for 5.x).

> WhitespaceTokenizer should tokenize on NBSP
> ---
>
> Key: LUCENE-6874
> URL: https://issues.apache.org/jira/browse/LUCENE-6874
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: David Smiley
>Priority: Minor
> Attachments: LUCENE-6874-jflex.patch, LUCENE-6874.patch, 
> LUCENE_6874_jflex.patch, icu-datasucker.patch, unicode-ws-tokenizer.patch, 
> unicode-ws-tokenizer.patch, unicode-ws-tokenizer.patch
>
>
> WhitespaceTokenizer uses [Character.isWhitespace 
> |http://docs.oracle.com/javase/8/docs/api/java/lang/Character.html#isWhitespace-int-]
>  to decide what is whitespace.  Here's a pertinent excerpt:
> bq. It is a Unicode space character (SPACE_SEPARATOR, LINE_SEPARATOR, or 
> PARAGRAPH_SEPARATOR) but is not also a non-breaking space ('\u00A0', 
> '\u2007', '\u202F')
> Perhaps Character.isWhitespace should have been called 
> isLineBreakableWhitespace?
> I think WhitespaceTokenizer should tokenize on this.  I am aware it's easy to 
> work around but why leave this trap in by default?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8278) ConfigSetService should use NIO2

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001909#comment-15001909
 ] 

ASF subversion and git services commented on SOLR-8278:
---

Commit 1713996 from [~romseygeek] in branch 'dev/trunk'
[ https://svn.apache.org/r1713996 ]

SOLR-8278: Use NIO2 APIs in ConfigSetService

> ConfigSetService should use NIO2 
> -
>
> Key: SOLR-8278
> URL: https://issues.apache.org/jira/browse/SOLR-8278
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-8278.patch
>
>
> Similar to SOLR-8260, using java.nio.file.Path instead of String and File 
> makes error handling and reporting a lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6874) WhitespaceTokenizer should tokenize on NBSP

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6874:
--
Attachment: unicode-ws-tokenizer.patch

New patch:
- Fix formatting (use hex) and line breaks in generator
- Move the bounds check to the returned Bitset impl
- Move UnicodeData to oal.analysis.util package

No tests until now, have to copy them from [~steve_rowe]'s patch later.

> WhitespaceTokenizer should tokenize on NBSP
> ---
>
> Key: LUCENE-6874
> URL: https://issues.apache.org/jira/browse/LUCENE-6874
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: David Smiley
>Priority: Minor
> Attachments: LUCENE-6874-jflex.patch, LUCENE-6874.patch, 
> LUCENE_6874_jflex.patch, icu-datasucker.patch, unicode-ws-tokenizer.patch, 
> unicode-ws-tokenizer.patch
>
>
> WhitespaceTokenizer uses [Character.isWhitespace 
> |http://docs.oracle.com/javase/8/docs/api/java/lang/Character.html#isWhitespace-int-]
>  to decide what is whitespace.  Here's a pertinent excerpt:
> bq. It is a Unicode space character (SPACE_SEPARATOR, LINE_SEPARATOR, or 
> PARAGRAPH_SEPARATOR) but is not also a non-breaking space ('\u00A0', 
> '\u2007', '\u202F')
> Perhaps Character.isWhitespace should have been called 
> isLineBreakableWhitespace?
> I think WhitespaceTokenizer should tokenize on this.  I am aware it's easy to 
> work around but why leave this trap in by default?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7539) Add a QueryAutofilteringComponent for query introspection using indexed metadata

[jmlucjav (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001893#comment-15001893
 ] 

jmlucjav commented on SOLR-7539:


This would be a great addition, I have used some custom handler to deal with 
these situations, wrote about it 
[here|https://medium.com/@jmlucjav/proper-handling-of-make-query-terms-in-e-commerce-search-39a9ac1bd4bc]
If I might suggest something, I would add some config to allow certain values 
to be excluded from being used as autofilters, this is useful when for example 
a brand name is also a common word that can occur naturally in other places etc.

> Add a QueryAutofilteringComponent for query introspection using indexed 
> metadata
> 
>
> Key: SOLR-7539
> URL: https://issues.apache.org/jira/browse/SOLR-7539
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ted Sullivan
>Priority: Minor
> Fix For: Trunk
>
> Attachments: SOLR-7539.patch, SOLR-7539.patch, SOLR-7539.patch
>
>
> The Query Autofiltering Component provides a method of inferring user intent 
> by matching noun phrases that are typically used for faceted-navigation into 
> Solr filter or boost queries (depending on configuration settings) so that 
> more precise user queries can be met with more precise results.
> The algorithm uses a "longest contiguous phrase match" strategy which allows 
> it to disambiguate queries where single terms are ambiguous but phrases are 
> not. It will work when there is structured information in the form of String 
> fields that are normally used for faceted navigation. It works across fields 
> by building a map of search term to index field using the Lucene FieldCache 
> (UninvertingReader). This enables users to create free text, multi-term 
> queries that combine attributes across facet fields - as if they had searched 
> and then navigated through several facet layers. To address the problem of 
> exact-match only semantics of String fields, support for synonyms (including 
> multi-term synonyms) and stemming was added. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8278) ConfigSetService should use NIO2

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated SOLR-8278:

Fix Version/s: 5.4

> ConfigSetService should use NIO2 
> -
>
> Key: SOLR-8278
> URL: https://issues.apache.org/jira/browse/SOLR-8278
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: 5.4
>
> Attachments: SOLR-8278.patch
>
>
> Similar to SOLR-8260, using java.nio.file.Path instead of String and File 
> makes error handling and reporting a lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8278) ConfigSetService should use NIO2

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001918#comment-15001918
 ] 

ASF subversion and git services commented on SOLR-8278:
---

Commit 1713999 from [~romseygeek] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1713999 ]

SOLR-8278: Use NIO2 APIs in ConfigSetService

> ConfigSetService should use NIO2 
> -
>
> Key: SOLR-8278
> URL: https://issues.apache.org/jira/browse/SOLR-8278
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-8278.patch
>
>
> Similar to SOLR-8260, using java.nio.file.Path instead of String and File 
> makes error handling and reporting a lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-8278) ConfigSetService should use NIO2

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward resolved SOLR-8278.
-
Resolution: Fixed

> ConfigSetService should use NIO2 
> -
>
> Key: SOLR-8278
> URL: https://issues.apache.org/jira/browse/SOLR-8278
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-8278.patch
>
>
> Similar to SOLR-8260, using java.nio.file.Path instead of String and File 
> makes error handling and reporting a lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Sharing a class across cores

[Ishan Chattopadhyaya]

> Or in a separate Solr core/collection.  :)
That support is available with this, I think:
https://cwiki.apache.org/confluence/display/solr/Blob+Store+API

On Thu, Nov 12, 2015 at 2:31 PM, Erik Hatcher 
wrote:

> Or in a separate Solr core/collection.  :)
>
> On Nov 11, 2015, at 19:05, Walter Underwood  wrote:
>
> Depending on how fast the access needs to be, you could put that big map
> in memcache.
>
> wunder
> Walter Underwood
> wun...@wunderwood.org
> http://observer.wunderwood.org/  (my blog)
>
>
> On Nov 11, 2015, at 4:04 PM, Gus Heck  wrote:
>
> P.S. I posted the original message concurrently with the chat session's
> occurance I beleive, certainly before I had read it, so no I haven't
> actually tried what you suggest yet.
>
> On Wed, Nov 11, 2015 at 7:02 PM, Gus Heck  wrote:
>
>> Yes asked by a colleague :). The chat session is now in our jira ticket
>> :).
>>
>> However, my take on it is that this seems like a pretty broad brush to
>> paint with to move *all* our classes up and out of the normal core loading
>> process. I assume there are good reasons for segregating this stuff into
>> separate class loaders to begin with. It would also be fairly burdensom to
>> make a separate jar file to break out this one component...
>>
>> I really just want a way to stash the map in a place where other cores
>> can see it (and thus I can appropriately synchronize things so that the
>> loading only happens once). I'm asking because it seems like surely this
>> must be a solved problem... if not, it might be easiest to just solve it by
>> adding some sort of shared resources facility to CoreContainer?
>>
>> -Gus
>>
>> On Wed, Nov 11, 2015 at 6:54 PM, Shawn Heisey 
>> wrote:
>>
>>> On 11/11/2015 4:11 PM, Gus Heck wrote:
>>> > I have a case where a component loads up a large CSV file (2.5 million
>>> > lines) to build a map. This worked ok in a case where we had a single
>>> > core, but it isn't working so well with 40 cores because each core
>>> loads
>>> > a new copy of the component in a new classloader and I get 40 new
>>> > versions of the same class each holding it's own private static final
>>> > map (one for each core). Each line is small, but a billion of anything
>>> > gets kinda heavy. Is this the intended class loading behavior?
>>> >
>>> > Is there some where that one can cause a class to be loaded in a parent
>>> > classloader above the core so that it's loaded just once? I want to
>>> load
>>> > it in some way that leverages standard solr resource loading, so that
>>> > I'm not hard coding or setting sysprops just to be able to find it.
>>> >
>>> > This is in a copy of trunk from about a month ago... so 6.x stuff is
>>> > mostly available.
>>>
>>> This sounds like a question that I just recently answered on IRC.
>>>
>>> If you remove all  elements from your solrconfig.xml files and
>>> place all extra jars for Solr into ${solr.solr.home}/lib ... Solr will
>>> load those jars before any cores are created and they will be available
>>> to all cores.
>>>
>>> There is a minor bug with this that will be fixed in Solr 5.4.0.  It is
>>> unlikely that this will affect third-party components, but be aware that
>>> until 5.4, jars in that lib directory will be loaded twice by older 5.x
>>> versions.
>>>
>>> https://issues.apache.org/jira/browse/SOLR-6188
>>>
>>> Thanks,
>>> Shawn
>>>
>>>
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
>>> For additional commands, e-mail: dev-h...@lucene.apache.org
>>>
>>>
>>
>>
>> --
>> http://www.the111shift.com
>>
>
>
>
> --
> http://www.the111shift.com
>
>
>

[jira] [Commented] (SOLR-8273) deprecate implicitly uninverted fields, force people to either use docValues, or be explicit that they want query time uninversion

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003069#comment-15003069
 ] 

Varun Thacker commented on SOLR-8273:
-

bq. sample schemas should be updated to use docValues on any field where the 
examples using those schemas suggest using those fields in that way (ie: 
sorting, faceting, etc...)

Does this mean we won't enable docValues to be true by default? If thats a case 
then after this change what happens if you sort/facet on a field? Will it throw 
an error?

Personally I think we should make docValues=true by default but that might not 
be part of the scope for this Jira?

> deprecate implicitly uninverted fields, force people to either use docValues, 
> or be explicit that they want query time uninversion
> --
>
> Key: SOLR-8273
> URL: https://issues.apache.org/jira/browse/SOLR-8273
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>
> once upon a time, there was nothing we could do to *stop* people from using 
> the FieldCache - even if they didn't realize they were using it.
> Then DocValues was added - and now people have a choice: they can set 
> {{docValues=true}} on a field/fieldtype and know that when they do 
> functions/sorting/faceting on that field, it won't require a big hunk of ram 
> and a big stall everytime a reader was reopened.  But it's easy to overlook 
> when clients might be doing something that required the FieldCache w/o 
> realizing it -- and there is no way to stop them, because Solr automatically 
> uses UninvertingReader under the covers and automatically allows every field 
> to be uninverted in this way.
> we should change that.
> 
> Straw man proposal...
> * introduce a new boolean fieldType/field property {{uninvertable}}
> * all existing FieldType classes should default to {{uninvertable==false}}
> * a field or fieldType that contains {{indexed="false" uninvertable="true"}} 
> should be an error.
> * the Schema {{version}} value should be incremented, such that any Schema 
> with an older version is treated as if every field with {{docValues==false}} 
> has an implict {{uninvertable="true"}} on it.
> * the Map passed to UninvertedReader should now only list items that have an 
> effective value of {{uninvertable==true}}
> * sample schemas should be updated to use docValues on any field where the 
> examples using those schemas suggest using those fields in that way (ie: 
> sorting, faceting, etc...)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003174#comment-15003174
 ] 

Mark Miller commented on SOLR-8075:
---

This node has to be the leader in the zk election line at this place it sets 
active - so there cannot be another valid leader.

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8185) Add operations support to streaming metrics

[Dennis Gove (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14968357#comment-14968357
 ] 

Dennis Gove edited comment on SOLR-8185 at 11/13/15 1:41 AM:
-

Full patch. 


was (Author: dpgove):
Full patch. All tests pass.

> Add operations support to streaming metrics
> ---
>
> Key: SOLR-8185
> URL: https://issues.apache.org/jira/browse/SOLR-8185
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrJ
>Reporter: Dennis Gove
>Assignee: Dennis Gove
>Priority: Minor
> Attachments: SOLR-8185.patch
>
>
> Adds support for operations on stream metrics.
> With this feature one can modify tuple values before applying to the computed 
> metric. There are a lot of use-cases I can see with this - I'll describe one 
> here.
> Imagine you have a RollupStream which is computing the average over some 
> field but you cannot be sure that all documents have a value for that field, 
> ie the value is null. When the value is null you want to treat it as a 0. 
> With this feature you can accomplish that like this
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0)),
>   count(*),
> )
> {code}
> The operations are applied to the tuple for each metric in the stream which 
> means you perform different operations on different metrics without being 
> impacted by operations on other metrics. 
> Adding to our previous example, imagine you want to also get the min of a 
> field but do not consider null values.
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0)),
>   min(a_i),
>   count(*),
> )
> {code}
> Also, the tuple is not modified for streams that might wrap this one. Ie, the 
> only thing that sees the applied operation is that particular metric. If you 
> want to apply operations for wrapping streams you can still achieve that with 
> the SelectStream (SOLR-7669).
> One feature I'm investigating but this patch DOES NOT add is the ability to 
> assign names to the resulting metric value. For example, to allow for 
> something like this
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0), as="avg_a_i_null_as_0"),
>   avg(a_i),
>   count(*, as="totalCount"),
> )
> {code}
> Right now that isn't possible because the identifier for each metric would be 
> the same "avg_a_i" and as such both couldn't be returned. It's relatively 
> easy to add but I have to investigate its impact on the SQL and FacetStream 
> areas.
> Depends on SOLR-7669 (SelectStream)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8273) deprecate implicitly uninverted fields, force people to either use docValues, or be explicit that they want query time uninversion

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003107#comment-15003107
 ] 

Yonik Seeley commented on SOLR-8273:


bq. Does this mean we won't enable docValues to be true by default?

No, I think we should enable docValues for the majority of field types... 
pretty much everything except full-text fields?


> deprecate implicitly uninverted fields, force people to either use docValues, 
> or be explicit that they want query time uninversion
> --
>
> Key: SOLR-8273
> URL: https://issues.apache.org/jira/browse/SOLR-8273
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>
> once upon a time, there was nothing we could do to *stop* people from using 
> the FieldCache - even if they didn't realize they were using it.
> Then DocValues was added - and now people have a choice: they can set 
> {{docValues=true}} on a field/fieldtype and know that when they do 
> functions/sorting/faceting on that field, it won't require a big hunk of ram 
> and a big stall everytime a reader was reopened.  But it's easy to overlook 
> when clients might be doing something that required the FieldCache w/o 
> realizing it -- and there is no way to stop them, because Solr automatically 
> uses UninvertingReader under the covers and automatically allows every field 
> to be uninverted in this way.
> we should change that.
> 
> Straw man proposal...
> * introduce a new boolean fieldType/field property {{uninvertable}}
> * all existing FieldType classes should default to {{uninvertable==false}}
> * a field or fieldType that contains {{indexed="false" uninvertable="true"}} 
> should be an error.
> * the Schema {{version}} value should be incremented, such that any Schema 
> with an older version is treated as if every field with {{docValues==false}} 
> has an implict {{uninvertable="true"}} on it.
> * the Map passed to UninvertedReader should now only list items that have an 
> effective value of {{uninvertable==true}}
> * sample schemas should be updated to use docValues on any field where the 
> examples using those schemas suggest using those fields in that way (ie: 
> sorting, faceting, etc...)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6889) BooleanQuery.rewrite could easily optimize some simple cases

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003169#comment-15003169
 ] 

Hoss Man commented on LUCENE-6889:
--

bq. It works fine because rewrite's contract is that it should be called until 
the return value is the same as the provided query...

Ah ... yeah, I totally forgot about that. sorry for the noise.

> BooleanQuery.rewrite could easily optimize some simple cases
> 
>
> Key: LUCENE-6889
> URL: https://issues.apache.org/jira/browse/LUCENE-6889
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6889.patch
>
>
> Follow-up of SOLR-8251: APIs and user interfaces sometimes encourage to write 
> BooleanQuery instances that are not optimal, for instance a typical case that 
> happens often with Solr/Elasticsearch is to send a request that has a 
> MatchAllDocsQuery as a query and some filter, which could be executed more 
> efficiently by directly wrapping the filter into a ConstantScoreQuery.
> Here are some ideas of rewrite operations that BooleanQuery could perform:
>  - remove FILTER clauses when they are also a MUST clause
>  - rewrite queries of the form "+*:* #filter" to a ConstantScoreQuery(filter)
>  - rewrite to a MatchNoDocsQuery when a clause that is a MUST or FILTER 
> clause is also a MUST_NOT clause



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003197#comment-15003197
 ] 

Mark Miller commented on SOLR-8075:
---

I'd file a JIRA issue to track the idea. It wouldn't be bad to tighten this up.

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-5.x - Build # 1015 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-5.x/1015/

1 tests failed.
FAILED:  
org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR

Error Message:
Captured an uncaught exception in thread: Thread[id=3099, 
name=coreZkRegister-780-thread-2, state=RUNNABLE, 
group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=3099, name=coreZkRegister-780-thread-2, 
state=RUNNABLE, group=TGRP-LeaderInitiatedRecoveryOnShardRestartTest]
Caused by: java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([345BA5D120782BD9]:0)
at 
org.apache.solr.cloud.ZkController.updateLeaderInitiatedRecoveryState(ZkController.java:2133)
at 
org.apache.solr.cloud.ShardLeaderElectionContext.runLeaderProcess(ElectionContext.java:434)
at 
org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:197)
at 
org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:157)
at 
org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:346)
at 
org.apache.solr.cloud.ZkController.joinElection(ZkController.java:1113)
at org.apache.solr.cloud.ZkController.register(ZkController.java:926)
at org.apache.solr.cloud.ZkController.register(ZkController.java:881)
at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:183)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:231)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10457 lines...]
   [junit4] Suite: 
org.apache.solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest
   [junit4]   2> Creating dataDir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-5.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderInitiatedRecoveryOnShardRestartTest_345BA5D120782BD9-001/init-core-data-001
   [junit4]   2> 662956 INFO  
(SUITE-LeaderInitiatedRecoveryOnShardRestartTest-seed#[345BA5D120782BD9]-worker)
 [] o.a.s.BaseDistributedSearchTestCase Setting hostContext system 
property: /
   [junit4]   2> 662962 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 662962 INFO  (Thread-625) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 662962 INFO  (Thread-625) [] o.a.s.c.ZkTestServer Starting 
server
   [junit4]   2> 663062 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.ZkTestServer start zk server on port:49107
   [junit4]   2> 663063 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663063 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663066 INFO  (zkCallback-885-thread-1) [] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@454c2a6d 
name:ZooKeeperConnection Watcher:127.0.0.1:49107 got event WatchedEvent 
state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663067 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 663067 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 663067 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 663071 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 663072 INFO  
(TEST-LeaderInitiatedRecoveryOnShardRestartTest.testRestartWithAllInLIR-seed#[345BA5D120782BD9])
 [] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 663074 INFO  (zkCallback-886-thread-1) [] 
o.a.s.c.c.ConnectionManager Watcher 
org.apache.solr.common.cloud.ConnectionManager@19461ed name:ZooKeeperConnection 
Watcher:127.0.0.1:49107/solr got event WatchedEvent state:SyncConnected 
type:None path:null path:null type:None
   [junit4]   2> 663074 INFO  

[jira] [Updated] (LUCENE-6276) Add matchCost() api to TwoPhaseDocIdSetIterator

[Paul Elschot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Paul Elschot updated LUCENE-6276:
-
Attachment: LUCENE-6276.patch

Patch of 12 Nov 2015.
Adds ScoringWrapperSpans.asTwoPhaseIterator().

This missing method could be a bug in itself.

> Add matchCost() api to TwoPhaseDocIdSetIterator
> ---
>
> Key: LUCENE-6276
> URL: https://issues.apache.org/jira/browse/LUCENE-6276
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
> Attachments: LUCENE-6276-ExactPhraseOnly.patch, 
> LUCENE-6276-NoSpans.patch, LUCENE-6276-NoSpans2.patch, LUCENE-6276.patch, 
> LUCENE-6276.patch, LUCENE-6276.patch, LUCENE-6276.patch, LUCENE-6276.patch, 
> LUCENE-6276.patch, LUCENE-6276.patch, LUCENE-6276.patch
>
>
> We could add a method like TwoPhaseDISI.matchCost() defined as something like 
> estimate of nanoseconds or similar. 
> ConjunctionScorer could use this method to sort its 'twoPhaseIterators' array 
> so that cheaper ones are called first. Today it has no idea if one scorer is 
> a simple phrase scorer on a short field vs another that might do some geo 
> calculation or more expensive stuff.
> PhraseScorers could implement this based on index statistics (e.g. 
> totalTermFreq/maxDoc)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6889) BooleanQuery.rewrite could easily optimize some simple cases

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-6889:
-
Attachment: LUCENE-6889.patch

Here is a patch that does the following rewrites:

Removal of FILTER clauses that are also MUST clauses
{noformat}
#a +a -> +a
{noformat}

FilteredQuery rewrite when the query is a MatchAllDocsQuery
{noformat}
+*:*^b #f -> ConstantScoreQuery(f)^b
{noformat}

Removal of filters on MatchAllDocsQuery if they are a MUST clause as well
{noformat}
+a #*:* -> +a
{noformat}

Deduplication of FILTER and MUST_NOT clauses
{noformat}
+a #f #f -f -f -> +a #f -f
{noformat}

They have the nice property of being able to execute things that we used to 
execute as a disjunction or a conjunction as a simple term query.

I also wanted to rewrite queries to a MatchAllDocsQuery when there was an 
intersection between required and prohibited clauses (Terry's rule 3) or when 
the mininumShouldMatch is greater than the number of SHOULD clauses but this 
broke weight normalization. We can probably solve the MUST_NOT/MUST 
intersection at the Scorer level but I propose to defer it to another issue.

The patch includes unit tests for the above rewrite rules as well as a random 
test that makes sure that the same set of matches and scores are produced if no 
rewriting is performed.

> BooleanQuery.rewrite could easily optimize some simple cases
> 
>
> Key: LUCENE-6889
> URL: https://issues.apache.org/jira/browse/LUCENE-6889
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6889.patch
>
>
> Follow-up of SOLR-8251: APIs and user interfaces sometimes encourage to write 
> BooleanQuery instances that are not optimal, for instance a typical case that 
> happens often with Solr/Elasticsearch is to send a request that has a 
> MatchAllDocsQuery as a query and some filter, which could be executed more 
> efficiently by directly wrapping the filter into a ConstantScoreQuery.
> Here are some ideas of rewrite operations that BooleanQuery could perform:
>  - remove FILTER clauses when they are also a MUST clause
>  - rewrite queries of the form "+*:* #filter" to a ConstantScoreQuery(filter)
>  - rewrite to a MatchNoDocsQuery when a clause that is a MUST or FILTER 
> clause is also a MUST_NOT clause



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-6894) Improve DISI.cost() by assuming independence for match probabilities

[Paul Elschot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003039#comment-15003039
 ] 

Paul Elschot edited comment on LUCENE-6894 at 11/12/15 10:27 PM:
-

Another reason why I started this is that the result of cost() is also used as 
weights for matchCost() at LUCENE-6276, and I'd prefer those weights to be as 
accurate as reasonably possible.

I think we can keep this (assuming independence for conjunctions and 
disjunctions) as a possible alternative until the current implementation gives 
a bad result.

For the proximity queries (Phrases, Spans) this reduces the conjunction cost() 
using the allowed slop.
Would it be worthwhile to open a separate issue for that?



was (Author: paul.elsc...@xs4all.nl):
Another reason why I started this is that the result of cost() is also used as 
weights for matchCost() at LUCENE-6276, and I'd prefer those weights to be as 
accurate as reasonably possible.

I think we can keep this alternative (assuming independence for conjunctions 
and disjunctions) as a possible alternative until the current implementation 
gives a bad result.

For the proximity queries (Phrases, Spans) this reduces the conjunction cost() 
using the allowed slop.
Would it be worthwhile to open a separate issue for that?


> Improve DISI.cost() by assuming independence for match probabilities
> 
>
> Key: LUCENE-6894
> URL: https://issues.apache.org/jira/browse/LUCENE-6894
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Paul Elschot
>Priority: Minor
> Attachments: LUCENE-6894.patch
>
>
> The DocIdSetIterator.cost() method returns an estimation of the number of 
> matching docs. Currently conjunctions use the minimum cost, and disjunctions 
> use the sum of the costs, and both are too high.
> The probability of a match is estimated by dividing available cost() by the 
> number of docs in a segment.
> The conjunction probability is then the product of the inputs, and the 
> disjunction probability follows from De Morgan's rule:
> "not (A and B)" is the same as "(not A) or (not B)"
> with the probability for "not" computed as 1 minus the input probability.
> The independence that is assumed is normally not there. However, the cost() 
> results are only used to order the input DISIs/Scorers for optimization, and 
> for that I expect this assumption to work nicely.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003133#comment-15003133
 ] 

Gregory Chanan commented on SOLR-8075:
--

Patch looks good, thanks Mark!

A comment outside the scope of this JIRA (I know this is pre-existing logic), 
but which I can't find a better place for:
{code}
+  // We can do this before registering as leader because only setting 
DOWN requires that
+  // we are leader, and here we are setting ACTIVE
+  zkController.updateLeaderInitiatedRecoveryState(collection, shardId,
+  leaderProps.getStr(ZkStateReader.CORE_NODE_NAME_PROP), 
Replica.State.ACTIVE, core.getCoreDescriptor(), true);
{code}

This seems difficult to reason about given that there are multiple 
non-commutative writers potentially racing here: a leader setting DOWN and this 
node setting ACTIVE.  It would be easier to reason about if there were two 
states:
1) leaders view of the world
2) replicas view of the world (i.e. telling the Overseer I know the leader 
thinks I'm in LIR but I know some special information and I'm telling you for 
this election # I'm OK).  That could go in the ZKNodeProps sent to the Overseer 
(or a separate znode) and the Overseer could do the correct logic with it.

Anyway, outside of the scope of the jira, just wanted to jot my thoughts down.  
if you think this is a valid improvement I can file a jira for it.

> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-5.x-Linux (32bit/jdk1.9.0-ea-b90) - Build # 14594 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/14594/
Java: 32bit/jdk1.9.0-ea-b90 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([55A3C6C696013124:BCF97DFE0898A18C]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:766)
at 
org.apache.solr.core.TestArbitraryIndexDir.testLoadNewIndexDir(TestArbitraryIndexDir.java:128)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:520)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:747)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=*[count(//doc)=1]
xml response was: 

00


request was:q=id:2=standard=0=20=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:759)
... 40 more




Build Log:
[...truncated 9917 lines...]
   [junit4] Suite: 

[jira] [Commented] (SOLR-6168) enhance collapse QParser so that "group head" documents can be selected by more complex sort options

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003189#comment-15003189
 ] 

Hoss Man commented on SOLR-6168:



Finally getting back to this ... 

Did a bunch of manual testing offline w/o spotting any problems, so i went 
ahead and commited to trunk.

backport to 5x was fairly clean, testing/precommiting now.

Assuming nothing weird  jumps out, I'll let jenkins hammer on trunk overnight 
and then commit the  backport tomorrow.

> enhance collapse QParser so that "group head" documents can be selected by 
> more complex sort options
> 
>
> Key: SOLR-6168
> URL: https://issues.apache.org/jira/browse/SOLR-6168
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 4.7.1, 4.8.1
>Reporter: Umesh Prasad
>Assignee: Joel Bernstein
> Attachments: CollapsingQParserPlugin-6168.patch.1-1stcut, 
> SOLR-6168-group-head-inconsistent-with-sort.patch, SOLR-6168.patch, 
> SOLR-6168.patch, SOLR-6168.patch, SOLR-6168.patch, SOLR-6168.patch
>
>
> The fundemental goal of this issue is add additional support to the 
> CollapseQParser so that as an alternative to the existing min/max localparam 
> options, more robust sort syntax can be used to sort on multiple criteria 
> when selecting the "group head" documents used to represent each collapsed 
> group.
> Since support for arbitrary, multi-clause, sorting is almost certainly going 
> to require more RAM then the existing min/max functionaly, this new 
> functionality should be in addition to the existing min/max localparam 
> implementation, not a replacement of it.
> (NOTE: early comments made in this jira may be confusing in historical 
> context due to the way this issue was originally filed as a bug report)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8273) deprecate implicitly uninverted fields, force people to either use docValues, or be explicit that they want query time uninversion

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003141#comment-15003141
 ] 

Yonik Seeley commented on SOLR-8273:


bq. There is only one valid use case I can think of where a TextField should 
have docValues enabled

Right, and more than that... some fields that are used for simple things like 
tags, that will be faceted on.
I was thinking we'd have a separate fieldType + dynamicField for such things 
that had uninverting=true by default.   This need not apply to all text fields.

> deprecate implicitly uninverted fields, force people to either use docValues, 
> or be explicit that they want query time uninversion
> --
>
> Key: SOLR-8273
> URL: https://issues.apache.org/jira/browse/SOLR-8273
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>
> once upon a time, there was nothing we could do to *stop* people from using 
> the FieldCache - even if they didn't realize they were using it.
> Then DocValues was added - and now people have a choice: they can set 
> {{docValues=true}} on a field/fieldtype and know that when they do 
> functions/sorting/faceting on that field, it won't require a big hunk of ram 
> and a big stall everytime a reader was reopened.  But it's easy to overlook 
> when clients might be doing something that required the FieldCache w/o 
> realizing it -- and there is no way to stop them, because Solr automatically 
> uses UninvertingReader under the covers and automatically allows every field 
> to be uninverted in this way.
> we should change that.
> 
> Straw man proposal...
> * introduce a new boolean fieldType/field property {{uninvertable}}
> * all existing FieldType classes should default to {{uninvertable==false}}
> * a field or fieldType that contains {{indexed="false" uninvertable="true"}} 
> should be an error.
> * the Schema {{version}} value should be incremented, such that any Schema 
> with an older version is treated as if every field with {{docValues==false}} 
> has an implict {{uninvertable="true"}} on it.
> * the Map passed to UninvertedReader should now only list items that have an 
> effective value of {{uninvertable==true}}
> * sample schemas should be updated to use docValues on any field where the 
> examples using those schemas suggest using those fields in that way (ie: 
> sorting, faceting, etc...)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-8185) Add operations support to streaming metrics

[Dennis Gove (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dennis Gove reassigned SOLR-8185:
-

Assignee: Dennis Gove

> Add operations support to streaming metrics
> ---
>
> Key: SOLR-8185
> URL: https://issues.apache.org/jira/browse/SOLR-8185
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrJ
>Reporter: Dennis Gove
>Assignee: Dennis Gove
>Priority: Minor
> Attachments: SOLR-8185.patch
>
>
> Adds support for operations on stream metrics.
> With this feature one can modify tuple values before applying to the computed 
> metric. There are a lot of use-cases I can see with this - I'll describe one 
> here.
> Imagine you have a RollupStream which is computing the average over some 
> field but you cannot be sure that all documents have a value for that field, 
> ie the value is null. When the value is null you want to treat it as a 0. 
> With this feature you can accomplish that like this
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0)),
>   count(*),
> )
> {code}
> The operations are applied to the tuple for each metric in the stream which 
> means you perform different operations on different metrics without being 
> impacted by operations on other metrics. 
> Adding to our previous example, imagine you want to also get the min of a 
> field but do not consider null values.
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0)),
>   min(a_i),
>   count(*),
> )
> {code}
> Also, the tuple is not modified for streams that might wrap this one. Ie, the 
> only thing that sees the applied operation is that particular metric. If you 
> want to apply operations for wrapping streams you can still achieve that with 
> the SelectStream (SOLR-7669).
> One feature I'm investigating but this patch DOES NOT add is the ability to 
> assign names to the resulting metric value. For example, to allow for 
> something like this
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0), as="avg_a_i_null_as_0"),
>   avg(a_i),
>   count(*, as="totalCount"),
> )
> {code}
> Right now that isn't possible because the identifier for each metric would be 
> the same "avg_a_i" and as such both couldn't be returned. It's relatively 
> easy to add but I have to investigate its impact on the SQL and FacetStream 
> areas.
> Depends on SOLR-7669 (SelectStream)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8075) Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003209#comment-15003209
 ] 

Gregory Chanan commented on SOLR-8075:
--

bq. This node has to be the leader in the zk election line at this place it 
sets active - so there cannot be another valid leader.

Agreed, my point is just:
1) that's true now, maybe not always true
2) that's more to reason about, better to keep things as simple as possible, at 
least where state is concerned

bq. I'd file a JIRA issue to track the idea. It wouldn't be bad to tighten this 
up.

Will do, thanks.


> Leader Initiated Recovery should not stop a leader that participated in an 
> election with all of it's replicas from becoming a valid leader.
> ---
>
> Key: SOLR-8075
> URL: https://issues.apache.org/jira/browse/SOLR-8075
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, 
> SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch, SOLR-8075.patch
>
>
> Currently, because of SOLR-8069, all the replicas in a shard can be put into 
> LIR.
> If you restart such a shard, the valid leader will will win the election and 
> sync with the shard and then be blocked from registering as ACTIVE because it 
> is in LIR.
> I think that is a little wonky because I don't think it even tries another 
> candidate because the leader that cannot publish ACTIVE does not have it's 
> election canceled.
> While SOLR-8069 should prevent this situation, we should add logic to allow a 
> leader that can sync with it's full shard to become leader and publish ACTIVE 
> regardless of LIR.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8286) Remove instances of solr.hdfs.blockcache.write.enabled from tests and docs

[Gregory Chanan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gregory Chanan updated SOLR-8286:
-
Attachment: SOLR-8286.patch

Patch that removes references to the write side of the block cache from the 
test configs.  Once this is in, I'll remove it from the docs as well.

> Remove instances of solr.hdfs.blockcache.write.enabled from tests and docs
> --
>
> Key: SOLR-8286
> URL: https://issues.apache.org/jira/browse/SOLR-8286
> Project: Solr
>  Issue Type: Bug
>  Components: documentation, Tests
>Affects Versions: 5.0
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8286.patch
>
>
> solr.hdfs.blockcache.write.enabled is currently disabled whether or not you 
> set it in the solr configs.  It makes sense to just avoid mentioning it in 
> the docs (it's still there on 
> https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS) and 
> the morphlines examples.  Best case, it's unnecessary information.  Worse 
> case, it causes people trouble on versions where it's not disabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6889) BooleanQuery.rewrite could easily optimize some simple cases

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003129#comment-15003129
 ] 

Hoss Man commented on LUCENE-6889:
--

maybe i'm reading the patch wrong, but it looks like the "actuallyRewritten" 
check from the recursive rewriting will return before several of the 
optimizations.  Shouldn't things like "remove duplicate FILTER and MUST_NOT 
clauses" and "remove FILTER clauses that are also MUST clauses" still be 
tested/done against the rewritten sub-clauses?

likewise isn't doing the "remove FILTER clauses that are also MUST clauses" 
optimization still worthwhile even if the "remove duplicate FILTER and MUST_NOT 
clauses" optimization finds & removes things? (it also looks like it has a 
short-circuit return)

bq. ... as well as a random test that makes sure that the same set of matches 
and scores are produced if no rewriting is performed.

why not randomize the docs/fields in the index as well?  At first glance one 
concern i have is that no doc has a single term more then once, so spotting 
subtle score discrepancies between the query and it's optimize version may 
never come into play with this test.

other small concerns about the current random query generation:
* is {{rarely()}} really appropriate for the BoostQuery wrapping? is that 
something that really makes sense to increase depending on wether the test is 
nightly? ... seems like something more straight forward like 
{{0==TestUtil.nextInt(random(), 0, 10)}} would make more sense for these tests
* randomizing setMinimumNumberShouldMatch between 0 and numClauses means that 
it's going to be very rare for the minimumNumberShouldMatch setting to actually 
impact the query unless there are also a lot of random SHOULD clauses (ie: if 
there are 5 clauses but only 2 SHOULD clauses there's only a 2:5 chance of that 
setting getting a random value that actually affects anything) ... probably 
better to count the actual # of SHOULD clauses generated randomly and then 
randomize the setting between 0 and #+1.
* MatchNoDocsQuery should probably be included in the randomization to better 
coverage of all the optimization situations.

> BooleanQuery.rewrite could easily optimize some simple cases
> 
>
> Key: LUCENE-6889
> URL: https://issues.apache.org/jira/browse/LUCENE-6889
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6889.patch
>
>
> Follow-up of SOLR-8251: APIs and user interfaces sometimes encourage to write 
> BooleanQuery instances that are not optimal, for instance a typical case that 
> happens often with Solr/Elasticsearch is to send a request that has a 
> MatchAllDocsQuery as a query and some filter, which could be executed more 
> efficiently by directly wrapping the filter into a ConstantScoreQuery.
> Here are some ideas of rewrite operations that BooleanQuery could perform:
>  - remove FILTER clauses when they are also a MUST clause
>  - rewrite queries of the form "+*:* #filter" to a ConstantScoreQuery(filter)
>  - rewrite to a MatchNoDocsQuery when a clause that is a MUST or FILTER 
> clause is also a MUST_NOT clause



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6168) enhance collapse QParser so that "group head" documents can be selected by more complex sort options

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003130#comment-15003130
 ] 

ASF subversion and git services commented on SOLR-6168:
---

Commit 1714133 from hoss...@apache.org in branch 'dev/trunk'
[ https://svn.apache.org/r1714133 ]

SOLR-6168: Add a 'sort' local param to the collapse QParser to support using 
complex sort options to select the representitive doc for each collapsed group

> enhance collapse QParser so that "group head" documents can be selected by 
> more complex sort options
> 
>
> Key: SOLR-6168
> URL: https://issues.apache.org/jira/browse/SOLR-6168
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 4.7.1, 4.8.1
>Reporter: Umesh Prasad
>Assignee: Joel Bernstein
> Attachments: CollapsingQParserPlugin-6168.patch.1-1stcut, 
> SOLR-6168-group-head-inconsistent-with-sort.patch, SOLR-6168.patch, 
> SOLR-6168.patch, SOLR-6168.patch, SOLR-6168.patch, SOLR-6168.patch
>
>
> The fundemental goal of this issue is add additional support to the 
> CollapseQParser so that as an alternative to the existing min/max localparam 
> options, more robust sort syntax can be used to sort on multiple criteria 
> when selecting the "group head" documents used to represent each collapsed 
> group.
> Since support for arbitrary, multi-clause, sorting is almost certainly going 
> to require more RAM then the existing min/max functionaly, this new 
> functionality should be in addition to the existing min/max localparam 
> implementation, not a replacement of it.
> (NOTE: early comments made in this jira may be confusing in historical 
> context due to the way this issue was originally filed as a bug report)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8273) deprecate implicitly uninverted fields, force people to either use docValues, or be explicit that they want query time uninversion

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003131#comment-15003131
 ] 

Varun Thacker commented on SOLR-8273:
-

bq. pretty much everything except full-text fields?

There is only one valid use case I can think of where a TextField should have 
docValues enabled : A field which uses a Keyword Tokenizer + Lowecase filter 
just to normalize the data.

> deprecate implicitly uninverted fields, force people to either use docValues, 
> or be explicit that they want query time uninversion
> --
>
> Key: SOLR-8273
> URL: https://issues.apache.org/jira/browse/SOLR-8273
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>
> once upon a time, there was nothing we could do to *stop* people from using 
> the FieldCache - even if they didn't realize they were using it.
> Then DocValues was added - and now people have a choice: they can set 
> {{docValues=true}} on a field/fieldtype and know that when they do 
> functions/sorting/faceting on that field, it won't require a big hunk of ram 
> and a big stall everytime a reader was reopened.  But it's easy to overlook 
> when clients might be doing something that required the FieldCache w/o 
> realizing it -- and there is no way to stop them, because Solr automatically 
> uses UninvertingReader under the covers and automatically allows every field 
> to be uninverted in this way.
> we should change that.
> 
> Straw man proposal...
> * introduce a new boolean fieldType/field property {{uninvertable}}
> * all existing FieldType classes should default to {{uninvertable==false}}
> * a field or fieldType that contains {{indexed="false" uninvertable="true"}} 
> should be an error.
> * the Schema {{version}} value should be incremented, such that any Schema 
> with an older version is treated as if every field with {{docValues==false}} 
> has an implict {{uninvertable="true"}} on it.
> * the Map passed to UninvertedReader should now only list items that have an 
> effective value of {{uninvertable==true}}
> * sample schemas should be updated to use docValues on any field where the 
> examples using those schemas suggest using those fields in that way (ie: 
> sorting, faceting, etc...)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [jira] [Commented] (SOLR-4905) Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded collection that has a replica on all nodes

[Trey Grainger]

Just to add another voice to the discussion, I have the exact same use case
described by Paul and Mikhail that I'm working through a Proof of Concept
for right now. I'd very much like to see the "single shard collection with
a replica on all nodes" restriction removed.

On Thu, Nov 12, 2015, 3:29 PM Mikhail Khludnev (JIRA) 
wrote:

>
> [
> https://issues.apache.org/jira/browse/SOLR-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002865#comment-15002865
> ]
>
> Mikhail Khludnev commented on SOLR-4905:
> 
>
> [~p...@search-solutions.net] would you mind to raise a separate jira for
> this?
>
> > Allow fromIndex parameter to JoinQParserPlugin to refer to a
> single-sharded collection that has a replica on all nodes
> >
> --
> >
> > Key: SOLR-4905
> > URL: https://issues.apache.org/jira/browse/SOLR-4905
> > Project: Solr
> >  Issue Type: Improvement
> >  Components: SolrCloud
> >Reporter: Philip K. Warren
> >Assignee: Timothy Potter
> > Fix For: 5.1, Trunk
> >
> > Attachments: SOLR-4905.patch, SOLR-4905.patch, patch.txt
> >
> >
> > Using a non-SolrCloud setup, it is possible to perform cross core joins (
> http://wiki.apache.org/solr/Join). When testing with SolrCloud, however,
> neither the collection name, alias name (we have created aliases to
> SolrCloud collections), or the automatically generated core name (i.e.
> _shard1_replica1) work as the fromIndex parameter for a
> cross-core join.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
> For additional commands, e-mail: dev-h...@lucene.apache.org
>
>

[jira] [Created] (SOLR-8286) Remove instances of solr.hdfs.blockcache.write.enabled from tests and docs

[Gregory Chanan (JIRA)]

Gregory Chanan created SOLR-8286:


 Summary: Remove instances of solr.hdfs.blockcache.write.enabled 
from tests and docs
 Key: SOLR-8286
 URL: https://issues.apache.org/jira/browse/SOLR-8286
 Project: Solr
  Issue Type: Bug
  Components: documentation, Tests
Affects Versions: 5.0
Reporter: Gregory Chanan
Assignee: Gregory Chanan
 Fix For: 5.4, Trunk


solr.hdfs.blockcache.write.enabled is currently disabled whether or not you set 
it in the solr configs.  It makes sense to just avoid mentioning it in the docs 
(it's still there on 
https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS) and the 
morphlines examples.  Best case, it's unnecessary information.  Worse case, it 
causes people trouble on versions where it's not disabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7255) Index Corruption on HDFS whenever online bulk indexing (from Hive)

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003299#comment-15003299
 ] 

Gregory Chanan commented on SOLR-7255:
--

bq. If you want to take ownership and change it to a doc issue, feel free

SOLR-8286.

> Index Corruption on HDFS whenever online bulk indexing (from Hive)
> --
>
> Key: SOLR-7255
> URL: https://issues.apache.org/jira/browse/SOLR-7255
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 4.10.3
> Environment: HDP 2.2 / HDP Search + LucidWorks hadoop-lws-job.jar
>Reporter: Hari Sekhon
>Priority: Blocker
>
> When running SolrCloud on HDFS and using the LucidWorks hadoop-lws-job.jar to 
> index a Hive table (620M rows) to Solr it runs for about 1500 secs and then 
> gets this exception:
> {code}Exception in thread "Lucene Merge Thread #2191" 
> org.apache.lucene.index.MergePolicy$MergeException: 
> org.apache.lucene.index.CorruptIndexException: codec header mismatch: actual 
> header=1494817490 vs expected header=1071082519 (resource: 
> BufferedChecksumIndexInput(_r3.nvm))
> at 
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:549)
> at 
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:522)
> Caused by: org.apache.lucene.index.CorruptIndexException: codec header 
> mismatch: actual header=1494817490 vs expected header=1071082519 (resource: 
> BufferedChecksumIndexInput(_r3.nvm))
> at org.apache.lucene.codecs.CodecUtil.checkHeader(CodecUtil.java:136)
> at 
> org.apache.lucene.codecs.lucene49.Lucene49NormsProducer.(Lucene49NormsProducer.java:75)
> at 
> org.apache.lucene.codecs.lucene49.Lucene49NormsFormat.normsProducer(Lucene49NormsFormat.java:112)
> at 
> org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:127)
> at 
> org.apache.lucene.index.SegmentReader.(SegmentReader.java:108)
> at 
> org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
> at 
> org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:282)
> at 
> org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:3951)
> at 
> org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:3913)
> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3766)
> at 
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:409)
> at 
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:486)
> {code}
> So I deleted the whole index, re-create it and re-ran the job to send Hive 
> table contents to Solr again and it returned exactly the same exception the 
> first time after trying to send a lot of updates to Solr.
> I moved off HDFS to a normal dataDir backend and then re-indexed the full 
> table in 2 hours successfully without index corruptions.
> This implies that this is some sort of stability issue on the HDFS 
> DirectoryFactory implementation.
> Regards,
> Hari Sekhon
> http://www.linkedin.com/in/harisekhon



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8185) Add operations support to streaming metrics

[Dennis Gove (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003390#comment-15003390
 ] 

Dennis Gove commented on SOLR-8185:
---

Running into some issues turning the expression into something that would 
perform the expected .equals()

{code}
 avg(a_f, replace(10, withValue=0))
{code}
In this example, what type is 10? Is it a long or a float or a double? The 
field is a float (as noted by the _f) so one would expect 10 to be a float as 
well. However, in converting 10 to some Object that we can call .equals(...) on 
we are not sure what the type is. This has been a persistent problem with this 
patch.

But I think I've come up with something that puts some of the decision making 
in the hands of the expression writer.

{code}
 avg(a_f, replace(10f, withValue=0f))
{code}
In this case the value can only be converted to a float so it will be created 
as a float object.

However, to add this new requirement on the expression creator I want to take a 
deeper look at what this might impact and make sure the documentation is very 
clear. If a user doesn't do the correct thing (gives us 10 instead of 10f) and 
the value in the tuple is a float then float.equals(long) == false every single 
time.

Anyway, this note is somewhat of a rant. 

> Add operations support to streaming metrics
> ---
>
> Key: SOLR-8185
> URL: https://issues.apache.org/jira/browse/SOLR-8185
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrJ
>Reporter: Dennis Gove
>Assignee: Dennis Gove
>Priority: Minor
> Attachments: SOLR-8185.patch
>
>
> Adds support for operations on stream metrics.
> With this feature one can modify tuple values before applying to the computed 
> metric. There are a lot of use-cases I can see with this - I'll describe one 
> here.
> Imagine you have a RollupStream which is computing the average over some 
> field but you cannot be sure that all documents have a value for that field, 
> ie the value is null. When the value is null you want to treat it as a 0. 
> With this feature you can accomplish that like this
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0)),
>   count(*),
> )
> {code}
> The operations are applied to the tuple for each metric in the stream which 
> means you perform different operations on different metrics without being 
> impacted by operations on other metrics. 
> Adding to our previous example, imagine you want to also get the min of a 
> field but do not consider null values.
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0)),
>   min(a_i),
>   count(*),
> )
> {code}
> Also, the tuple is not modified for streams that might wrap this one. Ie, the 
> only thing that sees the applied operation is that particular metric. If you 
> want to apply operations for wrapping streams you can still achieve that with 
> the SelectStream (SOLR-7669).
> One feature I'm investigating but this patch DOES NOT add is the ability to 
> assign names to the resulting metric value. For example, to allow for 
> something like this
> {code}
> rollup(
>   search(collection1, q=*:*, fl=\"a_s,a_i,a_f\", sort=\"a_s asc\"),
>   over=\"a_s\",
>   avg(a_i, replace(null, withValue=0), as="avg_a_i_null_as_0"),
>   avg(a_i),
>   count(*, as="totalCount"),
> )
> {code}
> Right now that isn't possible because the identifier for each metric would be 
> the same "avg_a_i" and as such both couldn't be returned. It's relatively 
> easy to add but I have to investigate its impact on the SQL and FacetStream 
> areas.
> Depends on SOLR-7669 (SelectStream)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8273) deprecate implicitly uninverted fields, force people to either use docValues, or be explicit that they want query time uninversion

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003136#comment-15003136
 ] 

Yonik Seeley commented on SOLR-8273:


bq. Put a WARN-level log message in any code just before it does reinversion. 

Although people who are alerting on WARNINGs and do know what they are doing 
might not be so happy.

Also, I'm not sure we have the right hooks to do things just before 
uninversion... We ask for docValues from the FieldCache / UninvertingReader and 
if they exist they will be returned, and if not, the field cache entry will be 
built.

> deprecate implicitly uninverted fields, force people to either use docValues, 
> or be explicit that they want query time uninversion
> --
>
> Key: SOLR-8273
> URL: https://issues.apache.org/jira/browse/SOLR-8273
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>
> once upon a time, there was nothing we could do to *stop* people from using 
> the FieldCache - even if they didn't realize they were using it.
> Then DocValues was added - and now people have a choice: they can set 
> {{docValues=true}} on a field/fieldtype and know that when they do 
> functions/sorting/faceting on that field, it won't require a big hunk of ram 
> and a big stall everytime a reader was reopened.  But it's easy to overlook 
> when clients might be doing something that required the FieldCache w/o 
> realizing it -- and there is no way to stop them, because Solr automatically 
> uses UninvertingReader under the covers and automatically allows every field 
> to be uninverted in this way.
> we should change that.
> 
> Straw man proposal...
> * introduce a new boolean fieldType/field property {{uninvertable}}
> * all existing FieldType classes should default to {{uninvertable==false}}
> * a field or fieldType that contains {{indexed="false" uninvertable="true"}} 
> should be an error.
> * the Schema {{version}} value should be incremented, such that any Schema 
> with an older version is treated as if every field with {{docValues==false}} 
> has an implict {{uninvertable="true"}} on it.
> * the Map passed to UninvertedReader should now only list items that have an 
> effective value of {{uninvertable==true}}
> * sample schemas should be updated to use docValues on any field where the 
> examples using those schemas suggest using those fields in that way (ie: 
> sorting, faceting, etc...)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6889) BooleanQuery.rewrite could easily optimize some simple cases

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003151#comment-15003151
 ] 

Adrien Grand commented on LUCENE-6889:
--

bq. but it looks like the "actuallyRewritten" check from the recursive 
rewriting will return before several of the optimizations
bq. likewise isn't doing the "remove FILTER clauses that are also MUST clauses" 
optimization still worthwhile even if the "remove duplicate FILTER and MUST_NOT 
clauses" optimization finds & removes things? (it also looks like it has a 
short-circuit return)

It works fine because rewrite's contract is that it should be called until the 
return value is the same as the provided query. So for instance maybe one 
optimization will be performed on the first call, then another one, etc.

Other comments make sense to me, I'll update the patch.

> BooleanQuery.rewrite could easily optimize some simple cases
> 
>
> Key: LUCENE-6889
> URL: https://issues.apache.org/jira/browse/LUCENE-6889
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Attachments: LUCENE-6889.patch
>
>
> Follow-up of SOLR-8251: APIs and user interfaces sometimes encourage to write 
> BooleanQuery instances that are not optimal, for instance a typical case that 
> happens often with Solr/Elasticsearch is to send a request that has a 
> MatchAllDocsQuery as a query and some filter, which could be executed more 
> efficiently by directly wrapping the filter into a ConstantScoreQuery.
> Here are some ideas of rewrite operations that BooleanQuery could perform:
>  - remove FILTER clauses when they are also a MUST clause
>  - rewrite queries of the form "+*:* #filter" to a ConstantScoreQuery(filter)
>  - rewrite to a MatchNoDocsQuery when a clause that is a MUST or FILTER 
> clause is also a MUST_NOT clause



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8286) Remove instances of solr.hdfs.blockcache.write.enabled from tests and docs

[Gregory Chanan (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003307#comment-15003307
 ] 

Gregory Chanan commented on SOLR-8286:
--

Also: I didn't remove:
{code}  public static final String BLOCKCACHE_WRITE_ENABLED = 
"solr.hdfs.blockcache.write.enabled"; // currently buggy and disabled{code}

In case someone is depending on that.

> Remove instances of solr.hdfs.blockcache.write.enabled from tests and docs
> --
>
> Key: SOLR-8286
> URL: https://issues.apache.org/jira/browse/SOLR-8286
> Project: Solr
>  Issue Type: Bug
>  Components: documentation, Tests
>Affects Versions: 5.0
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-8286.patch
>
>
> solr.hdfs.blockcache.write.enabled is currently disabled whether or not you 
> set it in the solr configs.  It makes sense to just avoid mentioning it in 
> the docs (it's still there on 
> https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS) and 
> the morphlines examples.  Best case, it's unnecessary information.  Worse 
> case, it causes people trouble on versions where it's not disabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Solaris (64bit/jdk1.8.0) - Build # 183 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Solaris/183/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.security.PKIAuthenticationIntegrationTest.testPkiAuth

Error Message:
There are still nodes recoverying - waited for 10 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 10 
seconds
at 
__randomizedtesting.SeedInfo.seed([CA4757D498DE9F88:FAF9CE8578681829]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:170)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:836)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1392)
at 
org.apache.solr.security.PKIAuthenticationIntegrationTest.testPkiAuth(PKIAuthenticationIntegrationTest.java:51)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 

[jira] [Commented] (LUCENE-6894) Improve DISI.cost() by assuming independence for match probabilities

[Stefan Pohl (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001984#comment-15001984
 ] 

Stefan Pohl commented on LUCENE-6894:
-

There are very likely situations where you can still decrease query runtime 
even further with a different order of clauses than the one based on current 
worst-case estimates, and I agree that the naming 'cost()' doesn't really 
reflect the conservative estimates. However, any other non-worst-case estimate 
might err very badly and make queries that are currently reasonably fast 
extremely slow.

It comes down to trading in worst-case behavior to gain average/throughput, but 
usually people care more about the slowest/hardest queries. However, maybe we 
can have worst-case and other estimates too and choose to use the latter only 
in cases where even making the wrong decision won't be too bad, so that you're 
speculative on the fast queries to gain throughput, but conservative on 
potentially slow queries.

> Improve DISI.cost() by assuming independence for match probabilities
> 
>
> Key: LUCENE-6894
> URL: https://issues.apache.org/jira/browse/LUCENE-6894
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Paul Elschot
>Priority: Minor
> Attachments: LUCENE-6894.patch
>
>
> The DocIdSetIterator.cost() method returns an estimation of the number of 
> matching docs. Currently conjunctions use the minimum cost, and disjunctions 
> use the sum of the costs, and both are too high.
> The probability of a match is estimated by dividing available cost() by the 
> number of docs in a segment.
> The conjunction probability is then the product of the inputs, and the 
> disjunction probability follows from De Morgan's rule:
> "not (A and B)" is the same as "(not A) or (not B)"
> with the probability for "not" computed as 1 minus the input probability.
> The independence that is assumed is normally not there. However, the cost() 
> results are only used to order the input DISIs/Scorers for optimization, and 
> for that I expect this assumption to work nicely.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-6895) TestGeo3DPointField.testBasic() failure

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001988#comment-15001988
 ] 

ASF subversion and git services commented on LUCENE-6895:
-

Commit 1714021 from [~mikemccand] in branch 'dev/trunk'
[ https://svn.apache.org/r1714021 ]

LUCENE-6895: fix test bug

> TestGeo3DPointField.testBasic() failure
> ---
>
> Key: LUCENE-6895
> URL: https://issues.apache.org/jira/browse/LUCENE-6895
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial3d
>Affects Versions: Trunk
>Reporter: Steve Rowe
>
> {noformat}
>[junit4] Suite: org.apache.lucene.geo3d.TestGeo3DPointField
>[junit4] IGNOR/A 0.01s J5 | TestGeo3DPointField.testRandomBig
>[junit4]> Assumption #1: 'nightly' test group is disabled (@Nightly())
>[junit4]   2> NOTE: download the large Jenkins line-docs file by running 
> 'ant get-jenkins-line-docs' in the lucene directory.
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestGeo3DPointField -Dtests.method=testBasic 
> -Dtests.seed=F318BB6A659E2E19 -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=ar_DZ -Dtests.timezone=America/North_Dakota/New_Salem 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.01s J5 | TestGeo3DPointField.testBasic <<<
>[junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> maxMBSortInHeap=0.14646630717120193 only allows for maxPointsSortInHeap=1066, 
> but this is less than maxPointsInLeafNode=1736; either increase 
> maxMBSortInHeap or decrease maxPointsInLeafNode
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([F318BB6A659E2E19:58E2A67FBA42A837]:0)
>[junit4]>  at 
> org.apache.lucene.util.bkd.BKDWriter.(BKDWriter.java:153)
>[junit4]>  at 
> org.apache.lucene.codecs.lucene60.Lucene60DimensionalWriter.writeField(Lucene60DimensionalWriter.java:83)
>[junit4]>  at 
> org.apache.lucene.index.DimensionalValuesWriter.flush(DimensionalValuesWriter.java:68)
>[junit4]>  at 
> org.apache.lucene.index.DefaultIndexingChain.writeDimensionalValues(DefaultIndexingChain.java:146)
>[junit4]>  at 
> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96)
>[junit4]>  at 
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
>[junit4]>  at 
> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502)
>[junit4]>  at 
> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:614)
>[junit4]>  at 
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>[junit4]>  at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>[junit4]>  at 
> org.apache.lucene.geo3d.TestGeo3DPointField.testBasic(TestGeo3DPointField.java:112)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]   2> NOTE: test params are: codec=Lucene60, 
> sim=ClassicSimilarity, locale=ar_DZ, timezone=America/North_Dakota/New_Salem
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_45 (64-bit)/cpus=16,threads=1,free=402059552,total=521142272
>[junit4]   2> NOTE: All tests run in this JVM: [TestGeo3DPointField]
>[junit4] Completed [9/9] on J5 in 4.25s, 5 tests, 1 error, 1 skipped <<< 
> FAILURES!
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8263) Tlog replication could interfere with the replay of buffered updates

[Renaud Delbru (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Renaud Delbru updated SOLR-8263:

Attachment: SOLR-8263-trunk-1.patch

[~shalinmangar][~erickerickson] An initial first patch for this issue. It 
includes a unit test that was able to produce the describe issue, and an 
initial fix for the issue.
The index fetcher is now taking care of moving the buffered updates of the 
previous update log to the new one. During the move, the index fetcher is 
blocking updates to ensure that no buffered updates will be missed.

> Tlog replication could interfere with the replay of buffered updates
> 
>
> Key: SOLR-8263
> URL: https://issues.apache.org/jira/browse/SOLR-8263
> Project: Solr
>  Issue Type: Sub-task
>Reporter: Renaud Delbru
>Assignee: Erick Erickson
> Attachments: SOLR-8263-trunk-1.patch
>
>
> The current implementation of the tlog replication might interfere with the 
> replay of the buffered updates. The current tlog replication works as follow:
> 1) Fetch the the tlog files from the master
> 2) reset the update log before switching the tlog directory
> 3) switch the tlog directory and re-initialise the update log with the new 
> directory.
> Currently there is no logic to keep "buffered updates" while resetting and 
> reinitializing the update log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-4905) Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded collection that has a replica on all nodes

[Paul Blanchaert (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001977#comment-15001977
 ] 

Paul Blanchaert commented on SOLR-4905:
---

Today, I came across the same issue as reported by Mikhail Khludnev ("SolrCloud 
join: multiple shards not yet supported join").

I would expect that there is no issue to perform a cross-core join over sharded 
collections when the following conditions are met:
1) both collections are sharded with the same replicationFactor and numShards
2) router.field of the collection is set to the same "key-field"
3) the cross-core join is based on that same (from/to) "key-field"

The router.field setup ensures that documents with the same "key-field" are 
routed to the same node. 
So the combination based on the "key-field" should always be available within 
the same node.

>From a user perspective, I believe these assumptions seem to be the "normal" 
>use-case in the cross-core join.
So, when these assumptions are correct, the next question is probably how 
feasable it is to implement this logic in the JoinQParserPlugin?

Thanks for your feedback.

> Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded 
> collection that has a replica on all nodes
> --
>
> Key: SOLR-4905
> URL: https://issues.apache.org/jira/browse/SOLR-4905
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Philip K. Warren
>Assignee: Timothy Potter
> Fix For: 5.1, Trunk
>
> Attachments: SOLR-4905.patch, SOLR-4905.patch, patch.txt
>
>
> Using a non-SolrCloud setup, it is possible to perform cross core joins 
> (http://wiki.apache.org/solr/Join). When testing with SolrCloud, however, 
> neither the collection name, alias name (we have created aliases to SolrCloud 
> collections), or the automatically generated core name (i.e. 
> _shard1_replica1) work as the fromIndex parameter for a 
> cross-core join.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-8263) Tlog replication could interfere with the replay of buffered updates

[Renaud Delbru (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002018#comment-15002018
 ] 

Renaud Delbru edited comment on SOLR-8263 at 11/12/15 12:50 PM:


[~shalinmangar] [~erickerickson] An initial first patch for this issue. It 
includes a unit test that was able to reproduce the described issue, and an 
initial fix for the issue.
The index fetcher is now taking care of moving the buffered updates from the 
previous update log to the new one. During the move, the index fetcher is 
blocking updates to ensure that no buffered updates are missed.


was (Author: rendel):
[~shalinmangar][~erickerickson] An initial first patch for this issue. It 
includes a unit test that was able to produce the describe issue, and an 
initial fix for the issue.
The index fetcher is now taking care of moving the buffered updates of the 
previous update log to the new one. During the move, the index fetcher is 
blocking updates to ensure that no buffered updates will be missed.

> Tlog replication could interfere with the replay of buffered updates
> 
>
> Key: SOLR-8263
> URL: https://issues.apache.org/jira/browse/SOLR-8263
> Project: Solr
>  Issue Type: Sub-task
>Reporter: Renaud Delbru
>Assignee: Erick Erickson
> Attachments: SOLR-8263-trunk-1.patch
>
>
> The current implementation of the tlog replication might interfere with the 
> replay of the buffered updates. The current tlog replication works as follow:
> 1) Fetch the the tlog files from the master
> 2) reset the update log before switching the tlog directory
> 3) switch the tlog directory and re-initialise the update log with the new 
> directory.
> Currently there is no logic to keep "buffered updates" while resetting and 
> reinitializing the update log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-6874) WhitespaceTokenizer should tokenize on NBSP

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6874:
--
Attachment: LUCENE-6874-chartokenizer.patch

Adds more tests (random strings) and adds UnicodeWhitespaceAnalyzer (for 
consistency & to test).

> WhitespaceTokenizer should tokenize on NBSP
> ---
>
> Key: LUCENE-6874
> URL: https://issues.apache.org/jira/browse/LUCENE-6874
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: David Smiley
>Priority: Minor
> Attachments: LUCENE-6874-chartokenizer.patch, 
> LUCENE-6874-chartokenizer.patch, LUCENE-6874-jflex.patch, LUCENE-6874.patch, 
> LUCENE_6874_jflex.patch, icu-datasucker.patch, unicode-ws-tokenizer.patch, 
> unicode-ws-tokenizer.patch, unicode-ws-tokenizer.patch
>
>
> WhitespaceTokenizer uses [Character.isWhitespace 
> |http://docs.oracle.com/javase/8/docs/api/java/lang/Character.html#isWhitespace-int-]
>  to decide what is whitespace.  Here's a pertinent excerpt:
> bq. It is a Unicode space character (SPACE_SEPARATOR, LINE_SEPARATOR, or 
> PARAGRAPH_SEPARATOR) but is not also a non-breaking space ('\u00A0', 
> '\u2007', '\u202F')
> Perhaps Character.isWhitespace should have been called 
> isLineBreakableWhitespace?
> I think WhitespaceTokenizer should tokenize on this.  I am aware it's easy to 
> work around but why leave this trap in by default?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-6895) TestGeo3DPointField.testBasic() failure

[Michael McCandless (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael McCandless resolved LUCENE-6895.

   Resolution: Fixed
 Assignee: Michael McCandless
Fix Version/s: Trunk

Thanks [~sar...@syr.edu], I committed a fix (test bug).

> TestGeo3DPointField.testBasic() failure
> ---
>
> Key: LUCENE-6895
> URL: https://issues.apache.org/jira/browse/LUCENE-6895
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: modules/spatial3d
>Affects Versions: Trunk
>Reporter: Steve Rowe
>Assignee: Michael McCandless
> Fix For: Trunk
>
>
> {noformat}
>[junit4] Suite: org.apache.lucene.geo3d.TestGeo3DPointField
>[junit4] IGNOR/A 0.01s J5 | TestGeo3DPointField.testRandomBig
>[junit4]> Assumption #1: 'nightly' test group is disabled (@Nightly())
>[junit4]   2> NOTE: download the large Jenkins line-docs file by running 
> 'ant get-jenkins-line-docs' in the lucene directory.
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=TestGeo3DPointField -Dtests.method=testBasic 
> -Dtests.seed=F318BB6A659E2E19 -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=ar_DZ -Dtests.timezone=America/North_Dakota/New_Salem 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.01s J5 | TestGeo3DPointField.testBasic <<<
>[junit4]> Throwable #1: java.lang.IllegalArgumentException: 
> maxMBSortInHeap=0.14646630717120193 only allows for maxPointsSortInHeap=1066, 
> but this is less than maxPointsInLeafNode=1736; either increase 
> maxMBSortInHeap or decrease maxPointsInLeafNode
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([F318BB6A659E2E19:58E2A67FBA42A837]:0)
>[junit4]>  at 
> org.apache.lucene.util.bkd.BKDWriter.(BKDWriter.java:153)
>[junit4]>  at 
> org.apache.lucene.codecs.lucene60.Lucene60DimensionalWriter.writeField(Lucene60DimensionalWriter.java:83)
>[junit4]>  at 
> org.apache.lucene.index.DimensionalValuesWriter.flush(DimensionalValuesWriter.java:68)
>[junit4]>  at 
> org.apache.lucene.index.DefaultIndexingChain.writeDimensionalValues(DefaultIndexingChain.java:146)
>[junit4]>  at 
> org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:96)
>[junit4]>  at 
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:423)
>[junit4]>  at 
> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:502)
>[junit4]>  at 
> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:614)
>[junit4]>  at 
> org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:422)
>[junit4]>  at 
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:86)
>[junit4]>  at 
> org.apache.lucene.geo3d.TestGeo3DPointField.testBasic(TestGeo3DPointField.java:112)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]   2> NOTE: test params are: codec=Lucene60, 
> sim=ClassicSimilarity, locale=ar_DZ, timezone=America/North_Dakota/New_Salem
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_45 (64-bit)/cpus=16,threads=1,free=402059552,total=521142272
>[junit4]   2> NOTE: All tests run in this JVM: [TestGeo3DPointField]
>[junit4] Completed [9/9] on J5 in 4.25s, 5 tests, 1 error, 1 skipped <<< 
> FAILURES!
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_66) - Build # 5394 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/5394/
Java: 32bit/jdk1.8.0_66 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.core.TestConfigSets.testDefaultConfigSetBasePathResolution

Error Message:
 Expected: is <\path\to\solr\home\configsets>  got: 
 

Stack Trace:
java.lang.AssertionError: 
Expected: is <\path\to\solr\home\configsets>
 got: 

at 
__randomizedtesting.SeedInfo.seed([727CA5586C39897C:265A579411016070]:0)
at org.junit.Assert.assertThat(Assert.java:780)
at org.junit.Assert.assertThat(Assert.java:738)
at 
org.apache.solr.core.TestConfigSets.testDefaultConfigSetBasePathResolution(TestConfigSets.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:

[jira] [Updated] (LUCENE-6874) WhitespaceTokenizer should tokenize on NBSP

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler updated LUCENE-6874:
--
Attachment: LUCENE-6874-chartokenizer.patch

LUCENE-6874-chartokenizer.patch is now the merged patch:
- Contains the WhitespaceTokenizerFactory from [~dsmiley], but without 
maxTokenLength (was specific to jflex)
- Adds the tests by [~steve_rowe]
- The ICUWhitespaceTokenizer is gone (as [~rcmuir] said)
- Modifies the ALG file to exclude this tokenizer. I have no yet tested the ALG 
file. I also did no performance test up to now.

I like the new solution more than the Jflex-based tokenizer, because it is 
smaller and maybe also faster (not yet tested). Using JFlex for this is not 
really a good idea, because the tokenization algorithm is very simple. 
CharTokenizer is the better match for that, thoroughly tested.

> WhitespaceTokenizer should tokenize on NBSP
> ---
>
> Key: LUCENE-6874
> URL: https://issues.apache.org/jira/browse/LUCENE-6874
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>Reporter: David Smiley
>Priority: Minor
> Attachments: LUCENE-6874-chartokenizer.patch, 
> LUCENE-6874-jflex.patch, LUCENE-6874.patch, LUCENE_6874_jflex.patch, 
> icu-datasucker.patch, unicode-ws-tokenizer.patch, unicode-ws-tokenizer.patch, 
> unicode-ws-tokenizer.patch
>
>
> WhitespaceTokenizer uses [Character.isWhitespace 
> |http://docs.oracle.com/javase/8/docs/api/java/lang/Character.html#isWhitespace-int-]
>  to decide what is whitespace.  Here's a pertinent excerpt:
> bq. It is a Unicode space character (SPACE_SEPARATOR, LINE_SEPARATOR, or 
> PARAGRAPH_SEPARATOR) but is not also a non-breaking space ('\u00A0', 
> '\u2007', '\u202F')
> Perhaps Character.isWhitespace should have been called 
> isLineBreakableWhitespace?
> I think WhitespaceTokenizer should tokenize on this.  I am aware it's easy to 
> work around but why leave this trap in by default?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-trunk-Linux (32bit/jdk1.9.0-ea-b90) - Build # 14889 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/14889/
Java: 32bit/jdk1.9.0-ea-b90 -client -XX:+UseConcMarkSweepGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest

Error Message:
5 threads leaked from SUITE scope at 
org.apache.solr.cloud.SaslZkACLProviderTest: 1) Thread[id=4063, 
name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
 at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:218) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2103)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1136)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:853)
 at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1083)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1143) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632) 
at java.lang.Thread.run(Thread.java:747)2) Thread[id=4062, 
name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest] at 
java.lang.Object.wait(Native Method) at 
java.lang.Object.wait(Object.java:516) at 
java.util.TimerThread.mainLoop(Timer.java:526) at 
java.util.TimerThread.run(Timer.java:505)3) Thread[id=4066, 
name=changePwdReplayCache.data, state=TIMED_WAITING, 
group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native 
Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:218) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2103)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1136)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:853)
 at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1083)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1143) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632) 
at java.lang.Thread.run(Thread.java:747)4) Thread[id=4064, 
name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest] 
at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:218) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2103)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1136)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:853)
 at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1083)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1143) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632) 
at java.lang.Thread.run(Thread.java:747)5) Thread[id=4065, 
name=kdcReplayCache.data, state=TIMED_WAITING, 
group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native 
Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:218) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2103)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1136)
 at 
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:853)
 at 
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1083)   
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1143) 
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:632) 
at java.lang.Thread.run(Thread.java:747)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE 
scope at org.apache.solr.cloud.SaslZkACLProviderTest: 
   1) Thread[id=4063, name=groupCache.data, state=TIMED_WAITING, 
group=TGRP-SaslZkACLProviderTest]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:218)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2103)
at 

[jira] [Commented] (SOLR-4905) Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded collection that has a replica on all nodes

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15003657#comment-15003657
 ] 

Mikhail Khludnev commented on SOLR-4905:


[~solrtrey] responded to the list, reposting here
bq. Just to add another voice to the discussion, I have the exact same use case 
described by Paul and Mikhail that I'm working through a Proof of Concept for 
right now. I'd very much like to see the "single shard collection with a 
replica on all nodes" restriction removed.

> Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded 
> collection that has a replica on all nodes
> --
>
> Key: SOLR-4905
> URL: https://issues.apache.org/jira/browse/SOLR-4905
> Project: Solr
>  Issue Type: Improvement
>  Components: SolrCloud
>Reporter: Philip K. Warren
>Assignee: Timothy Potter
> Fix For: 5.1, Trunk
>
> Attachments: SOLR-4905.patch, SOLR-4905.patch, patch.txt
>
>
> Using a non-SolrCloud setup, it is possible to perform cross core joins 
> (http://wiki.apache.org/solr/Join). When testing with SolrCloud, however, 
> neither the collection name, alias name (we have created aliases to SolrCloud 
> collections), or the automatically generated core name (i.e. 
> _shard1_replica1) work as the fromIndex parameter for a 
> cross-core join.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8287) TrieLongField and TrieDoubleField should override toNativeType

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-8287:
---
Attachment: SOLR-8287.patch

This patch adds the overridden methods to the field types.

> TrieLongField and TrieDoubleField should override toNativeType
> --
>
> Key: SOLR-8287
> URL: https://issues.apache.org/jira/browse/SOLR-8287
> Project: Solr
>  Issue Type: Bug
>Reporter: Ishan Chattopadhyaya
> Attachments: SOLR-8287.patch
>
>
> Although the TrieIntField and TrieFloatField override the toNativeType() 
> method, the TrieLongField and TrieDoubleField do not do so. 
> This method is called during atomic updates by the AtomicUpdateDocumentMerger 
> for the "set" operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Solaris (multiarch/jdk1.7.0) - Build # 185 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Solaris/185/
Java: multiarch/jdk1.7.0 -d32 -server -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.update.AutoCommitTest.testMaxDocs

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([215FAC1F573C2E63:98DE7AC07BD62AE9]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:766)
at 
org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:194)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound=1]
xml response was: 

00


request was:start=0=standard=2.2=20=id:14
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:759)
... 40 more




Build Log:
[...truncated 10190 lines...]
   [junit4] Suite: org.apache.solr.update.AutoCommitTest
   [junit4]   

[jira] [Created] (SOLR-8287) TrieLongField and TrieDoubleField should override toNativeType

[Ishan Chattopadhyaya (JIRA)]

Ishan Chattopadhyaya created SOLR-8287:
--

 Summary: TrieLongField and TrieDoubleField should override 
toNativeType
 Key: SOLR-8287
 URL: https://issues.apache.org/jira/browse/SOLR-8287
 Project: Solr
  Issue Type: Bug
Reporter: Ishan Chattopadhyaya


Although the TrieIntField and TrieFloatField override the toNativeType() 
method, the TrieLongField and TrieDoubleField do not do so. 

This method is called during atomic updates by the AtomicUpdateDocumentMerger 
for the "set" operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Do we really need all the unused Solr analyzer stacks in multiple examples?

[Alexandre Rafalovitch]

Triggered by:
https://github.com/apache/lucene-solr/blob/lucene_solr_5_3_1/solr/example/example-DIH/solr/tika/conf/schema.xml#L658
vs
https://github.com/apache/lucene-solr/blob/lucene_solr_5_3_1/solr/example/example-DIH/solr/db/conf/schema.xml#L851

One of them has an extra Filter, neither of them are actually used in
the example itself.

Given that we ship ALL of the examples together anyway, would it make
sense to have one Uber example and the rest only keeping what they
use. It is really hard to navigate nearly 2000 lines schema.xml and
the duplicates and discrepancies are creeping in.

This would reduce the number of referenced mostly-empty files those
unused stacks use as well.

Regards,
   Alex.


Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter:
http://www.solr-start.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.7.0_80) - Build # 14597 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/14597/
Java: 64bit/jdk1.7.0_80 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.search.join.TestScoreJoinQPNoScore.testJoin

Error Message:
mismatch: '1'!='5' @ response/docs/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: '1'!='5' @ response/docs/[0]/id
at 
__randomizedtesting.SeedInfo.seed([1E082395A7F0F0F9:233B161FC6A7F76F]:0)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:854)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:801)
at 
org.apache.solr.search.join.TestScoreJoinQPNoScore.testJoin(TestScoreJoinQPNoScore.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 9824 lines...]
   [junit4] Suite: org.apache.solr.search.join.TestScoreJoinQPNoScore
   [junit4]   2> Creating dataDir: 

[jira] [Created] (SOLR-8282) Move Solr internal APIs to NIO2, and ban use of java.io.File

[Alan Woodward (JIRA)]

Alan Woodward created SOLR-8282:
---

 Summary: Move Solr internal APIs to NIO2, and ban use of 
java.io.File
 Key: SOLR-8282
 URL: https://issues.apache.org/jira/browse/SOLR-8282
 Project: Solr
  Issue Type: Improvement
Reporter: Alan Woodward


This is an umbrella issue for removing all usage of java.io.File from Solr, and 
replacing it with NIO2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8260) Use NIO2 APIs in core discovery

[Kevin Risden (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002074#comment-15002074
 ] 

Kevin Risden commented on SOLR-8260:


[~romseygeek] I see that you are going through and fixing a bunch to nio from 
java.io. Is there a JIRA outlining what still needs to be done?

> Use NIO2 APIs in core discovery
> ---
>
> Key: SOLR-8260
> URL: https://issues.apache.org/jira/browse/SOLR-8260
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: 5.4
>
> Attachments: SOLR-8260.patch
>
>
> CorePropertiesLocator currently does all its file system interaction using 
> java.io.File and friends, which have all sorts of drawbacks with regard to 
> error handling and reporting.  We've been on java 7 for a while now, so we 
> should use the nio2 Path APIs instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-trunk - Build # 850 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-trunk/850/

2 tests failed.
FAILED:  org.apache.lucene.index.TestIndexWriterDelete.testDeletesOnDiskFull

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([1B924C7595D9712]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.index.TestIndexWriterDelete

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([1B924C7595D9712]:0)




Build Log:
[...truncated 1875 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDelete
   [junit4]   2> Nov 13, 2015 12:31:14 AM 
com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
   [junit4]   2> WARNING: Suite execution timed out: 
org.apache.lucene.index.TestIndexWriterDelete
   [junit4]   2>1) Thread[id=9, name=JUnit4-serializer-daemon, 
state=TIMED_WAITING, group=main]
   [junit4]   2> at java.lang.Thread.sleep(Native Method)
   [junit4]   2> at 
com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:47)
   [junit4]   2>2) Thread[id=11173, 
name=SUITE-TestIndexWriterDelete-seed#[1B924C7595D9712], state=RUNNABLE, 
group=TGRP-TestIndexWriterDelete]
   [junit4]   2> at java.lang.Thread.getStackTrace(Thread.java:1552)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:688)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:685)
   [junit4]   2> at java.security.AccessController.doPrivileged(Native 
Method)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:685)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:701)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:681)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:64)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:414)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:141)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:588)
   [junit4]   2>3) Thread[id=11174, 
name=TEST-TestIndexWriterDelete.testDeletesOnDiskFull-seed#[1B924C7595D9712], 
state=TIMED_WAITING, group=TGRP-TestIndexWriterDelete]
   [junit4]   2> at java.lang.Thread.sleep(Native Method)
   [junit4]   2> at 
org.apache.lucene.store.SlowClosingMockIndexInputWrapper.close(SlowClosingMockIndexInputWrapper.java:40)
   [junit4]   2> at 
org.apache.lucene.index.IndexWriter.slowFileExists(IndexWriter.java:4771)
   [junit4]   2> at 
org.apache.lucene.index.IndexWriter.filesExist(IndexWriter.java:4312)
   [junit4]   2> at 
org.apache.lucene.index.IndexWriter.(IndexWriter.java:940)
   [junit4]   2> at 
org.apache.lucene.index.TestIndexWriterDelete.doTestOperationsOnDiskFull(TestIndexWriterDelete.java:536)
   [junit4]   2> at 
org.apache.lucene.index.TestIndexWriterDelete.testDeletesOnDiskFull(TestIndexWriterDelete.java:482)
   [junit4]   2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
   [junit4]   2> at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
   [junit4]   2> at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
   [junit4]   2> at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> at 

[jira] [Commented] (SOLR-8260) Use NIO2 APIs in core discovery

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002079#comment-15002079
 ] 

Alan Woodward commented on SOLR-8260:
-

There isn't, but an overarching JIRA ticket isn't a bad idea.  Will open one up!

> Use NIO2 APIs in core discovery
> ---
>
> Key: SOLR-8260
> URL: https://issues.apache.org/jira/browse/SOLR-8260
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: 5.4
>
> Attachments: SOLR-8260.patch
>
>
> CorePropertiesLocator currently does all its file system interaction using 
> java.io.File and friends, which have all sorts of drawbacks with regard to 
> error handling and reporting.  We've been on java 7 for a while now, so we 
> should use the nio2 Path APIs instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8282) Move Solr internal APIs to NIO2, and ban use of java.io.File

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002125#comment-15002125
 ] 

Alan Woodward commented on SOLR-8282:
-

There are a whole bunch of smallish steps that can be taken here (SOLR-8260 and 
SOLR-8278 are the first ones).  I think it's important to try and do this 
piece-wise, as changing everything at once ends up with massive patches that 
are difficult to review.

I see a number of steps here:
* Firstly, try and move our internal APIs that deal with file paths away from 
using Strings to represent those paths, using Path instead.  For example, 
SolrResourceLoader takes a String to point to its instance directory, as does 
CoreDescriptor.  I have a half-done patch for SRL already, which I will open a 
separate ticket for.  SolrCore.getDataDir() and CoreContainer.getSolrHome() are 
also candidates here.
* Ban use of java.io.File in core, and try and load everything either through 
SolrResourceLoader or via SolrCore.getDataDir().
* Ban use of java.io.File in core tests.  This will likely take the longest.
* Ban use of Paths.get() in core and core tests, expect for a few places (eg 
SolrDispatchFilter).  This will mean that all file system resolution is done 
via SolrResourceLoader, and we can try and do interesting things like use the 
lucene exception-throwing file systems to test resilience.


> Move Solr internal APIs to NIO2, and ban use of java.io.File
> 
>
> Key: SOLR-8282
> URL: https://issues.apache.org/jira/browse/SOLR-8282
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>
> This is an umbrella issue for removing all usage of java.io.File from Solr, 
> and replacing it with NIO2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8278) ConfigSetService should use NIO2

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002055#comment-15002055
 ] 

ASF subversion and git services commented on SOLR-8278:
---

Commit 1714042 from [~romseygeek] in branch 'dev/branches/branch_5x'
[ https://svn.apache.org/r1714042 ]

SOLR-8278: Fix Windows test bug

> ConfigSetService should use NIO2 
> -
>
> Key: SOLR-8278
> URL: https://issues.apache.org/jira/browse/SOLR-8278
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: 5.4
>
> Attachments: SOLR-8278.patch
>
>
> Similar to SOLR-8260, using java.nio.file.Path instead of String and File 
> makes error handling and reporting a lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8278) ConfigSetService should use NIO2

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002054#comment-15002054
 ] 

ASF subversion and git services commented on SOLR-8278:
---

Commit 1714041 from [~romseygeek] in branch 'dev/trunk'
[ https://svn.apache.org/r1714041 ]

SOLR-8278: Fix Windows test bug

> ConfigSetService should use NIO2 
> -
>
> Key: SOLR-8278
> URL: https://issues.apache.org/jira/browse/SOLR-8278
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: 5.4
>
> Attachments: SOLR-8278.patch
>
>
> Similar to SOLR-8260, using java.nio.file.Path instead of String and File 
> makes error handling and reporting a lot easier.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Windows (32bit/jdk1.8.0_66) - Build # 5264 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5264/
Java: 32bit/jdk1.8.0_66 -client -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.core.TestConfigSets.testDefaultConfigSetBasePathResolution

Error Message:
 Expected: is <\path\to\solr\home\configsets>  got: 
 

Stack Trace:
java.lang.AssertionError: 
Expected: is <\path\to\solr\home\configsets>
 got: 

at 
__randomizedtesting.SeedInfo.seed([FEA42BECCEE4968B:AA82D920B3DC7F87]:0)
at org.junit.Assert.assertThat(Assert.java:780)
at org.junit.Assert.assertThat(Assert.java:738)
at 
org.apache.solr.core.TestConfigSets.testDefaultConfigSetBasePathResolution(TestConfigSets.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 

[jira] [Commented] (LUCENE-6276) Add matchCost() api to TwoPhaseDocIdSetIterator

[Paul Elschot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002694#comment-15002694
 ] 

Paul Elschot commented on LUCENE-6276:
--

The test failure reproduces here. I'll take a look, thanks.

> Add matchCost() api to TwoPhaseDocIdSetIterator
> ---
>
> Key: LUCENE-6276
> URL: https://issues.apache.org/jira/browse/LUCENE-6276
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Robert Muir
> Attachments: LUCENE-6276-ExactPhraseOnly.patch, 
> LUCENE-6276-NoSpans.patch, LUCENE-6276-NoSpans2.patch, LUCENE-6276.patch, 
> LUCENE-6276.patch, LUCENE-6276.patch, LUCENE-6276.patch, LUCENE-6276.patch, 
> LUCENE-6276.patch, LUCENE-6276.patch
>
>
> We could add a method like TwoPhaseDISI.matchCost() defined as something like 
> estimate of nanoseconds or similar. 
> ConjunctionScorer could use this method to sort its 'twoPhaseIterators' array 
> so that cheaper ones are called first. Today it has no idea if one scorer is 
> a simple phrase scorer on a short field vs another that might do some geo 
> calculation or more expensive stuff.
> PhraseScorers could implement this based on index statistics (e.g. 
> totalTermFreq/maxDoc)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-5.x-Linux (32bit/jdk1.7.0_80) - Build # 14592 - Still Failing!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/14592/
Java: 32bit/jdk1.7.0_80 -server -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:60500/collection1: Bad Request
request: 
http://127.0.0.1:49011/collection1/update?update.chain=distrib-dup-test-chain-explicit=TOLEADER=http%3A%2F%2F127.0.0.1%3A60500%2Fcollection1%2F=javabin=2

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:60500/collection1: Bad Request



request: 
http://127.0.0.1:49011/collection1/update?update.chain=distrib-dup-test-chain-explicit=TOLEADER=http%3A%2F%2F127.0.0.1%3A60500%2Fcollection1%2F=javabin=2
at 
__randomizedtesting.SeedInfo.seed([B121E995DA3447C8:3975D64F74C82A30]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:575)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:150)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:167)
at 
org.apache.solr.BaseDistributedSearchTestCase.add(BaseDistributedSearchTestCase.java:511)
at 
org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:640)
at 
org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:351)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[JENKINS] Lucene-Solr-trunk-Solaris (64bit/jdk1.8.0) - Build # 185 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Solaris/185/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.security.PKIAuthenticationIntegrationTest.testPkiAuth

Error Message:
No match for authorization/class = 
org.apache.solr.security.MockAuthorizationPlugin, full response = {   
"responseHeader":{ "status":0, "QTime":0},   "errorMessages":["No 
authorization configured"]} 

Stack Trace:
java.lang.AssertionError: No match for authorization/class = 
org.apache.solr.security.MockAuthorizationPlugin, full response = {
  "responseHeader":{
"status":0,
"QTime":0},
  "errorMessages":["No authorization configured"]}

at 
__randomizedtesting.SeedInfo.seed([35E4B0F18D24BC07:55A29A06D923BA6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.security.TestAuthorizationFramework.verifySecurityStatus(TestAuthorizationFramework.java:107)
at 
org.apache.solr.security.PKIAuthenticationIntegrationTest.testPkiAuth(PKIAuthenticationIntegrationTest.java:63)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1660)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:866)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:902)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:916)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:963)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:938)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:875)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:777)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:822)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at 

[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_66) - Build # 14887 - Failure!

[Policeman Jenkins Server]

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/14887/
Java: 32bit/jdk1.8.0_66 -client -XX:+UseParallelGC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestCloudPivotFacet

Error Message:
12 threads leaked from SUITE scope at 
org.apache.solr.cloud.TestCloudPivotFacet: 1) Thread[id=10145, 
name=qtp22576580-10145, state=BLOCKED, group=TGRP-TestCloudPivotFacet] 
at 
org.apache.lucene.uninverting.FieldCacheImpl$Cache.get(FieldCacheImpl.java:170) 
at 
org.apache.lucene.uninverting.FieldCacheImpl.getDocTermOrds(FieldCacheImpl.java:923)
 at 
org.apache.lucene.uninverting.UninvertingReader.getSortedSetDocValues(UninvertingReader.java:273)
 at 
org.apache.lucene.index.FilterLeafReader.getSortedSetDocValues(FilterLeafReader.java:459)
 at 
org.apache.lucene.index.SlowCompositeReaderWrapper.getSortedSetDocValues(SlowCompositeReaderWrapper.java:185)
 at 
org.apache.solr.request.DocValuesStats.getCounts(DocValuesStats.java:87)
 at 
org.apache.solr.handler.component.StatsField.computeLocalStatsValues(StatsField.java:424)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.doPivots(PivotFacetProcessor.java:328)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.doPivots(PivotFacetProcessor.java:322)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.doPivots(PivotFacetProcessor.java:322)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.processSingle(PivotFacetProcessor.java:228)
 at 
org.apache.solr.handler.component.PivotFacetProcessor.process(PivotFacetProcessor.java:170)
 at 
org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:262)
 at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:276)
 at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
 at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079) 
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:667) 
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460) at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:220)
 at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
 at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:109)
 at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
 at 
org.eclipse.jetty.servlets.UserAgentFilter.doFilter(UserAgentFilter.java:83)
 at org.eclipse.jetty.servlets.GzipFilter.doFilter(GzipFilter.java:300) 
at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
 at 
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)  
   at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
 at 
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)   
  at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
 at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
 at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)   
  at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)  
   at org.eclipse.jetty.server.Server.handle(Server.java:499) at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) 
at 
org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)  
   at 
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
 at 
org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) 
at java.lang.Thread.run(Thread.java:745)2) Thread[id=10230, 
name=qtp12050735-10230, state=RUNNABLE, group=TGRP-TestCloudPivotFacet] 
at java.lang.Object.clone(Native Method) at 
org.apache.lucene.store.DataInput.clone(DataInput.java:253) at 
org.apache.lucene.store.IndexInput.clone(IndexInput.java:93) at 
org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:78)
 at 
org.apache.lucene.store.MockIndexInputWrapper.clone(MockIndexInputWrapper.java:30)
 at 
org.apache.lucene.codecs.blocktree.SegmentTermsEnum.initIndexInput(SegmentTermsEnum.java:115)
 at 
org.apache.lucene.codecs.blocktree.SegmentTermsEnumFrame.loadBlock(SegmentTermsEnumFrame.java:148)
 at 

[jira] [Commented] (SOLR-7989) Down replica elected leader, stays down after successful election

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002241#comment-15002241
 ] 

Ishan Chattopadhyaya commented on SOLR-7989:


bq.  I don't think this is a valid test at all. Why does it clear LIR while 
Solr is running? That is not a valid real world scenario.
Indeed, that's why I didn't include it in the patch. I just wanted to bring out 
that there is a problem, but then I couldn't replicate it in a test in any 
other way that represents a real world situation. However, just that fact that 
there were two messages passed as one (LEADER message had a state param) 
indicated to me that we should, in theory, split them out.

+1 to your patch, I hadn't considered the core registration scenario. Also, the 
change from only DOWN to ACTIVE makes sense, and excluding the RECOVERING to 
ACTIVE change.

> Down replica elected leader, stays down after successful election
> -
>
> Key: SOLR-7989
> URL: https://issues.apache.org/jira/browse/SOLR-7989
> Project: Solr
>  Issue Type: Bug
>Reporter: Ishan Chattopadhyaya
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: DownLeaderTest.java, DownLeaderTest.java, 
> SOLR-7989.patch, SOLR-7989.patch, SOLR-7989.patch, SOLR-7989.patch, 
> SOLR-7989.patch, SOLR-7989.patch, SOLR-7989.patch, SOLR-8233.patch
>
>
> It is possible that a down replica gets elected as a leader, and that it 
> stays down after the election.
> Here's how I hit upon this:
> * There are 3 replicas: leader, notleader0, notleader1
> * Introduced network partition to isolate notleader0, notleader1 from leader 
> (leader puts these two in LIR via zk).
> * Kill leader, remove partition. Now leader is dead, and both of notleader0 
> and notleader1 are down. There is no leader.
> * Remove LIR znodes in zk.
> * Wait a while, and there happens a (flawed?) leader election.
> * Finally, the state is such that one of notleader0 or notleader1 (which were 
> down before) become leader, but stays down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-6874) WhitespaceTokenizer should tokenize on NBSP

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002248#comment-15002248
 ] 

Uwe Schindler edited comment on LUCENE-6874 at 11/12/15 3:39 PM:
-

Here is the output of the reuters test:

{noformat}
> Report Sum By (any) Name and Round (28 about 33 out of 34)
Operationround   runCnt   
recsPerRunrec/s  elapsedSecavgUsedMemavgTotalMem
AnalyzerFactory(name:WhitespaceTokenizer,WhitespaceTokenizer(rule:java))
   010 0.000.00 9,569,344
124,256,256
AnalyzerFactory(name:UnicodeWhitespaceTokenizer,WhitespaceTokenizer(rule:unicode))
 -   0 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -   9,569,344 -  
124,256,256
Rounds_5  01 
24493540   360,841.19   67.8816,566,472124,256,256
NewAnalyzer(WhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 -  - 0 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -   9,569,344 -  
124,256,256
[Character.isWhitespace()] WhitespaceTokenizer  
   01  2449354   331,038.537.4022,121,256
124,256,256
Seq_2 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 0 -  -   2 -  - 
2449354 - 344,131.22 -  -  14.23 -  22,121,256 -  118,489,088
NewAnalyzer(UnicodeWhitespaceTokenizer) 
   010 0.000.0022,121,256
112,721,920
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer -  -  -  -  -  -  -  
-  -   0 -  -   1 -  - 2449354 - 358,302.22 -  -   6.84 -  22,121,256 -  
112,721,920
NewAnalyzer(WhitespaceTokenizer)
  110 0.000.0012,138,024
112,721,920
[Character.isWhitespace()] WhitespaceTokenizer -  -  -  -  -  -  -  -  -  -  -  
-  -   1 -  -   1 -  - 2449354 - 366,724.66 -  -   6.68 -  22,374,536 -  
112,721,920
Seq_2  12  
2449354   365,139.25   13.4227,477,352117,702,656
NewAnalyzer(UnicodeWhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  
-  -  - 1 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -  22,374,536 -  
111,673,344
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer  
   11  2449354   363,567.476.7432,580,168
122,683,392
NewAnalyzer(WhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 -  - 2 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -  32,580,168 -  
122,683,392
[Character.isWhitespace()] WhitespaceTokenizer  
   21  2449354   365,793.596.7033,461,280
122,683,392
Seq_2 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 2 -  -   2 -  - 
2449354 - 365,112.03 -  -  13.42 -  33,461,280 -  117,178,368
NewAnalyzer(UnicodeWhitespaceTokenizer) 
   210 0.000.0033,461,280
111,673,344
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer -  -  -  -  -  -  -  
-  -   2 -  -   1 -  - 2449354 - 364,432.97 -  -   6.72 -  33,461,280 -  
111,673,344
NewAnalyzer(WhitespaceTokenizer)
  310 0.000.0010,836,464
111,673,344
[Character.isWhitespace()] WhitespaceTokenizer -  -  -  -  -  -  -  -  -  -  -  
-  -   3 -  -   1 -  - 2449354 - 367,660.47 -  -   6.66 -  12,451,400 -  
111,673,344
Seq_2  32  
2449354   365,820.94   13.3913,235,672111,673,344
NewAnalyzer(UnicodeWhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  
-  -  - 3 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -  12,451,400 -  
111,673,344
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer  
   31  2449354   363,999.696.7314,019,944
111,673,344
NewAnalyzer(WhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 -  - 4 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -  14,019,944 -  
111,673,344
[Character.isWhitespace()] WhitespaceTokenizer  
   41  2449354   367,329.626.6715,061,368
111,673,344
Seq_2 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 4 -  -   2 -  - 
2449354 - 365,057.59 -  -  13.42 -  15,813,920 -  111,673,344
NewAnalyzer(UnicodeWhitespaceTokenizer) 
   410 0.000.0015,061,368
111,673,344
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer -  -  -  -  -  -  -  
-  - 

[jira] [Commented] (LUCENE-6874) WhitespaceTokenizer should tokenize on NBSP

[Uwe Schindler (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002248#comment-15002248
 ] 

Uwe Schindler commented on LUCENE-6874:
---

Here is the output of the reuters test:

{noformat}
> Report Sum By (any) Name and Round (28 about 33 out of 34)
Operationround   runCnt   
recsPerRunrec/s  elapsedSecavgUsedMemavgTotalMem
AnalyzerFactory(name:WhitespaceTokenizer,WhitespaceTokenizer(rule:java))
   010 0.000.00 9,569,344
124,256,256
AnalyzerFactory(name:UnicodeWhitespaceTokenizer,WhitespaceTokenizer(rule:unicode))
 -   0 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -   9,569,344 -  
124,256,256
Rounds_5  01 
24493540   360,841.19   67.8816,566,472124,256,256
NewAnalyzer(WhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 -  - 0 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -   9,569,344 -  
124,256,256
[Character.isWhitespace()] WhitespaceTokenizer  
   01  2449354   331,038.537.4022,121,256
124,256,256
Seq_2 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 0 -  -   2 -  - 
2449354 - 344,131.22 -  -  14.23 -  22,121,256 -  118,489,088
NewAnalyzer(UnicodeWhitespaceTokenizer) 
   010 0.000.0022,121,256
112,721,920
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer -  -  -  -  -  -  -  
-  -   0 -  -   1 -  - 2449354 - 358,302.22 -  -   6.84 -  22,121,256 -  
112,721,920
NewAnalyzer(WhitespaceTokenizer)
  110 0.000.0012,138,024
112,721,920
[Character.isWhitespace()] WhitespaceTokenizer -  -  -  -  -  -  -  -  -  -  -  
-  -   1 -  -   1 -  - 2449354 - 366,724.66 -  -   6.68 -  22,374,536 -  
112,721,920
Seq_2  12  
2449354   365,139.25   13.4227,477,352117,702,656
NewAnalyzer(UnicodeWhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  
-  -  - 1 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -  22,374,536 -  
111,673,344
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer  
   11  2449354   363,567.476.7432,580,168
122,683,392
NewAnalyzer(WhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 -  - 2 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -  32,580,168 -  
122,683,392
[Character.isWhitespace()] WhitespaceTokenizer  
   21  2449354   365,793.596.7033,461,280
122,683,392
Seq_2 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 2 -  -   2 -  - 
2449354 - 365,112.03 -  -  13.42 -  33,461,280 -  117,178,368
NewAnalyzer(UnicodeWhitespaceTokenizer) 
   210 0.000.0033,461,280
111,673,344
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer -  -  -  -  -  -  -  
-  -   2 -  -   1 -  - 2449354 - 364,432.97 -  -   6.72 -  33,461,280 -  
111,673,344
NewAnalyzer(WhitespaceTokenizer)
  310 0.000.0010,836,464
111,673,344
[Character.isWhitespace()] WhitespaceTokenizer -  -  -  -  -  -  -  -  -  -  -  
-  -   3 -  -   1 -  - 2449354 - 367,660.47 -  -   6.66 -  12,451,400 -  
111,673,344
Seq_2  32  
2449354   365,820.94   13.3913,235,672111,673,344
NewAnalyzer(UnicodeWhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  
-  -  - 3 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -  12,451,400 -  
111,673,344
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer  
   31  2449354   363,999.696.7314,019,944
111,673,344
NewAnalyzer(WhitespaceTokenizer) -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 
 -  - 4 -  -   1 -  -  -  - 0 -  -  - 0.00 -  -   0.00 -  14,019,944 -  
111,673,344
[Character.isWhitespace()] WhitespaceTokenizer  
   41  2449354   367,329.626.6715,061,368
111,673,344
Seq_2 -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  -  - 4 -  -   2 -  - 
2449354 - 365,057.59 -  -  13.42 -  15,813,920 -  111,673,344
NewAnalyzer(UnicodeWhitespaceTokenizer) 
   410 0.000.0015,061,368
111,673,344
[UnicodeProps.WHITESPACE.get()] UnicodeWhitespaceTokenizer -  -  -  -  -  -  -  
-  -   4 -  -   1 -  - 2449354 - 362,813.50 -  -   6.75 

[jira] [Commented] (SOLR-7569) Create an API to force a leader election between nodes

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002251#comment-15002251
 ] 

Ishan Chattopadhyaya commented on SOLR-7569:


bq. I've taken a crack at making SOLR-7989 work.
Thanks!

bq. Perhaps the last thing the API should do is run through each shard and see 
if the registered leader is DOWN, and if it is make it ACTIVE (preferably by 
asking it to publish itself as ACTIVE - we don't want to publish for someone 
else). If the call waits around to make sure all the leaders come up, this 
should be simple.
This makes sense. I think this is something that Shalin alluded to (please 
excuse me if I'm mistaken) when he said, {{1. Leader is live but 'down' -> mark 
it 'active'}}. The suggestion for the replicas to mark themselves ACTIVE 
instead of someone else marking them down seems like a good thing to do.

> Create an API to force a leader election between nodes
> --
>
> Key: SOLR-7569
> URL: https://issues.apache.org/jira/browse/SOLR-7569
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
>  Labels: difficulty-medium, impact-high
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-7569-testfix.patch, SOLR-7569.patch, 
> SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, 
> SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, 
> SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, 
> SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, 
> SOLR-7569_lir_down_state_test.patch
>
>
> There are many reasons why Solr will not elect a leader for a shard e.g. all 
> replicas' last published state was recovery or due to bugs which cause a 
> leader to be marked as 'down'. While the best solution is that they never get 
> into this state, we need a manual way to fix this when it does get into this  
> state. Right now we can do a series of dance involving bouncing the node 
> (since recovery paths between bouncing and REQUESTRECOVERY are different), 
> but that is difficult when running a large cluster. Although it is possible 
> that such a manual API may lead to some data loss but in some cases, it is 
> the only possible option to restore availability.
> This issue proposes to build a new collection API which can be used to force 
> replicas into recovering a leader while avoiding data loss on a best effort 
> basis.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-6406) ConcurrentUpdateSolrServer hang in blockUntilFinished.

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15001429#comment-15001429
 ] 

Yonik Seeley edited comment on SOLR-6406 at 11/12/15 2:54 PM:
--

I was analyzing another "shards-out-of-sync" failure on trunk.
It looks like that certain update are just not being forwarded from the leader 
to a certain replica.

Working theory: the max connections per host of the HttpClient is being hit, 
starving updates from certain update threads.
This could account for why shutdownNow on the update executor service is having 
such an impact.  In an orderly shutdown, all scheduled jobs will still be run 
(I think), which means that connections will be released, and the updates that 
were being starved will get to proceed.  But it's for exactly this reason that 
we should probably keep the shutdownNow... it mimics much better what will 
happen in real world situations when a node goes down.

>From this, it looks like max connections per host is 20:

{code}
13404 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[A22375CC545D2B82]) [] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 9,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
1,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
{code}

edit: oops the above is for *search* not updates.  The default for updates 
looks like it's 100, so harder to hit.  Although if we have a mix of streaming 
and non-streaming, and connections are not reused immediately, perhaps still 
possible.  Still digging along this line of logic.

The test used 12 nodes (and 2 shards)... increasing the chance of hitting the 
max connections (since all nodes run on the same host).



was (Author: ysee...@gmail.com):
I was analyzing another "shards-out-of-sync" failure on trunk.
It looks like that certain update are just not being forwarded from the leader 
to a certain replica.

Working theory: the max connections per host of the HttpClient is being hit, 
starving updates from certain update threads.
This could account for why shutdownNow on the update executor service is having 
such an impact.  In an orderly shutdown, all scheduled jobs will still be run 
(I think), which means that connections will be released, and the updates that 
were being starved will get to proceed.  But it's for exactly this reason that 
we should probably keep the shutdownNow... it mimics much better what will 
happen in real world situations when a node goes down.

>From this, it looks like max connections per host is 20:

{code}
13404 INFO  
(TEST-HdfsChaosMonkeyNothingIsSafeTest.test-seed#[A22375CC545D2B82]) [] 
o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 9,urlScheme 
: ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 
1,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 
5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,
{code}

The test used 12 nodes (and 2 shards)... increasing the chance of hitting the 
max connections (since all nodes run on the same host).


> ConcurrentUpdateSolrServer hang in blockUntilFinished.
> --
>
> Key: SOLR-6406
> URL: https://issues.apache.org/jira/browse/SOLR-6406
> Project: Solr
>  Issue Type: Bug
>Reporter: Mark Miller
>Assignee: Yonik Seeley
> Fix For: 5.4, Trunk
>
> Attachments: CPU Sampling.png, SOLR-6406.patch, SOLR-6406.patch, 
> SOLR-6406.patch
>
>
> Not sure what is causing this, but SOLR-6136 may have taken us a step back 
> here. I see this problem occasionally pop up in ChaosMonkeyNothingIsSafeTest 
> now - test fails because of a thread leak, thread leak is due to a 
> ConcurrentUpdateSolrServer hang in blockUntilFinished. Only started popping 
> up recently.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-7569) Create an API to force a leader election between nodes

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002251#comment-15002251
 ] 

Ishan Chattopadhyaya edited comment on SOLR-7569 at 11/12/15 3:38 PM:
--

bq. I've taken a crack at making SOLR-7989 work.
Thanks!

bq. Perhaps the last thing the API should do is run through each shard and see 
if the registered leader is DOWN, and if it is make it ACTIVE (preferably by 
asking it to publish itself as ACTIVE - we don't want to publish for someone 
else). If the call waits around to make sure all the leaders come up, this 
should be simple.
This makes sense. I think this is something that Shalin alluded to (please 
excuse me if I'm mistaken) when he said, {{1. Leader is live but 'down' -> mark 
it 'active'}}. The suggestion for the replicas to mark themselves ACTIVE 
instead of someone else marking them ACTIVE seems like a good thing to do.


was (Author: ichattopadhyaya):
bq. I've taken a crack at making SOLR-7989 work.
Thanks!

bq. Perhaps the last thing the API should do is run through each shard and see 
if the registered leader is DOWN, and if it is make it ACTIVE (preferably by 
asking it to publish itself as ACTIVE - we don't want to publish for someone 
else). If the call waits around to make sure all the leaders come up, this 
should be simple.
This makes sense. I think this is something that Shalin alluded to (please 
excuse me if I'm mistaken) when he said, {{1. Leader is live but 'down' -> mark 
it 'active'}}. The suggestion for the replicas to mark themselves ACTIVE 
instead of someone else marking them down seems like a good thing to do.

> Create an API to force a leader election between nodes
> --
>
> Key: SOLR-7569
> URL: https://issues.apache.org/jira/browse/SOLR-7569
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Reporter: Shalin Shekhar Mangar
>Assignee: Noble Paul
>  Labels: difficulty-medium, impact-high
> Fix For: 5.4, Trunk
>
> Attachments: SOLR-7569-testfix.patch, SOLR-7569.patch, 
> SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, 
> SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, 
> SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, 
> SOLR-7569.patch, SOLR-7569.patch, SOLR-7569.patch, 
> SOLR-7569_lir_down_state_test.patch
>
>
> There are many reasons why Solr will not elect a leader for a shard e.g. all 
> replicas' last published state was recovery or due to bugs which cause a 
> leader to be marked as 'down'. While the best solution is that they never get 
> into this state, we need a manual way to fix this when it does get into this  
> state. Right now we can do a series of dance involving bouncing the node 
> (since recovery paths between bouncing and REQUESTRECOVERY are different), 
> but that is difficult when running a large cluster. Although it is possible 
> that such a manual API may lead to some data loss but in some cases, it is 
> the only possible option to restore availability.
> This issue proposes to build a new collection API which can be used to force 
> replicas into recovering a leader while avoiding data loss on a best effort 
> basis.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7989) Down replica elected leader, stays down after successful election

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002288#comment-15002288
 ] 

Mark Miller commented on SOLR-7989:
---

bq. Indeed, that's why I didn't include it in the patch.

I think that doing that is fine in a test - we just have to carefully 
understand what we are testing and what we are fixing. This one was 
particularly tricky - I did not catch the issue the first few times I looked at 
it. Many things are impossible to test without some of this artificial hand 
waving though.



> Down replica elected leader, stays down after successful election
> -
>
> Key: SOLR-7989
> URL: https://issues.apache.org/jira/browse/SOLR-7989
> Project: Solr
>  Issue Type: Bug
>Reporter: Ishan Chattopadhyaya
>Assignee: Mark Miller
> Fix For: 5.4, Trunk
>
> Attachments: DownLeaderTest.java, DownLeaderTest.java, 
> SOLR-7989.patch, SOLR-7989.patch, SOLR-7989.patch, SOLR-7989.patch, 
> SOLR-7989.patch, SOLR-7989.patch, SOLR-7989.patch, SOLR-8233.patch
>
>
> It is possible that a down replica gets elected as a leader, and that it 
> stays down after the election.
> Here's how I hit upon this:
> * There are 3 replicas: leader, notleader0, notleader1
> * Introduced network partition to isolate notleader0, notleader1 from leader 
> (leader puts these two in LIR via zk).
> * Kill leader, remove partition. Now leader is dead, and both of notleader0 
> and notleader1 are down. There is no leader.
> * Remove LIR znodes in zk.
> * Wait a while, and there happens a (flawed?) leader election.
> * Finally, the state is such that one of notleader0 or notleader1 (which were 
> down before) become leader, but stays down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-6273) Cross Data Center Replication

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-6273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002518#comment-15002518
 ] 

ASF subversion and git services commented on SOLR-6273:
---

Commit 1714099 from [~erickoerickson] in branch 'dev/trunk'
[ https://svn.apache.org/r1714099 ]

SOLR-6273: Removed unused imports, no code changes

> Cross Data Center Replication
> -
>
> Key: SOLR-6273
> URL: https://issues.apache.org/jira/browse/SOLR-6273
> Project: Solr
>  Issue Type: New Feature
>Reporter: Yonik Seeley
>Assignee: Erick Erickson
> Attachments: SOLR-6273-trunk-testfix1.patch, 
> SOLR-6273-trunk-testfix2.patch, SOLR-6273-trunk-testfix3.patch, 
> SOLR-6273-trunk-testfix6.patch, SOLR-6273-trunk-testfix7.patch, 
> SOLR-6273-trunk.patch, SOLR-6273-trunk.patch, SOLR-6273.patch, 
> SOLR-6273.patch, SOLR-6273.patch, SOLR-6273.patch, forShalin.patch
>
>
> This is the master issue for Cross Data Center Replication (CDCR)
> described at a high level here: 
> http://heliosearch.org/solr-cross-data-center-replication/



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Sharing a class across cores

[Gus Heck]

This is for use in code in a search component, while I could fire up an
http client or solr client to get stuff out of a blob store, that will make
it hard to ensure that concurrent core startups don't duplicate each
other's work and all process the same file into the blob store etc...  and
the blob is meant to be a map in which the component will look things up,
so can one query into the blob in the blob store? I suspect not.

@Erik, This has been discussed as a possible fallback, though it also
involves creating a client that makes requests inside the search component
while it processes queries. It also creates a need for manual steps to
create the index that stands in place of a map and then that has to be
scripted for our continuous deployment...

I have a patch that I am testing locally, and my unit tests like it so far
today I should get it into a running instance for testing. We will at least
use it locally. If it goes well I'll probably contribute it back as a patch
to SOLR-3443.

On Thu, Nov 12, 2015 at 5:07 AM, Ishan Chattopadhyaya <
ichattopadhy...@gmail.com> wrote:

> > Or in a separate Solr core/collection.  :)
> That support is available with this, I think:
> https://cwiki.apache.org/confluence/display/solr/Blob+Store+API
>
> On Thu, Nov 12, 2015 at 2:31 PM, Erik Hatcher 
> wrote:
>
>> Or in a separate Solr core/collection.  :)
>>
>> On Nov 11, 2015, at 19:05, Walter Underwood 
>> wrote:
>>
>> Depending on how fast the access needs to be, you could put that big map
>> in memcache.
>>
>> wunder
>> Walter Underwood
>> wun...@wunderwood.org
>> http://observer.wunderwood.org/  (my blog)
>>
>>
>> On Nov 11, 2015, at 4:04 PM, Gus Heck  wrote:
>>
>> P.S. I posted the original message concurrently with the chat session's
>> occurance I beleive, certainly before I had read it, so no I haven't
>> actually tried what you suggest yet.
>>
>> On Wed, Nov 11, 2015 at 7:02 PM, Gus Heck  wrote:
>>
>>> Yes asked by a colleague :). The chat session is now in our jira ticket
>>> :).
>>>
>>> However, my take on it is that this seems like a pretty broad brush to
>>> paint with to move *all* our classes up and out of the normal core loading
>>> process. I assume there are good reasons for segregating this stuff into
>>> separate class loaders to begin with. It would also be fairly burdensom to
>>> make a separate jar file to break out this one component...
>>>
>>> I really just want a way to stash the map in a place where other cores
>>> can see it (and thus I can appropriately synchronize things so that the
>>> loading only happens once). I'm asking because it seems like surely this
>>> must be a solved problem... if not, it might be easiest to just solve it by
>>> adding some sort of shared resources facility to CoreContainer?
>>>
>>> -Gus
>>>
>>> On Wed, Nov 11, 2015 at 6:54 PM, Shawn Heisey 
>>> wrote:
>>>
 On 11/11/2015 4:11 PM, Gus Heck wrote:
 > I have a case where a component loads up a large CSV file (2.5 million
 > lines) to build a map. This worked ok in a case where we had a single
 > core, but it isn't working so well with 40 cores because each core
 loads
 > a new copy of the component in a new classloader and I get 40 new
 > versions of the same class each holding it's own private static final
 > map (one for each core). Each line is small, but a billion of anything
 > gets kinda heavy. Is this the intended class loading behavior?
 >
 > Is there some where that one can cause a class to be loaded in a
 parent
 > classloader above the core so that it's loaded just once? I want to
 load
 > it in some way that leverages standard solr resource loading, so that
 > I'm not hard coding or setting sysprops just to be able to find it.
 >
 > This is in a copy of trunk from about a month ago... so 6.x stuff is
 > mostly available.

 This sounds like a question that I just recently answered on IRC.

 If you remove all  elements from your solrconfig.xml files and
 place all extra jars for Solr into ${solr.solr.home}/lib ... Solr will
 load those jars before any cores are created and they will be available
 to all cores.

 There is a minor bug with this that will be fixed in Solr 5.4.0.  It is
 unlikely that this will affect third-party components, but be aware that
 until 5.4, jars in that lib directory will be loaded twice by older 5.x
 versions.

 https://issues.apache.org/jira/browse/SOLR-6188

 Thanks,
 Shawn


 -
 To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
 For additional commands, e-mail: dev-h...@lucene.apache.org


>>>
>>>
>>> --
>>> http://www.the111shift.com
>>>
>>
>>
>>
>> --
>> http://www.the111shift.com
>>
>>
>>
>


-- 

[jira] [Created] (SOLR-8283) factor out SortSpecParsing[Test] from QueryParsing[Test]

[Christine Poerschke (JIRA)]

Christine Poerschke created SOLR-8283:
-

 Summary: factor out SortSpecParsing[Test] from QueryParsing[Test]
 Key: SOLR-8283
 URL: https://issues.apache.org/jira/browse/SOLR-8283
 Project: Solr
  Issue Type: Task
Reporter: Christine Poerschke
Assignee: Christine Poerschke


patch to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8283) factor out SortSpecParsing[Test] from QueryParsing[Test]

[Christine Poerschke (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke updated SOLR-8283:
--
Attachment: SOLR-8283.patch

attaching proposed patch against trunk (the patch was generated from git, when 
applying the patch I would use 'svn cp' also to preserve the log history of the 
code factored out into new classes)

> factor out SortSpecParsing[Test] from QueryParsing[Test]
> 
>
> Key: SOLR-8283
> URL: https://issues.apache.org/jira/browse/SOLR-8283
> Project: Solr
>  Issue Type: Task
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
> Attachments: SOLR-8283.patch
>
>
> patch to follow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8212) Standard Highlighter Inconsistent with NGram Tokenizer

[Esther Quansah (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esther Quansah updated SOLR-8212:
-
Description: 
Noticing some inconsistent behavior with the Standard Highlighter and its 
function on terms that use the NGram Tokenizer. Ex: 
I created a field called "title_contains" which uses the NGram Tokenizer and I 
indexed the term "bronchoscopy". Querying "co" on the title_contains field 
should return "bronchoscopy", but the Standard highlighter returns 
"bronchoscopy" without the highlighting information.
I created a test called testNgram() which tests the above example using (1) the 
Standard Highlighter on the ngram field type and (2) the Fast Vector 
Highlighter on the ngram field type. The first fails and the second passes. 


Problem identified: 

  was:
Noticing some inconsistent behavior with the Standard Highlighter and its 
function on terms that use the NGram Tokenizer. Ex: 
I created a field called "title_contains" which uses the NGram Tokenizer and I 
indexed the term "bronchoscopy". Querying "co" on the title_contains field 
should return "bronchoscopy", but the Standard highlighter returns 
"bronchoscopy" without the highlighting information.
I created a test called testNgram() which tests the above example using (1) the 
Standard Highlighter on the ngram field type and (2) the Fast Vector 
Highlighter on the ngram field type. The first fails and the second passes. 


> Standard Highlighter Inconsistent with NGram Tokenizer
> --
>
> Key: SOLR-8212
> URL: https://issues.apache.org/jira/browse/SOLR-8212
> Project: Solr
>  Issue Type: Bug
>Reporter: Esther Quansah
>Priority: Minor
> Attachments: SOLR-8212.patch
>
>
> Noticing some inconsistent behavior with the Standard Highlighter and its 
> function on terms that use the NGram Tokenizer. Ex: 
> I created a field called "title_contains" which uses the NGram Tokenizer and 
> I indexed the term "bronchoscopy". Querying "co" on the title_contains field 
> should return "bronchoscopy", but the Standard highlighter returns 
> "bronchoscopy" without the highlighting information.
> I created a test called testNgram() which tests the above example using (1) 
> the Standard Highlighter on the ngram field type and (2) the Fast Vector 
> Highlighter on the ngram field type. The first fails and the second passes. 
> Problem identified: 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8212) Standard Highlighter Inconsistent with NGram Tokenizer

[Esther Quansah (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Esther Quansah updated SOLR-8212:
-
Description: 
Noticing some inconsistent behavior with the Standard Highlighter and its 
function on terms that use the NGram Tokenizer. Ex: 
I created a field called "title_contains" which uses the NGram Tokenizer and I 
indexed the term "bronchoscopy". Querying "co" on the title_contains field 
should return "bronchoscopy", but the Standard highlighter returns 
"bronchoscopy" without the highlighting information.
I created a test called testNgram() which tests the above example using (1) the 
Standard Highlighter on the ngram field type and (2) the Fast Vector 
Highlighter on the ngram field type. The first fails and the second passes. 


Problem identified: MAX_NUM_TOKENS_PER_GROUP = 50 (in TokenGroup.Java) and for 
some terms numTokens >=50...this causes incorrect match start and end offsets 
and therefore no highlighting on found term. 

  was:
Noticing some inconsistent behavior with the Standard Highlighter and its 
function on terms that use the NGram Tokenizer. Ex: 
I created a field called "title_contains" which uses the NGram Tokenizer and I 
indexed the term "bronchoscopy". Querying "co" on the title_contains field 
should return "bronchoscopy", but the Standard highlighter returns 
"bronchoscopy" without the highlighting information.
I created a test called testNgram() which tests the above example using (1) the 
Standard Highlighter on the ngram field type and (2) the Fast Vector 
Highlighter on the ngram field type. The first fails and the second passes. 


Problem identified: 


> Standard Highlighter Inconsistent with NGram Tokenizer
> --
>
> Key: SOLR-8212
> URL: https://issues.apache.org/jira/browse/SOLR-8212
> Project: Solr
>  Issue Type: Bug
>Reporter: Esther Quansah
>Priority: Minor
> Attachments: SOLR-8212.patch
>
>
> Noticing some inconsistent behavior with the Standard Highlighter and its 
> function on terms that use the NGram Tokenizer. Ex: 
> I created a field called "title_contains" which uses the NGram Tokenizer and 
> I indexed the term "bronchoscopy". Querying "co" on the title_contains field 
> should return "bronchoscopy", but the Standard highlighter returns 
> "bronchoscopy" without the highlighting information.
> I created a test called testNgram() which tests the above example using (1) 
> the Standard Highlighter on the ngram field type and (2) the Fast Vector 
> Highlighter on the ngram field type. The first fails and the second passes. 
> Problem identified: MAX_NUM_TOKENS_PER_GROUP = 50 (in TokenGroup.Java) and 
> for some terms numTokens >=50...this causes incorrect match start and end 
> offsets and therefore no highlighting on found term. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8260) Use NIO2 APIs in core discovery

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002454#comment-15002454
 ] 

Shawn Heisey commented on SOLR-8260:


I added the same exclusions found in the forbiddenapi config for Lucene to 
Solr, and started going through the tagged classes in trunk to change to NIO2.  
I don't think I've touched anything for core discovery yet, so I'm hoping the 
commits here won't overlap.


> Use NIO2 APIs in core discovery
> ---
>
> Key: SOLR-8260
> URL: https://issues.apache.org/jira/browse/SOLR-8260
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Fix For: 5.4
>
> Attachments: SOLR-8260.patch
>
>
> CorePropertiesLocator currently does all its file system interaction using 
> java.io.File and friends, which have all sorts of drawbacks with regard to 
> error handling and reporting.  We've been on java 7 for a while now, so we 
> should use the nio2 Path APIs instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8212) Standard Highlighter Inconsistent with NGram Tokenizer

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15002447#comment-15002447
 ] 

David Smiley commented on SOLR-8212:


Do the Postings or FastVector highlighters work properly for you?  I know they 
don't have this specific deficiency but I'm wondering if they highlight NGram 
based analysis the same way as the Standard highlighter.
https://cwiki.apache.org/confluence/display/solr/Highlighting
note that postings highlighter effectively only supports 
{{hl.usePhraseHighlighter=false}} at this time.

> Standard Highlighter Inconsistent with NGram Tokenizer
> --
>
> Key: SOLR-8212
> URL: https://issues.apache.org/jira/browse/SOLR-8212
> Project: Solr
>  Issue Type: Bug
>Reporter: Esther Quansah
>Priority: Minor
> Attachments: SOLR-8212.patch
>
>
> Noticing some inconsistent behavior with the Standard Highlighter and its 
> function on terms that use the NGram Tokenizer. Ex: 
> I created a field called "title_contains" which uses the NGram Tokenizer and 
> I indexed the term "bronchoscopy". Querying "co" on the title_contains field 
> should return "bronchoscopy", but the Standard highlighter returns 
> "bronchoscopy" without the highlighting information.
> I created a test called testNgram() which tests the above example using (1) 
> the Standard Highlighter on the ngram field type and (2) the Fast Vector 
> Highlighter on the ngram field type. The first fails and the second passes. 
> Problem identified: MAX_NUM_TOKENS_PER_GROUP = 50 (in TokenGroup.Java) and 
> for some terms numTokens >=50...this causes incorrect match start and end 
> offsets and therefore no highlighting on found term. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org