Re: 6.4.2 release?

[Ere Maijala]

Please make LUCENE-7698 a blocker if possible. It's a regression that 
makes Solr pretty much useless for anyone with CommonGramsQueryFilter in 
the analysis chain.


--Ere

21.2.2017, 21.46, Ishan Chattopadhyaya kirjoitti:

Actually, LUCENE-7698 was not a blocker, just marked for a 6.4.2
release. Should we make it a blocker?
As per an offline discussion with Andrzej, I've added SOLR-10182 as a
blocker. Tentatively, I'll cut a RC for 6.4.2 by Tuesday.

On Tue, Feb 21, 2017 at 11:35 PM, Ishan Chattopadhyaya
<ichattopadhy...@gmail.com <mailto:ichattopadhy...@gmail.com>> wrote:

I would like to volunteer for this 6.4.2 release. Planning to cut a
RC as soon as blockers are resolved.
One of the unresolved blocker issues seems to be LUCENE-7698 (I'll
take a look to see if there are more). If there are more issues that
should be part of the release, please let me know or mark as
blockers in jira.

Thanks,
Ishan


On Thu, Feb 16, 2017 at 3:48 AM, Adrien Grand <jpou...@gmail.com
<mailto:jpou...@gmail.com>> wrote:

I had initially planned on releasing tomorrow but the mirrors
replicated faster than I had thought they would so I finished
the release today, including the addition of the new 5.5.4
indices for backward testing so I am good with proceeding with a
new release now.

Le mer. 15 févr. 2017 à 16:13, Adrien Grand <jpou...@gmail.com
<mailto:jpou...@gmail.com>> a écrit :

+1

One ask I have is to wait for the 5.5.4 release process to
be complete so that branch_6_4 has the 5.5.4 backward
indices when we cut the first RC. I will let you know when I
am done.

Le mer. 15 févr. 2017 à 15:53, Christine Poerschke
(BLOOMBERG/ LONDON) <cpoersc...@bloomberg.net
<mailto:cpoersc...@bloomberg.net>> a écrit :

Hi,

These two could be minor candidates for inclusion:

* https://issues.apache.org/jira/browse/SOLR-10083
<https://issues.apache.org/jira/browse/SOLR-10083>
Fix instanceof check in ConstDoubleSource.equals

* https://issues.apache.org/jira/browse/LUCENE-7676
<https://issues.apache.org/jira/browse/LUCENE-7676>
FilterCodecReader to override more super-class methods

The former had narrowly missed the 6.4.1 release.

Regards,

Christine

From: dev@lucene.apache.org
<mailto:dev@lucene.apache.org> At: 02/15/17 14:27:52
To: dev@lucene.apache.org
Subject: Re:6.4.2 release?

Hi devs,

These two issues seem serious enough to warrant a
new release from branch_6_4:
* SOLR-10130: Serious performance degradation in
Solr 6.4.1 due to the new metrics collection
* SOLR-10138: Transaction log replay can hit an NPE
due to new Metrics code.

What do you think? Anything else that should go there?

---
        Best regards,

Andrzej Bialecki





--
Ere Maijala
Kansalliskirjasto / The National Library of Finland

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

6.4.2 release?

[Ere Maijala]

Hi,

I'd like to nominate LUCENE-7698 (CommonGramsQueryFilter doesn't work 
properly, caused by LUCENE-7603 and a regression from 6.3.0 where it 
worked fine). It doesn't have a fix, but at least we got seriously 
bitten by it.


Regards,
Ere

P.S. Sorry, no proper references in this email since I just joined the list.


Hi devs,

These two issues seem serious enough to warrant a new release from branch_6_4:
* SOLR-10130: Serious performance degradation in Solr 6.4.1 due to the new 
metrics collection
* SOLR-10138: Transaction log replay can hit an NPE due to new Metrics code.

What do you think? Anything else that should go there?

---
Best regards,

Andrzej Bialecki




-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Review Request 65942: SOLR-11982: Add support for shards.sort parameter

[Ere Maijala]

> On maalis 8, 2018, 8:54 ip, Tomás Fernández Löbbe wrote:
> >

I believe I've addressed the comments.


> On maalis 8, 2018, 8:54 ip, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Lines 309 (patched)
> > 
> >
> > I'm debating with myself on whether we should allow both params to be 
> > specified or not. I think it would be better to throw an exception in case 
> > of both parameters specified (since preferLocalShards would now be 
> > deprecated)

Agreed. The parameters are now mutually exclusive.


> On maalis 8, 2018, 8:54 ip, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Line 364 (original), 325 (patched)
> > 
> >
> > I think we should surround this with a try/catch and throw a 
> > SolrException(SolrError.BAD_REQUEST) in case of an IllegalArgumentException

Done.


> On maalis 8, 2018, 8:54 ip, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Lines 379 (patched)
> > 
> >
> > This should be an inner class or have it's own file

Right, fixed.


> On maalis 8, 2018, 8:54 ip, Tomás Fernández Löbbe wrote:
> > solr/core/src/test/org/apache/solr/handler/component/HttpShardHandlerFactoryTest.java
> > Lines 31 (patched)
> > 
> >
> > Note that there is a `TestHttpShardHandlerFactory`, did you 
> > intentionally not use that one?

No, I just failed to find it due to its naming. Fixed.


- Ere


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65942/#review198909
---


On maalis 8, 2018, 5:38 ip, Tomás Fernández Löbbe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65942/
> ---
> 
> (Updated maalis 8, 2018, 5:38 ip)
> 
> 
> Review request for lucene.
> 
> 
> Repository: lucene-solr
> 
> 
> Description
> ---
> 
> Creating this Review request on Ere Maijala's patch. See SOLR-11982 for 
> previous discussion.
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> shards.sort parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with ``shards.sor=replicaType:PULL|TLOG``(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).
> 
> 
> Diffs
> -
> 
>   solr/CHANGES.txt c1b5161c9c 
>   
> solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
>  6bfd36af94 
>   
> solr/core/src/test/org/apache/solr/handler/component/HttpShardHandlerFactoryTest.java
>  PRE-CREATION 
>   solr/solr-ref-guide/src/distributed-requests.adoc f5aaff469e 
>   solr/solrj/src/java/org/apache/solr/common/params/ShardParams.java 
> cbc33f41f4 
>   
> solr/solrj/src/test/org/apache/solr/client/solrj/impl/CloudSolrClientTest.java
>  9539846f88 
> 
> 
> Diff: https://reviews.apache.org/r/65942/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Tomás Fernández Löbbe
> 
>

Re: Review Request 65942: SOLR-11982: Add support for shards.sort parameter

[Ere Maijala]

> On maalis 7, 2018, 2:49 ap, Tomás Fernández Löbbe wrote:
> >

I believe I've addressed the review comments in the latest patch.


> On maalis 7, 2018, 2:49 ap, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Line 308 (original), 311 (patched)
> > 
> >
> > We could make this class package private and do better unit tests, 
> > should be easier to test all the possible branches, specially failure 
> > scenarios.

Done. The only thing missing from the new tests is the "replicaLocation:local" 
rule. I didn't add that since it would require way more infrastructure in the 
form of a valid request and ZK.


> On maalis 7, 2018, 2:49 ap, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Line 309 (original), 312 (patched)
> > 
> >
> > This class can be made static

Done.


> On maalis 7, 2018, 2:49 ap, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Lines 310-311 (original), 313-314 (patched)
> > 
> >
> > These two could be made final, right?

Right, done.


> On maalis 7, 2018, 2:49 ap, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Lines 327 (patched)
> > 
> >
> > You could set the size of the array to ``sortRules.size()``, right?

Done.


> On maalis 7, 2018, 2:49 ap, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Line 323 (original)
> > 
> >
> > I think we should throw an exception if the rule.name is unknown (not 
> > one of the expected values)

True. Done.


> On maalis 7, 2018, 2:49 ap, Tomás Fernández Löbbe wrote:
> > solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
> > Lines 404-406 (patched)
> > 
> >
> > I don't think this can really happen (someone comparing with the 
> > replica type directly) or am I missing something?

True. It seems it's still possible to have a situation where the list contains 
only shard addresses, so I left a type check there.


> On maalis 7, 2018, 2:49 ap, Tomás Fernández Löbbe wrote:
> > solr/solr-ref-guide/src/distributed-requests.adoc
> > Lines 149 (patched)
> > 
> >
> > ``[...]replicas in the given order of precedence``. Maybe add "within 
> > each shard"? Not sure if that's clear enough. WDYT?

Agreed. I added that.


- Ere


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/65942/#review198758
---


On maalis 7, 2018, 2:17 ap, Tomás Fernández Löbbe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/65942/
> ---
> 
> (Updated maalis 7, 2018, 2:17 ap)
> 
> 
> Review request for lucene.
> 
> 
> Repository: lucene-solr
> 
> 
> Description
> ---
> 
> Creating this Review request on Ere Maijala's patch. See SOLR-11982 for 
> previous discussion.
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> shards.sort parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with ``shards.sor=replicaType:PULL|TLOG``(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).
> 
> 
> Diffs
> -
> 
>   solr/CHANGES.txt 99e61f1d16 
>   
> solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java
>  6bfd36af94 
>   solr/solr-ref-guide/src/distributed-requests.adoc f5aaff469e 
>   solr/solrj/src/java/org/apache/solr/common/params/ShardParams.java 
> cbc33f41f4 
>   
> solr/solrj/src/test/org/apache/solr/client/solrj/impl/CloudSolrClientTest.java
>  9539846f88 
> 
> 
> Diff: https://reviews.apache.org/r/65942/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Tomás Fernández Löbbe
> 
>

[jira] [Commented] (LUCENE-5648) Index/search multi-valued time durations

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13991652#comment-13991652
 ] 

Ere Maijala commented on LUCENE-5648:
-

Since it will be date-based, can you make sure it also works with BC dates? 
Last time looked at the date implementation, it didn't handle the minus sign 
properly in ISO 8601 dates.

 Index/search multi-valued time durations
 

 Key: LUCENE-5648
 URL: https://issues.apache.org/jira/browse/LUCENE-5648
 Project: Lucene - Core
  Issue Type: New Feature
  Components: modules/spatial
Reporter: David Smiley
Assignee: David Smiley

 If you need to index a date/time duration, then the way to do that is to have 
 a pair of date fields; one for the start and one for the end -- pretty 
 straight-forward. But if you need to index a variable number of durations per 
 document, then the options aren't pretty, ranging from denormalization, to 
 joins, to using Lucene spatial with 2D as described 
 [here|http://wiki.apache.org/solr/SpatialForTimeDurations].  Ideally it would 
 be easier to index durations, and work in a more optimal way.
 This issue implements the aforementioned feature using Lucene-spatial with a 
 new single-dimensional SpatialPrefixTree implementation. Unlike the other two 
 SPT implementations, it's not based on floating point numbers. It will have a 
 Date based customization that indexes levels at meaningful quantities like 
 seconds, minutes, hours, etc.  The point of that alignment is to make it 
 faster to query across meaningful ranges (i.e. [2000 TO 2014]) and to enable 
 a follow-on issue to facet on the data in a really fast way.
 I'll expect to have a working patch up this week.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-6532) Possibility to return an error immediately from ping handler if no searcher is available

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-6532:
-

 Summary: Possibility to return an error immediately from ping 
handler if no searcher is available
 Key: SOLR-6532
 URL: https://issues.apache.org/jira/browse/SOLR-6532
 Project: Solr
  Issue Type: Improvement
  Components: search
Affects Versions: 4.10
 Environment: Load-balanced service
Reporter: Ere Maijala
Priority: Minor


Especially in a load-balanced environment it would be useful if it was possible 
to configure PingRequestHandler to return right away with an error status when 
a searcher is not (yet) available. This would allow the load-balancer to 
quickly fail over to a Solr instance that's able to serve the requests.

Currently the ping handler waits for the searcher to become available, which 
means the load-balancer has to either keep waiting or use a suitably short 
timeout, which is difficult to define in a way that provides a timely failover 
without false negatives.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-2522) Change max() and min() to work on multiValued fields

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14314151#comment-14314151
 ] 

Ere Maijala commented on SOLR-2522:
---

Voted too.

 Change max() and min() to work on multiValued fields 
 -

 Key: SOLR-2522
 URL: https://issues.apache.org/jira/browse/SOLR-2522
 Project: Solr
  Issue Type: Improvement
Reporter: Bill Bell

 Switch max() and min() functions to work on multiValued fields so we can 
 do sort=min(fieldname) asc and the sort would work on multiValued fields...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-7392) SOLR_JAVA_MEM setting in solr.in.sh ignored

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-7392:
-

 Summary: SOLR_JAVA_MEM setting in solr.in.sh ignored
 Key: SOLR-7392
 URL: https://issues.apache.org/jira/browse/SOLR-7392
 Project: Solr
  Issue Type: Bug
  Components: scripts and tools
Affects Versions: 5.1
Reporter: Ere Maijala


In Solr 5.1 the following line was added to bin/solr line 1262:

SOLR_JAVA_MEM=()

This overrides any setting previously read from solr.in.sh. Only SOLR_HEAP 
setting could have effect, but it's not even mentioned in solr.in.sh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-7392) SOLR_JAVA_MEM setting in solr.in.sh ignored

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14495946#comment-14495946
 ] 

Ere Maijala commented on SOLR-7392:
---

True, looks like SOLR_OPTS has the same issue. 

 SOLR_JAVA_MEM setting in solr.in.sh ignored
 ---

 Key: SOLR-7392
 URL: https://issues.apache.org/jira/browse/SOLR-7392
 Project: Solr
  Issue Type: Bug
  Components: scripts and tools
Affects Versions: 5.1
Reporter: Ere Maijala
Assignee: Ramkumar Aiyengar
 Attachments: SOLR-7392.patch


 In Solr 5.1 the following line was added to bin/solr line 1262:
 SOLR_JAVA_MEM=()
 This overrides any setting previously read from solr.in.sh. Only SOLR_HEAP 
 setting could have effect, but it's not even mentioned in solr.in.sh.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-5855) re-use document term-vector Fields instance across fields in the DefaultSolrHighlighter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14578437#comment-14578437
 ] 

Ere Maijala commented on SOLR-5855:
---

I was hoping, perhaps naively, that this would help with the highlighter 
performance problems we're having with Solr 5. Unfortunately this doesn't seems 
to have made a difference. Using hl.usePhraseHighlighter=false has a 
significant effect, but obviously with downsides and still much slower than 
4.10.2.

For what it's worth, here is some additional information:

Timing from Solr 4.10.2 (42.5 million records):

process: {
time: 1711,
query: {
time: 0
},
facet: {
time: 66
},
mlt: {
time: 0
},
highlight: {
time: 708
},
stats: {
time: 0
},
expand: {
time: 0
},
spellcheck: {
time: 433
},
debug: {
time: 503
}
}

Timing from Solr 5.2.0 (38.8 million records):

process: {
time: 10172,
query: {
time: 0
},
facet: {
time: 45
},
facet_module: {
time: 0
},
mlt: {
time: 0
},
highlight: {
time: 9310
},
stats: {
time: 0
},
expand: {
time: 0
},
spellcheck: {
time: 345
},
debug: {
time: 472
}
}

A couple of jstack outputs during the query execution are here: 
http://pastebin.com/8FJiq5R3. The schema and solrconfig are at 
https://github.com/NatLibFi/NDL-VuFind-Solr/tree/master/vufind/biblio/conf. 

 re-use document term-vector Fields instance across fields in the 
 DefaultSolrHighlighter
 ---

 Key: SOLR-5855
 URL: https://issues.apache.org/jira/browse/SOLR-5855
 Project: Solr
  Issue Type: Improvement
  Components: highlighter
Affects Versions: Trunk
Reporter: Daniel Debray
Assignee: David Smiley
 Fix For: 5.2

 Attachments: SOLR-5855-without-cache.patch, 
 SOLR-5855_with_FVH_support.patch, SOLR-5855_with_FVH_support.patch, 
 highlight.patch


 Hi folks,
 while investigating possible performance bottlenecks in the highlight 
 component i discovered two places where we can save some cpu cylces.
 Both are in the class org.apache.solr.highlight.DefaultSolrHighlighter
 First in method doHighlighting (lines 411-417):
 In the loop we try to highlight every field that has been resolved from the 
 params on each document. Ok, but why not skip those fields that are not 
 present on the current document? 
 So i changed the code from:
 for (String fieldName : fieldNames) {
   fieldName = fieldName.trim();
   if( useFastVectorHighlighter( params, schema, fieldName ) )
 doHighlightingByFastVectorHighlighter( fvh, fieldQuery, req, 
 docSummaries, docId, doc, fieldName );
   else
 doHighlightingByHighlighter( query, req, docSummaries, docId, doc, 
 fieldName );
 }
 to:
 for (String fieldName : fieldNames) {
   fieldName = fieldName.trim();
   if (doc.get(fieldName) != null) {
 if( useFastVectorHighlighter( params, schema, fieldName ) )
   doHighlightingByFastVectorHighlighter( fvh, fieldQuery, req, 
 docSummaries, docId, doc, fieldName );
 else
   doHighlightingByHighlighter( query, req, docSummaries, docId, doc, 
 fieldName );
   }
 }
 The second place is where we try to retrieve the TokenStream from the 
 document for a specific field.
 line 472:
 TokenStream tvStream = 
 TokenSources.getTokenStreamWithOffsets(searcher.getIndexReader(), docId, 
 fieldName);
 where..
 public static TokenStream getTokenStreamWithOffsets(IndexReader reader, int 
 docId, String field) throws IOException {
   Fields vectors = reader.getTermVectors(docId);
   if (vectors == null) {
 return null;
   }
   Terms vector = vectors.terms(field);
   if (vector == null) {
 return null;
   }
   if (!vector.hasPositions() || !vector.hasOffsets()) {
 return null;
   }
   return getTokenStream(vector);
 }
 keep in mind that we currently hit the IndexReader n times where n = 
 requested rows(documents) * requested amount of highlight fields.
 in my

[jira] [Commented] (SOLR-7655) Perf bug- DefaultSolrHighlighter.getSpanQueryScorer triggers MultiFields.getMergedFieldInfos

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-7655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580428#comment-14580428
 ] 

Ere Maijala commented on SOLR-7655:
---

Took me a while to get a build environment up, but results look very promising. 
My very unscientific benchmark shows that the highlighter may now be even a bit 
faster than in Solr 4.10.2 at least in some cases. Thanks for the great work, 
[~dsmiley]!

 Perf bug- DefaultSolrHighlighter.getSpanQueryScorer triggers 
 MultiFields.getMergedFieldInfos
 

 Key: SOLR-7655
 URL: https://issues.apache.org/jira/browse/SOLR-7655
 Project: Solr
  Issue Type: Bug
  Components: highlighter
Affects Versions: 5.0
Reporter: David Smiley
Assignee: David Smiley
 Attachments: SOLR-7655.patch


 It appears grabbing the FieldInfos from the SlowCompositeReaderWrapper is 
 slow.  It isn't cached.  The DefaultSolrHighligher in SOLR-6196 (v5.0) uses 
 it to ascertain if there are payloads.  Instead it can grab it from the Terms 
 instance, which is cached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-8065) Improve the start/stop script shutdown logic to avoid forcefully killing Solr

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-8065:
-

 Summary: Improve the start/stop script shutdown logic to avoid 
forcefully killing Solr
 Key: SOLR-8065
 URL: https://issues.apache.org/jira/browse/SOLR-8065
 Project: Solr
  Issue Type: Improvement
  Components: scripts and tools
Affects Versions: 5.3
 Environment: Unix/Linux
Reporter: Ere Maijala
Priority: Minor


Currently the start/stop script only waits 5 seconds before forcefully killing 
the Solr process. In our experience this is not enough for graceful shutdown on 
an active SolrCloud node. I propose making the shutdown wait for up to 30 
seconds and check the process status every second to avoid unnecessary delay.

I will attach a patch to accomplish this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8065) Improve the start/stop script shutdown logic to avoid forcefully killing Solr

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14791705#comment-14791705
 ] 

Ere Maijala commented on SOLR-8065:
---

The patch also changes the delay after kill -9 to 5 seconds to make sure the 
process has time to die. In practice I've seen the old 1 second delay to not be 
long enough causing the script to fail.

> Improve the start/stop script shutdown logic to avoid forcefully killing Solr
> -
>
> Key: SOLR-8065
> URL: https://issues.apache.org/jira/browse/SOLR-8065
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 5.3
> Environment: Unix/Linux
>Reporter: Ere Maijala
>Priority: Minor
> Attachments: solr_stop_patch.diff
>
>
> Currently the start/stop script only waits 5 seconds before forcefully 
> killing the Solr process. In our experience this is not enough for graceful 
> shutdown on an active SolrCloud node. I propose making the shutdown wait for 
> up to 30 seconds and check the process status every second to avoid 
> unnecessary delay.
> I will attach a patch to accomplish this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8065) Improve the start/stop script shutdown logic to avoid forcefully killing Solr

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-8065:
--
Attachment: solr_stop_patch.diff

Proposed patch to improve the logic of solr_stop function.

> Improve the start/stop script shutdown logic to avoid forcefully killing Solr
> -
>
> Key: SOLR-8065
> URL: https://issues.apache.org/jira/browse/SOLR-8065
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 5.3
> Environment: Unix/Linux
>Reporter: Ere Maijala
>Priority: Minor
> Attachments: solr_stop_patch.diff
>
>
> Currently the start/stop script only waits 5 seconds before forcefully 
> killing the Solr process. In our experience this is not enough for graceful 
> shutdown on an active SolrCloud node. I propose making the shutdown wait for 
> up to 30 seconds and check the process status every second to avoid 
> unnecessary delay.
> I will attach a patch to accomplish this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8065) Improve the start/stop script shutdown logic to avoid forcefully killing Solr

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-8065:
--
Labels: patch  (was: )

> Improve the start/stop script shutdown logic to avoid forcefully killing Solr
> -
>
> Key: SOLR-8065
> URL: https://issues.apache.org/jira/browse/SOLR-8065
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 5.3
> Environment: Unix/Linux
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch
> Attachments: solr_stop_patch.diff
>
>
> Currently the start/stop script only waits 5 seconds before forcefully 
> killing the Solr process. In our experience this is not enough for graceful 
> shutdown on an active SolrCloud node. I propose making the shutdown wait for 
> up to 30 seconds and check the process status every second to avoid 
> unnecessary delay.
> I will attach a patch to accomplish this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-8065) Improve the start/stop script shutdown logic to avoid forcefully killing Solr

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-8065:
--
Flags: Patch

> Improve the start/stop script shutdown logic to avoid forcefully killing Solr
> -
>
> Key: SOLR-8065
> URL: https://issues.apache.org/jira/browse/SOLR-8065
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 5.3
> Environment: Unix/Linux
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch
> Attachments: solr_stop_patch.diff
>
>
> Currently the start/stop script only waits 5 seconds before forcefully 
> killing the Solr process. In our experience this is not enough for graceful 
> shutdown on an active SolrCloud node. I propose making the shutdown wait for 
> up to 30 seconds and check the process status every second to avoid 
> unnecessary delay.
> I will attach a patch to accomplish this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8065) Improve the start/stop script shutdown logic to avoid forcefully killing Solr

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14934685#comment-14934685
 ] 

Ere Maijala commented on SOLR-8065:
---

So, as far as I can see the Windows solr.cmd is already using SolrCLI at 
startup but the unix script isn't. Is there a reason for that? Could that be a 
follow-up issue or should everything be cleaned up here?

> Improve the start/stop script shutdown logic to avoid forcefully killing Solr
> -
>
> Key: SOLR-8065
> URL: https://issues.apache.org/jira/browse/SOLR-8065
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 5.3
> Environment: Unix/Linux
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch
> Attachments: solr_stop_patch.diff
>
>
> Currently the start/stop script only waits 5 seconds before forcefully 
> killing the Solr process. In our experience this is not enough for graceful 
> shutdown on an active SolrCloud node. I propose making the shutdown wait for 
> up to 30 seconds and check the process status every second to avoid 
> unnecessary delay.
> I will attach a patch to accomplish this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8418) BoostQuery cannot be cast to TermQuery

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15088958#comment-15088958
 ] 

Ere Maijala commented on SOLR-8418:
---

Confirmed fixed for me too. Thanks!

> BoostQuery cannot be cast to TermQuery
> --
>
> Key: SOLR-8418
> URL: https://issues.apache.org/jira/browse/SOLR-8418
> Project: Solr
>  Issue Type: Bug
>  Components: MoreLikeThis
>Affects Versions: 5.4
>Reporter: Jens Wille
>Assignee: Ramkumar Aiyengar
> Fix For: 5.5, Trunk
>
> Attachments: SOLR-8418.patch, SOLR-8418.patch
>
>
> As a consequence of LUCENE-6590, {{MoreLikeThisHandler}} was changed in 
> r1701621 to use the new API. In SOLR-7912, I adapted that code for 
> {{CloudMLTQParser}} and {{SimpleMLTQParser}}. However, setting the {{boost}} 
> parameter just failed for me after updating to 5.4 with the following error 
> message:
> {code}
> java.lang.ClassCastException: org.apache.lucene.search.BoostQuery cannot be 
> cast to org.apache.lucene.search.TermQuery
> at 
> org.apache.solr.search.mlt.SimpleMLTQParser.parse(SimpleMLTQParser.java:139)
> at org.apache.solr.search.QParser.getQuery(QParser.java:141)
> at 
> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:160)
> at 
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:254)
> at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2073)
> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:457)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:222)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:181)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at 
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at 
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at 
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10172) Sorl 6 with jetty issues

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874516#comment-15874516
 ] 

Ere Maijala commented on SOLR-10172:


Duplicate of SOLR-10130?

> Sorl 6 with jetty issues
> 
>
> Key: SOLR-10172
> URL: https://issues.apache.org/jira/browse/SOLR-10172
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Abc 
>Priority: Blocker
>
> Issues with solr settings while migrating from solr 4.0 to solr6.0.
> Issue Faced : My cpu consumption goes to unacceptable levels. ie. load on 
> solr4.0 is between 6 to 10 while load on solr 6 reaches 100 and since its the 
> production i rolled back quickly.
> My Solr4 setting
>  - Running on tomcat
>  - JVM Memory : 16GB
>  - 24 core cpu
>  - JVM settings :
>- JVM Runtime Java HotSpot(TM) 64-Bit Server VM (24.45-b08) 
>- Processors   24 
>- Args : Paths mentioned here
> **My Solr6 setting**
>  - Running on jetty
>  - JVM Memory : 20GB
>  - 32 core cpu
>  - JVM settings :
>- Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.8.0_45 25.45-b02
>- Processors   32
>- Args
>   - DSTOP.KEY=solrrocks
>   - DSTOP.PORT=7983
>   - Djetty.home=/usr/local/solr-6.4.1/server-Djetty.port=8983
>   - 
> Dlog4j.configuration=file:/usr/local/solr-6.4.1/example/resources/log4j.properties
>   - 
> Dsolr.install.dir=/usr/local/solr-6.4.1-Dsolr.log.dir=/usr/local/solr-6.4.1/example/techproducts/solr/../logs
>   - Dsolr.log.muteconsole
>   - 
> Dsolr.solr.home=/usr/local/solr-6.4.1/example/techproducts/solr-Duser.timezone=US/Eastern
>   - XX:+AggressiveOpts
>   - XX:+CMSParallelRemarkEnabled
>   - XX:+CMSScavengeBeforeRemark
>   - XX:+ParallelRefProcEnabled
>   - XX:+PrintGCApplicationStoppedTime
>   - XX:+PrintGCDateStamps
>   - XX:+PrintGCDetails
>   - XX:+PrintGCTimeStamps
>   - XX:+PrintHeapAtGC
>   - XX:+PrintTenuringDistribution
>   - XX:+UseCMSInitiatingOccupancyOnly
>   - XX:+UseConcMarkSweepGC
>   - XX:+UseGCLogFileRotation
>   - XX:-UseSuperWord
>   - XX:CMSFullGCsBeforeCompaction=1
>   - XX:CMSInitiatingOccupancyFraction=70
>   - XX:CMSMaxAbortablePrecleanTime=6000
>   - XX:CMSTriggerPermRatio=80
>   - XX:GCLogFileSize=20M
>   - XX:MaxTenuringThreshold=8
>   - XX:NewRatio=2
>   - XX:NumberOfGCLogFiles=9
>   - XX:OnOutOfMemoryError=/usr/local/solr-6.4.1/bin/oom_solr.sh 8983 
> /usr/local/solr-6.4.1/example/techproducts/solr/../logs
>   - XX:PretenureSizeThreshold=64m
>   - XX:SurvivorRatio=15
>   - 
> XX:TargetSurvivorRatio=90-Xloggc:/usr/local/solr-6.4.1/example/techproducts/solr/../logs/solr_gc.log-Xms21g-Xmx21g-Xss256k-verbose:gc
> Bug :
> In this version, there must be issue regarding cpu utilization.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7698) CommonGramsQueryFilter in the query analyzer chain breaks phrase queries

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874209#comment-15874209
 ] 

Ere Maijala commented on LUCENE-7698:
-

[~mikemccand], thanks for the fix. An initial check indicates that the patch 
fixes my use case. I ran the tests in branch_6x. The patch didn't quite apply 
cleanly to branch_6_4 and after applying manually a test didn't compile:

{code}
common.compile-test:
[mkdir] Created dir: 
/Users/eremaijala/src/solr/lucene/build/analysis/common/classes/test
[javac] Compiling 279 source files to 
/Users/eremaijala/src/solr/lucene/build/analysis/common/classes/test
[javac] 
/Users/eremaijala/src/solr/lucene/analysis/common/src/test/org/apache/lucene/analysis/commongrams/TestCommonGramsQueryFilterFactory.java:103:
 error: cannot find symbol
[javac] assertGraphStrings(stream, "testing_the the_factory factory 
works");
[javac] ^
[javac]   symbol:   method assertGraphStrings(TokenStream,String)
[javac]   location: class TestCommonGramsQueryFilterFactory
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] 1 error
{code}

> CommonGramsQueryFilter in the query analyzer chain breaks phrase queries
> 
>
> Key: LUCENE-7698
> URL: https://issues.apache.org/jira/browse/LUCENE-7698
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 6.4, 6.4.1
>Reporter: Ere Maijala
>  Labels: regression
> Fix For: master (7.0), 6.4.2
>
> Attachments: LUCENE-7698.patch
>
>
> (Please pardon me if the project or component are wrong!)
> CommonGramsQueryFilter breaks phrase queries. The behavior also seems to 
> change with addition or removal of adjacent terms.
> Steps to reproduce:
> 1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
> 2.) Modify 
> server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
> modify text_general fieldType by adding CommonGrams(Query)Filter before 
> stopWordFilter:
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  words="stopwords.txt" />
> 
> 
>   
>   
> 
>  words="stopwords.txt"/>
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 3.) Add "with" to 
> server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and 
> make sure the file has correct line endings (extracted from Solr zip it seems 
> to contain DOS/Windows lien endings which may break things).
> 4.) Run the techproducts example with "bin/solr -e techproducts"
> 5.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>
> 6.) Observe that parsedquery in the debug output is empty
> 7.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>
> 8.) Observe that parsedquery contains ipod_with as expected but not 
> with_video.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15867402#comment-15867402
 ] 

Ere Maijala commented on SOLR-10130:


I don't have proper benchmarks at hand, but I can support others' findings 
about the serious query performance degradation. I suppose it's easily 
overlooked when testing with light concurrency, but with enough concurrent 
queries being served it gets CPU-heavy. We use queries with a lot of filters so 
that may play a role too. I'll see if I came come up with a reproducible-enough 
test results from our actual queries.

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-10130:
---
Attachment: solr-8983-console-f1.log

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>  Labels: perfomance
> Attachments: solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-10130:
--

 Summary: Serious performance degradation in Solr 6.4.1 due to the 
new metrics collection
 Key: SOLR-10130
 URL: https://issues.apache.org/jira/browse/SOLR-10130
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: metrics
Affects Versions: 6.4.1
 Environment: Centos 7, OpenJDK 1.8.0 update 111
Reporter: Ere Maijala
 Attachments: solr-8983-console-f1.log

We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
Looks like the new metrics collection system in MetricsDirectoryFactory is 
causing a major slowdown. This happens with an index configuration that, as far 
as I can see, has no metrics specific configuration and uses luceneMatchVersion 
5.5.0. In practice a moderate load will completely bog down the server with 
Solr threads constantly using up all CPU (600% on 6 core machine) capacity with 
a load that normally  where we normally see an average load of < 50%.

I took stack traces (I'll attach them) and noticed that the threads are 
spending time in com.codahale.metrics.Meter.mark. I tested building Solr 6.4.1 
with the metrics collection disabled in MetricsDirectoryFactory getByte and 
getBytes methods and was unable to reproduce the issue.

As far as I can see there are several issues:
1. Collecting metrics on every single byte read is slow.
2. Having it enabled by default is not a good idea.
3. The comment "enable coarse-grained metrics by default" at 
https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
 implies that only coarse-grained metrics should be enabled by default, and 
this contradicts with collecting metrics on every single byte read.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7698) CommonGramsQueryFilter in the query analyzer chain breaks phrase queries

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871575#comment-15871575
 ] 

Ere Maijala commented on LUCENE-7698:
-

Looks to me like LUCENE-7603 broke this.

> CommonGramsQueryFilter in the query analyzer chain breaks phrase queries
> 
>
> Key: LUCENE-7698
> URL: https://issues.apache.org/jira/browse/LUCENE-7698
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 6.4, 6.4.1
>    Reporter: Ere Maijala
>  Labels: regression
>
> (Please pardon me if the project or component are wrong!)
> CommonGramsQueryFilter breaks phrase queries. The behavior also seems to 
> change with addition or removal of adjacent terms.
> Steps to reproduce:
> 1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
> 2.) Modify 
> server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
> modify text_general fieldType by adding CommonGrams(Query)Filter before 
> stopWordFilter:
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  words="stopwords.txt" />
> 
> 
>   
>   
> 
>  words="stopwords.txt"/>
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 3.) Add "with" to 
> server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and 
> make sure the file has correct line endings (extracted from Solr zip it seems 
> to contain DOS/Windows lien endings which may break things).
> 4.) Run the techproducts example with "bin/solr -e techproducts"
> 5.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>
> 6.) Observe that parsedquery in the debug output is empty
> 7.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>
> 8.) Observe that parsedquery contains ipod_with as expected but not 
> with_video.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7698) CommonGramsQueryFilter in the query analyzer chain breaks phrase queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated LUCENE-7698:

Description: 
(Please pardon me if the project or component are wrong!)

CommonGramsQueryFilter breaks phrase queries. The behavior also seems to change 
with addition or removal of adjacent terms.

Steps to reproduce:

1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
2.) Modify 
server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
modify text_general fieldType by adding CommonGrams(Query)Filter before 
stopWordFilter:


  





  
  





  


3.) Add "with" to 
server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and make 
sure the file has correct line endings (extracted from Solr zip it seems to 
contain DOS/Windows lien endings which may break things).

4.) Run the techproducts example with "bin/solr -e techproducts"

5.) Browse to 
<http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>

6.) Observe that parsedquery in the debug output is empty

7.) Browse to 
<http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>

8.) Observe that parsedquery contains ipod_with as expected but not with_video.

  was:
CommonGramsQueryFilter breaks phrase queries. The behavior also seems to change 
with addition or removal of adjacent terms.

Steps to reproduce:

1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
2.) Modify 
server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
modify text_general fieldType by adding CommonGrams(Query)Filter before 
stopWordFilter:


  





  
  





  


3.) Add "with" to 
server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and make 
sure the file has correct line endings (extracted from Solr zip it seems to 
contain DOS/Windows lien endings which may break things).

4.) Run the techproducts example with "bin/solr -e techproducts"

5.) Browse to 
<http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>

6.) Observe that parsedquery in the debug output is empty

7.) Browse to 
<http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>

8.) Observe that parsedquery contains ipod_with as expected but not with_video.


> CommonGramsQueryFilter in the query analyzer chain breaks phrase queries
> 
>
> Key: LUCENE-7698
> URL: https://issues.apache.org/jira/browse/LUCENE-7698
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 6.4, 6.4.1
>Reporter: Ere Maijala
>  Labels: regression
>
> (Please pardon me if the project or component are wrong!)
> CommonGramsQueryFilter breaks phrase queries. The behavior also seems to 
> change with addition or removal of adjacent terms.
> Steps to reproduce:
> 1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
> 2.) Modify 
> server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
> modify text_general fieldType by adding CommonGrams(Query)Filter before 
> stopWordFilter:
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  words="stopwords.txt" />
> 
> 
>   
>   
> 
>  words="stopwords.txt"/>
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 3.) Add "with" to 
> server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and 
> make sure the file has correct line endings (extracted from Solr zip it seems 
> to contain DOS/Windows lien endings which may break things).
> 4.) Run the techproducts example with "bin/solr -e techproducts"
> 5.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>
> 6.) Observe that parsedquery in the debug output is empty
> 7.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>
> 8.) Observe that parsedquery contains ipod_with as expected but not 
> with_video.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7698) CommonGramsQueryFilter in the query analyzer chain breaks phrase queries

[Ere Maijala (JIRA)]

Ere Maijala created LUCENE-7698:
---

 Summary: CommonGramsQueryFilter in the query analyzer chain breaks 
phrase queries
 Key: LUCENE-7698
 URL: https://issues.apache.org/jira/browse/LUCENE-7698
 Project: Lucene - Core
  Issue Type: Bug
  Components: core/queryparser
Affects Versions: 6.4.1
Reporter: Ere Maijala


CommonGramsQueryFilter breaks phrase queries. The behavior also seems to change 
with addition or removal of adjacent terms.

Steps to reproduce:

1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
2.) Modify 
server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
modify text_general fieldType by adding CommonGrams(Query)Filter before 
stopWordFilter:


  





  
  





  


3.) Add "with" to 
server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and make 
sure the file has correct line endings (extracted from Solr zip it seems to 
contain DOS/Windows lien endings which may break things).

4.) Run the techproducts example with "bin/solr -e techproducts"

5.) Browse to 
<http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>

6.) Observe that parsedquery in the debug output is empty

7.) Browse to 
<http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>

8.) Observe that parsedquery contains ipod_with as expected but not with_video.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7698) CommonGramsQueryFilter in the query analyzer chain breaks phrase queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated LUCENE-7698:

Affects Version/s: 6.4

> CommonGramsQueryFilter in the query analyzer chain breaks phrase queries
> 
>
> Key: LUCENE-7698
> URL: https://issues.apache.org/jira/browse/LUCENE-7698
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 6.4, 6.4.1
>    Reporter: Ere Maijala
>
> CommonGramsQueryFilter breaks phrase queries. The behavior also seems to 
> change with addition or removal of adjacent terms.
> Steps to reproduce:
> 1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
> 2.) Modify 
> server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
> modify text_general fieldType by adding CommonGrams(Query)Filter before 
> stopWordFilter:
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  words="stopwords.txt" />
> 
> 
>   
>   
> 
>  words="stopwords.txt"/>
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 3.) Add "with" to 
> server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and 
> make sure the file has correct line endings (extracted from Solr zip it seems 
> to contain DOS/Windows lien endings which may break things).
> 4.) Run the techproducts example with "bin/solr -e techproducts"
> 5.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>
> 6.) Observe that parsedquery in the debug output is empty
> 7.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>
> 8.) Observe that parsedquery contains ipod_with as expected but not 
> with_video.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7698) CommonGramsQueryFilter in the query analyzer chain breaks phrase queries

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871485#comment-15871485
 ] 

Ere Maijala edited comment on LUCENE-7698 at 2/17/17 9:07 AM:
--

This seems to be a regression in Solr 6.4.0. At least a quick test shows 
correct results in 6.3.0.


was (Author: emaijala):
This seems be a regression in Solr 6.4.0. At least a quick test shows correct 
results in 6.3.0.

> CommonGramsQueryFilter in the query analyzer chain breaks phrase queries
> 
>
> Key: LUCENE-7698
> URL: https://issues.apache.org/jira/browse/LUCENE-7698
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 6.4, 6.4.1
>    Reporter: Ere Maijala
>
> CommonGramsQueryFilter breaks phrase queries. The behavior also seems to 
> change with addition or removal of adjacent terms.
> Steps to reproduce:
> 1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
> 2.) Modify 
> server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
> modify text_general fieldType by adding CommonGrams(Query)Filter before 
> stopWordFilter:
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  words="stopwords.txt" />
> 
> 
>   
>   
> 
>  words="stopwords.txt"/>
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 3.) Add "with" to 
> server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and 
> make sure the file has correct line endings (extracted from Solr zip it seems 
> to contain DOS/Windows lien endings which may break things).
> 4.) Run the techproducts example with "bin/solr -e techproducts"
> 5.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>
> 6.) Observe that parsedquery in the debug output is empty
> 7.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>
> 8.) Observe that parsedquery contains ipod_with as expected but not 
> with_video.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7698) CommonGramsQueryFilter in the query analyzer chain breaks phrase queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated LUCENE-7698:

Labels: regression  (was: )

> CommonGramsQueryFilter in the query analyzer chain breaks phrase queries
> 
>
> Key: LUCENE-7698
> URL: https://issues.apache.org/jira/browse/LUCENE-7698
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 6.4, 6.4.1
>    Reporter: Ere Maijala
>  Labels: regression
>
> CommonGramsQueryFilter breaks phrase queries. The behavior also seems to 
> change with addition or removal of adjacent terms.
> Steps to reproduce:
> 1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
> 2.) Modify 
> server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
> modify text_general fieldType by adding CommonGrams(Query)Filter before 
> stopWordFilter:
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  words="stopwords.txt" />
> 
> 
>   
>   
> 
>  words="stopwords.txt"/>
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 3.) Add "with" to 
> server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and 
> make sure the file has correct line endings (extracted from Solr zip it seems 
> to contain DOS/Windows lien endings which may break things).
> 4.) Run the techproducts example with "bin/solr -e techproducts"
> 5.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>
> 6.) Observe that parsedquery in the debug output is empty
> 7.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>
> 8.) Observe that parsedquery contains ipod_with as expected but not 
> with_video.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7698) CommonGramsQueryFilter in the query analyzer chain breaks phrase queries

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871485#comment-15871485
 ] 

Ere Maijala commented on LUCENE-7698:
-

This seems be a regression in Solr 6.4.0. At least a quick test shows correct 
results in 6.3.0.

> CommonGramsQueryFilter in the query analyzer chain breaks phrase queries
> 
>
> Key: LUCENE-7698
> URL: https://issues.apache.org/jira/browse/LUCENE-7698
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/queryparser
>Affects Versions: 6.4, 6.4.1
>    Reporter: Ere Maijala
>
> CommonGramsQueryFilter breaks phrase queries. The behavior also seems to 
> change with addition or removal of adjacent terms.
> Steps to reproduce:
> 1.) Download and extract Solr (in my test case version 6.4.1) somewhere.
> 2.) Modify 
> server/solr/configsets/sample_techproducts_configs/conf/managed-schema and 
> modify text_general fieldType by adding CommonGrams(Query)Filter before 
> stopWordFilter:
>  positionIncrementGap="100">
>   
> 
>  words="stopwords.txt" />
>  words="stopwords.txt" />
> 
> 
>   
>   
> 
>  words="stopwords.txt"/>
>  words="stopwords.txt" />
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 3.) Add "with" to 
> server/solr/configsets/sample_techproducts_configs/conf/stopwords.txt and 
> make sure the file has correct line endings (extracted from Solr zip it seems 
> to contain DOS/Windows lien endings which may break things).
> 4.) Run the techproducts example with "bin/solr -e techproducts"
> 5.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22iPod%20with%20Video%22=true>
> 6.) Observe that parsedquery in the debug output is empty
> 7.) Browse to 
> <http://localhost:8983/solr/techproducts/select?q=%22Apple%2060%20GB%20iPod%20with%20Video%20Playback%20Black%22=true>
> 8.) Observe that parsedquery contains ipod_with as expected but not 
> with_video.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10130) Serious performance degradation in Solr 6.4.1 due to the new metrics collection

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871772#comment-15871772
 ] 

Ere Maijala commented on SOLR-10130:


I still don't have proper benchmarks, but I've tested enough to say with fair 
confidence that this is fixed for us.

> Serious performance degradation in Solr 6.4.1 due to the new metrics 
> collection
> ---
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
>Reporter: Ere Maijala
>Assignee: Andrzej Bialecki 
>Priority: Blocker
>  Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, SOLR-10130.patch, 
> solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1. 
> Looks like the new metrics collection system in MetricsDirectoryFactory is 
> causing a major slowdown. This happens with an index configuration that, as 
> far as I can see, has no metrics specific configuration and uses 
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog 
> down the server with Solr threads constantly using up all CPU (600% on 6 core 
> machine) capacity with a load that normally  where we normally see an average 
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are 
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr 
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte 
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at 
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
>  implies that only coarse-grained metrics should be enabled by default, and 
> this contradicts with collecting metrics on every single byte read.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9818) Solr admin UI rapidly retries any request(s) if it loses connection with the server

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15885278#comment-15885278
 ] 

Ere Maijala commented on SOLR-9818:
---

I initially encountered this issue when trying to reload a collection, so 
fixing ADDREPLICA of course only fixes one instance where this might happen.

In my case I would have been happy with an error message that says that the 
connection timed out and that there's no guarantee that the request was 
processed or whether it's still being processed. It would be ok to poll the 
server automatically to see if it's available again, but only like at max once 
a second or so and with a safe request.

> Solr admin UI rapidly retries any request(s) if it loses connection with the 
> server
> ---
>
> Key: SOLR-9818
> URL: https://issues.apache.org/jira/browse/SOLR-9818
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Admin UI
>Affects Versions: 6.3
>Reporter: Ere Maijala
>
> It seems that whenever the Solr admin UI loses connection with the server, be 
> the reason that the server is too slow to answer or that it's gone away 
> completely, it starts hammering the server with the previous request until it 
> gets a success response, it seems. That can be especially bad if the last 
> attempted action was something like collection reload with a SolrCloud 
> instance. The admin UI will quickly add hundreds of reload commands to 
> overseer/collection-queue-work, which may essentially cause the replicas to 
> get overloaded when they're trying to handle all the reload commands.
> I believe the UI should never retry the previous command blindly when the 
> connection is lost, but instead just ping the server until it responds again.
> Steps to reproduce:
> 1.) Fire up Solr
> 2.) Open the admin UI in browser
> 3.) Open a web console in the browser to see the requests it sends
> 4.) Stop solr
> 5.) Try an action in the admin UI
> 6.) Observe the web console in browser quickly fill up with repeats of the 
> originally attempted request



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9267) Cloud MLT field boost not working

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575129#comment-15575129
 ] 

Ere Maijala edited comment on SOLR-9267 at 10/14/16 12:02 PM:
--

This patch fixes the handling of qf so that fields and any boosts are always 
extracted. It also fixes the filteredDocument creation so that IndexableField 
type is not directly cast to a string as that would include strings like 
"indexed" and "stored" and throw the results off if those are included in the 
indexed records.

As far as I can see this should work in the 6_2 branch too (apart from 
CHANGES.txt obviously).


was (Author: emaijala):
This patch fixes the handling of qf so that fields and any boosts are always 
extracted. It also fixes the filteredDocument creation so that IndexableField 
type is not directly cast to a string as that would include strings like 
"indexed" and "stored" and throw the results off if those are included in the 
indexed records.

> Cloud MLT field boost not working
> -
>
> Key: SOLR-9267
> URL: https://issues.apache.org/jira/browse/SOLR-9267
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.3.2, 5.4, 5.4.1, 
> 5.5, 5.5.1, 5.5.2, 5.5.3, 5.6, 6.0, 6.0.1, 6.0.2, 6.1, 6.1.1, 6.2
>Reporter: Brian Feldman
> Attachments: SOLR-9267.patch
>
>
> When boosting by field "fieldname otherFieldName^4.0" the boost is not 
> stripped from the field name when adding to fieldNames ArrayList.  So on line 
> 133 of CloudMLTQParser when adding field content to the filteredDocument the 
> field is not found (incorrectly trying to find 'otherFieldName^4.0').
> The easiest but perhaps hackiest solution is to overwrite qf:
> {code}
> if (localParams.get("boost") != null) {
>   mlt.setBoost(localParams.getBool("boost"));
>   boostFields = SolrPluginUtils.parseFieldBoosts(qf);
>   qf = boostFields.keySet().toArray(qf);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9644) MoreLikeThis parser doesn't handle boosts properly

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9644:
--
Remaining Estimate: (was: 4h)
 Original Estimate: (was: 4h)

> MoreLikeThis parser doesn't handle boosts properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>
> It seems SimpleMLTQParser should be able to handle boost parameters, but it's 
> not working properly. I've added a proposed patch to fix similar issue in 
> CloudMLTQParser in issue https://issues.apache.org/jira/browse/SOLR-9267 and 
> will attach a patch here too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9644) MoreLikeThis parser doesn't handle boosts properly

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-9644:
-

 Summary: MoreLikeThis parser doesn't handle boosts properly
 Key: SOLR-9644
 URL: https://issues.apache.org/jira/browse/SOLR-9644
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: MoreLikeThis
Affects Versions: 6.2.1
Reporter: Ere Maijala


It seems SimpleMLTQParser should be able to handle boost parameters, but it's 
not working properly. I've added a proposed patch to fix similar issue in 
CloudMLTQParser in issue https://issues.apache.org/jira/browse/SOLR-9267 and 
will attach a patch here too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9267) Cloud MLT field boost not working

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9267:
--
Attachment: SOLR-9267.patch

This patch fixes the handling of qf so that fields and any boosts are always 
extracted. It also fixes the filteredDocument creation so that IndexableField 
type is not directly cast to a string as that would include strings like 
"indexed" and "stored" and throw the results off if those are included in the 
indexed records.

> Cloud MLT field boost not working
> -
>
> Key: SOLR-9267
> URL: https://issues.apache.org/jira/browse/SOLR-9267
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.3.2, 5.4, 5.4.1, 
> 5.5, 5.5.1, 5.5.2, 5.5.3, 5.6, 6.0, 6.0.1, 6.0.2, 6.1, 6.1.1, 6.2
>Reporter: Brian Feldman
> Attachments: SOLR-9267.patch
>
>
> When boosting by field "fieldname otherFieldName^4.0" the boost is not 
> stripped from the field name when adding to fieldNames ArrayList.  So on line 
> 133 of CloudMLTQParser when adding field content to the filteredDocument the 
> field is not found (incorrectly trying to find 'otherFieldName^4.0').
> The easiest but perhaps hackiest solution is to overwrite qf:
> {code}
> if (localParams.get("boost") != null) {
>   mlt.setBoost(localParams.getBool("boost"));
>   boostFields = SolrPluginUtils.parseFieldBoosts(qf);
>   qf = boostFields.keySet().toArray(qf);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (SOLR-9644) MoreLikeThis parser doesn't handle boosts properly

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9644:
--
Comment: was deleted

(was: Pull request now open: https://github.com/apache/lucene-solr/pull/98)

> MoreLikeThis parser doesn't handle boosts properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>
> It seems SimpleMLTQParser and CouldMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9644) MoreLikeThis parser doesn't handle boosts properly

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9644:
--
Labels: patch  (was: )

> MoreLikeThis parser doesn't handle boosts properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>  Labels: patch
>
> It seems SimpleMLTQParser and CouldMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9644) MoreLikeThis parser doesn't handle boosts properly

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15581504#comment-15581504
 ] 

Ere Maijala commented on SOLR-9644:
---

Pull request now open: https://github.com/apache/lucene-solr/pull/98

> MoreLikeThis parser doesn't handle boosts properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>
> It seems SimpleMLTQParser and CouldMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9371) Fix bin/solr script calculations - start/stop wait time and RMI_PORT

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15601833#comment-15601833
 ] 

Ere Maijala commented on SOLR-9371:
---

Please do both. I've been hoping to get this fixed for over a year (see the 
linked issue SOLR-8065).

> Fix bin/solr script calculations - start/stop wait time and RMI_PORT
> 
>
> Key: SOLR-9371
> URL: https://issues.apache.org/jira/browse/SOLR-9371
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: 6.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9371.patch, SOLR-9371.patch
>
>
> The bin/solr script doesn't wait long enough for Solr to stop before it sends 
> the KILL signal to the process.  The start could use a longer wait too.
> Also, the RMI_PORT is calculated by simply prefixing the port number with a 
> "1" instead of adding 1.  If the solr port has five digits, then the rmi 
> port will be invalid, because it will be greater than 65535.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9371) Fix bin/solr script calculations - start/stop wait time and RMI_PORT

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15602032#comment-15602032
 ] 

Ere Maijala commented on SOLR-9371:
---

Oops, sorry for being so vague. With "both" I meant "committing this" and "put 
it on 6_2 as well". I can't help with Windows, and I think that's why SOLR-8065 
got stalled, but if you agree on doing this separately for Windows, that would 
be great. It's not easy to maintain your own version of the solr script when 
it's being enhanced all the time, and this issue has existed way too long 
already without anyone stepping up to do something about the Windows version.

> Fix bin/solr script calculations - start/stop wait time and RMI_PORT
> 
>
> Key: SOLR-9371
> URL: https://issues.apache.org/jira/browse/SOLR-9371
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: scripts and tools
>Affects Versions: 6.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Fix For: 6.2, master (7.0)
>
> Attachments: SOLR-9371.patch, SOLR-9371.patch
>
>
> The bin/solr script doesn't wait long enough for Solr to stop before it sends 
> the KILL signal to the process.  The start could use a longer wait too.
> Also, the RMI_PORT is calculated by simply prefixing the port number with a 
> "1" instead of adding 1.  If the solr port has five digits, then the rmi 
> port will be invalid, because it will be greater than 65535.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9644) MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts properly

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9644:
--
  Flags: Patch
Description: It seems SimpleMLTQParser and CloudMLTQParser should be able 
to handle boost parameters, but it's not working properly. I'll make a pull 
request to add tests and fix both.  (was: It seems SimpleMLTQParser and 
CouldMLTQParser should be able to handle boost parameters, but it's not working 
properly. I'll make a pull request to add tests and fix both.)
Summary: MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser 
don't handle boosts properly  (was: MoreLikeThis parser doesn't handle boosts 
properly)

> MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts 
> properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>  Labels: patch
>
> It seems SimpleMLTQParser and CloudMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9267) Cloud MLT field boost not working

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9267:
--
Attachment: (was: SOLR-9267.patch)

> Cloud MLT field boost not working
> -
>
> Key: SOLR-9267
> URL: https://issues.apache.org/jira/browse/SOLR-9267
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.3.2, 5.4, 5.4.1, 
> 5.5, 5.5.1, 5.5.2, 5.5.3, 5.6, 6.0, 6.0.1, 6.0.2, 6.1, 6.1.1, 6.2
>Reporter: Brian Feldman
>
> When boosting by field "fieldname otherFieldName^4.0" the boost is not 
> stripped from the field name when adding to fieldNames ArrayList.  So on line 
> 133 of CloudMLTQParser when adding field content to the filteredDocument the 
> field is not found (incorrectly trying to find 'otherFieldName^4.0').
> The easiest but perhaps hackiest solution is to overwrite qf:
> {code}
> if (localParams.get("boost") != null) {
>   mlt.setBoost(localParams.getBool("boost"));
>   boostFields = SolrPluginUtils.parseFieldBoosts(qf);
>   qf = boostFields.keySet().toArray(qf);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9267) Cloud MLT field boost not working

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15581365#comment-15581365
 ] 

Ere Maijala commented on SOLR-9267:
---

I'm moving my patch to address this to SOLR-9644 since the same functionality 
in the SimpleMLTQParser is affected too.

> Cloud MLT field boost not working
> -
>
> Key: SOLR-9267
> URL: https://issues.apache.org/jira/browse/SOLR-9267
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.3.2, 5.4, 5.4.1, 
> 5.5, 5.5.1, 5.5.2, 5.5.3, 5.6, 6.0, 6.0.1, 6.0.2, 6.1, 6.1.1, 6.2
>Reporter: Brian Feldman
>
> When boosting by field "fieldname otherFieldName^4.0" the boost is not 
> stripped from the field name when adding to fieldNames ArrayList.  So on line 
> 133 of CloudMLTQParser when adding field content to the filteredDocument the 
> field is not found (incorrectly trying to find 'otherFieldName^4.0').
> The easiest but perhaps hackiest solution is to overwrite qf:
> {code}
> if (localParams.get("boost") != null) {
>   mlt.setBoost(localParams.getBool("boost"));
>   boostFields = SolrPluginUtils.parseFieldBoosts(qf);
>   qf = boostFields.keySet().toArray(qf);
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9644) MoreLikeThis parser doesn't handle boosts properly

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9644:
--
Description: It seems SimpleMLTQParser and CouldMLTQParser should be able 
to handle boost parameters, but it's not working properly. I'll make a pull 
request to add tests and fix both.  (was: It seems SimpleMLTQParser should be 
able to handle boost parameters, but it's not working properly. I've added a 
proposed patch to fix similar issue in CloudMLTQParser in issue 
https://issues.apache.org/jira/browse/SOLR-9267 and will attach a patch here 
too.)

> MoreLikeThis parser doesn't handle boosts properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>
> It seems SimpleMLTQParser and CouldMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9818) Solr admin UI must not retry any request if it loses connection with the server

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-9818:
-

 Summary: Solr admin UI must not retry any request if it loses 
connection with the server
 Key: SOLR-9818
 URL: https://issues.apache.org/jira/browse/SOLR-9818
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: web gui
Affects Versions: 6.3
Reporter: Ere Maijala


It seems that whenever the Solr admin UI loses connection with the server, be 
the reason that the server is too slow to answer or that it's gone away 
completely, it starts hammering the server with the previous request until it 
gets a success response, it seems. That can be especially bad if the last 
attempted action was something like collection reload with a SolrCloud 
instance. The admin UI will quickly add hundreds of reload commands to 
overseer/collection-queue-work, which may essentially cause the replicas to get 
overloaded when they're trying to handle all the reload commands.

I believe the UI should never retry the previous command blindly when the 
connection is lost, but instead just ping the server until it responds again.

Steps to reproduce:
1.) Fire up Solr
2.) Open the admin UI in browser
3.) Open a web console in the browser to see the requests it sends
4.) Stop solr
5.) Try an action in the admin UI
6.) Observe the web console in browser quickly fill up with repeats of the 
originally attempted request



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9818) Solr admin UI rapidly retries any request(s) if it loses connection with the server

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9818:
--
Summary: Solr admin UI rapidly retries any request(s) if it loses 
connection with the server  (was: Solr admin UI must not retry any request if 
it loses connection with the server)

> Solr admin UI rapidly retries any request(s) if it loses connection with the 
> server
> ---
>
> Key: SOLR-9818
> URL: https://issues.apache.org/jira/browse/SOLR-9818
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: web gui
>Affects Versions: 6.3
>Reporter: Ere Maijala
>
> It seems that whenever the Solr admin UI loses connection with the server, be 
> the reason that the server is too slow to answer or that it's gone away 
> completely, it starts hammering the server with the previous request until it 
> gets a success response, it seems. That can be especially bad if the last 
> attempted action was something like collection reload with a SolrCloud 
> instance. The admin UI will quickly add hundreds of reload commands to 
> overseer/collection-queue-work, which may essentially cause the replicas to 
> get overloaded when they're trying to handle all the reload commands.
> I believe the UI should never retry the previous command blindly when the 
> connection is lost, but instead just ping the server until it responds again.
> Steps to reproduce:
> 1.) Fire up Solr
> 2.) Open the admin UI in browser
> 3.) Open a web console in the browser to see the requests it sends
> 4.) Stop solr
> 5.) Try an action in the admin UI
> 6.) Observe the web console in browser quickly fill up with repeats of the 
> originally attempted request



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9826) Shutting down leader when it's sending updates makes another active node go into recovery

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15721813#comment-15721813
 ] 

Ere Maijala commented on SOLR-9826:
---

The attached log is from node finna-index-2.

> Shutting down leader when it's sending updates makes another active node go 
> into recovery
> -
>
> Key: SOLR-9826
> URL: https://issues.apache.org/jira/browse/SOLR-9826
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.3
>Reporter: Ere Maijala
>  Labels: solrcloud
> Attachments: failure.log
>
>
> If the leader in SolrCloud is sending updates to a follower when it's shut 
> down, it forces the replica it can't communicate with (due to being shut 
> down, I assume) to go into recovery. I'll attach a log excerpt that shows the 
> related messages.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9826) Shutting down leader when it's sending updates makes another active node go into recovery

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15721808#comment-15721808
 ] 

Ere Maijala edited comment on SOLR-9826 at 12/5/16 9:58 AM:


Added a Solr log showing how the leader gets confused and sends another node 
into recovery.


was (Author: emaijala):
Solr log showing how the leader gets confused and sends another node into 
recovery.

> Shutting down leader when it's sending updates makes another active node go 
> into recovery
> -
>
> Key: SOLR-9826
> URL: https://issues.apache.org/jira/browse/SOLR-9826
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.3
>Reporter: Ere Maijala
>  Labels: solrcloud
> Attachments: failure.log
>
>
> If the leader in SolrCloud is sending updates to a follower when it's shut 
> down, it forces the replica it can't communicate with (due to being shut 
> down, I assume) to go into recovery. I'll attach a log excerpt that shows the 
> related messages.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9826) Shutting down leader when it's sending updates makes another active node go into recovery

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9826:
--
Attachment: failure.log

Solr log showing how the leader gets confused and sends another node into 
recovery.

> Shutting down leader when it's sending updates makes another active node go 
> into recovery
> -
>
> Key: SOLR-9826
> URL: https://issues.apache.org/jira/browse/SOLR-9826
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.3
>Reporter: Ere Maijala
>  Labels: solrcloud
> Attachments: failure.log
>
>
> If the leader in SolrCloud is sending updates to a follower when it's shut 
> down, it forces the replica it can't communicate with (due to being shut 
> down, I assume) to go into recovery. I'll attach a log excerpt that shows the 
> related messages.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9826) Shutting down leader when it's sending updates makes another active node go into recovery

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-9826:
-

 Summary: Shutting down leader when it's sending updates makes 
another active node go into recovery
 Key: SOLR-9826
 URL: https://issues.apache.org/jira/browse/SOLR-9826
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.3
Reporter: Ere Maijala


If the leader in SolrCloud is sending updates to a follower when it's shut 
down, it forces the replica it can't communicate with (due to being shut down, 
I assume) to go into recovery. I'll attach a log excerpt that shows the related 
messages.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9644) MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts properly

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-9644:
--
Attachment: SOLR-9644-branch_6x.patch
SOLR-9644-master.patch

Added patches for master and branch_6x. For some reason GitHub said that there 
are no conflicts, but the patch couldn't be applied, so I decided to attach 
proper patches.

> MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts 
> properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>  Labels: patch
> Attachments: SOLR-9644-branch_6x.patch, SOLR-9644-master.patch
>
>
> It seems SimpleMLTQParser and CloudMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9644) MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts properly

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15797989#comment-15797989
 ] 

Ere Maijala commented on SOLR-9644:
---

Sorry about that. I'm not sure why GitHub thought there were no conflicts, but 
proper patches now attached.

> MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts 
> properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>  Labels: patch
> Attachments: SOLR-9644-branch_6x.patch, SOLR-9644-master.patch
>
>
> It seems SimpleMLTQParser and CloudMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9644) MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts properly

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782944#comment-15782944
 ] 

Ere Maijala commented on SOLR-9644:
---

Thanks! Note that the commits from 17th should apply to 6x branch too (or at 
least they did at the time).

> MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts 
> properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>  Labels: patch
>
> It seems SimpleMLTQParser and CloudMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9120) o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_NNN] -- NoSuchFileException

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994650#comment-15994650
 ] 

Ere Maijala commented on SOLR-9120:
---

We also have encounter this often when trying to create backups, but I don't 
have a consistently reproducible test case. It's more like "index enough stuff 
and then try to create a backup". This is with a SolrCloud with three nodes and 
a collection with 1 shard, 3 replicas.

> o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_NNN] -- 
> NoSuchFileException
> 
>
> Key: SOLR-9120
> URL: https://issues.apache.org/jira/browse/SOLR-9120
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 6.0
>Reporter: Markus Jelsma
> Attachments: SOLR-9120.patch
>
>
> On Solr 6.0, we frequently see the following errors popping up:
> {code}
> java.nio.file.NoSuchFileException: 
> /var/lib/solr/logs_shard2_replica1/data/index/segments_2c5
>   at 
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>   at 
> sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
>   at 
> sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
>   at 
> sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
>   at java.nio.file.Files.readAttributes(Files.java:1737)
>   at java.nio.file.Files.size(Files.java:2332)
>   at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
>   at 
> org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:131)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:597)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:585)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:137)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
>   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2033)
>   at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>   at org.eclipse.jetty.server.Server.handle(Server.java:518)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
>   at 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
>   at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
>   at 
> org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
>   at 
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
>   at 
> org.eclipse.jett

[jira] [Comment Edited] (SOLR-9120) o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_NNN] -- NoSuchFileException

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15994650#comment-15994650
 ] 

Ere Maijala edited comment on SOLR-9120 at 5/3/17 10:49 AM:


We also encounter this often when trying to create backups, but I don't have a 
consistently reproducible test case. It's more like "index enough stuff and 
then try to create a backup". This is with a SolrCloud with three nodes and a 
collection with 1 shard, 3 replicas.


was (Author: emaijala):
We also have encounter this often when trying to create backups, but I don't 
have a consistently reproducible test case. It's more like "index enough stuff 
and then try to create a backup". This is with a SolrCloud with three nodes and 
a collection with 1 shard, 3 replicas.

> o.a.s.h.a.LukeRequestHandler Error getting file length for [segments_NNN] -- 
> NoSuchFileException
> 
>
> Key: SOLR-9120
> URL: https://issues.apache.org/jira/browse/SOLR-9120
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 6.0
>Reporter: Markus Jelsma
> Attachments: SOLR-9120.patch
>
>
> On Solr 6.0, we frequently see the following errors popping up:
> {code}
> java.nio.file.NoSuchFileException: 
> /var/lib/solr/logs_shard2_replica1/data/index/segments_2c5
>   at 
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>   at 
> sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
>   at 
> sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
>   at 
> sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
>   at java.nio.file.Files.readAttributes(Files.java:1737)
>   at java.nio.file.Files.size(Files.java:2332)
>   at org.apache.lucene.store.FSDirectory.fileLength(FSDirectory.java:243)
>   at 
> org.apache.lucene.store.NRTCachingDirectory.fileLength(NRTCachingDirectory.java:131)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.getFileLength(LukeRequestHandler.java:597)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:585)
>   at 
> org.apache.solr.handler.admin.LukeRequestHandler.handleRequestBody(LukeRequestHandler.java:137)
>   at 
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
>   at org.apache.solr.core.SolrCore.execute(SolrCore.java:2033)
>   at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
>   at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
>   at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
>   at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
>   at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
>   at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
>   at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
>   at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
>   at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
>   at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
>   at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
>   at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
>   at org.eclipse.jetty.server.Server.handle(Server.java:518)
>   at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
>   at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
>   at 
> org

[jira] [Commented] (SOLR-8096) Major faceting performance regressions

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153272#comment-16153272
 ] 

Ere Maijala commented on SOLR-8096:
---

[~guenterh], I believe one should be able to use Solr's date range fields as 
intended. Besides, it's not possible to handle multivalued date ranges with 
ints.

> Major faceting performance regressions
> --
>
> Key: SOLR-8096
> URL: https://issues.apache.org/jira/browse/SOLR-8096
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.3, 6.0
>Reporter: Yonik Seeley
>Priority: Critical
> Attachments: facetcache.diff, simple_facets.diff
>
>
> Use of the highly optimized faceting that Solr had for multi-valued fields 
> over relatively static indexes was removed as part of LUCENE-5666, causing 
> severe performance regressions.
> Here are some quick benchmarks to gauge the damage, on a 5M document index, 
> with each field having between 0 and 5 values per document.  *Higher numbers 
> represent worse 5x performance*.
> Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time  
> ||...|| Percent of index being faceted
> ||num_unique_values|| 10% || 50% || 90% ||
> |10   | 351.17%   | 1587.08%  | 3057.28% |
> |100  | 158.10%   | 203.61%   | 1421.93% |
> |1000 | 143.78%   | 168.01%   | 1325.87% |
> |1| 137.98%   | 175.31%   | 1233.97% |
> |10   | 142.98%   | 159.42%   | 1252.45% |
> |100  | 255.15%   | 165.17%   | 1236.75% |
> For example, a field with 1000 unique values in the whole index, faceting 
> with 5x took 143% of the 4x time, when ~10% of the docs in the index were 
> faceted.
> One user who brought the performance problem to our attention: 
> http://markmail.org/message/ekmqh4ocbkwxv3we
> "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3)
> The disabling of the UnInvertedField algorithm was previously discovered in 
> SOLR-7190, but we didn't know just how bad the problem was at that time.
> edit: removed "secret" adverb by request



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-8096) Major faceting performance regressions

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150525#comment-16150525
 ] 

Ere Maijala commented on SOLR-8096:
---

Chiming in as one of those affected by performance issues with faceting. I've 
been testing with a 57 million record index of bibliographic data. A faceting 
request that used to take around 20ms in Solr 4.10.2 is at least 2600ms in Solr 
6.6.0. While in general I find it fine to change the default behavior to 
something that works better than before for a majority of use cases, there 
should be a way to maintain performance in other cases. 

My main issue at the moment is that even facet.method=uif is slow if you 
request more than a few items. In a smaller test index of 6 million records I 
can get the top 20 results in 4ms, but facet.limit=200 takes ~100ms and 
facet.limit=2000 takes ~1300ms (the facet has 1960 buckets). Params user for 
the query:

q=*:*=0=true=building=1=[20-2000]=true=uif
 

Anyway, here's a list of issues that, for me, seem to be contribute to all the 
confusion around faceting performance:

# As far as I can see, facet.method=uif is completely undocumented apart from a 
short entry in release notes.
# Also undocumented is the fact (as observed during testing) that docValues 
must not be enabled for facet.method=uif to do any good. Otherwise the 
performance can be even worse than with FC.
# There's no proper documentation on what the introduction of docValues means 
in practice. There are several articles about what good it brings but I 
couldn't find much of analysis on any possible downsides.
# facet.method=uif with Solr 6.6.0 is still very slow compared to that in Solr 
4.10.2 if you request more than a few entries.
# There was no way to get back UIF before SOLR-8466.
# Changes in behavior haven't really been documented. This is how the 
introduction of docValues was documented in the release notes of Solr 4.2.0: 
"SOLR-3855, SOLR-4490: Doc values support". That doesn't help a poor developer 
like me to get the big picture. Then I read in 
https://lucidworks.com/2013/04/02/fun-with-docvalues-in-solr-4-2/ that compared 
to what we used to have _"DocValues aim to alleviate both of these problems 
while keeping performance comparable."_ Of course that's just something I read 
on internet, but so far it's the best description of docValues I've read and 
makes it sound like there won't be significant performance differences.
# It should be possible to make an informed decision to go with something that 
uses more JVM memory and is slower to warm up if required by the use-case. This 
is difficult because information is so scattered and the Solr reference guide 
doesn't go into much detail. For instance the effect of docValues is not 
mentioned in the reference guide where facet.method is described.
# Solr'd documentation on DocValues 
(https://lucene.apache.org/solr/guide/6_6/docvalues.html) highlights the 
positive effects it has on performance, memory consumption etc. It starts with 
_"DocValues are a way of recording field values internally that is more 
efficient for some purposes, such as sorting and faceting, than traditional 
indexing."_ That sounds like something you should enable as quickly as possible 
to reap the benefits!
# Discussions about docValues in solr-user list also mostly recomment enabling 
docValues without discussing any caveats.

> Major faceting performance regressions
> --
>
> Key: SOLR-8096
> URL: https://issues.apache.org/jira/browse/SOLR-8096
> Project: Solr
>  Issue Type: Bug
>Affects Versions: 5.0, 5.1, 5.2, 5.3, 6.0
>Reporter: Yonik Seeley
>Priority: Critical
> Attachments: facetcache.diff, simple_facets.diff
>
>
> Use of the highly optimized faceting that Solr had for multi-valued fields 
> over relatively static indexes was removed as part of LUCENE-5666, causing 
> severe performance regressions.
> Here are some quick benchmarks to gauge the damage, on a 5M document index, 
> with each field having between 0 and 5 values per document.  *Higher numbers 
> represent worse 5x performance*.
> Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time  
> ||...|| Percent of index being faceted
> ||num_unique_values|| 10% || 50% || 90% ||
> |10   | 351.17%   | 1587.08%  | 3057.28% |
> |100  | 158.10%   | 203.61%   | 1421.93% |
> |1000 | 143.78%   | 168.01%   | 1325.87% |
> |1| 137.98%   | 175.31%   | 1233.97% |
> |10   | 142.98%   | 159.42%   | 1252.45% |
> |100  | 255.15%   | 165.17%   | 1236.75% |
> For example, a field with 1000 unique values

[jira] [Commented] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500222#comment-16500222
 ] 

Ere Maijala commented on SOLR-11982:


That's an issue if its type is still TLOG while it's serving as the leader. I'm 
under the impression that this should be a temporary situation, however, so it 
might very well be justified e.g. if the replica is on hardware that has the 
best capacity for handling queries. That's a hairy situation, though, since 
it's difficult to say what's the best thing to do. The logic could probably be 
extended with an option to prefer non-leaders.

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.4, master (8.0)
>Reporter: Ere Maijala
>Assignee: Tomás Fernández Löbbe
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16501390#comment-16501390
 ] 

Ere Maijala commented on SOLR-11982:


I think it would be good to open a new issue for e.g the isLeader option and 
any further discussion on how the different cases should work.

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.4, master (8.0)
>Reporter: Ere Maijala
>Assignee: Tomás Fernández Löbbe
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11811) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11811:
---
Description: 
While ICUNormalizer2FilterFactory supports a filter attribute to define a 
Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
allows one to e.g. exclude a set of characters from being folded. E.g. for 
Finnish and Swedish the filter could be defined like this:

  

Note: An additional MappingCharFilterFactory or solr.LowerCaseFilterFactory 
would be needed for lowercasing the characters excluded from folding. This is 
similar to what ElasticSearch provides (see 
https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu-folding.html).

I'll add a patch that does this similar to ICUNormalizer2FilterFactory. Applies 
at least to master and branch_7x.



  was:
While ICUNormalizer2FilterFactory supports a filter attribute to define a 
Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
allows one to e.g. exclude a set of characters from being folded. E.g. for 
Finnish and Swedish the filter could be defined like this:

  

(Note: An additional MappingCharFilterFactory for lowercasing the characters 
excluded from folding would be needed for perfect results.)

I'll add a patch that does this similar to ICUNormalizer2FilterFactory. Applies 
at least to master and branch_7x.




> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: SOLR-11811
> URL: https://issues.apache.org/jira/browse/SOLR-11811
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory
> Attachments: SOLR-11811.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> Note: An additional MappingCharFilterFactory or solr.LowerCaseFilterFactory 
> would be needed for lowercasing the characters excluded from folding. This is 
> similar to what ElasticSearch provides (see 
> https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu-folding.html).
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11811) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11811:
---
Attachment: SOLR-11811.patch

Added a patch that includes a test for the filter parameter.

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: SOLR-11811
> URL: https://issues.apache.org/jira/browse/SOLR-11811
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory
> Attachments: SOLR-11811.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> (Note: An additional MappingCharFilterFactory for lowercasing the characters 
> excluded from folding would be needed for perfect results.)
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11811) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11811:
---
Attachment: (was: SOLR-11811.patch)

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: SOLR-11811
> URL: https://issues.apache.org/jira/browse/SOLR-11811
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory
> Attachments: SOLR-11811.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> (Note: An additional MappingCharFilterFactory for lowercasing the characters 
> excluded from folding would be needed for perfect results.)
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11870) Ref Guide: Add filter parameter to ICU filters

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-11870:
--

 Summary: Ref Guide: Add filter parameter to ICU filters
 Key: SOLR-11870
 URL: https://issues.apache.org/jira/browse/SOLR-11870
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: documentation
Affects Versions: master (8.0), 7.3
Reporter: Ere Maijala






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11870) Ref Guide: Add filter parameter to ICU filters

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11870:
---
Attachment: SOLR-11870.patch

> Ref Guide: Add filter parameter to ICU filters
> --
>
> Key: SOLR-11870
> URL: https://issues.apache.org/jira/browse/SOLR-11870
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
> Attachments: SOLR-11870.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8129) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated LUCENE-8129:

Lucene Fields: New,Patch Available  (was: New)

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: LUCENE-8129
> URL: https://issues.apache.org/jira/browse/LUCENE-8129
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory, patch-available, patch-with-test
> Attachments: LUCENE-8129.patch, LUCENE-8129.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> Note: An additional MappingCharFilterFactory or solr.LowerCaseFilterFactory 
> would be needed for lowercasing the characters excluded from folding. This is 
> similar to what ElasticSearch provides (see 
> https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu-folding.html).
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11881) Connection Reset Causing LIR

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334040#comment-16334040
 ] 

Ere Maijala commented on SOLR-11881:


Does this also fix SOLR-9826?

> Connection Reset Causing LIR
> 
>
> Key: SOLR-11881
> URL: https://issues.apache.org/jira/browse/SOLR-11881
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Priority: Major
> Attachments: SOLR-11881.patch
>
>
> We can see that a connection reset is causing LIR.
> If a leader -> replica update get's a connection like this the leader will 
> initiate LIR
> {code}
> 2018-01-08 17:39:16.980 ERROR (qtp1010030701-468988) [c:person s:shard7_1 
> r:core_node56 x:person_shard7_1_replica1] 
> o.a.s.u.p.DistributedUpdateProcessor Setting up to try to start recovery on 
> replica https://host08.domain:8985/solr/person_shard7_1_replica2/
> java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
> at sun.security.ssl.InputRecord.read(InputRecord.java:503)
> at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
> at 
> sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
> at 
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
> at 
> sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
> at 
> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:543)
> at 
> org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:409)
> at 
> org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
> at 
> org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
> at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
> at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at 
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:312)
> at 
> org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:185)
> at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> From https://issues.apache.org/jira/browse/SOLR-6931 Mark says "On a heavy 
> working SolrCloud cluster, even a rare response like this from a replica can 
> cause a recovery and heavy cluster disruption" . 
> Looking at SOLR-6931 we added a http retry handler but we only retry on GET 
> requests. Updates are POST requests 
> {{ConcurrentUpdateSolrClient#sendUpdateStream}}
> Update requests between the leader and replica should be retry-able since 
> they have been versioned. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8129) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated LUCENE-8129:

Attachment: LUCENE-8129.patch

Updated patch with {{normalizer}} renamed to {{NORMALIZER}}.

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: LUCENE-8129
> URL: https://issues.apache.org/jira/browse/LUCENE-8129
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory, patch-available, patch-with-test
> Attachments: LUCENE-8129.patch, LUCENE-8129.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> Note: An additional MappingCharFilterFactory or solr.LowerCaseFilterFactory 
> would be needed for lowercasing the characters excluded from folding. This is 
> similar to what ElasticSearch provides (see 
> https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu-folding.html).
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11849) Core recovery fails to complete if warmup query fails due to exceeding timeAllowed

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-11849:
--

 Summary: Core recovery fails to complete if warmup query fails due 
to exceeding timeAllowed
 Key: SOLR-11849
 URL: https://issues.apache.org/jira/browse/SOLR-11849
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: search
Affects Versions: 7.2
Reporter: Ere Maijala
Priority: Minor


Core init or recovery never completes if a warmup query fails to complete when 
timeAllowed is specified as a default parameter in requestHandler settings and 
the warmup query execution exceeds it. In this case an exception is logged but 
the recovery never completes. It's of course possible to include another value 
for timeAllowed in the warmup query, but I believe this could be handled in a 
more robust manner, such as ignoring timeAllowed for warmup or binging the core 
online regardless of the timeout.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11849) Core recovery fails to complete if warmup query fails due to exceeding timeAllowed

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11849:
---

Related to SOLR-4408 but not the same, I think.

> Core recovery fails to complete if warmup query fails due to exceeding 
> timeAllowed
> --
>
> Key: SOLR-11849
> URL: https://issues.apache.org/jira/browse/SOLR-11849
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: search
>Affects Versions: 7.2
>Reporter: Ere Maijala
>Priority: Minor
>
> Core init or recovery never completes if a warmup query fails to complete 
> when timeAllowed is specified as a default parameter in requestHandler 
> settings and the warmup query execution exceeds it. In this case an exception 
> is logged but the recovery never completes. It's of course possible to 
> include another value for timeAllowed in the warmup query, but I believe this 
> could be handled in a more robust manner, such as ignoring timeAllowed for 
> warmup or binging the core online regardless of the timeout.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8129) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated LUCENE-8129:

Attachment: (was: SOLR-11811.patch)

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: LUCENE-8129
> URL: https://issues.apache.org/jira/browse/LUCENE-8129
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory, patch-available, patch-with-test
> Attachments: LUCENE-8129.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> Note: An additional MappingCharFilterFactory or solr.LowerCaseFilterFactory 
> would be needed for lowercasing the characters excluded from folding. This is 
> similar to what ElasticSearch provides (see 
> https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu-folding.html).
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8129) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated LUCENE-8129:

Attachment: LUCENE-8129.patch

Thanks, it was indeed bad. 

I checked that Normalizer2.getInstance calls Norm2AllModes.getInstance which 
returns a cached instance if available, so I believe you're right about it 
being immutable. An improved patch is attached.

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: LUCENE-8129
> URL: https://issues.apache.org/jira/browse/LUCENE-8129
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory, patch-available, patch-with-test
> Attachments: LUCENE-8129.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> Note: An additional MappingCharFilterFactory or solr.LowerCaseFilterFactory 
> would be needed for lowercasing the characters excluded from folding. This is 
> similar to what ElasticSearch provides (see 
> https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu-folding.html).
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-11982:
--

 Summary: Add support for preferReplicaTypes parameter
 Key: SOLR-11982
 URL: https://issues.apache.org/jira/browse/SOLR-11982
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: SolrCloud
Affects Versions: master (8.0), 7.3
Reporter: Ere Maijala


It would be nice to have the possibility to easily prefer certain replica types 
in a similar fashion to preferLocalShards. I'll be coming up with a patch that 
allows one to specify e.g. preferReplicaTypes=PULL,TLOG which would mean that 
NRT replicas wouldn't be hit with queries unless they're the only ones 
available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982.patch

> Add support for preferReplicaTypes parameter
> 
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
> Attachments: SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica 
> types in a similar fashion to preferLocalShards. I'll be coming up with a 
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which 
> would mean that NRT replicas wouldn't be hit with queries unless they're the 
> only ones available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
 Flags: Patch
Labels: patch-available patch-with-test  (was: )

> Add support for preferReplicaTypes parameter
> 
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica 
> types in a similar fashion to preferLocalShards. I'll be coming up with a 
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which 
> would mean that NRT replicas wouldn't be hit with queries unless they're the 
> only ones available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982-v3.patch

> Add support for preferReplicaTypes parameter
> 
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-v2.patch, SOLR-11982-v3.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica 
> types in a similar fashion to preferLocalShards. I'll be coming up with a 
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which 
> would mean that NRT replicas wouldn't be hit with queries unless they're the 
> only ones available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982-v2.patch

> Add support for preferReplicaTypes parameter
> 
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-v2.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica 
> types in a similar fashion to preferLocalShards. I'll be coming up with a 
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which 
> would mean that NRT replicas wouldn't be hit with queries unless they're the 
> only ones available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371431#comment-16371431
 ] 

Ere Maijala commented on SOLR-11982:


Patch v3 attached. This makes sure to use a new collection for the replica type 
test and that there are always replicas for all types.

> Add support for preferReplicaTypes parameter
> 
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-v2.patch, SOLR-11982-v3.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica 
> types in a similar fashion to preferLocalShards. I'll be coming up with a 
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which 
> would mean that NRT replicas wouldn't be hit with queries unless they're the 
> only ones available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374052#comment-16374052
 ] 

Ere Maijala commented on SOLR-11982:


[~tomasflobbe] I agree that it makes sense to unify the syntax. I'm not sure, 
though, that this should include preferLocalShards. I want to be able to say 
that I prefer local PULL replicas over remote PULL replicas, and with your 
proposed API that's not possible, right?

> Add support for preferReplicaTypes parameter
> 
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-v2.patch, SOLR-11982-v3.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica 
> types in a similar fashion to preferLocalShards. I'll be coming up with a 
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which 
> would mean that NRT replicas wouldn't be hit with queries unless they're the 
> only ones available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371370#comment-16371370
 ] 

Ere Maijala commented on SOLR-11982:


I've attached a new patch that should address all the comments. Sorry for the 
whitespace changes, they should be reverted now.

[~cpoerschke] Thanks for the feedback! The original patch didn't take the order 
of the preferred replica types to account, but the new patch does, since it can 
be useful too. I also tried to clarify the documentation so that the use case 
for both parameters is more clear. Does this help?

[~erickerickson] Thanks, that was indeed a typo, now fixed.

> Add support for preferReplicaTypes parameter
> 
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-v2.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica 
> types in a similar fashion to preferLocalShards. I'll be coming up with a 
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which 
> would mean that NRT replicas wouldn't be hit with queries unless they're the 
> only ones available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for preferReplicaTypes parameter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16371415#comment-16371415
 ] 

Ere Maijala commented on SOLR-11982:


I'm getting intermittent test failures with the v2 patch, investigating...

> Add support for preferReplicaTypes parameter
> 
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-v2.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily prefer certain replica 
> types in a similar fashion to preferLocalShards. I'll be coming up with a 
> patch that allows one to specify e.g. preferReplicaTypes=PULL,TLOG which 
> would mean that NRT replicas wouldn't be hit with queries unless they're the 
> only ones available.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-11811) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

Ere Maijala created SOLR-11811:
--

 Summary: Support for defining a Unicode set filter when using 
ICUFoldingFilter
 Key: SOLR-11811
 URL: https://issues.apache.org/jira/browse/SOLR-11811
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Schema and Analysis
Reporter: Ere Maijala
Priority: Minor


ne a Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
allows one to e.g. exclude a set of characters from being folded. E.g. for 
Finnish and Swedish the filter could be defined like this:

  

(Note: An additional MappingCharFilterFactory for lowercasing the characters 
excluded from folding would be needed for perfect results.)

I'll add a patch that does this similar to ICUNormalizer2FilterFactory. Applies 
at least to master and branch_7x.





--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11811) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11811:
---
 Flags: Patch
Attachment: SOLR-11811.patch

Attached a patch.

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: SOLR-11811
> URL: https://issues.apache.org/jira/browse/SOLR-11811
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory
> Attachments: SOLR-11811.patch
>
>
> ne a Unicode set filter, ICUFoldingFilterFactory does not support it. A 
> filter allows one to e.g. exclude a set of characters from being folded. E.g. 
> for Finnish and Swedish the filter could be defined like this:
>   
> (Note: An additional MappingCharFilterFactory for lowercasing the characters 
> excluded from folding would be needed for perfect results.)
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11811) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309512#comment-16309512
 ] 

Ere Maijala commented on SOLR-11811:


Tests are still passing with the patch on branch_7x and master.

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: SOLR-11811
> URL: https://issues.apache.org/jira/browse/SOLR-11811
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory
> Attachments: SOLR-11811.patch
>
>
> ne a Unicode set filter, ICUFoldingFilterFactory does not support it. A 
> filter allows one to e.g. exclude a set of characters from being folded. E.g. 
> for Finnish and Swedish the filter could be defined like this:
>   
> (Note: An additional MappingCharFilterFactory for lowercasing the characters 
> excluded from folding would be needed for perfect results.)
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11811) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11811:
---
Description: 
While ICUNormalizer2FilterFactory supports a filter attribute to define a 
Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
allows one to e.g. exclude a set of characters from being folded. E.g. for 
Finnish and Swedish the filter could be defined like this:

  

(Note: An additional MappingCharFilterFactory for lowercasing the characters 
excluded from folding would be needed for perfect results.)

I'll add a patch that does this similar to ICUNormalizer2FilterFactory. Applies 
at least to master and branch_7x.



  was:
ne a Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
allows one to e.g. exclude a set of characters from being folded. E.g. for 
Finnish and Swedish the filter could be defined like this:

  

(Note: An additional MappingCharFilterFactory for lowercasing the characters 
excluded from folding would be needed for perfect results.)

I'll add a patch that does this similar to ICUNormalizer2FilterFactory. Applies 
at least to master and branch_7x.




> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: SOLR-11811
> URL: https://issues.apache.org/jira/browse/SOLR-11811
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory
> Attachments: SOLR-11811.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> (Note: An additional MappingCharFilterFactory for lowercasing the characters 
> excluded from folding would be needed for perfect results.)
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11811) Support for defining a Unicode set filter when using ICUFoldingFilter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11811:
---
Labels: ICUFoldingFilterFactory patch-available patch-with-test  (was: 
ICUFoldingFilterFactory)

> Support for defining a Unicode set filter when using ICUFoldingFilter
> -
>
> Key: SOLR-11811
> URL: https://issues.apache.org/jira/browse/SOLR-11811
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>    Reporter: Ere Maijala
>Priority: Minor
>  Labels: ICUFoldingFilterFactory, patch-available, patch-with-test
> Attachments: SOLR-11811.patch
>
>
> While ICUNormalizer2FilterFactory supports a filter attribute to define a 
> Unicode set filter, ICUFoldingFilterFactory does not support it. A filter 
> allows one to e.g. exclude a set of characters from being folded. E.g. for 
> Finnish and Swedish the filter could be defined like this:
>   
> Note: An additional MappingCharFilterFactory or solr.LowerCaseFilterFactory 
> would be needed for lowercasing the characters excluded from folding. This is 
> similar to what ElasticSearch provides (see 
> https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-icu-folding.html).
> I'll add a patch that does this similar to ICUNormalizer2FilterFactory. 
> Applies at least to master and branch_7x.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for shards.sort parameter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16391134#comment-16391134
 ] 

Ere Maijala commented on SOLR-11982:


(somehow failed to send this comment yesterday)

Thanks for the review. I believe I've addressed the review comments, but please 
let me know if there are any remaining issues.

> Add support for shards.sort parameter
> -
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with {{shards.sor=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982.patch

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982-preferReplicaTypes.patch

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Affects Version/s: (was: 7.3)
   7.4

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.4, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982.patch

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.4, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395136#comment-16395136
 ] 

Ere Maijala commented on SOLR-11982:


I moved the CHANGES.txt entry to version 7.4.

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.4, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16395076#comment-16395076
 ] 

Ere Maijala commented on SOLR-11982:


Added a new patch that should address the latest review comments. I also put 
all the strings to ShardParams and renamed replicaType to replica.type and 
replicaLocation to replica.location.

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982.patch

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, 
> SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for shards.sort parameter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392528#comment-16392528
 ] 

Ere Maijala commented on SOLR-11982:


I don't feel strongly about either naming convention, but how do we proceed 
from here to agree on something?

> Add support for shards.sort parameter
> -
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for shards.sort parameter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Description: It would be nice to have the possibility to easily sort the 
shards in the preferred order e.g. by replica type. The attached patch adds 
support for {{shards.sort}} parameter that allows one to sort e.g. PULL and 
TLOG replicas first with \{{shards.sort=replicaType:PULL|TLOG }}(which would 
mean that NRT replicas wouldn't be hit with queries unless they're the only 
ones available) and/or to sort by replica location (like preferLocalShards=true 
but more versatile).  (was: It would be nice to have the possibility to easily 
sort the shards in the preferred order e.g. by replica type. The attached patch 
adds support for {{shards.sort}} parameter that allows one to sort e.g. PULL 
and TLOG replicas first with {{shards.sor=replicaType:PULL|TLOG }}(which would 
mean that NRT replicas wouldn't be hit with queries unless they're the only 
ones available) and/or to sort by replica location (like preferLocalShards=true 
but more versatile).)

> Add support for shards.sort parameter
> -
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for indicating preferred replica types for queries

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Summary: Add support for indicating preferred replica types for queries  
(was: Add support for shards.sort parameter)

> Add support for indicating preferred replica types for queries
> --
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for shards.sort parameter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982-preferReplicaTypes.patch

> Add support for shards.sort parameter
> -
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11982) Add support for shards.sort parameter

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16392615#comment-16392615
 ] 

Ere Maijala commented on SOLR-11982:


I attached SOLR-11982-preferReplicaTypes.patch, which is a refined version of 
the original patch to add preferReplicaTypes parameter. This is a smaller 
change to the current functionality and does not require deprecating 
preferLocalShards, but then it doesn't allow as much flexibility as the 
shards.sort version. Both version fulfill my needs, but I'd really appreciate a 
clear decision which way is to be preferred.

[~tomasflobbe], I'll wait for the decision before addressing the latest review. 
You're right in that this doesn't work for single-shard collections but I don't 
think it needs to since the client, whatever it is, has full control over where 
to send the query.

> Add support for shards.sort parameter
> -
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.3, master (8.0)
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982-preferReplicaTypes.patch, SOLR-11982.patch, 
> SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with \{{shards.sort=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11982) Add support for shards.sort parameter

[Ere Maijala (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ere Maijala updated SOLR-11982:
---
Attachment: SOLR-11982.patch

> Add support for shards.sort parameter
> -
>
> Key: SOLR-11982
> URL: https://issues.apache.org/jira/browse/SOLR-11982
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (8.0), 7.3
>Reporter: Ere Maijala
>Priority: Minor
>  Labels: patch-available, patch-with-test
> Attachments: SOLR-11982.patch, SOLR-11982.patch, SOLR-11982.patch, 
> SOLR-11982.patch
>
>
> It would be nice to have the possibility to easily sort the shards in the 
> preferred order e.g. by replica type. The attached patch adds support for 
> {{shards.sort}} parameter that allows one to sort e.g. PULL and TLOG replicas 
> first with {{shards.sor=replicaType:PULL|TLOG }}(which would mean that NRT 
> replicas wouldn't be hit with queries unless they're the only ones available) 
> and/or to sort by replica location (like preferLocalShards=true but more 
> versatile).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org