[jira] [Commented] (LUCENE-2461) Forrest skin resource docs/skin/fontsize.js causes annoying javascript alert when embedded docs are viewed locally with Chrome

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550944#comment-15550944
 ] 

Lewis John McGibbney commented on LUCENE-2461:
--

This issue can be closed as wont fix?

> Forrest skin resource docs/skin/fontsize.js causes annoying javascript alert 
> when embedded docs are viewed locally with Chrome
> --
>
> Key: LUCENE-2461
> URL: https://issues.apache.org/jira/browse/LUCENE-2461
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/other
>Affects Versions: 3.0.1
> Environment: linux/Google chrome
>Reporter: Sami Siren
>Priority: Trivial
> Attachments: LUCENE-2461.patch
>
>
> When displaying the documentation from local filesystem (file:// urls) a 
> JavaScript alert popup is displayed on every forrest generated page:
> "Error: SECURITY_ERR: DOM Exception 18"
> This seems to be related to Chrome not allowing reading cookies for such urls.
> One could fix this by patching the javascript file (docs/skin/fontsize.js) to 
> check for the protocol. 
> It does not seem to be currently possible to alter the text size on the skin 
> at all? If so then perhaps the whole .js file could be disabled instead?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 889 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/889/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.client.solrj.embedded.LargeVolumeBinaryJettyTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [TransactionLog] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
  at org.apache.solr.update.TransactionLog.(TransactionLog.java:188)  at 
org.apache.solr.update.UpdateLog.newTransactionLog(UpdateLog.java:344)  at 
org.apache.solr.update.UpdateLog.ensureLog(UpdateLog.java:859)  at 
org.apache.solr.update.UpdateLog.add(UpdateLog.java:428)  at 
org.apache.solr.update.UpdateLog.add(UpdateLog.java:415)  at 
org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:299)
  at 
org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:211)
  at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:166)
  at 
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67)
  at 
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
  at 
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:957)
  at 
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1112)
  at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:738)
  at 
org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
  at 
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:97)  
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:179)
  at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:135)
  at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:275)  
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:121)
  at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:240)  
at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:158)  
at 
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:186)
  at 
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107)
  at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54)  
at 
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
  at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
  at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:153)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:2106)  at 
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:653)  at 
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:302)
  at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:253)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
  at 
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:111)
  at 
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) 
 at 
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)  
at 
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
  at 
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
  at 
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)  
at 
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)  
at 
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) 
 at org.eclipse.jetty.server.Server.handle(Server.java:518)  at 
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)  at 
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)  at 
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
  at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)  at 
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) 
 at 

[jira] [Updated] (SOLR-9511) Retire using individual versions to request updates during PeerSync

[Pushkar Raste (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pushkar Raste updated SOLR-9511:

Attachment: SOLR-9511.patch

> Retire using individual versions to request updates during PeerSync
> ---
>
> Key: SOLR-9511
> URL: https://issues.apache.org/jira/browse/SOLR-9511
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: master (7.0)
>Reporter: Pushkar Raste
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: SOLR-9511.patch
>
>
> We started using version ranges to request updates during PeerSync in 
> [SOLR-9207| https://issues.apache.org/jira/browse/SOLR-9207]. Using version 
> ranges was also made default.
> There is no need to have code that uses individual versions start Solr 7. 
> Decommission (remove unnecessary code)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9546) There is a lot of unnecessary boxing/unboxing going on in {{SolrParams}} class

[Pushkar Raste (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pushkar Raste updated SOLR-9546:

Attachment: SOLR-9546.patch

> There is a lot of unnecessary boxing/unboxing going on in {{SolrParams}} class
> --
>
> Key: SOLR-9546
> URL: https://issues.apache.org/jira/browse/SOLR-9546
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Pushkar Raste
>Priority: Minor
> Attachments: SOLR-9546.patch
>
>
> Here is an excerpt 
> {code}
>   public Long getLong(String param, Long def) {
> String val = get(param);
> try {
>   return val== null ? def : Long.parseLong(val);
> }
> catch( Exception ex ) {
>   throw new SolrException( SolrException.ErrorCode.BAD_REQUEST, 
> ex.getMessage(), ex );
> }
>   }
> {code}
> {{Long.parseLong()}} returns a primitive type but since method expect to 
> return a {{Long}}, it needs to be wrapped. There are many more method like 
> that. We might be creating a lot of unnecessary objects here.
> I am not sure if JVM catches upto it and somehow optimizes it if these 
> methods are called enough times (or may be compiler does some modifications 
> at compile time)
> Let me know if I am thinking of some premature optimization



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9592) decorateDocValues cause serious performance issue because of using slowCompositeReaderWrapper

[Takahiro Ishikawa (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550289#comment-15550289
 ] 

Takahiro Ishikawa commented on SOLR-9592:
-

Thanks for commit! [~yo...@apache.org]!

> decorateDocValues cause serious performance issue because of using 
> slowCompositeReaderWrapper
> -
>
> Key: SOLR-9592
> URL: https://issues.apache.org/jira/browse/SOLR-9592
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers, search
>Affects Versions: 6.0, 6.1, 6.2
>Reporter: Takahiro Ishikawa
>Assignee: Yonik Seeley
>  Labels: performance
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9592.patch, SOLR-9592.patch, SOLR-9592_6x.patch
>
>
> I have serious performance issue using AtomicUpdate (and RealtimeGet) with 
> non stored docValues.
> Because decorateDocValues try to merge each leafLeader on the fly via 
> slowCompositeReaderWrapper and it’s extremely slow (> 10sec).
> Simply access docValues via nonCompositeReader could resolve this 
> issue.(patch) 
> AtomicUpdate performance(or RealtimeGet performance)
> * Environment
> ** solr version : 6.0.0
> ** schema ~ 100 fields(90% docValues, some of those are multi valued)
> ** index : 5,000,000
> * Performance
> ** original :  > 10sec per query
> ** patched : at least 100msec per query
> This patch will also enhance search performance, because DocStreamer also 
> fetch docValues via decorateDocValues.
> Though it depends on each environment, I could take 20% search performance 
> gain.
> (This patch originally written for solr 6.0.0, and now rewritten for master)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-4736) Support group.mincount for Result Grouping

[jefferyyuan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-4736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jefferyyuan updated SOLR-4736:
--
Issue Type: Improvement  (was: Bug)

> Support group.mincount for Result Grouping
> --
>
> Key: SOLR-4736
> URL: https://issues.apache.org/jira/browse/SOLR-4736
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Affects Versions: 4.2
>Reporter: jefferyyuan
>Priority: Minor
>  Labels: group, solr
> Fix For: 4.9, 6.0
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> Result Grouping is a very useful feature: we can use it to find duplicate 
> data in index, but it lacks of one feature-group.mincount. 
> With group.mincount, we can specify that only groups that has equal or more 
> than ${mincount} for the group field will be returned.
> Specially, we can use group.mincount=2 to only return duplicate data.
> Could we add this in future release? Thanks. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5701) Allow DocTransformer to add arbitrary fields

[jefferyyuan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jefferyyuan updated SOLR-5701:
--
Issue Type: Improvement  (was: Bug)

> Allow DocTransformer to add arbitrary fields
> 
>
> Key: SOLR-5701
> URL: https://issues.apache.org/jira/browse/SOLR-5701
> Project: Solr
>  Issue Type: Improvement
>  Components: search
>Reporter: jefferyyuan
>  Labels: search
> Fix For: 4.9, 6.0
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> DocTransformer is very powerful, and allow us to add/remove or update fields 
> before returning.
> One limit we don't like is that it can only add one field, and the field name 
> must be [transformer_name].
> We may want to add multiple fields in one DocTransformer.
> One possible solution is to add method getFieldNames into DocTransformer.
> public abstract class DocTransformer{
>public void List getFieldNames() { return null; }
> }
> Then in SolrReturnFields.add(String, NamedList, DocTransformers, 
> SolrQueryRequest)
> Change augmenters.addTransformer( factory.create(disp, augmenterParams, req) 
> ); like below:
> DocTransformer docTransfomer = factory.create(disp, augmenterParams, req);
> SolrReturnFields.add(docTransfomer);
> then read fi3eldnames: docTransfomer.getFieldNames(); add them into 
> SolrReturnFields.
> DocTransfomer implementation would add all fields via doc.addField.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9463) SolrJ: Support Converter and make it easier to extend DocumentObjectBinder

[jefferyyuan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

jefferyyuan updated SOLR-9463:
--
Priority: Major  (was: Minor)

> SolrJ: Support Converter and make it easier to extend DocumentObjectBinder
> --
>
> Key: SOLR-9463
> URL: https://issues.apache.org/jira/browse/SOLR-9463
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: jefferyyuan
>  Labels: extensibility, solrj
>
> In our old project, we use Spring-Solr, it provides some good function such 
> as allow us to define converters to serialize java enum to solr string and 
> vice verse, serialize object as json string and vice verse.
> But it doesn't support latest solr, solr cloud and child documents.
> We would like to use pure solrj, but we do like spring solr's converters 
> function.
> Is it possible that SolrJ can support custom convert in SolrJ?
> Also SolrJ should make it easier to extend DocumentObjectBinder, such as make 
> DocField, infocache, getDocFields etc accessible in sub class.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_102) - Build # 6160 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6160/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseG1GC

5 tests failed.
FAILED:  org.apache.solr.TestDistributedSearch.test

Error Message:
Error from server at http://127.0.0.1:63531/kb/qx/collection1: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://[ff01::083]:2/kb/qx, 
http://[ff01::114]:2/kb/qx, http://127.0.0.1:63534/kb/qx/collection1, 
http://[ff01::213]:2/kb/qx]

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:63531/kb/qx/collection1: 
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available 
to handle this request:[http://[ff01::083]:2/kb/qx, 
http://[ff01::114]:2/kb/qx, http://127.0.0.1:63534/kb/qx/collection1, 
http://[ff01::213]:2/kb/qx]
at 
__randomizedtesting.SeedInfo.seed([996DD9CF787E3CFA:1139E615D6825102]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:608)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:260)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:249)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase.queryServer(BaseDistributedSearchTestCase.java:557)
at 
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:605)
at 
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:587)
at 
org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:566)
at 
org.apache.solr.TestDistributedSearch.test(TestDistributedSearch.java:839)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1011)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 

[jira] [Comment Edited] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549940#comment-15549940
 ] 

Mikhail Khludnev edited comment on SOLR-9604 at 10/5/16 9:04 PM:
-

attaching [^SOLR-9604-6x.patch] 
HttpSolrClient*ConPoolTest works (and fails) as expected.
ConnectionReuseTest asserts makes me wonder (I swapped two lines to fix one 
failure for null metrics). It fails so far SOLR-9608.  


was (Author: mkhludnev):
attaching [^SOLR-9604-6x.patch] 
HttpSolrClient*ConPoolTest works (and fails) as expected.
ConnectionReuseTest asserts makes me wonder. It fails so far. 

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604-6x.patch, SOLR-9604.patch, SOLR-9604.patch, 
> fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9604:
---
Attachment: SOLR-9604-6x.patch

attaching [^SOLR-9604-6x.patch] 
HttpSolrClient*ConPoolTest works (and fails) as expected.
ConnectionReuseTest asserts makes me wonder. It fails so far. 

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604-6x.patch, SOLR-9604.patch, SOLR-9604.patch, 
> fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7475) Sparse norms

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549939#comment-15549939
 ] 

Adrien Grand commented on LUCENE-7475:
--

No worries, it's a good change! I was just going to ask whether you would be 
against making {{longValue()}} throw an exception. :)

> Sparse norms
> 
>
> Key: LUCENE-7475
> URL: https://issues.apache.org/jira/browse/LUCENE-7475
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: LUCENE-7475.patch
>
>
> Even though norms now have an iterator API, they are still always dense in 
> practice since documents that do not have a value get assigned 0 as a norm 
> value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: lucene-solr:master: SOLR-9132: RulesTest must tear down collections at the end of each test

[Chris Hostetter]

It's not immediately obvious to me why these collection deletions can't be 
done in an @After method -- but if they need to live in each test method 
can we at least have an @After method that asserts no collections exist 
(via a STATUS call) so if someone writes a new test method but forgets to 
delete that collection then the @After method will catch it and give them 
a self explanatory failure instead of some future confusing/trappy error 
that depends on test order or what not?



: Date: Wed,  5 Oct 2016 14:28:21 + (UTC)
: From: romseyg...@apache.org
: Reply-To: dev@lucene.apache.org
: To: comm...@lucene.apache.org
: Subject: lucene-solr:master: SOLR-9132: RulesTest must tear down collections
: at the end of each test
: 
: Repository: lucene-solr
: Updated Branches:
:   refs/heads/master b6610b9f0 -> d398617be
: 
: 
: SOLR-9132: RulesTest must tear down collections at the end of each test
: 
: 
: Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
: Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/d398617b
: Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/d398617b
: Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/d398617b
: 
: Branch: refs/heads/master
: Commit: d398617be891c9bc4ac72f85bf6ba4bff81f4f89
: Parents: b6610b9
: Author: Alan Woodward 
: Authored: Wed Oct 5 15:18:46 2016 +0100
: Committer: Alan Woodward 
: Committed: Wed Oct 5 15:19:22 2016 +0100
: 
: --
:  .../src/test/org/apache/solr/cloud/rule/RulesTest.java| 10 ++
:  1 file changed, 10 insertions(+)
: --
: 
: 
: 
http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/d398617b/solr/core/src/test/org/apache/solr/cloud/rule/RulesTest.java
: --
: diff --git a/solr/core/src/test/org/apache/solr/cloud/rule/RulesTest.java 
b/solr/core/src/test/org/apache/solr/cloud/rule/RulesTest.java
: index 1b74cbe..eec3d93 100644
: --- a/solr/core/src/test/org/apache/solr/cloud/rule/RulesTest.java
: +++ b/solr/core/src/test/org/apache/solr/cloud/rule/RulesTest.java
: @@ -80,6 +80,8 @@ public class RulesTest extends SolrCloudTestCase {
:  CollectionAdminRequest.createShard(rulesColl, 
"shard2").process(cluster.getSolrClient());
:  CollectionAdminRequest.addReplicaToShard(rulesColl, 
"shard2").process(cluster.getSolrClient());
:  
: +
CollectionAdminRequest.deleteCollection(rulesColl).process(cluster.getSolrClient());
: +
:}
:  
:@Test
: @@ -102,6 +104,8 @@ public class RulesTest extends SolrCloudTestCase {
:  list = (List) rulesCollection.get("snitch");
:  assertEquals(1, list.size());
:  assertEquals ( "ImplicitSnitch", ((Map)list.get(0)).get("class"));
: +
: +
CollectionAdminRequest.deleteCollection(rulesColl).process(cluster.getSolrClient());
:}
:  
:@Test
: @@ -129,6 +133,8 @@ public class RulesTest extends SolrCloudTestCase {
:  list = (List) rulesCollection.get("snitch");
:  assertEquals(1, list.size());
:  assertEquals("ImplicitSnitch", list.get(0).get("class"));
: +
: +
CollectionAdminRequest.deleteCollection(rulesColl).process(cluster.getSolrClient());
:}
:  
:  
: @@ -151,6 +157,8 @@ public class RulesTest extends SolrCloudTestCase {
:  .setSnitch("class:ImplicitSnitch")
:  .process(cluster.getSolrClient());
:  
: +
CollectionAdminRequest.deleteCollection(rulesColl).process(cluster.getSolrClient());
: +
:}
:  
:  
: @@ -192,5 +200,7 @@ public class RulesTest extends SolrCloudTestCase {
:  list = (List) rulesCollection.get("snitch");
:  assertEquals(1, list.size());
:  assertEquals("ImplicitSnitch", ((Map) list.get(0)).get("class"));
: +
: +
CollectionAdminRequest.deleteCollection(rulesColl).process(cluster.getSolrClient());
:}
:  }
: 
: 

-Hoss
http://www.lucidworks.com/

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [DISCUSS] JIRA maintenance and response times

[Jeff Wartes]

I was thinking about this some more last night, and since I’m in software, 
here’s the software I think I’d want:

Create a list of Alerts, each of which know how to do two things: 1) Determine 
whether a given jira issue matches the alert 2) Determine an Alertee for a 
matching issue.
One of Alexandre’s examples in this context: 1) If the issue hasn’t had an 
update in a week, and the last update was by a committer, it’s a match. 2) The 
committer with the last comment is the Alertee.
Another example: 1) If the issue has a patch file or a github pull request 
comment, but isn’t assigned to anyone, it’s a match. 2) This dev mailing list 
is the Alertee

Every week, iterate through all unresolved issues, and give every Alert 
definition the chance to gather the list of matching tickets.
Then:
1. Dump each Alert’s list of issues on a web page someplace. Include diffs vs 
the last run, so you can see if a given Alert’s list is growing over time.
2. Get the distinct list of Alertee for all issues that matched any Alerts, and 
send each Alertee one email with the list of issues mapped to that Alertee, 
grouped by Alert type.

So you give periodic reminders to exactly the people who should get them, and 
you also have a place you can see this week’s status for everything. For bonus 
points, people could squelch getting the email if it only contained issues 
derived from certain Alerts.


On 10/5/16, 10:42 AM, "Alexandre Rafalovitch"  wrote:

 I believe the first 3 dashboards can be done in JIRA itself. I have
something similar, but unfortunately cannot share the dashboard (no
permission it seems).

The last one (breakdown of creation date? by year) is something that
JIRA does not seem to be able to give.

In terms of pulling data, I was able to build an API to pull up to 200
issues from JIRA at a time with full case information in the - as
Cassandra discovered - rather convoluted XML. So, it would not be too
hard to run several APIs and merge the results to get full export. And
then maybe run daily/weekly export and merge/override. Then, it is
pre-process and distribute.

It is totally doable. We just need to firm up the business
requirements. What kind of reports people feel would be useful for
_them_ to be more proactive. I've listed my ideas earlier, but not
sure if they would be useful to others.

Regards,
   Alex.
P.s. Do we need a JIRA for this?

Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/


On 6 October 2016 at 00:06, Cassandra Targett  wrote:
> In terms of a report, I found a template that uses JIRA's API to
> import issues to a Google Sheet:
> 
http://www.littlebluemonkey.com/blog/automatically-import-jira-backlog-into-google-spreadsheet
>
> I made some modifications to the original and have shared it for
> public viewing:
> 
https://docs.google.com/spreadsheets/d/1i1_XgabWM4CFnDLGyp3xNVXcZNvGDjww3HPk2hA4uNc/edit#gid=0
>
> I was able to pull in issues with my JIRA login without asking INFRA,
> so I think committers at least already have permissions to use the
> JIRA API.
>
> The template Sheet was initially created to make a board with cards
> for planning, but I have removed those tabs and the script that
> generates the cards. So ignore anything about "story cards" that you
> might see in instructions or anywhere else.
>
> There are a few issues with the API output that makes the data a
> little difficult to work with. Most of the fields I wanted include a
> bunch of metadata JIRA uses for display. And the date field won't
> convert from a timestamp to a date. The date issue is the worst of
> these, for aging purposes, but there might be something we can do in
> the script  that pulls the data to fix this.
>
> Despite that, it's still possible to make a simple report from this.
> I've added a tab "Stats" that shows some initial (easy) calculations -
> # of Issues by Type, # older than 2016 by types, # unassigned, a
> breakdown by year.
>
> Google's flavor of javascript (Google Apps Script) runs the sheet, so
> I think we could add to the main script, or add other scripts, if we
> want to. I know pretty close to zero javascript, however, so would
> need help there.
>
> JIRA can do a lot of this same stuff in its own dashboards (especially
> basic stuff), but that's very much opt-in by individual users. It
> sounds like some folks would prefer something pushed to the community.
> We could do a combination, however - I could take a stab at a shared
> dashboard people could opt into, and also send out a summary weekly
> based on a report generated somehow.
>
> I have no ego invested in the import-to-spreadsheet example I'm
> sharing 

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549766#comment-15549766
 ] 

Alan Woodward commented on SOLR-9604:
-

My commit didn't take into account CloudSolrClient, which also needs fixing 
here, and broke TestConnectionReuse (except when it selected CSC to test, which 
it obviously did when I ran things locally), so I've backed it out.

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9608) ConnectionReuseTest failures

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549763#comment-15549763
 ] 

Alan Woodward commented on SOLR-9608:
-

OK, I've reverted SOLR-9604 while I try and fix this.  It's a combination of 
test bug and error in CloudSolrClient, which meant that the test randomly 
passed with CSC (which wasn't re-using its connections properly!) and failed 
when either HttpSolrClient or ConcurrentUpdateSolrClient were used.

> ConnectionReuseTest failures
> 
>
> Key: SOLR-9608
> URL: https://issues.apache.org/jira/browse/SOLR-9608
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Alan Woodward
>
> My Jenkins has seen this test fail three times in the last couple hours, all 
> three reproduce for me - this is new, last failures were from May - here's 
> the first failure:
> {noformat}
> Checking out Revision 0eb6b1c823d347319cc0894b5fea95f085d4c8d4 
> (refs/remotes/origin/master)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=ConnectionReuseTest -Dtests.method=testConnectionReuse 
> -Dtests.seed=FA03A6C483303E7D -Dtests.slow=true -Dtests.locale=ga 
> -Dtests.timezone=Etc/GMT-9 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.09s J9  | ConnectionReuseTest.testConnectionReuse <<<
>[junit4]> Throwable #1: java.lang.AssertionError: We expected all 
> communication to use one connection! HttpSolrClient 1
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([FA03A6C483303E7D:DB3BCDAAAB786730]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.ConnectionReuseTest.testConnectionReuse(ConnectionReuseTest.java:158)
> {noformat}
> This test failed for me twice without specifying a seed, so it's likely 
> failing (close to) 100% of the time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549757#comment-15549757
 ] 

ASF subversion and git services commented on SOLR-9604:
---

Commit 36b3b0884a8f745bc137db1b1dc9890a59fa0895 in lucene-solr's branch 
refs/heads/master from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=36b3b08 ]

Revert "SOLR-9604: Ensure SSL connections are re-used"

This reverts commit 0eb6b1c823d347319cc0894b5fea95f085d4c8d4, which was
causing test failures in ConnectionReuseTest - see SOLR-9608


> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9608) ConnectionReuseTest failures

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549758#comment-15549758
 ] 

ASF subversion and git services commented on SOLR-9608:
---

Commit 36b3b0884a8f745bc137db1b1dc9890a59fa0895 in lucene-solr's branch 
refs/heads/master from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=36b3b08 ]

Revert "SOLR-9604: Ensure SSL connections are re-used"

This reverts commit 0eb6b1c823d347319cc0894b5fea95f085d4c8d4, which was
causing test failures in ConnectionReuseTest - see SOLR-9608


> ConnectionReuseTest failures
> 
>
> Key: SOLR-9608
> URL: https://issues.apache.org/jira/browse/SOLR-9608
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Alan Woodward
>
> My Jenkins has seen this test fail three times in the last couple hours, all 
> three reproduce for me - this is new, last failures were from May - here's 
> the first failure:
> {noformat}
> Checking out Revision 0eb6b1c823d347319cc0894b5fea95f085d4c8d4 
> (refs/remotes/origin/master)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=ConnectionReuseTest -Dtests.method=testConnectionReuse 
> -Dtests.seed=FA03A6C483303E7D -Dtests.slow=true -Dtests.locale=ga 
> -Dtests.timezone=Etc/GMT-9 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.09s J9  | ConnectionReuseTest.testConnectionReuse <<<
>[junit4]> Throwable #1: java.lang.AssertionError: We expected all 
> communication to use one connection! HttpSolrClient 1
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([FA03A6C483303E7D:DB3BCDAAAB786730]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.ConnectionReuseTest.testConnectionReuse(ConnectionReuseTest.java:158)
> {noformat}
> This test failed for me twice without specifying a seed, so it's likely 
> failing (close to) 100% of the time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9608) ConnectionReuseTest failures

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward reassigned SOLR-9608:
---

Assignee: Alan Woodward

> ConnectionReuseTest failures
> 
>
> Key: SOLR-9608
> URL: https://issues.apache.org/jira/browse/SOLR-9608
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Alan Woodward
>
> My Jenkins has seen this test fail three times in the last couple hours, all 
> three reproduce for me - this is new, last failures were from May - here's 
> the first failure:
> {noformat}
> Checking out Revision 0eb6b1c823d347319cc0894b5fea95f085d4c8d4 
> (refs/remotes/origin/master)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=ConnectionReuseTest -Dtests.method=testConnectionReuse 
> -Dtests.seed=FA03A6C483303E7D -Dtests.slow=true -Dtests.locale=ga 
> -Dtests.timezone=Etc/GMT-9 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.09s J9  | ConnectionReuseTest.testConnectionReuse <<<
>[junit4]> Throwable #1: java.lang.AssertionError: We expected all 
> communication to use one connection! HttpSolrClient 1
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([FA03A6C483303E7D:DB3BCDAAAB786730]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.ConnectionReuseTest.testConnectionReuse(ConnectionReuseTest.java:158)
> {noformat}
> This test failed for me twice without specifying a seed, so it's likely 
> failing (close to) 100% of the time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9608) ConnectionReuseTest failures

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549720#comment-15549720
 ] 

Alan Woodward commented on SOLR-9608:
-

Almost certainly SOLR-9604, I will dig.

> ConnectionReuseTest failures
> 
>
> Key: SOLR-9608
> URL: https://issues.apache.org/jira/browse/SOLR-9608
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>
> My Jenkins has seen this test fail three times in the last couple hours, all 
> three reproduce for me - this is new, last failures were from May - here's 
> the first failure:
> {noformat}
> Checking out Revision 0eb6b1c823d347319cc0894b5fea95f085d4c8d4 
> (refs/remotes/origin/master)
> [...]
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=ConnectionReuseTest -Dtests.method=testConnectionReuse 
> -Dtests.seed=FA03A6C483303E7D -Dtests.slow=true -Dtests.locale=ga 
> -Dtests.timezone=Etc/GMT-9 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] FAILURE 1.09s J9  | ConnectionReuseTest.testConnectionReuse <<<
>[junit4]> Throwable #1: java.lang.AssertionError: We expected all 
> communication to use one connection! HttpSolrClient 1
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([FA03A6C483303E7D:DB3BCDAAAB786730]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.ConnectionReuseTest.testConnectionReuse(ConnectionReuseTest.java:158)
> {noformat}
> This test failed for me twice without specifying a seed, so it's likely 
> failing (close to) 100% of the time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_102) - Build # 17978 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17978/
Java: 64bit/jdk1.8.0_102 -XX:-UseCompressedOops -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.client.solrj.ConnectionReuseTest.testConnectionReuse

Error Message:
We expected all communication to use one connection! HttpSolrClient 1

Stack Trace:
java.lang.AssertionError: We expected all communication to use one connection! 
HttpSolrClient 1
at 
__randomizedtesting.SeedInfo.seed([65368EDF46370983:440EE5B16E7F50CE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.client.solrj.ConnectionReuseTest.testConnectionReuse(ConnectionReuseTest.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs

Error Message:
IOException occured when talking to server at: 

[jira] [Resolved] (SOLR-9592) decorateDocValues cause serious performance issue because of using slowCompositeReaderWrapper

[Yonik Seeley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley resolved SOLR-9592.

Resolution: Fixed

Committed.  Thanks!

> decorateDocValues cause serious performance issue because of using 
> slowCompositeReaderWrapper
> -
>
> Key: SOLR-9592
> URL: https://issues.apache.org/jira/browse/SOLR-9592
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers, search
>Affects Versions: 6.0, 6.1, 6.2
>Reporter: Takahiro Ishikawa
>Assignee: Yonik Seeley
>  Labels: performance
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9592.patch, SOLR-9592.patch, SOLR-9592_6x.patch
>
>
> I have serious performance issue using AtomicUpdate (and RealtimeGet) with 
> non stored docValues.
> Because decorateDocValues try to merge each leafLeader on the fly via 
> slowCompositeReaderWrapper and it’s extremely slow (> 10sec).
> Simply access docValues via nonCompositeReader could resolve this 
> issue.(patch) 
> AtomicUpdate performance(or RealtimeGet performance)
> * Environment
> ** solr version : 6.0.0
> ** schema ~ 100 fields(90% docValues, some of those are multi valued)
> ** index : 5,000,000
> * Performance
> ** original :  > 10sec per query
> ** patched : at least 100msec per query
> This patch will also enhance search performance, because DocStreamer also 
> fetch docValues via decorateDocValues.
> Though it depends on each environment, I could take 20% search performance 
> gain.
> (This patch originally written for solr 6.0.0, and now rewritten for master)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9592) decorateDocValues cause serious performance issue because of using slowCompositeReaderWrapper

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549690#comment-15549690
 ] 

ASF subversion and git services commented on SOLR-9592:
---

Commit 59d83f57e12f72ff8db503a0c01b77cb5df354fd in lucene-solr's branch 
refs/heads/branch_6x from [~yo...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=59d83f5 ]

SOLR-9592: use correct leaf reader rather than top-level reader in 
SolrIndexReaderm.decorateDocValues


> decorateDocValues cause serious performance issue because of using 
> slowCompositeReaderWrapper
> -
>
> Key: SOLR-9592
> URL: https://issues.apache.org/jira/browse/SOLR-9592
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers, search
>Affects Versions: 6.0, 6.1, 6.2
>Reporter: Takahiro Ishikawa
>Assignee: Yonik Seeley
>  Labels: performance
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9592.patch, SOLR-9592.patch, SOLR-9592_6x.patch
>
>
> I have serious performance issue using AtomicUpdate (and RealtimeGet) with 
> non stored docValues.
> Because decorateDocValues try to merge each leafLeader on the fly via 
> slowCompositeReaderWrapper and it’s extremely slow (> 10sec).
> Simply access docValues via nonCompositeReader could resolve this 
> issue.(patch) 
> AtomicUpdate performance(or RealtimeGet performance)
> * Environment
> ** solr version : 6.0.0
> ** schema ~ 100 fields(90% docValues, some of those are multi valued)
> ** index : 5,000,000
> * Performance
> ** original :  > 10sec per query
> ** patched : at least 100msec per query
> This patch will also enhance search performance, because DocStreamer also 
> fetch docValues via decorateDocValues.
> Though it depends on each environment, I could take 20% search performance 
> gain.
> (This patch originally written for solr 6.0.0, and now rewritten for master)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9608) ConnectionReuseTest failures

[Steve Rowe (JIRA)]

Steve Rowe created SOLR-9608:


 Summary: ConnectionReuseTest failures
 Key: SOLR-9608
 URL: https://issues.apache.org/jira/browse/SOLR-9608
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Steve Rowe


My Jenkins has seen this test fail three times in the last couple hours, all 
three reproduce for me - this is new, last failures were from May - here's the 
first failure:

{noformat}
Checking out Revision 0eb6b1c823d347319cc0894b5fea95f085d4c8d4 
(refs/remotes/origin/master)
[...]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ConnectionReuseTest 
-Dtests.method=testConnectionReuse -Dtests.seed=FA03A6C483303E7D 
-Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=Etc/GMT-9 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 1.09s J9  | ConnectionReuseTest.testConnectionReuse <<<
   [junit4]> Throwable #1: java.lang.AssertionError: We expected all 
communication to use one connection! HttpSolrClient 1
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([FA03A6C483303E7D:DB3BCDAAAB786730]:0)
   [junit4]>at 
org.apache.solr.client.solrj.ConnectionReuseTest.testConnectionReuse(ConnectionReuseTest.java:158)
{noformat}

This test failed for me twice without specifying a seed, so it's likely failing 
(close to) 100% of the time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9592) decorateDocValues cause serious performance issue because of using slowCompositeReaderWrapper

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549673#comment-15549673
 ] 

ASF subversion and git services commented on SOLR-9592:
---

Commit cae6b49a065bfbc5789d8bdcf3706c158c0fc10d in lucene-solr's branch 
refs/heads/master from [~yo...@apache.org]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cae6b49 ]

SOLR-9592: use correct leaf reader rather than top-level reader in 
SolrIndexReaderm.decorateDocValues


> decorateDocValues cause serious performance issue because of using 
> slowCompositeReaderWrapper
> -
>
> Key: SOLR-9592
> URL: https://issues.apache.org/jira/browse/SOLR-9592
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers, search
>Affects Versions: 6.0, 6.1, 6.2
>Reporter: Takahiro Ishikawa
>Assignee: Yonik Seeley
>  Labels: performance
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9592.patch, SOLR-9592.patch, SOLR-9592_6x.patch
>
>
> I have serious performance issue using AtomicUpdate (and RealtimeGet) with 
> non stored docValues.
> Because decorateDocValues try to merge each leafLeader on the fly via 
> slowCompositeReaderWrapper and it’s extremely slow (> 10sec).
> Simply access docValues via nonCompositeReader could resolve this 
> issue.(patch) 
> AtomicUpdate performance(or RealtimeGet performance)
> * Environment
> ** solr version : 6.0.0
> ** schema ~ 100 fields(90% docValues, some of those are multi valued)
> ** index : 5,000,000
> * Performance
> ** original :  > 10sec per query
> ** patched : at least 100msec per query
> This patch will also enhance search performance, because DocStreamer also 
> fetch docValues via decorateDocValues.
> Though it depends on each environment, I could take 20% search performance 
> gain.
> (This patch originally written for solr 6.0.0, and now rewritten for master)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9605) Add field literal support to /update/json/docs

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549637#comment-15549637
 ] 

Noble Paul commented on SOLR-9605:
--

bq.Hopefully the 'literal string' in the examples can still be full JSON though.

yes, it could be a single value or an array of values. it should not accept 
objects like 
{code}
f=y@/child1:{x:y, p:q}
{code}

so the possible values are 
# a string literal specified in single quotes
# an array of primitive values specified in json array format 

> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Assignee: Noble Paul
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549559#comment-15549559
 ] 

Mikhail Khludnev commented on SOLR-9604:


bq. OK, not being able to reproduce this behaviour at all locally
And I probably know [why|SOLR-9039] 

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7475) Sparse norms

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549558#comment-15549558
 ] 

Michael McCandless commented on LUCENE-7475:


Woops, I just pushed a small speedup to the old norms format 
({{Lucene43NormsProducer}}) to avoid the wrapper class (over dense norms) 
before seeing this new issue :)

> Sparse norms
> 
>
> Key: LUCENE-7475
> URL: https://issues.apache.org/jira/browse/LUCENE-7475
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: LUCENE-7475.patch
>
>
> Even though norms now have an iterator API, they are still always dense in 
> practice since documents that do not have a value get assigned 0 as a norm 
> value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7407) Explore switching doc values to an iterator API

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549552#comment-15549552
 ] 

ASF subversion and git services commented on LUCENE-7407:
-

Commit 001a3ca55b30656e0e42f612d927a7923f5370e9 in lucene-solr's branch 
refs/heads/master from Mike McCandless
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=001a3ca ]

LUCENE-7407: speed up iterating norms a bit by having default codec implement 
the iterator directly


> Explore switching doc values to an iterator API
> ---
>
> Key: LUCENE-7407
> URL: https://issues.apache.org/jira/browse/LUCENE-7407
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
>  Labels: docValues
> Fix For: master (7.0)
>
> Attachments: LUCENE-7407.patch
>
>
> I think it could be compelling if we restricted doc values to use an
> iterator API at read time, instead of the more general random access
> API we have today:
>   * It would make doc values disk usage more of a "you pay for what
> what you actually use", like postings, which is a compelling
> reduction for sparse usage.
>   * I think codecs could compress better and maybe speed up decoding
> of doc values, even in the non-sparse case, since the read-time
> API is more restrictive "forward only" instead of random access.
>   * We could remove {{getDocsWithField}} entirely, since that's
> implicit in the iteration, and the awkward "return 0 if the
> document didn't have this field" would go away.
>   * We can remove the annoying thread locals we must make today in
> {{CodecReader}}, and close the trappy "I accidentally shared a
> single XXXDocValues instance across threads", since an iterator is
> inherently "use once".
>   * We could maybe leverage the numerous optimizations we've done for
> postings over time, since the two problems ("iterate over doc ids
> and store something interesting for each") are very similar.
> This idea has come up many in the past, e.g. LUCENE-7253 is a recent
> example, and very early iterations of doc values started with exactly
> this ;)
> However, it's a truly enormous change, likely 7.0 only.  Or maybe we
> could have the new iterator APIs also ported to 6.x side by side with
> the deprecate existing random-access APIs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9605) Add field literal support to /update/json/docs

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549525#comment-15549525
 ] 

Yonik Seeley commented on SOLR-9605:


Ah, so what's missing from the smart json merging is adding for *every* child 
(and array handling... they go hand-in-hand).

Hopefully the 'literal string' in the examples can still be full JSON though... 
in order to handle multi-valued fields and fields with types other than string.
{code}
f=y@/child1:[5,7,11]
{code}

> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Assignee: Noble Paul
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Apache Tika's public regression corpus

[Allison, Timothy B.]

All,

I recently blogged about some of the work we're doing with a large scale 
regression corpus to make Tika, POI and PDFBox more robust and to identify 
regressions before release.  If you'd like to chip in with recommendations, 
requests or Hadoop/Spark clusters (why not shoot for the stars), please do!

  
http://openpreservation.org/blog/2016/10/04/apache-tikas-regression-corpus-tika-1302/

Many thanks, again, to Rackspace for our vm and to Common Crawl and govdocs1 
for most of our files!

Cheers,

 Tim

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9605) Add field literal support to /update/json/docs

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549417#comment-15549417
 ] 

Noble Paul edited comment on SOLR-9605 at 10/5/16 5:50 PM:
---

it's not that simple [~ysee...@gmail.com]

we have a lot of flexibility in our syntax .  I can specify 
{{child=/|/child1|child2}} . what if wish to add literal 'A' to the parent doc 
and literal 'B' to docs found under {{/child1}} and literal 'C' to docs found 
under {{/child2}}. 


I suggest the following 

{code}
f=x:'literal string' // this will add field 'x' as 'literal string' to root 
doc. The most common usecase equivalent to f=x@/:'literal string'
f=y@/child1:'literal string 2' // will add field 'y' to docs found under 
/child1 as 'literal srtring 2'
f=z@/child2:'literal string 3'  // will add field 'z' to docs found under 
/child2 as 'literal srtring 3'
{code}


was (Author: noble.paul):
it's not that simple [~ysee...@gmail.com]

we have a lot of flexibility in our syntax .  I can specify 
{{child=/|/child1|child2}} . what if wish to add literal 'A' to the parent doc 
and literal 'B' to docs found under {{/child1}} and literal 'C' to docs found 
under {{/child2}}. 


I suggest the following 

{code}
f=x:'literal string' // this will add field 'x' as 'literal string' to root 
doc. The most common usecase equivalent to f=x@/:'literal string'
f=y@/child1:'literal string 2' // will add field 'y' to docs found under 
/child1 as 'literal srtring 2'
f=z@/child2:'literal string 2'  // will add field 'z' to docs found under 
/child2 as 'literal srtring 3'
{code}

> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Assignee: Noble Paul
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [DISCUSS] JIRA maintenance and response times

[Alexandre Rafalovitch]

 I believe the first 3 dashboards can be done in JIRA itself. I have
something similar, but unfortunately cannot share the dashboard (no
permission it seems).

The last one (breakdown of creation date? by year) is something that
JIRA does not seem to be able to give.

In terms of pulling data, I was able to build an API to pull up to 200
issues from JIRA at a time with full case information in the - as
Cassandra discovered - rather convoluted XML. So, it would not be too
hard to run several APIs and merge the results to get full export. And
then maybe run daily/weekly export and merge/override. Then, it is
pre-process and distribute.

It is totally doable. We just need to firm up the business
requirements. What kind of reports people feel would be useful for
_them_ to be more proactive. I've listed my ideas earlier, but not
sure if they would be useful to others.

Regards,
   Alex.
P.s. Do we need a JIRA for this?

Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/


On 6 October 2016 at 00:06, Cassandra Targett  wrote:
> In terms of a report, I found a template that uses JIRA's API to
> import issues to a Google Sheet:
> http://www.littlebluemonkey.com/blog/automatically-import-jira-backlog-into-google-spreadsheet
>
> I made some modifications to the original and have shared it for
> public viewing:
> https://docs.google.com/spreadsheets/d/1i1_XgabWM4CFnDLGyp3xNVXcZNvGDjww3HPk2hA4uNc/edit#gid=0
>
> I was able to pull in issues with my JIRA login without asking INFRA,
> so I think committers at least already have permissions to use the
> JIRA API.
>
> The template Sheet was initially created to make a board with cards
> for planning, but I have removed those tabs and the script that
> generates the cards. So ignore anything about "story cards" that you
> might see in instructions or anywhere else.
>
> There are a few issues with the API output that makes the data a
> little difficult to work with. Most of the fields I wanted include a
> bunch of metadata JIRA uses for display. And the date field won't
> convert from a timestamp to a date. The date issue is the worst of
> these, for aging purposes, but there might be something we can do in
> the script  that pulls the data to fix this.
>
> Despite that, it's still possible to make a simple report from this.
> I've added a tab "Stats" that shows some initial (easy) calculations -
> # of Issues by Type, # older than 2016 by types, # unassigned, a
> breakdown by year.
>
> Google's flavor of javascript (Google Apps Script) runs the sheet, so
> I think we could add to the main script, or add other scripts, if we
> want to. I know pretty close to zero javascript, however, so would
> need help there.
>
> JIRA can do a lot of this same stuff in its own dashboards (especially
> basic stuff), but that's very much opt-in by individual users. It
> sounds like some folks would prefer something pushed to the community.
> We could do a combination, however - I could take a stab at a shared
> dashboard people could opt into, and also send out a summary weekly
> based on a report generated somehow.
>
> I have no ego invested in the import-to-spreadsheet example I'm
> sharing here; if it won't work for helping us to stay on top of aging
> issues, I can toss it. It was just an option to explore since there is
> an easy-to-use template already out in the world. If we do think it
> will work, though, I can give edit permissions to the folks who want
> to help get it into shape.
>
> On Wed, Oct 5, 2016 at 11:24 AM, Alexandre Rafalovitch
>  wrote:
>> From my reviews, I see a lot of issues where the discussion just
>> petered out without clear handover of next action you would see in the
>> support cases. So, I am not sure explicit flags would be something
>> people use enough for this to work.
>>
>> But, as I believe I mentioned before, I think two basic reports may
>> make a difference:
>> 1) Case older than X days, where all comments are by the same author
>> (as in, nobody else looked at it yet)
>> 2) Case where "I" did the last response and that response is older
>> than X. This will help to restart the conversations where I said "what
>> do you think?" and the other person(s) did not reply, therefore
>> causing a long conversation lull. This will not help with "we are
>> waiting for issue Y", but perhaps that could be an edge case to code
>> around.
>>
>> Unfortunately, I don't think JIRA's JQL supports either of the
>> queries. I poked around a bit and don't see relevant fields/functions.
>> So, it would have to be a 3rd party app based on JIRA exports with
>> committers individually subscribing to receive the reports.
>>
>> Regards,
>>Alex.
>> P.s. If we can tell what flags are Solr specific, it would be great to
>> review those and perhaps remove all irrelevant ones.
>>
>> 
>> Newsletter and resources for Solr beginners and intermediates:
>> http://www.solr-start.com/
>>
>>
>> 

[jira] [Commented] (SOLR-9605) Add field literal support to /update/json/docs

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549417#comment-15549417
 ] 

Noble Paul commented on SOLR-9605:
--

it's not that simple [~ysee...@gmail.com]

we have a lot of flexibility in our syntax .  I can specify 
{{child=/|/child1|child2}} . what if wish to add literal 'A' to the parent doc 
and literal 'B' to docs found under {{/child1}} and literal 'C' to docs found 
under {{/child2}}. 


I suggest the following 

{code}
f=x:'literal string' // this will add field 'x' as 'literal string' to root 
doc. The most common usecase equivalent to f=x@/:'literal string'
f=y@/child1:'literal string 2' // will add field 'y' to docs found under 
/child1 as 'literal srtring 2'
f=z@/child2:'literal string 2'  // will add field 'z' to docs found under 
/child2 as 'literal srtring 3'
{code}

> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Assignee: Noble Paul
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9193) Add scoreNodes Streaming Expression

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549358#comment-15549358
 ] 

Alexandre Rafalovitch commented on SOLR-9193:
-

Great. I created SOLR-9607 to add the parameters and cleanup config files.

> Add scoreNodes Streaming Expression
> ---
>
> Key: SOLR-9193
> URL: https://issues.apache.org/jira/browse/SOLR-9193
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrJ
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.2
>
> Attachments: SOLR-9193.patch
>
>
> The scoreNodes Streaming Expression is another *GraphExpression*. It will 
> decorate a gatherNodes expression and use a tf-idf scoring algorithm to score 
> the nodes.
> The gatherNodes expression only gathers nodes and aggregations. This is 
> similar in nature to tf in search ranking, where the number of times a node 
> appears in the traversal represents the tf. But this skews recommendations 
> towards nodes that appear frequently in the index.
> Using the idf for each node we can score each node as a function of tf-idf. 
> This will provide a boost to nodes that appear less frequently in the index. 
> The scoreNodes expression will gather the idf's from the shards for each node 
> emitted by the underlying gatherNodes expression. It will then assign the 
> score to each node. 
> The computed score will be added to each node in the *nodeScore* field. The 
> docFreq of the node across the entire collection will be added to each node 
> in the *docFreq* field. Other streaming expressions can then perform a 
> ranking based on the nodeScore or compute their own score using the nodeFreq.
> proposed syntax:
> {code}
> top(n="10",
>   sort="nodeScore desc",
>   scoreNodes(gatherNodes(...))) 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9607) Finalize /terms implicit definition and remove explicit one from configsets

[Alexandre Rafalovitch (JIRA)]

Alexandre Rafalovitch created SOLR-9607:
---

 Summary: Finalize /terms implicit definition and remove explicit 
one from configsets
 Key: SOLR-9607
 URL: https://issues.apache.org/jira/browse/SOLR-9607
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 6.2
Reporter: Alexandre Rafalovitch
Assignee: Alexandre Rafalovitch
Priority: Minor


SOLR-9193 has created implicit definitions for terms SearchComponent and /terms 
RequestHandler. The RequestHandler definition is missing a couple of parameters 
to match current explicit definitions (terms=true, distrib=false).

Once those parameters are added, both SearchComponent and RequestHandler 
sections can be removed from example configuration files, making them a bit 
easier to read.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [DISCUSS] JIRA maintenance and response times

[Cassandra Targett]

In terms of a report, I found a template that uses JIRA's API to
import issues to a Google Sheet:
http://www.littlebluemonkey.com/blog/automatically-import-jira-backlog-into-google-spreadsheet

I made some modifications to the original and have shared it for
public viewing:
https://docs.google.com/spreadsheets/d/1i1_XgabWM4CFnDLGyp3xNVXcZNvGDjww3HPk2hA4uNc/edit#gid=0

I was able to pull in issues with my JIRA login without asking INFRA,
so I think committers at least already have permissions to use the
JIRA API.

The template Sheet was initially created to make a board with cards
for planning, but I have removed those tabs and the script that
generates the cards. So ignore anything about "story cards" that you
might see in instructions or anywhere else.

There are a few issues with the API output that makes the data a
little difficult to work with. Most of the fields I wanted include a
bunch of metadata JIRA uses for display. And the date field won't
convert from a timestamp to a date. The date issue is the worst of
these, for aging purposes, but there might be something we can do in
the script  that pulls the data to fix this.

Despite that, it's still possible to make a simple report from this.
I've added a tab "Stats" that shows some initial (easy) calculations -
# of Issues by Type, # older than 2016 by types, # unassigned, a
breakdown by year.

Google's flavor of javascript (Google Apps Script) runs the sheet, so
I think we could add to the main script, or add other scripts, if we
want to. I know pretty close to zero javascript, however, so would
need help there.

JIRA can do a lot of this same stuff in its own dashboards (especially
basic stuff), but that's very much opt-in by individual users. It
sounds like some folks would prefer something pushed to the community.
We could do a combination, however - I could take a stab at a shared
dashboard people could opt into, and also send out a summary weekly
based on a report generated somehow.

I have no ego invested in the import-to-spreadsheet example I'm
sharing here; if it won't work for helping us to stay on top of aging
issues, I can toss it. It was just an option to explore since there is
an easy-to-use template already out in the world. If we do think it
will work, though, I can give edit permissions to the folks who want
to help get it into shape.

On Wed, Oct 5, 2016 at 11:24 AM, Alexandre Rafalovitch
 wrote:
> From my reviews, I see a lot of issues where the discussion just
> petered out without clear handover of next action you would see in the
> support cases. So, I am not sure explicit flags would be something
> people use enough for this to work.
>
> But, as I believe I mentioned before, I think two basic reports may
> make a difference:
> 1) Case older than X days, where all comments are by the same author
> (as in, nobody else looked at it yet)
> 2) Case where "I" did the last response and that response is older
> than X. This will help to restart the conversations where I said "what
> do you think?" and the other person(s) did not reply, therefore
> causing a long conversation lull. This will not help with "we are
> waiting for issue Y", but perhaps that could be an edge case to code
> around.
>
> Unfortunately, I don't think JIRA's JQL supports either of the
> queries. I poked around a bit and don't see relevant fields/functions.
> So, it would have to be a 3rd party app based on JIRA exports with
> committers individually subscribing to receive the reports.
>
> Regards,
>Alex.
> P.s. If we can tell what flags are Solr specific, it would be great to
> review those and perhaps remove all irrelevant ones.
>
> 
> Newsletter and resources for Solr beginners and intermediates:
> http://www.solr-start.com/
>
>
> On 5 October 2016 at 21:56, Jan Høydahl  wrote:
>> Thanks for voicing your experience here. Hopefully more people find the
>> discussion useful.
>>
>> but unfortunately some also just rot, with the chances of them being useful
>> going down every day. I’d rather get a No.
>>
>>
>> I agree with you, it is much better to get a +1 or -1 reply early than just
>> silence :)
>>
>> Anything to help call out issues that need attention would be greatly
>> appreciated
>>
>>
>> Today most people just add a comment “Can a committer please look at this?”,
>> and if it is still silent some time later there may be an email to dev@
>> calling out for committer attention for SOLR-.
>>
>> If we get ourselves a dashboard, it would be nice to have a column listing
>> issues that are explicitly awaiting review. But how should that be
>> populated? The proper JIRA way would be through a different Status workflow,
>> where an issue can be changed from OPEN —> NEEDS REVIEW. Or we could agree
>> on a label or even a “Flag” that can be filtered on.
>>
>> Speaking of flags, the SOLR Jira has two boolean flags: “Important” (sic)
>> and “Patch”. Should probably just delete them as they are not being used..
>>

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549340#comment-15549340
 ] 

Mikhail Khludnev commented on SOLR-9604:


Sigh..

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1906) Posibility to store actual geohash values in the geohash field type

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549334#comment-15549334
 ] 

Alexandre Rafalovitch commented on SOLR-1906:
-

Thanks David for the feedback.

I'll monitor this issue and close it if nobody jumps on it, as you suggest.

> Posibility to store actual geohash values in the geohash field type
> ---
>
> Key: SOLR-1906
> URL: https://issues.apache.org/jira/browse/SOLR-1906
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 1.5
> Environment: NA
>Reporter: Stian Berger
>Assignee: Alexandre Rafalovitch
>Priority: Trivial
>  Labels: geohash, spatialsearch
>
> Tried to index some data, containing already encoded geohashes, into a 
> geohash field type.
> To my surprise I could not make it work...
> A sneak peak at the source, revealed to me that this field type takes a 
> lat/lng pair as it's value, not a geohash...
> Could this be fixed, so the field type also can take actual geohashes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-1906) Posibility to store actual geohash values in the geohash field type

[Alexandre Rafalovitch (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexandre Rafalovitch reassigned SOLR-1906:
---

Assignee: Alexandre Rafalovitch

> Posibility to store actual geohash values in the geohash field type
> ---
>
> Key: SOLR-1906
> URL: https://issues.apache.org/jira/browse/SOLR-1906
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 1.5
> Environment: NA
>Reporter: Stian Berger
>Assignee: Alexandre Rafalovitch
>Priority: Trivial
>  Labels: geohash, spatialsearch
>
> Tried to index some data, containing already encoded geohashes, into a 
> geohash field type.
> To my surprise I could not make it work...
> A sneak peak at the source, revealed to me that this field type takes a 
> lat/lng pair as it's value, not a geohash...
> Could this be fixed, so the field type also can take actual geohashes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9605) Add field literal support to /update/json/docs

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549308#comment-15549308
 ] 

Yonik Seeley commented on SOLR-9605:


High-level idea: reuse the "Smart merging of multiple JSON parameters"
http://yonik.com/solr-json-request-api/

{code}
json.mystrfield="literal"
json.myintfield=42
json.my_multivalued_field=[1,2,5]
json.child_doc_path.child_field="hi"
json.child_doc_path={"field1":10, "field2":"setting multiple fields at once"}
{code}


> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Assignee: Noble Paul
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Hoss Man (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549309#comment-15549309
 ] 

Hoss Man commented on SOLR-9604:


bq. also, it's worth to remove @RandomSSL from HttpSolrClientSSLAuthConPoolTest 
to make it really random. 

bq.  I think it's worth keeping the RandomizeSSL annotation on this test 
though, as it's only really applicable to SSL+clientAuth situations.

Agreed, but I think I also see Mikhail's point: from a coverage standpoint, it 
would be nice to have a test that helps ensure we are re-using connection pools 
*regardless* of whether SSL+clientAuth is used or not (ie: as the test stands 
now, we only ensure they are re-used with SSL, some future bug could break 
connection re-use for non-ssl connections and we'd never know -- heck: in 
theory the changes in this fix for SSL connection reuse could be breaking 
connection reuse for non-SSL requests, and we wouldn't know it.)

So perhaps refactor the current {{HttpSolrClientSSLAuthConPoolTest}} class into 
a {{HttpSolrClientConPoolTest}} that uses the default SSL randomization, and 
then make {{HttpSolrClientSSLAuthConPoolTest}} a subclass that forces 
{{\@RandomizeSSL(1.0)}} ?
(along with a "test the test" sanity assertion that the urls start with "https")


> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1906) Posibility to store actual geohash values in the geohash field type

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549286#comment-15549286
 ] 

David Smiley commented on SOLR-1906:


bq. What would the URP do (using the latest available geo fields)? We could 
create an enhancement newdev JIRA for this.

Quite simply use GeoHashUtils to parse it to get a lat-lon, and then spit that 
out as "lat, lon".  I think I like this better than the spatial field type 
itself accepting a geohash.  Any way, I'm a little skeptical how much people 
would care about this feature.  If nobody further responds to this issue 
wanting this capability within a month, I suggest taking no further action and 
closing the issue.  Either way, I'm not signing up to do the issue, only to 
review & commit a suitable patch.

I filed SOLR-9606 to remove GeoHashField

> Posibility to store actual geohash values in the geohash field type
> ---
>
> Key: SOLR-1906
> URL: https://issues.apache.org/jira/browse/SOLR-1906
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 1.5
> Environment: NA
>Reporter: Stian Berger
>Priority: Trivial
>  Labels: geohash, spatialsearch
>
> Tried to index some data, containing already encoded geohashes, into a 
> geohash field type.
> To my surprise I could not make it work...
> A sneak peak at the source, revealed to me that this field type takes a 
> lat/lng pair as it's value, not a geohash...
> Could this be fixed, so the field type also can take actual geohashes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9605) Add field literal support to /update/json/docs

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549282#comment-15549282
 ] 

Noble Paul edited comment on SOLR-9605 at 10/5/16 4:48 PM:
---

one potential issue with syntax I see is with child docs. what does it mean 
when I specify {{split=/|/child-doc-path}} do we put the literal on both root 
and child docs?  what if we want to add different literal fields for parent, 
child1 and child2 ?



was (Author: noble.paul):
one potential issue with syntax I see is with child docs. what does it mean 
when I specify {{split=/|/child-doc-path}} do we put the literal on both root 
and child docs?  what if we want to add different literal fields for parent, 
shild1 and child2 ?


> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Assignee: Noble Paul
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9605) Add field literal support to /update/json/docs

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549282#comment-15549282
 ] 

Noble Paul commented on SOLR-9605:
--

one potential issue with syntax I see is with child docs. what does it mean 
when I specify {{split=/|/child-doc-path}} do we put the literal on both root 
and child docs?  what if we want to add different literal fields for parent, 
shild1 and child2 ?


> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Assignee: Noble Paul
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9506) cache IndexFingerprint for each segment

[Pushkar Raste (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pushkar Raste updated SOLR-9506:

Attachment: SOLR-9506.patch

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506.patch, SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9506) cache IndexFingerprint for each segment

[Pushkar Raste (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pushkar Raste updated SOLR-9506:

Attachment: (was: SOLR-9506.patch)

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9606) Remove GeoHashField from Solr

[David Smiley (JIRA)]

David Smiley created SOLR-9606:
--

 Summary: Remove GeoHashField from Solr
 Key: SOLR-9606
 URL: https://issues.apache.org/jira/browse/SOLR-9606
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
  Components: spatial
Reporter: David Smiley
Assignee: David Smiley
 Fix For: master (7.0)


I'd like to remove GeoHashField from Solr -- 7.0.  I see no use-case for it any 
more.  And it's presence is distracting from spatial fields you *should* be 
using.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9605) Add field literal support to /update/json/docs

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9605:
-
Description: [~noble.paul] suggests {{f=x:'literal string'}}  (was: 
[~noble.paul] suggests `f=x:'literal string'`)

> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9605) Add field literal support to /update/json/docs

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul reassigned SOLR-9605:


Assignee: Noble Paul

> Add field literal support to /update/json/docs
> --
>
> Key: SOLR-9605
> URL: https://issues.apache.org/jira/browse/SOLR-9605
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: update
>Reporter: Erik Hatcher
>Assignee: Noble Paul
>Priority: Minor
>
> [~noble.paul] suggests {{f=x:'literal string'}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1236) stop using strings for filepaths and start using File objects

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549263#comment-15549263
 ] 

Alexandre Rafalovitch commented on SOLR-1236:
-

Ok. Good to know. We are definitely well covered for this topic with other 
JIRAs then.

> stop using strings for filepaths and start using File objects
> -
>
> Key: SOLR-1236
> URL: https://issues.apache.org/jira/browse/SOLR-1236
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>Priority: Trivial
>
> Catching up on my email after a long hiatus i notice SOLR-1213 and it 
> reminded me that the last time i was digging arround in the code i was 
> frustrated by lots of places in Solr where "String" instances to store a file 
> or directory paths and then manipulated with String operations.  I would be 
> nice to switch all of these member variables and and function params to use 
> genuine File objects to improve the code base readibility, and reduce the 
> likelyhood of OS variant bugs or nonsensical path operations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549254#comment-15549254
 ] 

ASF subversion and git services commented on SOLR-9604:
---

Commit 0eb6b1c823d347319cc0894b5fea95f085d4c8d4 in lucene-solr's branch 
refs/heads/master from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0eb6b1c ]

SOLR-9604: Ensure SSL connections are re-used


> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1906) Posibility to store actual geohash values in the geohash field type

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549250#comment-15549250
 ] 

Alexandre Rafalovitch commented on SOLR-1906:
-

What would the URP do (using the latest available geo fields)? We could create 
an enhancement newdev JIRA for this.

On removing the GeoHashField, if we were to remove it, would we want to 
deprecate it first? Do we need a new Jira for that?

> Posibility to store actual geohash values in the geohash field type
> ---
>
> Key: SOLR-1906
> URL: https://issues.apache.org/jira/browse/SOLR-1906
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 1.5
> Environment: NA
>Reporter: Stian Berger
>Priority: Trivial
>  Labels: geohash, spatialsearch
>
> Tried to index some data, containing already encoded geohashes, into a 
> geohash field type.
> To my surprise I could not make it work...
> A sneak peak at the source, revealed to me that this field type takes a 
> lat/lng pair as it's value, not a geohash...
> Could this be fixed, so the field type also can take actual geohashes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1236) stop using strings for filepaths and start using File objects

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549249#comment-15549249
 ] 

Alan Woodward commented on SOLR-1236:
-

https://issues.apache.org/jira/browse/SOLR-8282 is what you want there, I 
think.  I got stuck trying to get DirectoryFactories converted, but it can 
probably be picked up again.

> stop using strings for filepaths and start using File objects
> -
>
> Key: SOLR-1236
> URL: https://issues.apache.org/jira/browse/SOLR-1236
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>Priority: Trivial
>
> Catching up on my email after a long hiatus i notice SOLR-1213 and it 
> reminded me that the last time i was digging arround in the code i was 
> frustrated by lots of places in Solr where "String" instances to store a file 
> or directory paths and then manipulated with String operations.  I would be 
> nice to switch all of these member variables and and function params to use 
> genuine File objects to improve the code base readibility, and reduce the 
> likelyhood of OS variant bugs or nonsensical path operations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1236) stop using strings for filepaths and start using File objects

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549240#comment-15549240
 ] 

Alexandre Rafalovitch commented on SOLR-1236:
-

I get 77 files, when I run:
{noformat}
ack "java.io.File;" --type java --ignore-dir="test-framework" 
--ignore-dir="test" |wc -l
{noformat}

I would say we are not ready for forbidden-api for this one. I am not sure if 
that means we should have a different issue(s) open for this discovery.

But the "Strings" issue is probably done as a specific "next-action" issue. If 
there is no further objections, I will close it.

> stop using strings for filepaths and start using File objects
> -
>
> Key: SOLR-1236
> URL: https://issues.apache.org/jira/browse/SOLR-1236
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>Priority: Trivial
>
> Catching up on my email after a long hiatus i notice SOLR-1213 and it 
> reminded me that the last time i was digging arround in the code i was 
> frustrated by lots of places in Solr where "String" instances to store a file 
> or directory paths and then manipulated with String operations.  I would be 
> nice to switch all of these member variables and and function params to use 
> genuine File objects to improve the code base readibility, and reduce the 
> likelyhood of OS variant bugs or nonsensical path operations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549231#comment-15549231
 ] 

Mikhail Khludnev commented on SOLR-9604:


No problem. Feel free to push in master what you have, I'll take care about 
porting to branch_6x

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549219#comment-15549219
 ] 

Alan Woodward commented on SOLR-9604:
-

OK, not being able to reproduce this behaviour at all locally is making this 
quite tricky to write - [~mkhludnev] do you think you'll have any time to 
backport the above patch to 6.x?

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [DISCUSS] JIRA maintenance and response times

[Alexandre Rafalovitch]

>From my reviews, I see a lot of issues where the discussion just
petered out without clear handover of next action you would see in the
support cases. So, I am not sure explicit flags would be something
people use enough for this to work.

But, as I believe I mentioned before, I think two basic reports may
make a difference:
1) Case older than X days, where all comments are by the same author
(as in, nobody else looked at it yet)
2) Case where "I" did the last response and that response is older
than X. This will help to restart the conversations where I said "what
do you think?" and the other person(s) did not reply, therefore
causing a long conversation lull. This will not help with "we are
waiting for issue Y", but perhaps that could be an edge case to code
around.

Unfortunately, I don't think JIRA's JQL supports either of the
queries. I poked around a bit and don't see relevant fields/functions.
So, it would have to be a 3rd party app based on JIRA exports with
committers individually subscribing to receive the reports.

Regards,
   Alex.
P.s. If we can tell what flags are Solr specific, it would be great to
review those and perhaps remove all irrelevant ones.


Newsletter and resources for Solr beginners and intermediates:
http://www.solr-start.com/


On 5 October 2016 at 21:56, Jan Høydahl  wrote:
> Thanks for voicing your experience here. Hopefully more people find the
> discussion useful.
>
> but unfortunately some also just rot, with the chances of them being useful
> going down every day. I’d rather get a No.
>
>
> I agree with you, it is much better to get a +1 or -1 reply early than just
> silence :)
>
> Anything to help call out issues that need attention would be greatly
> appreciated
>
>
> Today most people just add a comment “Can a committer please look at this?”,
> and if it is still silent some time later there may be an email to dev@
> calling out for committer attention for SOLR-.
>
> If we get ourselves a dashboard, it would be nice to have a column listing
> issues that are explicitly awaiting review. But how should that be
> populated? The proper JIRA way would be through a different Status workflow,
> where an issue can be changed from OPEN —> NEEDS REVIEW. Or we could agree
> on a label or even a “Flag” that can be filtered on.
>
> Speaking of flags, the SOLR Jira has two boolean flags: “Important” (sic)
> and “Patch”. Should probably just delete them as they are not being used..
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> 4. okt. 2016 kl. 23.02 skrev Jeff Wartes :
>
> I’m not a committer, but Solr is the first major open source project I’ve
> followed closely enough to get a feel for the actual community. The issues
> coming up in this thread have been rattling around in my head for at least
> the last year, and I’m absolutely thrilled to see this conversation. I
> nearly brought it up myself on couple of occasions, but I wasn’t sure how
> much of this was common to all open source (volunteer-driven) projects.
>
> I’ve filed Solr issues, some with patches, some of which have been accepted.
> Some went a different way than I expected, which is fine, but unfortunately
> some also just rot, with the chances of them being useful going down every
> day. I’d rather get a No.
>
> There have been a few cases where I’ve worked on a patch after discussing
> and getting a favorable opinion from a committer, but even that doesn’t seem
> to offer any assurance that the work will ever be reviewed.
>
> And frankly, yes, this effects how I deal with Solr issues. Among other
> things, it discourages me from contributing more work until I see attention
> to the stuff I’ve already provided.
>
> Anything to help call out issues that need attention would be greatly
> appreciated, I think. I have a suspicion that the Jira-notificaiton-firehose
> is the most common notification mechanism, and people generally just look at
> whatever came out most recently there. Meaning, if something blew past
> unnoticed, it’s gone forever.
>
>
>
> On 9/29/16, 11:32 AM, "Erick Erickson"  wrote:
>
>Big +1 to this statement:
>
>***
>To me, the most urgent aspect of the problem is that Bugs are not
>getting verified and fixed as soon as possible, and non-committers
>(particularly) who take the time to create a patch for an improvement
>are not seeing their efforts acknowledged, let alone reviewed or
>committed
>
>
>This hits the nail on the head IMO. I wonder how many potential
>committers we've lost through inaction? Yonik's line about "you
>get to be a committer by acting like a committer" comes to mind.
>We have people "acting like committers" by submitting
>patches and the like then don't get back to them.
>
>Of course we all have our day jobs, limited time and at least
>some of us have these things called "lives".

[jira] [Created] (SOLR-9605) Add field literal support to /update/json/docs

[Erik Hatcher (JIRA)]

Erik Hatcher created SOLR-9605:
--

 Summary: Add field literal support to /update/json/docs
 Key: SOLR-9605
 URL: https://issues.apache.org/jira/browse/SOLR-9605
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
  Components: update
Reporter: Erik Hatcher
Priority: Minor


[~noble.paul] suggests `f=x:'literal string'`



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9592) decorateDocValues cause serious performance issue because of using slowCompositeReaderWrapper

[Yonik Seeley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley reassigned SOLR-9592:
--

Assignee: Yonik Seeley

> decorateDocValues cause serious performance issue because of using 
> slowCompositeReaderWrapper
> -
>
> Key: SOLR-9592
> URL: https://issues.apache.org/jira/browse/SOLR-9592
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers, search
>Affects Versions: 6.0, 6.1, 6.2
>Reporter: Takahiro Ishikawa
>Assignee: Yonik Seeley
>  Labels: performance
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9592.patch, SOLR-9592.patch, SOLR-9592_6x.patch
>
>
> I have serious performance issue using AtomicUpdate (and RealtimeGet) with 
> non stored docValues.
> Because decorateDocValues try to merge each leafLeader on the fly via 
> slowCompositeReaderWrapper and it’s extremely slow (> 10sec).
> Simply access docValues via nonCompositeReader could resolve this 
> issue.(patch) 
> AtomicUpdate performance(or RealtimeGet performance)
> * Environment
> ** solr version : 6.0.0
> ** schema ~ 100 fields(90% docValues, some of those are multi valued)
> ** index : 5,000,000
> * Performance
> ** original :  > 10sec per query
> ** patched : at least 100msec per query
> This patch will also enhance search performance, because DocStreamer also 
> fetch docValues via decorateDocValues.
> Though it depends on each environment, I could take 20% search performance 
> gain.
> (This patch originally written for solr 6.0.0, and now rewritten for master)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1236) stop using strings for filepaths and start using File objects

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549149#comment-15549149
 ] 

Shawn Heisey commented on SOLR-1236:


I guess the question is whether we need a specific issue to focus on avoiding 
String and platform-specific representations of paths.  I think it will always 
be relevant to worry about this, but IMHO it's more of a general coding 
guideline than a specific issue.  That approach says we close this issue and 
check whatever documentation we have regarding coding practices.

File objects, the focus of this issue, are now thoroughly outdated.  Lucene has 
switched to NIO2, which utilizes Path.

If it hasn't been done already, then for both LUCENE and SOLR, I think we 
probably need to add File to the forbidden-api list to detect any unwanted 
stragglers and make sure the old APIs don't creep back in.


> stop using strings for filepaths and start using File objects
> -
>
> Key: SOLR-1236
> URL: https://issues.apache.org/jira/browse/SOLR-1236
> Project: Solr
>  Issue Type: Improvement
>Reporter: Hoss Man
>Priority: Trivial
>
> Catching up on my email after a long hiatus i notice SOLR-1213 and it 
> reminded me that the last time i was digging arround in the code i was 
> frustrated by lots of places in Solr where "String" instances to store a file 
> or directory paths and then manipulated with String operations.  I would be 
> nice to switch all of these member variables and and function params to use 
> genuine File objects to improve the code base readibility, and reduce the 
> likelyhood of OS variant bugs or nonsensical path operations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7475) Sparse norms

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand updated LUCENE-7475:
-
Attachment: LUCENE-7475.patch

Here is a patch that:
 - fixes NormValuesWriter to support sparse norms
 - adds a new Lucene70NormsFormat that supports sparsity and only encodes norms 
for documents that have a norm
 - adds a {{codecSupportsSparsity}} method to BaseNormsFormatTestCase so that 
modern norms formats can get proper testing of the sparse case
 - fixes SimpleTextNormsFormat to support sparsity
 - moves Lucene53NormsFormat to the backward-codecs module

Notes:
 - the current patch assigns a norm value of zero to fields that generate no 
tokens (can happen eg. with the empty string or if all tokens are stop words) 
and only considers that a document does not have norms if no text field were 
indexed at all. We could also decide that fields that generate no tokens are 
considered as missing too, I think both approaches can make sense.
 - the new Lucene70NormsFormat is only a first step, it can certainly be 
improved in further issues

> Sparse norms
> 
>
> Key: LUCENE-7475
> URL: https://issues.apache.org/jira/browse/LUCENE-7475
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Adrien Grand
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: LUCENE-7475.patch
>
>
> Even though norms now have an iterator API, they are still always dense in 
> practice since documents that do not have a value get assigned 0 as a norm 
> value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7475) Sparse norms

[Adrien Grand (JIRA)]

Adrien Grand created LUCENE-7475:


 Summary: Sparse norms
 Key: LUCENE-7475
 URL: https://issues.apache.org/jira/browse/LUCENE-7475
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Adrien Grand
Priority: Minor
 Fix For: master (7.0)


Even though norms now have an iterator API, they are still always dense in 
practice since documents that do not have a value get assigned 0 as a norm 
value.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9562) Minimize queried collections for time series alias

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549087#comment-15549087
 ] 

David Smiley commented on SOLR-9562:


Sorry, but IMO aliasing by the collection just doesn't make sense to me.  This 
approach would lead to potentially lots of collections -- no thanks; I'd much 
rather have lots of shards.  I think it better conceptually matches what's 
going on, and it doesn't pollute the collection namespace.  If somehow sharding 
by time would be difficult for clients (e.g. Banana), I'd like to learn more 
about how that would be so -- and if we find any issue there then I think 
fixing _that_ would make more sense than sharding by collection.

> Minimize queried collections for time series alias
> --
>
> Key: SOLR-9562
> URL: https://issues.apache.org/jira/browse/SOLR-9562
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Eungsop Yoo
>Priority: Minor
> Attachments: SOLR-9562-v2.patch, SOLR-9562.patch
>
>
> For indexing time series data(such as large log data), we can create a new 
> collection regularly(hourly, daily, etc.) with a write alias and create a 
> read alias for all of those collections. But all of the collections of the 
> read alias are queried even if we search over very narrow time window. In 
> this case, the docs to be queried may be stored in very small portion of 
> collections. So we don't need to do that.
> I suggest this patch for read alias to minimize queried collections. Three 
> parameters for CREATEALIAS action are added.
> || Key || Type || Required || Default || Description ||
> | timeField | string | No | | The time field name for time series data. It 
> should be date type. |
> | dateTimeFormat | string | No | | The format of timestamp for collection 
> creation. Every collection should has a suffix(start with "_") with this 
> format. 
> Ex. dateTimeFormat: MMdd, collectionName: col_20160927
> See 
> [DateTimeFormatter|https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html].
>  |
> | timeZone | string | No | | The time zone information for dateTimeFormat 
> parameter.
> Ex. GMT+9. 
> See 
> [DateTimeFormatter|https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html].
>  |
> And then when we query with filter query like this "timeField:\[fromTime TO 
> toTime\]", only the collections have the docs for a given time range will be 
> queried.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-1906) Posibility to store actual geohash values in the geohash field type

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-1906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549038#comment-15549038
 ] 

David Smiley commented on SOLR-1906:


Regarding accepting a geohash as input as an alternative to a lat,lon or WKT... 
I don't know... I guess so?  Patches welcome :-)  One could always write an 
UpdateRequestProcessor to do such things.

BTW I think GeoHashField should be removed from Solr 7.0; it's dated and I 
can't think of any reason why someone would use it.

> Posibility to store actual geohash values in the geohash field type
> ---
>
> Key: SOLR-1906
> URL: https://issues.apache.org/jira/browse/SOLR-1906
> Project: Solr
>  Issue Type: Improvement
>Affects Versions: 1.5
> Environment: NA
>Reporter: Stian Berger
>Priority: Trivial
>  Labels: geohash, spatialsearch
>
> Tried to index some data, containing already encoded geohashes, into a 
> geohash field type.
> To my surprise I could not make it work...
> A sneak peak at the source, revealed to me that this field type takes a 
> lat/lng pair as it's value, not a geohash...
> Could this be fixed, so the field type also can take actual geohashes?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9600) RulesTest.doIntegrationTest() failures

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe resolved SOLR-9600.
--
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.3

Thanks [~romseygeek], I got 0 out of 50 beasting failures with your fixed 
master version. 

> RulesTest.doIntegrationTest() failures
> --
>
> Key: SOLR-9600
> URL: https://issues.apache.org/jira/browse/SOLR-9600
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Alan Woodward
> Fix For: 6.3, master (7.0)
>
>
> My Jenkins has seen this test fail about 8 times today, mostly on branch_6x 
> but also on master, e.g. 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-6.x/3049/], 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-master/8833/].  This is new 
> - previous failure on my Jenkins was from August.  The failures aren't 100% 
> reproducible.
> From Policeman Jenkins 
> [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6158]:
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RulesTest 
> -Dtests.method=doIntegrationTest -Dtests.seed=D12AC7FA27544B42 
> -Dtests.slow=true -Dtests.locale=de-DE -Dtests.timezone=America/New_York 
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] ERROR   14.1s J0 | RulesTest.doIntegrationTest <<<
>[junit4]> Throwable #1: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://127.0.0.1:51451/solr: Could not identify nodes matching 
> the rules [{"cores":"<4"}, {
>[junit4]>   "replica":"<2",
>[junit4]>   "node":"*"}, {"freedisk":">1"}]
>[junit4]>  tag values{
>[junit4]>   "127.0.0.1:51451_solr":{
>[junit4]> "node":"127.0.0.1:51451_solr",
>[junit4]> "cores":3,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51444_solr":{
>[junit4]> "node":"127.0.0.1:51444_solr",
>[junit4]> "cores":1,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51461_solr":{
>[junit4]> "node":"127.0.0.1:51461_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51441_solr":{
>[junit4]> "node":"127.0.0.1:51441_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51454_solr":{
>[junit4]> "node":"127.0.0.1:51454_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31}}
>[junit4]> Initial state for the coll : {
>[junit4]>   "shard1":{
>[junit4]> "127.0.0.1:51454_solr":1,
>[junit4]> "127.0.0.1:51444_solr":1},
>[junit4]>   "shard2":{
>[junit4]> "127.0.0.1:51461_solr":1,
>[junit4]> "127.0.0.1:51441_solr":1}}
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([D12AC7FA27544B42:3419807B3B20B940]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1288)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1058)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1000)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
>[junit4]>  at 
> org.apache.solr.cloud.rule.RulesTest.doIntegrationTest(RulesTest.java:81)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Beasting current master with Miller's beasting script resulted in 6 failures 
> out of 50 iterations.
> I'm running {{git bisect}} in combination with beasting to see if I can find 
> the commit where this started happening.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: 

[jira] [Commented] (SOLR-3724) No highlighting for phrases with stop words when FVH is used

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15549004#comment-15549004
 ] 

David Smiley commented on SOLR-3724:


I wonder if this is still a problem?  I'm surprised the FVH would have a 
deficiency this significant.

FYI there will be a Solr adapter coming for the new UnifiedHighlighter added in 
LUCENE-7438 -- I can't promise exactly when.  It doesn't have the problem 
described here, and it's quite fast (see the benchmarks on that issue).  The 
main limitation at present is that hl.requireFieldMatch=true is only supported. 
 (same for PostingsHighlighter)

> No highlighting for phrases with stop words when FVH is used
> 
>
> Key: SOLR-3724
> URL: https://issues.apache.org/jira/browse/SOLR-3724
> Project: Solr
>  Issue Type: Bug
>  Components: highlighter
>Affects Versions: 3.6.1
>Reporter: Igor Motov
>
> To reproduce:
> - Index text "foo and bar" into the field "message" with the following schema 
> :
> {code:xml}
> 
>   
> 
>  positionIncrementGap="100">
>   
> 
>  words="lang/stopwords_en.txt" enablePositionIncrements="true"/>
> 
>   
>   
> 
>  words="lang/stopwords_en.txt" enablePositionIncrements="true"/>
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 
>   
>   
> 
>  required="true" termVectors="true" termPositions="true" termOffsets="true"/>
> 
>   
>   
> 
> {code}
> - Search for the {{message:"foo and bar"}} with highlighting enabled and 
> {{hl.useFastVectorHighlighter=true}}
> - The text is not highlighted
> Standard highlighter works fine. If I set {{enablePositionIncrements=false}} 
> in the analyzer, FVH starts to highlight the entire phrase. You can find 
> complete schema and test data files that I used to reproduce this issue here: 
> https://gist.github.com/3279879 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [DISCUSS] JIRA maintenance and response times

[Jan Høydahl]

Thanks for voicing your experience here. Hopefully more people find the 
discussion useful.

> but unfortunately some also just rot, with the chances of them being useful 
> going down every day. I’d rather get a No.


I agree with you, it is much better to get a +1 or -1 reply early than just 
silence :)

> Anything to help call out issues that need attention would be greatly 
> appreciated


Today most people just add a comment “Can a committer please look at this?”, 
and if it is still silent some time later there may be an email to dev@ calling 
out for committer attention for SOLR-.

If we get ourselves a dashboard, it would be nice to have a column listing 
issues that are explicitly awaiting review. But how should that be populated? 
The proper JIRA way would be through a different Status workflow, where an 
issue can be changed from OPEN —> NEEDS REVIEW. Or we could agree on a label or 
even a “Flag” that can be filtered on.

Speaking of flags, the SOLR Jira has two boolean flags: “Important” (sic) and 
“Patch”. Should probably just delete them as they are not being used..

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 4. okt. 2016 kl. 23.02 skrev Jeff Wartes :
> 
> I’m not a committer, but Solr is the first major open source project I’ve 
> followed closely enough to get a feel for the actual community. The issues 
> coming up in this thread have been rattling around in my head for at least 
> the last year, and I’m absolutely thrilled to see this conversation. I nearly 
> brought it up myself on couple of occasions, but I wasn’t sure how much of 
> this was common to all open source (volunteer-driven) projects.
> 
> I’ve filed Solr issues, some with patches, some of which have been accepted. 
> Some went a different way than I expected, which is fine, but unfortunately 
> some also just rot, with the chances of them being useful going down every 
> day. I’d rather get a No.
> 
> There have been a few cases where I’ve worked on a patch after discussing and 
> getting a favorable opinion from a committer, but even that doesn’t seem to 
> offer any assurance that the work will ever be reviewed.
> 
> And frankly, yes, this effects how I deal with Solr issues. Among other 
> things, it discourages me from contributing more work until I see attention 
> to the stuff I’ve already provided. 
> 
> Anything to help call out issues that need attention would be greatly 
> appreciated, I think. I have a suspicion that the Jira-notificaiton-firehose 
> is the most common notification mechanism, and people generally just look at 
> whatever came out most recently there. Meaning, if something blew past 
> unnoticed, it’s gone forever.
> 
> 
> 
> On 9/29/16, 11:32 AM, "Erick Erickson"  wrote:
> 
>Big +1 to this statement:
> 
>***
>To me, the most urgent aspect of the problem is that Bugs are not
>getting verified and fixed as soon as possible, and non-committers
>(particularly) who take the time to create a patch for an improvement
>are not seeing their efforts acknowledged, let alone reviewed or
>committed
>
> 
>This hits the nail on the head IMO. I wonder how many potential
>committers we've lost through inaction? Yonik's line about "you
>get to be a committer by acting like a committer" comes to mind.
>We have people "acting like committers" by submitting
>patches and the like then don't get back to them.
> 
>Of course we all have our day jobs, limited time and at least
>some of us have these things called "lives".
> 
>I'm not sure how to resolve the issue either. It can take
>significant time to even look at a patch and give any reasonable
>feedback
> 
>I'm glad for the conversation too, just wish I had a magic cure.
> 
>Erick
> 
> 
>On Thu, Sep 29, 2016 at 10:35 AM, Cassandra Targett
> wrote:
>> On Thu, Sep 29, 2016 at 7:01 AM, Stefan Matheis  wrote:
>> 
>>> first idea about it: we could bring a script or something that collects 
>>> once a week information about all new issues and sends it to the dev-list? 
>>> so get a quick overview about what happend last week w/o too much trouble?
>>> 
>> 
>> +1 to this idea - awareness of the problem is the first step to being
>> able to change it. And I agree it is a problem.
>> 
>> It's enough of a problem that at Lucidworks we have added it to our
>> priority list for the next year. Consequently, I've spent quite a bit
>> of time looking at old issues in the past couple of months.
>> 
>> To me, the most urgent aspect of the problem is that Bugs are not
>> getting verified and fixed as soon as possible, and non-committers
>> (particularly) who take the time to create a patch for an improvement
>> are not seeing their efforts acknowledged, let alone reviewed or
>> committed. I think this causes more bad impressions than someone's
>> 

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548917#comment-15548917
 ] 

Alan Woodward commented on SOLR-9604:
-

Ah, of course, 6.x is using a completely different way of configuring 
HttpClients...  I'll put up a separate 6.x patch shortly.

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9600) RulesTest.doIntegrationTest() failures

[Noble Paul (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9600:
-
Assignee: Alan Woodward  (was: Noble Paul)

> RulesTest.doIntegrationTest() failures
> --
>
> Key: SOLR-9600
> URL: https://issues.apache.org/jira/browse/SOLR-9600
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Alan Woodward
>
> My Jenkins has seen this test fail about 8 times today, mostly on branch_6x 
> but also on master, e.g. 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-6.x/3049/], 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-master/8833/].  This is new 
> - previous failure on my Jenkins was from August.  The failures aren't 100% 
> reproducible.
> From Policeman Jenkins 
> [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6158]:
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RulesTest 
> -Dtests.method=doIntegrationTest -Dtests.seed=D12AC7FA27544B42 
> -Dtests.slow=true -Dtests.locale=de-DE -Dtests.timezone=America/New_York 
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] ERROR   14.1s J0 | RulesTest.doIntegrationTest <<<
>[junit4]> Throwable #1: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://127.0.0.1:51451/solr: Could not identify nodes matching 
> the rules [{"cores":"<4"}, {
>[junit4]>   "replica":"<2",
>[junit4]>   "node":"*"}, {"freedisk":">1"}]
>[junit4]>  tag values{
>[junit4]>   "127.0.0.1:51451_solr":{
>[junit4]> "node":"127.0.0.1:51451_solr",
>[junit4]> "cores":3,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51444_solr":{
>[junit4]> "node":"127.0.0.1:51444_solr",
>[junit4]> "cores":1,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51461_solr":{
>[junit4]> "node":"127.0.0.1:51461_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51441_solr":{
>[junit4]> "node":"127.0.0.1:51441_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51454_solr":{
>[junit4]> "node":"127.0.0.1:51454_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31}}
>[junit4]> Initial state for the coll : {
>[junit4]>   "shard1":{
>[junit4]> "127.0.0.1:51454_solr":1,
>[junit4]> "127.0.0.1:51444_solr":1},
>[junit4]>   "shard2":{
>[junit4]> "127.0.0.1:51461_solr":1,
>[junit4]> "127.0.0.1:51441_solr":1}}
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([D12AC7FA27544B42:3419807B3B20B940]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1288)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1058)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1000)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
>[junit4]>  at 
> org.apache.solr.cloud.rule.RulesTest.doIntegrationTest(RulesTest.java:81)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Beasting current master with Miller's beasting script resulted in 6 failures 
> out of 50 iterations.
> I'm running {{git bisect}} in combination with beasting to see if I can find 
> the commit where this started happening.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9600) RulesTest.doIntegrationTest() failures

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548903#comment-15548903
 ] 

Noble Paul commented on SOLR-9600:
--

thanks [~romseygeek]

> RulesTest.doIntegrationTest() failures
> --
>
> Key: SOLR-9600
> URL: https://issues.apache.org/jira/browse/SOLR-9600
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Noble Paul
>
> My Jenkins has seen this test fail about 8 times today, mostly on branch_6x 
> but also on master, e.g. 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-6.x/3049/], 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-master/8833/].  This is new 
> - previous failure on my Jenkins was from August.  The failures aren't 100% 
> reproducible.
> From Policeman Jenkins 
> [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6158]:
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RulesTest 
> -Dtests.method=doIntegrationTest -Dtests.seed=D12AC7FA27544B42 
> -Dtests.slow=true -Dtests.locale=de-DE -Dtests.timezone=America/New_York 
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] ERROR   14.1s J0 | RulesTest.doIntegrationTest <<<
>[junit4]> Throwable #1: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://127.0.0.1:51451/solr: Could not identify nodes matching 
> the rules [{"cores":"<4"}, {
>[junit4]>   "replica":"<2",
>[junit4]>   "node":"*"}, {"freedisk":">1"}]
>[junit4]>  tag values{
>[junit4]>   "127.0.0.1:51451_solr":{
>[junit4]> "node":"127.0.0.1:51451_solr",
>[junit4]> "cores":3,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51444_solr":{
>[junit4]> "node":"127.0.0.1:51444_solr",
>[junit4]> "cores":1,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51461_solr":{
>[junit4]> "node":"127.0.0.1:51461_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51441_solr":{
>[junit4]> "node":"127.0.0.1:51441_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51454_solr":{
>[junit4]> "node":"127.0.0.1:51454_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31}}
>[junit4]> Initial state for the coll : {
>[junit4]>   "shard1":{
>[junit4]> "127.0.0.1:51454_solr":1,
>[junit4]> "127.0.0.1:51444_solr":1},
>[junit4]>   "shard2":{
>[junit4]> "127.0.0.1:51461_solr":1,
>[junit4]> "127.0.0.1:51441_solr":1}}
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([D12AC7FA27544B42:3419807B3B20B940]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1288)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1058)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1000)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
>[junit4]>  at 
> org.apache.solr.cloud.rule.RulesTest.doIntegrationTest(RulesTest.java:81)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Beasting current master with Miller's beasting script resulted in 6 failures 
> out of 50 iterations.
> I'm running {{git bisect}} in combination with beasting to see if I can find 
> the commit where this started happening.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9600) RulesTest.doIntegrationTest() failures

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548892#comment-15548892
 ] 

Alan Woodward commented on SOLR-9600:
-

I committed under SOLR-9132, commits are 
3d0f9425022b58337a96c2b9a347e16933ecc496 for branch_6x and 
d398617be891c9bc4ac72f85bf6ba4bff81f4f89 for master

> RulesTest.doIntegrationTest() failures
> --
>
> Key: SOLR-9600
> URL: https://issues.apache.org/jira/browse/SOLR-9600
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Noble Paul
>
> My Jenkins has seen this test fail about 8 times today, mostly on branch_6x 
> but also on master, e.g. 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-6.x/3049/], 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-master/8833/].  This is new 
> - previous failure on my Jenkins was from August.  The failures aren't 100% 
> reproducible.
> From Policeman Jenkins 
> [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6158]:
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RulesTest 
> -Dtests.method=doIntegrationTest -Dtests.seed=D12AC7FA27544B42 
> -Dtests.slow=true -Dtests.locale=de-DE -Dtests.timezone=America/New_York 
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] ERROR   14.1s J0 | RulesTest.doIntegrationTest <<<
>[junit4]> Throwable #1: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://127.0.0.1:51451/solr: Could not identify nodes matching 
> the rules [{"cores":"<4"}, {
>[junit4]>   "replica":"<2",
>[junit4]>   "node":"*"}, {"freedisk":">1"}]
>[junit4]>  tag values{
>[junit4]>   "127.0.0.1:51451_solr":{
>[junit4]> "node":"127.0.0.1:51451_solr",
>[junit4]> "cores":3,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51444_solr":{
>[junit4]> "node":"127.0.0.1:51444_solr",
>[junit4]> "cores":1,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51461_solr":{
>[junit4]> "node":"127.0.0.1:51461_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51441_solr":{
>[junit4]> "node":"127.0.0.1:51441_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51454_solr":{
>[junit4]> "node":"127.0.0.1:51454_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31}}
>[junit4]> Initial state for the coll : {
>[junit4]>   "shard1":{
>[junit4]> "127.0.0.1:51454_solr":1,
>[junit4]> "127.0.0.1:51444_solr":1},
>[junit4]>   "shard2":{
>[junit4]> "127.0.0.1:51461_solr":1,
>[junit4]> "127.0.0.1:51441_solr":1}}
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([D12AC7FA27544B42:3419807B3B20B940]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1288)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1058)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1000)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
>[junit4]>  at 
> org.apache.solr.cloud.rule.RulesTest.doIntegrationTest(RulesTest.java:81)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Beasting current master with Miller's beasting script resulted in 6 failures 
> out of 50 iterations.
> I'm running {{git bisect}} in combination with beasting to see if I can find 
> the commit where this started happening.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9132) Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548889#comment-15548889
 ] 

ASF subversion and git services commented on SOLR-9132:
---

Commit 3d0f9425022b58337a96c2b9a347e16933ecc496 in lucene-solr's branch 
refs/heads/branch_6x from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3d0f942 ]

SOLR-9132: RulesTest must tear down collections at the end of each test


> Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase
> -
>
> Key: SOLR-9132
> URL: https://issues.apache.org/jira/browse/SOLR-9132
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9132-deletereplicas.patch, SOLR-9132-rules.patch, 
> SOLR-9132.patch
>
>
> We need to remove AbstractDistribZkTestCase if we want to move away from 
> legacy cloud configurations.  This issue is for migrating tests to 
> SolrCloudTestCase instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9132) Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548883#comment-15548883
 ] 

ASF subversion and git services commented on SOLR-9132:
---

Commit d398617be891c9bc4ac72f85bf6ba4bff81f4f89 in lucene-solr's branch 
refs/heads/master from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d398617 ]

SOLR-9132: RulesTest must tear down collections at the end of each test


> Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase
> -
>
> Key: SOLR-9132
> URL: https://issues.apache.org/jira/browse/SOLR-9132
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9132-deletereplicas.patch, SOLR-9132-rules.patch, 
> SOLR-9132.patch
>
>
> We need to remove AbstractDistribZkTestCase if we want to move away from 
> legacy cloud configurations.  This issue is for migrating tests to 
> SolrCloudTestCase instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9600) RulesTest.doIntegrationTest() failures

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548836#comment-15548836
 ] 

Alan Woodward commented on SOLR-9600:
-

Ah, it's because the tests are run in random orders now, and doIntegrationTest 
is expecting to run on a cluster with no cores in it.  I'll edit the testcase 
to ensure everything is cleared down at the end of each test.

> RulesTest.doIntegrationTest() failures
> --
>
> Key: SOLR-9600
> URL: https://issues.apache.org/jira/browse/SOLR-9600
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Noble Paul
>
> My Jenkins has seen this test fail about 8 times today, mostly on branch_6x 
> but also on master, e.g. 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-6.x/3049/], 
> [http://jenkins.sarowe.net/job/Lucene-Solr-tests-master/8833/].  This is new 
> - previous failure on my Jenkins was from August.  The failures aren't 100% 
> reproducible.
> From Policeman Jenkins 
> [https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6158]:
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RulesTest 
> -Dtests.method=doIntegrationTest -Dtests.seed=D12AC7FA27544B42 
> -Dtests.slow=true -Dtests.locale=de-DE -Dtests.timezone=America/New_York 
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] ERROR   14.1s J0 | RulesTest.doIntegrationTest <<<
>[junit4]> Throwable #1: 
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
> from server at http://127.0.0.1:51451/solr: Could not identify nodes matching 
> the rules [{"cores":"<4"}, {
>[junit4]>   "replica":"<2",
>[junit4]>   "node":"*"}, {"freedisk":">1"}]
>[junit4]>  tag values{
>[junit4]>   "127.0.0.1:51451_solr":{
>[junit4]> "node":"127.0.0.1:51451_solr",
>[junit4]> "cores":3,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51444_solr":{
>[junit4]> "node":"127.0.0.1:51444_solr",
>[junit4]> "cores":1,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51461_solr":{
>[junit4]> "node":"127.0.0.1:51461_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51441_solr":{
>[junit4]> "node":"127.0.0.1:51441_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31},
>[junit4]>   "127.0.0.1:51454_solr":{
>[junit4]> "node":"127.0.0.1:51454_solr",
>[junit4]> "cores":2,
>[junit4]> "freedisk":31}}
>[junit4]> Initial state for the coll : {
>[junit4]>   "shard1":{
>[junit4]> "127.0.0.1:51454_solr":1,
>[junit4]> "127.0.0.1:51444_solr":1},
>[junit4]>   "shard2":{
>[junit4]> "127.0.0.1:51461_solr":1,
>[junit4]> "127.0.0.1:51441_solr":1}}
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([D12AC7FA27544B42:3419807B3B20B940]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1288)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1058)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1000)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
>[junit4]>  at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
>[junit4]>  at 
> org.apache.solr.cloud.rule.RulesTest.doIntegrationTest(RulesTest.java:81)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Beasting current master with Miller's beasting script resulted in 6 failures 
> out of 50 iterations.
> I'm running {{git bisect}} in combination with beasting to see if I can find 
> the commit where this started happening.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: 

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment

[Yonik Seeley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548767#comment-15548767
 ] 

Yonik Seeley commented on SOLR-9506:


A few random points after browsing this issue...

bq. We can not use current versionsHash (unless we cache all the individual 
version numbers), as it is not additive.

The current versionsHash is additive (it must be, because as you say segments 
may not line up between leader and replica, and document order may differ).  
When caching per segment, keep this property by simply adding the segment 
fingerprints together.  Am I missing something here?

bq. private final Map cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506.patch, SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9193) Add scoreNodes Streaming Expression

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548734#comment-15548734
 ] 

Joel Bernstein commented on SOLR-9193:
--

I just checked and the ScoreNodesStream should not need any adjustment when 
defaults are added.

> Add scoreNodes Streaming Expression
> ---
>
> Key: SOLR-9193
> URL: https://issues.apache.org/jira/browse/SOLR-9193
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrJ
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.2
>
> Attachments: SOLR-9193.patch
>
>
> The scoreNodes Streaming Expression is another *GraphExpression*. It will 
> decorate a gatherNodes expression and use a tf-idf scoring algorithm to score 
> the nodes.
> The gatherNodes expression only gathers nodes and aggregations. This is 
> similar in nature to tf in search ranking, where the number of times a node 
> appears in the traversal represents the tf. But this skews recommendations 
> towards nodes that appear frequently in the index.
> Using the idf for each node we can score each node as a function of tf-idf. 
> This will provide a boost to nodes that appear less frequently in the index. 
> The scoreNodes expression will gather the idf's from the shards for each node 
> emitted by the underlying gatherNodes expression. It will then assign the 
> score to each node. 
> The computed score will be added to each node in the *nodeScore* field. The 
> docFreq of the node across the entire collection will be added to each node 
> in the *docFreq* field. Other streaming expressions can then perform a 
> ranking based on the nodeScore or compute their own score using the nodeFreq.
> proposed syntax:
> {code}
> top(n="10",
>   sort="nodeScore desc",
>   scoreNodes(gatherNodes(...))) 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548675#comment-15548675
 ] 

Mikhail Khludnev edited comment on SOLR-9604 at 10/5/16 1:40 PM:
-

bq. Mikhail Khludnev, can you ensure that the test fails if you change 
HttpSolrClient so that it passes 'null' instead of 'this' to the context 
builder?
absolutely 
!fails with null.png!


was (Author: mkhludnev):
bq. Mikhail Khludnev, can you ensure that the test fails if you change 
HttpSolrClient so that it passes 'null' instead of 'this' to the context 
builder?
absolutely 
!fails with null.png|thumbnail!

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_102) - Build # 17976 - Failure!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17976/
Java: 64bit/jdk1.8.0_102 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.rule.RulesTest.doIntegrationTest

Error Message:
Error from server at http://127.0.0.1:37437/solr: Could not identify nodes 
matching the rules [{"cores":"<4"}, {   "replica":"<2",   "node":"*"}, 
{"freedisk":">0"}]  tag values{   "127.0.0.1:45453_solr":{ 
"node":"127.0.0.1:45453_solr", "cores":1, "freedisk":85},   
"127.0.0.1:37437_solr":{ "node":"127.0.0.1:37437_solr", "cores":1, 
"freedisk":85},   "127.0.0.1:43628_solr":{ "node":"127.0.0.1:43628_solr",   
  "cores":3, "freedisk":85},   "127.0.0.1:43036_solr":{ 
"node":"127.0.0.1:43036_solr", "cores":2, "freedisk":85},   
"127.0.0.1:34275_solr":{ "node":"127.0.0.1:34275_solr", "cores":1, 
"freedisk":85}} Initial state for the coll : {   "shard1":{ 
"127.0.0.1:37437_solr":1, "127.0.0.1:34275_solr":1},   "shard2":{ 
"127.0.0.1:45453_solr":1, "127.0.0.1:43036_solr":1}}

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:37437/solr: Could not identify nodes matching 
the rules [{"cores":"<4"}, {
  "replica":"<2",
  "node":"*"}, {"freedisk":">0"}]
 tag values{
  "127.0.0.1:45453_solr":{
"node":"127.0.0.1:45453_solr",
"cores":1,
"freedisk":85},
  "127.0.0.1:37437_solr":{
"node":"127.0.0.1:37437_solr",
"cores":1,
"freedisk":85},
  "127.0.0.1:43628_solr":{
"node":"127.0.0.1:43628_solr",
"cores":3,
"freedisk":85},
  "127.0.0.1:43036_solr":{
"node":"127.0.0.1:43036_solr",
"cores":2,
"freedisk":85},
  "127.0.0.1:34275_solr":{
"node":"127.0.0.1:34275_solr",
"cores":1,
"freedisk":85}}
Initial state for the coll : {
  "shard1":{
"127.0.0.1:37437_solr":1,
"127.0.0.1:34275_solr":1},
  "shard2":{
"127.0.0.1:45453_solr":1,
"127.0.0.1:43036_solr":1}}
at 
__randomizedtesting.SeedInfo.seed([24D8F120E31F777F:C1EBB6A1FF6B857D]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:439)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:391)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1288)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1058)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1000)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
at 
org.apache.solr.cloud.rule.RulesTest.doIntegrationTest(RulesTest.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548705#comment-15548705
 ] 

Alan Woodward commented on SOLR-9604:
-

Спасибо!

I'll commit this shortly, and then we can try removing the SuppressSSL 
annotation from some tests and see if they pass.  I think it's worth keeping 
the RandomizeSSL annotation on this test though, as it's only really applicable 
to SSL+clientAuth situations.

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9193) Add scoreNodes Streaming Expression

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548697#comment-15548697
 ] 

Joel Bernstein commented on SOLR-9193:
--

I think this is more of an oversight. Let's create a new ticket to add the 
default params. We may have to update the ScoreNodesStream to override the 
distrib=false param but it may already be sending this param. 

> Add scoreNodes Streaming Expression
> ---
>
> Key: SOLR-9193
> URL: https://issues.apache.org/jira/browse/SOLR-9193
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrJ
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.2
>
> Attachments: SOLR-9193.patch
>
>
> The scoreNodes Streaming Expression is another *GraphExpression*. It will 
> decorate a gatherNodes expression and use a tf-idf scoring algorithm to score 
> the nodes.
> The gatherNodes expression only gathers nodes and aggregations. This is 
> similar in nature to tf in search ranking, where the number of times a node 
> appears in the traversal represents the tf. But this skews recommendations 
> towards nodes that appear frequently in the index.
> Using the idf for each node we can score each node as a function of tf-idf. 
> This will provide a boost to nodes that appear less frequently in the index. 
> The scoreNodes expression will gather the idf's from the shards for each node 
> emitted by the underlying gatherNodes expression. It will then assign the 
> score to each node. 
> The computed score will be added to each node in the *nodeScore* field. The 
> docFreq of the node across the entire collection will be added to each node 
> in the *docFreq* field. Other streaming expressions can then perform a 
> ranking based on the nodeScore or compute their own score using the nodeFreq.
> proposed syntax:
> {code}
> top(n="10",
>   sort="nodeScore desc",
>   scoreNodes(gatherNodes(...))) 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-5944) Support updates of numeric DocValues

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-5944:
---
Attachment: SOLR-5944.patch

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9604:
---
Attachment: fails with null.png

bq. Mikhail Khludnev, can you ensure that the test fails if you change 
HttpSolrClient so that it passes 'null' instead of 'this' to the context 
builder?
absolutely 
!fails with null.png|thumbnail!

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch, fails with null.png
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5944) Support updates of numeric DocValues

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547986#comment-15547986
 ] 

Ishan Chattopadhyaya edited comment on SOLR-5944 at 10/5/16 1:12 PM:
-

Updated patch, brought up to master. Here are my replies inline (not all of 
them, but I'll keep editing this comment to provide all replies).


Ok -- it took a while, but here's my notes after reviewing the latest patch


{panel:title=DistributedUpdateProcessor}

* waitForDependentUpdates {color:green}FIXED{color}
** I know you & shalin went back and forth a bit on the wait call (ie: 
wait(100) with max retries vs wait(5000)) but i think the way things settled 
out {{bucket.wait(waitTimeout.timeLeft(TimeUnit.MILLISECONDS));}} would be 
better then a generic {{wait(5000)}}
*** consider the scenerio where: the dependent update is never going to come; a 
spurious notify/wake happens during the first "wait" call @ 4950ms; the 
lookupVersion call takes 45ms.  Now we've only got 5ms left on our original 
TimeOut, but we _could_ wind up "wait"ing another full 5s (total of 10s) unless 
we get another spurrious notify/wake inthe mean time.
** {{log.info("Fetched the update: " + missingUpdate);}} that's a really good 
candidate for templating since the AddUpdateCommand.toString() could be 
expensive if log.info winds up being a no-op (ie: {{log.info("Fetched the 
update: \{\}", missingUpdate);}})

* fetchMissingUpdateFromLeader
** In response to a previous question you said...{quote}
[FIXED. Initially, I wanted to fetch all missing updates, i.e. from what we 
have till what we want. Noble suggested that fetching only one at a time makes 
more sense.]
{quote} ... but from what i can tell skimming RTGC.processGetUpdates() it's 
still possible that multiple updates will be returned, notably in the case 
where: {{// Must return all delete-by-query commands that occur after the first 
add requested}}.  How is that possibility handled in the code paths that use 
fetchMissingUpdateFromLeader?
*** that seems like a scenerio that would be really easy to test for -- similar 
to how outOfOrderDeleteUpdatesIndividualReplicaTest works
** {{assert ((List) missingUpdates).size() == 1: "More than 1 update ...}}
*** based on my skimming of the code, an empty list is just as possible, so the 
assertion is missleading (ideally it should say how many updates it got, or 
maybe toString() the whole List ?)

{panel}


{panel:title=AtomicUpdateDocumentMerger}

* isSupportedFieldForInPlaceUpdate
** javadocs

* getFieldNamesFromIndex
** javadocs
** method name seems VERY missleading considering what it does 
{color:green}Changed it to getSearcherNonStoredDVs{color}

* isInPlaceUpdate
** javadocs should be clear what hapens to inPlaceUpdatedFields if result is 
false (even if answer is "undefined"
** based on usage, wouldn't it be simplier if instead of returning a boolean, 
this method just returned a (new) Set of inplace update fields found, and if 
the set is empty that means it's not an in place update? 
{color:green}FIXED{color}
** isn't getFieldNamesFromIndex kind of an expensive method to call on every 
AddUpdateCommand ?
*** couldn't this list of fields be created by the caller and re-used at least 
for the entire request (ie: when adding multiple docs) ? {color:green}The set 
returned is precomputed upon the opening of a searcher. The only cost I see is 
to create a new unmodifiableSet every time. I'd prefer to take up this 
optimization later, if needed.{color}
** {{if (indexFields.contains(fieldName) == false && 
schema.isDynamicField(fieldName))}}
*** why does it matter one way or the other if it's a dynamicField? 
{color:green}Changed the logic to check in the IW for presence of field. Added 
a comment: "// if dynamic field and this field doesn't exist, DV update can't 
work"{color}
** the special {{DELETED}} sentinal value still isn't being checked against the 
return value of {{getInputDocumentFromTlog}} {color:green}Not using 
getInputDocumentFromTlog call anymore{color}
** this method still seems like it could/should do "cheaper" validation (ie: 
not requiring SchemaField object creation, or tlog lookups) first.  (Ex: the 
set of supported atomic ops are checked after isSupportedFieldForInPlaceUpdate 
& a possible read from the tlog). {color:green}FIXED{color}
*** My suggested rewrite would be something like...{code}
Set candidateResults = new HashSet<>();
// first pass, check the things that are virtually free,
// and bail out early if anything is obviously not a valid in-place update
for (String fieldName : sdoc.getFieldNames()) {
  if (schema.getUniqueKeyField().getName().equals(fieldName)
  || fieldName.equals(DistributedUpdateProcessor.VERSION_FIELD)) {
continue;
  }
  Object fieldValue = sdoc.getField(fieldName).getValue();
  if (! (fieldValue instanceof Map) ) {
// not even an atomic update, definitely not an 

[jira] [Updated] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated SOLR-9604:

Attachment: SOLR-9604.patch

Updated with the old methods left in with deprecations.

[~mkhludnev], can you ensure that the test fails if you change HttpSolrClient 
so that it passes 'null' instead of 'this' to the context builder?  It still 
passes for me locally...

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch, SOLR-9604.patch
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548604#comment-15548604
 ] 

Mikhail Khludnev commented on SOLR-9604:


also, it's worth to remove \@RandomSSL from HttpSolrClientSSLAuthConPoolTest to 
make it really random. 

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9506) cache IndexFingerprint for each segment

[Pushkar Raste (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pushkar Raste updated SOLR-9506:

Attachment: SOLR-9506.patch

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506.patch, SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7438) UnifiedHighlighter

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-7438:
-
Attachment: LUCENE_7438_UH_benchmark.patch

This is an update to the benchmark patch.  I removed the existing benchmark 
highlighting abstraction (that I felt was a bit obsolete), and with it the 
existing two highlighting benchmark classes: SearchTravRetHighlightTask, 
SearchTravRetVectorHighlightTask.  The patch actually replaces 
SearchTravRetHighlightTask with the one from the previous patch, and so by 
class name it still exists, but is internally very different as it tests all 
highlighters in all offset modes.  It has the 2 highlighters-\*.alg added in 
the last patch, and I kept the 3 query-\*.txt files too.  I removed the 
existing highlight .alg files except for one which I updated -- 
standard-highlights-notv.alg -> highlights.alg.  I also added a "UH" highlight 
mode to the benchmark, which is the UH's default mode operation in which it 
detects the offset source based on FieldInfo.

I tweaked the build.xml & .gitignore to avoid work/ and temp/ and to allow them 
to be symbolic links.

The only thing I feel bad about was outright removing some tests related to the 
old highlight abstraction... meanwhile there are no new tests for this new one. 
 I rationalize this as it's better to finally have a more up-to-date way to 
highlight all highlighters in all modes (and in a consistent way) than it is to 
have something incomplete that is nevertheless tested.

I'll commit this in a couple days.

> UnifiedHighlighter
> --
>
> Key: LUCENE-7438
> URL: https://issues.apache.org/jira/browse/LUCENE-7438
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Affects Versions: 6.2
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.3
>
> Attachments: LUCENE-7438.patch, LUCENE_7438_UH_benchmark.patch, 
> LUCENE_7438_UH_benchmark.patch, LUCENE_7438_UH_small_changes.patch
>
>
> The UnifiedHighlighter is an evolution of the PostingsHighlighter that is 
> able to highlight using offsets in either postings, term vectors, or from 
> analysis (a TokenStream). Lucene’s existing highlighters are mostly 
> demarcated along offset source lines, whereas here it is unified -- hence 
> this proposed name. In this highlighter, the offset source strategy is 
> separated from the core highlighting functionalty. The UnifiedHighlighter 
> further improves on the PostingsHighlighter’s design by supporting accurate 
> phrase highlighting using an approach similar to the standard highlighter’s 
> WeightedSpanTermExtractor. The next major improvement is a hybrid offset 
> source strategythat utilizes postings and “light” term vectors (i.e. just the 
> terms) for highlighting multi-term queries (wildcards) without resorting to 
> analysis. Phrase highlighting and wildcard highlighting can both be disabled 
> if you’d rather highlight a little faster albeit not as accurately reflecting 
> the query.
> We’ve benchmarked an earlier version of this highlighter comparing it to the 
> other highlighters and the results were exciting! It’s tempting to share 
> those results but it’s definitely due for another benchmark, so we’ll work on 
> that. Performance was the main motivator for creating the UnifiedHighlighter, 
> as the standard Highlighter (the only one meeting Bloomberg Law’s accuracy 
> requirements) wasn’t fast enough, even with term vectors along with several 
> improvements we contributed back, and even after we forked it to highlight in 
> multiple threads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7438) UnifiedHighlighter

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated LUCENE-7438:
-
Fix Version/s: 6.3

> UnifiedHighlighter
> --
>
> Key: LUCENE-7438
> URL: https://issues.apache.org/jira/browse/LUCENE-7438
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Affects Versions: 6.2
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Fix For: 6.3
>
> Attachments: LUCENE-7438.patch, LUCENE_7438_UH_benchmark.patch, 
> LUCENE_7438_UH_small_changes.patch
>
>
> The UnifiedHighlighter is an evolution of the PostingsHighlighter that is 
> able to highlight using offsets in either postings, term vectors, or from 
> analysis (a TokenStream). Lucene’s existing highlighters are mostly 
> demarcated along offset source lines, whereas here it is unified -- hence 
> this proposed name. In this highlighter, the offset source strategy is 
> separated from the core highlighting functionalty. The UnifiedHighlighter 
> further improves on the PostingsHighlighter’s design by supporting accurate 
> phrase highlighting using an approach similar to the standard highlighter’s 
> WeightedSpanTermExtractor. The next major improvement is a hybrid offset 
> source strategythat utilizes postings and “light” term vectors (i.e. just the 
> terms) for highlighting multi-term queries (wildcards) without resorting to 
> analysis. Phrase highlighting and wildcard highlighting can both be disabled 
> if you’d rather highlight a little faster albeit not as accurately reflecting 
> the query.
> We’ve benchmarked an earlier version of this highlighter comparing it to the 
> other highlighters and the results were exciting! It’s tempting to share 
> those results but it’s definitely due for another benchmark, so we’ll work on 
> that. Performance was the main motivator for creating the UnifiedHighlighter, 
> as the standard Highlighter (the only one meeting Bloomberg Law’s accuracy 
> requirements) wasn’t fast enough, even with term vectors along with several 
> improvements we contributed back, and even after we forked it to highlight in 
> multiple threads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_102) - Build # 496 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/496/
Java: 32bit/jdk1.8.0_102 -client -XX:+UseG1GC

1 tests failed.
FAILED:  
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([8A116F9B1EED16C1:7D6281C3D805B927]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigAliasReplication(TestReplicationHandler.java:1331)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10680 lines...]
   [junit4] Suite: 

[jira] [Commented] (SOLR-9193) Add scoreNodes Streaming Expression

[Alexandre Rafalovitch (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548509#comment-15548509
 ] 

Alexandre Rafalovitch commented on SOLR-9193:
-

I know this issue is closed, but I wanted to check before I open a new one.

The implicit definition of "/terms" is now:
{noformat}
   "/terms": {
  "class": "solr.SearchHandler",
  "useParams":"_TERMS",
  "components": [
"terms"
  ]
},
{noformat}

This conflicts with all explicit definitions we currently have in 
solrconfig.xml file:
{noformat}

  
true
false
  
  
terms
  

{noformat}

Specifically, the existing definition is *terms=true* and *distrib=false*. As 
is, we cannot remove those definitions from the solrconfig. Any specific 
reasons those were not included when this ticket did the implicit definition 
(especially *distrib*) or was that just an oversight?

> Add scoreNodes Streaming Expression
> ---
>
> Key: SOLR-9193
> URL: https://issues.apache.org/jira/browse/SOLR-9193
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrJ
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
> Fix For: 6.2
>
> Attachments: SOLR-9193.patch
>
>
> The scoreNodes Streaming Expression is another *GraphExpression*. It will 
> decorate a gatherNodes expression and use a tf-idf scoring algorithm to score 
> the nodes.
> The gatherNodes expression only gathers nodes and aggregations. This is 
> similar in nature to tf in search ranking, where the number of times a node 
> appears in the traversal represents the tf. But this skews recommendations 
> towards nodes that appear frequently in the index.
> Using the idf for each node we can score each node as a function of tf-idf. 
> This will provide a boost to nodes that appear less frequently in the index. 
> The scoreNodes expression will gather the idf's from the shards for each node 
> emitted by the underlying gatherNodes expression. It will then assign the 
> score to each node. 
> The computed score will be added to each node in the *nodeScore* field. The 
> docFreq of the node across the entire collection will be added to each node 
> in the *docFreq* field. Other streaming expressions can then perform a 
> ranking based on the nodeScore or compute their own score using the nodeFreq.
> proposed syntax:
> {code}
> top(n="10",
>   sort="nodeScore desc",
>   scoreNodes(gatherNodes(...))) 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548510#comment-15548510
 ] 

Noble Paul commented on SOLR-9506:
--

https://github.com/apache/lucene-solr/pull/84

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548422#comment-15548422
 ] 

Mikhail Khludnev commented on SOLR-9604:


ok. my fault. Now I see. The patch amends arguments of public methods, what's 
about API compatibility?   

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548409#comment-15548409
 ] 

Alan Woodward commented on SOLR-9604:
-

It's there in the patch:
{code}
diff --git 
a/solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpClientUtil.java 
b/solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpClientUtil.java
index b9580b8..0f738c2 100644
--- a/solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpClientUtil.java
+++ b/solr/solrj/src/java/org/apache/solr/client/solrj/impl/HttpClientUtil.java
@@ -198,7 +198,12 @@ public class HttpClientUtil {
*  configuration (no additional configuration) is created. 
*/
   public static CloseableHttpClient createClient(SolrParams params) {
-return createClient(params, new 
PoolingHttpClientConnectionManager(schemaRegistryProvider.getSchemaRegistry()));
+return createClient(params, createPoolingConnectionManager());
+  }
+
+  /** test usage subject to change @lucene.experimental */ 
+  static PoolingHttpClientConnectionManager createPoolingConnectionManager() {
+return new 
PoolingHttpClientConnectionManager(schemaRegistryProvider.getSchemaRegistry());
   }
   
   public static CloseableHttpClient createClient(SolrParams params, 
PoolingHttpClientConnectionManager cm) {
@@ -396,10 +401,17 @@ public class HttpClientUtil {
   }
 
   /**
-   * 
+   * Create a HttpClientContext object
+   *
+   * If the client is going to be re-used, then you should pass in an object 
that
+   * can be used by internal connection pools as a cache key.  This is 
particularly
+   * important if client authentication is enabled, as SSL connections will not
+   * be re-used if no cache key is provided.
+   *
+   * @param cacheKey an Object to be used as a cache key for pooling 
connections
*/
-  public static HttpClientContext createNewHttpClientRequestContext() {
-return httpClientRequestContextBuilder.createContext();
+  public static HttpClientContext createNewHttpClientRequestContext(Object 
cacheKey) {
+return httpClientRequestContextBuilder.createContext(cacheKey);
   }
{code}

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-5944) Support updates of numeric DocValues

[Ishan Chattopadhyaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15547986#comment-15547986
 ] 

Ishan Chattopadhyaya edited comment on SOLR-5944 at 10/5/16 10:30 AM:
--

Updated patch, brought up to master. Here are my replies inline (not all of 
them, but I'll keep editing this comment to provide all replies).


{panel:title=DistributedUpdateProcessor}

* waitForDependentUpdates {color:green}FIXED{color}
** I know you & shalin went back and forth a bit on the wait call (ie: 
wait(100) with max retries vs wait(5000)) but i think the way things settled 
out {{bucket.wait(waitTimeout.timeLeft(TimeUnit.MILLISECONDS));}} would be 
better then a generic {{wait(5000)}}
*** consider the scenerio where: the dependent update is never going to come; a 
spurious notify/wake happens during the first "wait" call @ 4950ms; the 
lookupVersion call takes 45ms.  Now we've only got 5ms left on our original 
TimeOut, but we _could_ wind up "wait"ing another full 5s (total of 10s) unless 
we get another spurrious notify/wake inthe mean time.
** {{log.info("Fetched the update: " + missingUpdate);}} that's a really good 
candidate for templating since the AddUpdateCommand.toString() could be 
expensive if log.info winds up being a no-op (ie: {{log.info("Fetched the 
update: \{\}", missingUpdate);}})

* fetchMissingUpdateFromLeader
** In response to a previous question you said...{quote}
[FIXED. Initially, I wanted to fetch all missing updates, i.e. from what we 
have till what we want. Noble suggested that fetching only one at a time makes 
more sense.]
{quote} ... but from what i can tell skimming RTGC.processGetUpdates() it's 
still possible that multiple updates will be returned, notably in the case 
where: {{// Must return all delete-by-query commands that occur after the first 
add requested}}.  How is that possibility handled in the code paths that use 
fetchMissingUpdateFromLeader?
*** that seems like a scenerio that would be really easy to test for -- similar 
to how outOfOrderDeleteUpdatesIndividualReplicaTest works
** {{assert ((List) missingUpdates).size() == 1: "More than 1 update ...}}
*** based on my skimming of the code, an empty list is just as possible, so the 
assertion is missleading (ideally it should say how many updates it got, or 
maybe toString() the whole List ?)

{panel}


{panel:title=AtomicUpdateDocumentMerger}

* isSupportedFieldForInPlaceUpdate
** javadocs

* getFieldNamesFromIndex
** javadocs
** method name seems VERY missleading considering what it does 
{color:green}Changed it to getSearcherNonStoredDVs{color}

* isInPlaceUpdate
** javadocs should be clear what hapens to inPlaceUpdatedFields if result is 
false (even if answer is "undefined"
** based on usage, wouldn't it be simplier if instead of returning a boolean, 
this method just returned a (new) Set of inplace update fields found, and if 
the set is empty that means it's not an in place update? 
{color:green}FIXED{color}
** isn't getFieldNamesFromIndex kind of an expensive method to call on every 
AddUpdateCommand ?
*** couldn't this list of fields be created by the caller and re-used at least 
for the entire request (ie: when adding multiple docs) ? {color:green}The set 
returned is precomputed upon the opening of a searcher. The only cost I see is 
to create a new unmodifiableSet every time. I'd prefer to take up this 
optimization later, if needed.{color}
** {{if (indexFields.contains(fieldName) == false && 
schema.isDynamicField(fieldName))}}
*** why does it matter one way or the other if it's a dynamicField? 
{color:green}Changed the logic to check in the IW for presence of field. Added 
a comment: "// if dynamic field and this field doesn't exist, DV update can't 
work"{color}
** the special {{DELETED}} sentinal value still isn't being checked against the 
return value of {{getInputDocumentFromTlog}} {color:green}Not using 
getInputDocumentFromTlog call anymore{color}
** this method still seems like it could/should do "cheaper" validation (ie: 
not requiring SchemaField object creation, or tlog lookups) first.  (Ex: the 
set of supported atomic ops are checked after isSupportedFieldForInPlaceUpdate 
& a possible read from the tlog). {color:green}FIXED{color}
*** My suggested rewrite would be something like...{code}
Set candidateResults = new HashSet<>();
// first pass, check the things that are virtually free,
// and bail out early if anything is obviously not a valid in-place update
for (String fieldName : sdoc.getFieldNames()) {
  if (schema.getUniqueKeyField().getName().equals(fieldName)
  || fieldName.equals(DistributedUpdateProcessor.VERSION_FIELD)) {
continue;
  }
  Object fieldValue = sdoc.getField(fieldName).getValue();
  if (! (fieldValue instanceof Map) ) {
// not even an atomic update, definitely not an in-place update
return Collections.emptySet();
  }
  // else it's a atomic 

[jira] [Updated] (SOLR-5944) Support updates of numeric DocValues

[Ishan Chattopadhyaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-5944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-5944:
---
Attachment: SOLR-5944.patch

> Support updates of numeric DocValues
> 
>
> Key: SOLR-5944
> URL: https://issues.apache.org/jira/browse/SOLR-5944
> Project: Solr
>  Issue Type: New Feature
>Reporter: Ishan Chattopadhyaya
>Assignee: Shalin Shekhar Mangar
> Attachments: DUP.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, SOLR-5944.patch, 
> TestStressInPlaceUpdates.eb044ac71.beast-167-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.beast-587-failure.stdout.txt, 
> TestStressInPlaceUpdates.eb044ac71.failures.tar.gz, defensive-checks.log.gz, 
> hoss.62D328FA1DEA57FD.fail.txt, hoss.62D328FA1DEA57FD.fail2.txt, 
> hoss.62D328FA1DEA57FD.fail3.txt, hoss.D768DD9443A98DC.fail.txt, 
> hoss.D768DD9443A98DC.pass.txt
>
>
> LUCENE-5189 introduced support for updates to numeric docvalues. It would be 
> really nice to have Solr support this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548270#comment-15548270
 ] 

Mikhail Khludnev commented on SOLR-9604:


seems nice. but I couldn't find a method definition for 
{code}
HttpClientUtil.createNewHttpClientRequestContext(null) 
{code}
couldn't you forget to {{git add}} ? 

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 432 - Unstable!

[Michael McCandless]

I'll dig.

Mike McCandless

http://blog.mikemccandless.com


On Wed, Oct 5, 2016 at 4:12 AM, Policeman Jenkins Server
 wrote:
> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/432/
> Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
>
> 1 tests failed.
> FAILED:  
> org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2.testMergeCalledOnTwoFormats
>
> Error Message:
> expected:<1> but was:<2>
>
> Stack Trace:
> java.lang.AssertionError: expected:<1> but was:<2>
> at 
> __randomizedtesting.SeedInfo.seed([9C8B401EAF454DD:34F486948DF8FDF0]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.failNotEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:128)
> at org.junit.Assert.assertEquals(Assert.java:472)
> at org.junit.Assert.assertEquals(Assert.java:456)
> at 
> org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2.testMergeCalledOnTwoFormats(TestPerFieldPostingsFormat2.java:387)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
> at 
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at 
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
> at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
> at 
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
> at 
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at 
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at 
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at 
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at 
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at 
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at 
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at 
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 1052 lines...]
>[junit4] Suite: 
> 

[jira] [Commented] (SOLR-3724) No highlighting for phrases with stop words when FVH is used

[Daniel Aschauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548178#comment-15548178
 ] 

Daniel Aschauer commented on SOLR-3724:
---

I guess you should use hl.usePhraseHighlighter=false

> No highlighting for phrases with stop words when FVH is used
> 
>
> Key: SOLR-3724
> URL: https://issues.apache.org/jira/browse/SOLR-3724
> Project: Solr
>  Issue Type: Bug
>  Components: highlighter
>Affects Versions: 3.6.1
>Reporter: Igor Motov
>
> To reproduce:
> - Index text "foo and bar" into the field "message" with the following schema 
> :
> {code:xml}
> 
>   
> 
>  positionIncrementGap="100">
>   
> 
>  words="lang/stopwords_en.txt" enablePositionIncrements="true"/>
> 
>   
>   
> 
>  words="lang/stopwords_en.txt" enablePositionIncrements="true"/>
>  ignoreCase="true" expand="true"/>
> 
>   
> 
> 
>   
>   
> 
>  required="true" termVectors="true" termPositions="true" termOffsets="true"/>
> 
>   
>   
> 
> {code}
> - Search for the {{message:"foo and bar"}} with highlighting enabled and 
> {{hl.useFastVectorHighlighter=true}}
> - The text is not highlighted
> Standard highlighter works fine. If I set {{enablePositionIncrements=false}} 
> in the analyzer, FVH starts to highlight the entire phrase. You can find 
> complete schema and test data files that I used to reproduce this issue here: 
> https://gist.github.com/3279879 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9604) Pooled SSL connections are not being reused with client authentication

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15548153#comment-15548153
 ] 

Alan Woodward commented on SOLR-9604:
-

The concurrent client in DistributedUpdateProcessor actually already uses the 
HttpClient object from UpdateHandler, so I think this is ready to go.

> Pooled SSL connections are not being reused with client authentication
> --
>
> Key: SOLR-9604
> URL: https://issues.apache.org/jira/browse/SOLR-9604
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9604.patch
>
>
> Solr isn't setting user tokens on any of its HttpClientContext objects when 
> requested new http connections.  This means that when SSL + client 
> authentication is used, HttpClient is creating a new connection on every 
> request, to ensure that authentication tokens aren't shared between different 
> users.  We end up with lots of unused open connections in the connection 
> pool, leading to slowness and out-of-memory errors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org