date:20160926

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/869/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionStateFormat2Test.test

Error Message:
Could not find collection:.system

Stack Trace:
java.lang.AssertionError: Could not find collection:.system
at 
__randomizedtesting.SeedInfo.seed([F35D4CAB8F8F8441:7B0973712173E9B9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:153)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:138)
at 
org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:133)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:856)
at 
org.apache.solr.cloud.CollectionStateFormat2Test.testConfNameAndCollectionNameSame(CollectionStateFormat2Test.java:53)
at 
org.apache.solr.cloud.CollectionStateFormat2Test.test(CollectionStateFormat2Test.java:40)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_102) - Build # 17909 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17909/
Java: 32bit/jdk1.8.0_102 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.schema.TestManagedSchemaAPI.test

Error Message:
Error from server at http://127.0.0.1:32923/solr/testschemaapi_shard1_replica2: 
ERROR: [doc=2] unknown field 'myNewField1'

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from 
server at http://127.0.0.1:32923/solr/testschemaapi_shard1_replica2: ERROR: 
[doc=2] unknown field 'myNewField1'
at 
__randomizedtesting.SeedInfo.seed([2233D53E4D083504:AA67EAE4E3F458FC]:0)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:753)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1161)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1050)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:992)
at 
org.apache.solr.schema.TestManagedSchemaAPI.testAddFieldAndDocument(TestManagedSchemaAPI.java:86)
at 
org.apache.solr.schema.TestManagedSchemaAPI.test(TestManagedSchemaAPI.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at

[jira] [Updated] (SOLR-8969) SQLHandler causes NPE in non-cloud mode

2016-09-26 Thread Kevin Risden (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-8969:
---
Attachment: SOLR-8969.patch

Adding a patch with a test that prints "/sql Handler only works in Solr Cloud 
mode"

Not 100% sure this is the right approach but saw that I had this patch and 
forgot to attach it.

> SQLHandler causes NPE in non-cloud mode
> ---
>
> Key: SOLR-8969
> URL: https://issues.apache.org/jira/browse/SOLR-8969
> Project: Solr
>  Issue Type: Bug
>  Components: Parallell SQL
>Affects Versions: 6.0
>Reporter: Markus Jelsma
>Priority: Minor
> Fix For: master (7.0)
>
> Attachments: SOLR-8969.patch
>
>
> It should instead throw a proper error message. Stack trace:
> {code}
> 1233075 INFO  (qtp97730845-21) [   x:logs] o.a.s.c.S.Request [logs]  
> webapp=/solr path=/sql params={stmt=SELECT+uid+FROM+logs} status=0 QTime=18
> 1233095 INFO  (qtp97730845-21) [   x:logs] o.a.s.c.c.SolrZkClient Using 
> default ZkCredentialsProvider
> 1233109 ERROR (qtp97730845-21) [   x:logs] o.a.s.c.s.i.s.ExceptionStream 
> org.apache.solr.common.SolrException: java.lang.NullPointerException
> at 
> org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:169)
> at 
> org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:115)
> at 
> org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:105)
> at 
> org.apache.solr.common.cloud.ZkStateReader.(ZkStateReader.java:200)
> at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.connect(CloudSolrClient.java:466)
> at 
> org.apache.solr.client.solrj.io.SolrClientCache.getCloudSolrClient(SolrClientCache.java:48)
> at 
> org.apache.solr.client.solrj.io.stream.CloudSolrStream.open(CloudSolrStream.java:237)
> at 
> org.apache.solr.client.solrj.io.stream.SelectStream.open(SelectStream.java:153)
> at 
> org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:47)
> at 
> org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:362)
> at 
> org.apache.solr.response.TextResponseWriter.writeTupleStream(TextResponseWriter.java:301)
> at 
> org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:167)
> at 
> org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:183)
> at 
> org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:299)
> at 
> org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:95)
> at 
> org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:60)
> at 
> org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
> at 
> org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:725)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
> at 
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at 
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
> at 
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
> at 
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
> at 
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at 
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
> at 
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
> at 
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:518)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
> at 
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
> at 
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
> at

[jira] [Updated] (SOLR-9496) SolrJ + Kerberos Requires commons-codec

2016-09-26 Thread Kevin Risden (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Risden updated SOLR-9496:
---
Attachment: SOLR-9496.patch

Adding a quick patch if adding the dependency is the right direction.

> SolrJ + Kerberos Requires commons-codec
> ---
>
> Key: SOLR-9496
> URL: https://issues.apache.org/jira/browse/SOLR-9496
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: 6.2
>Reporter: Bryan Bende
>Priority: Minor
> Attachments: SOLR-9496.patch
>
>
> When using SolrJ 6.2 with Kerberos enabled on the server (also 6.2), the 
> following exception was encountered:
> {code}
> java.lang.NoClassDefFoundError: org/apache/commons/codec/binary/Base64
>   at 
> org.apache.http.impl.auth.GGSSchemeBase.(GGSSchemeBase.java:85) 
> ~[httpclient-4.4.1.jar:4.4.1]
>   at org.apache.http.impl.auth.SPNegoScheme.(SPNegoScheme.java:54) 
> ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.auth.SPNegoSchemeFactory.newInstance(SPNegoSchemeFactory.java:78)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.auth.AuthSchemeRegistry.getAuthScheme(AuthSchemeRegistry.java:113)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.auth.AuthSchemeRegistry$1.create(AuthSchemeRegistry.java:151) 
> ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.AuthenticationStrategyImpl.select(AuthenticationStrategyImpl.java:188)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.TargetAuthenticationStrategy.select(TargetAuthenticationStrategy.java:43)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.auth.HttpAuthenticator.handleAuthChallenge(HttpAuthenticator.java:154)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.HttpAuthenticator.authenticate(HttpAuthenticator.java:58)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.DefaultRequestDirector.handleResponse(DefaultRequestDirector.java:1057)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) 
> ~[na:na]
>   at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166) 
> ~[na:na]
> {code}
> Adding a dependency to my project on commons-codec resolved the issue:
> {code}
> 
> commons-codec
> commons-codec
> 1.10
> 
> {code}
> SolrJ should include this dependency if it is required for Kerberos 
> authentication. 
> If not we should consider updating the SolrJ section on the Wiki page here to 
> mention that client application needs to add it:
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9496) SolrJ + Kerberos Requires commons-codec

2016-09-26 Thread Kevin Risden (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524356#comment-15524356
 ] 

Kevin Risden commented on SOLR-9496:


[~ichattopadhyaya] / [~noble.paul] - Saw you added a dependency in SOLR-9542. I 
noticed this issue with missing base64. Any issues with adding commons-codec as 
a dependency to solrj? Is there a better way?

> SolrJ + Kerberos Requires commons-codec
> ---
>
> Key: SOLR-9496
> URL: https://issues.apache.org/jira/browse/SOLR-9496
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: 6.2
>Reporter: Bryan Bende
>Priority: Minor
>
> When using SolrJ 6.2 with Kerberos enabled on the server (also 6.2), the 
> following exception was encountered:
> {code}
> java.lang.NoClassDefFoundError: org/apache/commons/codec/binary/Base64
>   at 
> org.apache.http.impl.auth.GGSSchemeBase.(GGSSchemeBase.java:85) 
> ~[httpclient-4.4.1.jar:4.4.1]
>   at org.apache.http.impl.auth.SPNegoScheme.(SPNegoScheme.java:54) 
> ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.auth.SPNegoSchemeFactory.newInstance(SPNegoSchemeFactory.java:78)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.auth.AuthSchemeRegistry.getAuthScheme(AuthSchemeRegistry.java:113)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.auth.AuthSchemeRegistry$1.create(AuthSchemeRegistry.java:151) 
> ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.AuthenticationStrategyImpl.select(AuthenticationStrategyImpl.java:188)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.TargetAuthenticationStrategy.select(TargetAuthenticationStrategy.java:43)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.auth.HttpAuthenticator.handleAuthChallenge(HttpAuthenticator.java:154)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.HttpAuthenticator.authenticate(HttpAuthenticator.java:58)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.DefaultRequestDirector.handleResponse(DefaultRequestDirector.java:1057)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:515)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
>  ~[httpclient-4.4.1.jar:4.4.1]
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:355)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
>  ~[na:na]
>   at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149) 
> ~[na:na]
>   at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166) 
> ~[na:na]
> {code}
> Adding a dependency to my project on commons-codec resolved the issue:
> {code}
> 
> commons-codec
> commons-codec
> 1.10
> 
> {code}
> SolrJ should include this dependency if it is required for Kerberos 
> authentication. 
> If not we should consider updating the SolrJ section on the Wiki page here to 
> mention that client application needs to add it:
> https://cwiki.apache.org/confluence/display/solr/Kerberos+Authentication+Plugin



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9390) LeaderFailoverAfterPartitionTest failures: Connection refused, too few active replicas, and no registered leader


[ 
https://issues.apache.org/jira/browse/SOLR-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524338#comment-15524338
 ] 

Alan Woodward commented on SOLR-9390:
-

The ConnectionRefused failures have been fixed by disabling directToLeaders 
updates, but we're still getting occasional failures.  I think 
https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3566/ is caused by 
SOLR-9555, which as it's a race would explain why the failures don't reproduce.

> LeaderFailoverAfterPartitionTest failures: Connection refused, too few active 
> replicas, and no registered leader
> 
>
> Key: SOLR-9390
> URL: https://issues.apache.org/jira/browse/SOLR-9390
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>
> In manual testing, I've seen LeaderFailoverAfterPartitionTest fail a couple 
> times, and the seeds reproduce for me:
> {noformat}
>   [junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=LeaderFailoverAfterPartitionTest -Dtests.method=test 
> -Dtests.seed=49939C8CF78131E7 -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=en-AU -Dtests.timezone=America/Port-au-Prince 
> -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] ERROR   65.8s | LeaderFailoverAfterPartitionTest.test <<<
>[junit4]> Throwable #1: 
> org.apache.solr.client.solrj.SolrServerException: 
> org.apache.solr.client.solrj.SolrServerException: No live SolrServers 
> available to handle this 
> request:[http://127.0.0.1:60401/c8n_1x3_lf_shard1_replica3]
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([49939C8CF78131E7:C1C7A356597D5C1F]:0)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:774)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
>[junit4]>at 
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
>[junit4]>at 
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
>[junit4]>at 
> org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:178)
>[junit4]>at 
> org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
>[junit4]>at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
>[junit4]>at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
>[junit4]> Caused by: org.apache.solr.client.solrj.SolrServerException: 
> No live SolrServers available to handle this 
> request:[http://127.0.0.1:60401/c8n_1x3_lf_shard1_replica3]
>[junit4]>at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:393)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:768)
>[junit4]>... 48 more
>[junit4]> Caused by: org.apache.solr.client.solrj.SolrServerException: 
> Server refused connection at: 
> http://127.0.0.1:60401/c8n_1x3_lf_shard1_replica3
>[junit4]>at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:615)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
>[junit4]>at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:382)
>[junit4]>... 49 more
>[junit4]> Caused by: org.apache.http.conn.HttpHostConnectException: 
> Connect to 127.0.0.1:60401 [/127.0.0.1] failed: Connection refused
>[junit4]>at 
> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
>[junit4]>

[jira] [Resolved] (SOLR-9305) ForceLeaderTest.testReplicasInLIRNoLeader(): connection refused


 [ 
https://issues.apache.org/jira/browse/SOLR-9305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward resolved SOLR-9305.
-
   Resolution: Fixed
 Assignee: Alan Woodward
Fix Version/s: 6.3

This hasn't failed since I disabled directToLeaders indexing for the test, so 
I'm marking it as resolved.

> ForceLeaderTest.testReplicasInLIRNoLeader(): connection refused
> ---
>
> Key: SOLR-9305
> URL: https://issues.apache.org/jira/browse/SOLR-9305
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Steve Rowe
>Assignee: Alan Woodward
> Fix For: 6.3
>
>
> I saw the following while doing other testing on latest master, reproduces 
> for me:
> {noformat}
>[junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ForceLeaderTest 
> -Dtests.method=testReplicasInLIRNoLeader -Dtests.seed=64E138BA51B16ACE 
> -Dtests.slow=true 
> -Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt 
> -Dtests.locale=de-LU -Dtests.timezone=America/Indiana/Winamac 
> -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
>[junit4] ERROR   32.9s | ForceLeaderTest.testReplicasInLIRNoLeader <<<
>[junit4]> Throwable #1: 
> org.apache.solr.client.solrj.SolrServerException: 
> org.apache.solr.client.solrj.SolrServerException: No live SolrServers 
> available to handle this 
> request:[http://127.0.0.1:59064/forceleader_test_collection_shard1_replica1]
>[junit4]>  at 
> __randomizedtesting.SeedInfo.seed([64E138BA51B16ACE:82760C7A683393AF]:0)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:753)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1151)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1040)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:976)
>[junit4]>  at 
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
>[junit4]>  at 
> org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
>[junit4]>  at 
> org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
>[junit4]>  at 
> org.apache.solr.cloud.ForceLeaderTest.assertSendDocFails(ForceLeaderTest.java:315)
>[junit4]>  at 
> org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:110)
>[junit4]>  at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
>[junit4]>  at 
> org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
>[junit4]>  at java.lang.Thread.run(Thread.java:745)
>[junit4]> Caused by: org.apache.solr.client.solrj.SolrServerException: 
> No live SolrServers available to handle this 
> request:[http://127.0.0.1:59064/forceleader_test_collection_shard1_replica1]
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:393)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:747)
>[junit4]>  ... 49 more
>[junit4]> Caused by: org.apache.solr.client.solrj.SolrServerException: 
> Server refused connection at: 
> http://127.0.0.1:59064/forceleader_test_collection_shard1_replica1
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:613)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
>[junit4]>  at 
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:382)
>[junit4]>  ... 50 more
>[junit4]> Caused by: org.apache.http.conn.HttpHostConnectException: 
> Connect to 127.0.0.1:59064 [/127.0.0.1] failed: Connection refused
>[junit4]>  at 
> org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
>[junit4]>  at 
> org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
>[junit4]>

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3566 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3566/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:52731","node_name":"127.0.0.1:52731_","state":"active","leader":"true"}];
 clusterState: 
DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/19)={   
"replicationFactor":"3",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node1":{   "state":"down",   
"base_url":"http://127.0.0.1:52714;,   
"core":"c8n_1x3_lf_shard1_replica3",   "node_name":"127.0.0.1:52714_"}, 
"core_node2":{   "core":"c8n_1x3_lf_shard1_replica1",   
"base_url":"http://127.0.0.1:52721;,   "node_name":"127.0.0.1:52721_",  
 "state":"down"}, "core_node3":{   
"core":"c8n_1x3_lf_shard1_replica2",   
"base_url":"http://127.0.0.1:52731;,   "node_name":"127.0.0.1:52731_",  
 "state":"active",   "leader":"true",   
"router":{"name":"compositeId"},   "maxShardsPerNode":"1",   
"autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 
1; 
[core_node3:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:52731","node_name":"127.0.0.1:52731_","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/19)={
  "replicationFactor":"3",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node1":{
  "state":"down",
  "base_url":"http://127.0.0.1:52714;,
  "core":"c8n_1x3_lf_shard1_replica3",
  "node_name":"127.0.0.1:52714_"},
"core_node2":{
  "core":"c8n_1x3_lf_shard1_replica1",
  "base_url":"http://127.0.0.1:52721;,
  "node_name":"127.0.0.1:52721_",
  "state":"down"},
"core_node3":{
  "core":"c8n_1x3_lf_shard1_replica2",
  "base_url":"http://127.0.0.1:52731;,
  "node_name":"127.0.0.1:52731_",
  "state":"active",
  "leader":"true",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
at 
__randomizedtesting.SeedInfo.seed([6A6803506208B3:883E57D9FE9E654B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:170)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at

[jira] [Resolved] (SOLR-9548) solr.log should start with informative welcome message

2016-09-26 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl resolved SOLR-9548.
---
Resolution: Fixed

> solr.log should start with informative welcome message
> --
>
> Key: SOLR-9548
> URL: https://issues.apache.org/jira/browse/SOLR-9548
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>  Labels: logging
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9548-detailversion.patch, SOLR-9548.patch
>
>
> When starting Solr, the first log line should be more informative, such as
> {code}
> Welcome to Apache Solr™ version 7.0.0, running in standalone mode on port 
> 8983 from folder /Users/janhoy/git/lucene-solr/solr
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9548) solr.log should start with informative welcome message


[ 
https://issues.apache.org/jira/browse/SOLR-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524213#comment-15524213
 ] 

ASF subversion and git services commented on SOLR-9548:
---

Commit 6e1c82336f2fba7ea15ec37037a1515f6ef4ecb5 in lucene-solr's branch 
refs/heads/branch_6x from [~janhoy]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6e1c823 ]

SOLR-9548: Print full solr-impl version for SNAPSHOT builds

(cherry picked from commit c1553c2)


> solr.log should start with informative welcome message
> --
>
> Key: SOLR-9548
> URL: https://issues.apache.org/jira/browse/SOLR-9548
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>  Labels: logging
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9548-detailversion.patch, SOLR-9548.patch
>
>
> When starting Solr, the first log line should be more informative, such as
> {code}
> Welcome to Apache Solr™ version 7.0.0, running in standalone mode on port 
> 8983 from folder /Users/janhoy/git/lucene-solr/solr
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9548) solr.log should start with informative welcome message

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524212#comment-15524212
 ] 

ASF subversion and git services commented on SOLR-9548:
---

Commit 047b643d2baeb2e9011d0d1bdf40209c5f1ee2c2 in lucene-solr's branch 
refs/heads/branch_6x from [~janhoy]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=047b643 ]

SOLR-9548: The beginning of solr.log now starts with a more informative welcome 
message

(cherry picked from commit 4c7a8c4)


> solr.log should start with informative welcome message
> --
>
> Key: SOLR-9548
> URL: https://issues.apache.org/jira/browse/SOLR-9548
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>  Labels: logging
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9548-detailversion.patch, SOLR-9548.patch
>
>
> When starting Solr, the first log line should be more informative, such as
> {code}
> Welcome to Apache Solr™ version 7.0.0, running in standalone mode on port 
> 8983 from folder /Users/janhoy/git/lucene-solr/solr
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9548) solr.log should start with informative welcome message

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524210#comment-15524210
 ] 

ASF subversion and git services commented on SOLR-9548:
---

Commit c1553c20f0590cccaeb0571cb254efc7394f8275 in lucene-solr's branch 
refs/heads/master from [~janhoy]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c1553c2 ]

SOLR-9548: Print full solr-impl version for SNAPSHOT builds


> solr.log should start with informative welcome message
> --
>
> Key: SOLR-9548
> URL: https://issues.apache.org/jira/browse/SOLR-9548
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>  Labels: logging
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9548-detailversion.patch, SOLR-9548.patch
>
>
> When starting Solr, the first log line should be more informative, such as
> {code}
> Welcome to Apache Solr™ version 7.0.0, running in standalone mode on port 
> 8983 from folder /Users/janhoy/git/lucene-solr/solr
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9504) A replica with an empty index becomes the leader even when other more qualified replicas are in line

2016-09-26 Thread Shalin Shekhar Mangar (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-9504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Shalin Shekhar Mangar updated SOLR-9504:

Attachment: SOLR-9504.patch

Patch with a test that fails without the fix.

Basically, when we need to bail out because we have no versions, we peek at the 
other replicas. If even one has versions, then we return this bit of 
information to the ShardLeaderElectionContext.runLeaderProcess and rejoin the 
election, else we proceed as before. The hacky bit is that there is now a 
PeerSyncResult class which has a success flag as well as an optional 
otherHasVersions flag.

I'm going to run some tests in a loop to ensure I haven't broken anything.

> A replica with an empty index becomes the leader even when other more 
> qualified replicas are in line
> 
>
> Key: SOLR-9504
> URL: https://issues.apache.org/jira/browse/SOLR-9504
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: master (7.0)
>Reporter: Shalin Shekhar Mangar
>Priority: Critical
>  Labels: impact-high
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9504.patch
>
>
> I haven't tried branch_6x or any release yet. But this is trivially 
> reproducible on master with the following steps:
> # Start two solr nodes
> # Create a collection with 1 shard, 1 replica so that one node is empty.
> # Index some documents
> # Shutdown the leader node
> # Use addreplica API to create a replica of the collection on the 
> still-running node. For some reason this API hangs until you restart the 
> other node (possibly a bug itself) but do not wait for the API to complete.
> # Restart the former leader node
> You'll find that the replica with 0 docs has become the leader. The former 
> leader recovers from the leader without replicating any index files. It still 
> has the old index which has some docs.
> This is from the logs of the 0 doc replica:
> {code}
> 713102 INFO  (zkCallback-4-thread-5-processing-n:127.0.1.1:7574_solr) [   ] 
> o.a.s.c.c.ZkStateReader Updating data for [gettingstarted] from [9] to [10]
> 714377 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Enough 
> replicas found to continue.
> 714377 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I may be 
> the new leader - try and sync
> 714377 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.c.SyncStrategy Sync replicas to 
> http://127.0.1.1:7574/solr/gettingstarted_shard1_replica2/
> 714380 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.u.PeerSync PeerSync: 
> core=gettingstarted_shard1_replica2 url=http://127.0.1.1:7574/solr START 
> replicas=[http://127.0.1.1:8983/solr/gettingstarted_shard1_replica1/] 
> nUpdates=100
> 714381 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.u.PeerSync PeerSync: 
> core=gettingstarted_shard1_replica2 url=http://127.0.1.1:7574/solr DONE.  We 
> have no versions.  sync failed.
> 714382 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.c.SyncStrategy Leader's attempt to 
> sync with shard failed, moving to the next candidate
> 714382 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.c.ShardLeaderElectionContext We 
> failed sync, but we have no versions - we can't sync in that case - we were 
> active before, so become leader anyway
> 714387 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.c.ShardLeaderElectionContextBase 
> Creating leader registration node 
> /collections/gettingstarted/leaders/shard1/leader after winning as 
> /collections/gettingstarted/leader_elect/shard1/election/96579592334475268-core_node2-n_01
> 714398 INFO  (qtp110456297-15) [c:gettingstarted s:shard1 r:core_node2 
> x:gettingstarted_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I am the 
> new leader: http://127.0.1.1:7574/solr/gettingstarted_shard1_replica2/ shard1
> {code}
> It basically tries to sync but has no versions and because it was active 
> before (it is a new core starting up for the first time), it becomes the 
> leader and publishes itself as active.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail:

[jira] [Commented] (LUCENE-7438) UnifiedHighlighter


[ 
https://issues.apache.org/jira/browse/LUCENE-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524120#comment-15524120
 ] 

David Smiley commented on LUCENE-7438:
--

Thanks for your input Ryan!  I have not heard of Wikimedia's "experimental 
highlighter"; maybe I'll go poke around there to see what it's doing.

RE snippet delineation: This is customizable in the PH & UH & FVH via supplying 
a java.text.BreakIterator.

RE snippets with multiple keywords: Yes!  I've had that thought as well and 
some months ago I filed a TODO in my wish list of highlighter features to add a 
"coordination factor" to the UH algorithm (which at the moment is identical to 
the PH).  A simple coord factor would help a lot, but even better would be one 
that also considered term diversity across the multiple passages you ask for 
rather than scoring each separately.  To do this, it might internally track 
it's the leading candidate passage per term, in addition to the PriorityQueue 
it has now.  This would be very low overhead.

> UnifiedHighlighter
> --
>
> Key: LUCENE-7438
> URL: https://issues.apache.org/jira/browse/LUCENE-7438
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Affects Versions: 6.2
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Attachments: LUCENE_7438_UH_benchmark.patch
>
>
> The UnifiedHighlighter is an evolution of the PostingsHighlighter that is 
> able to highlight using offsets in either postings, term vectors, or from 
> analysis (a TokenStream). Lucene’s existing highlighters are mostly 
> demarcated along offset source lines, whereas here it is unified -- hence 
> this proposed name. In this highlighter, the offset source strategy is 
> separated from the core highlighting functionalty. The UnifiedHighlighter 
> further improves on the PostingsHighlighter’s design by supporting accurate 
> phrase highlighting using an approach similar to the standard highlighter’s 
> WeightedSpanTermExtractor. The next major improvement is a hybrid offset 
> source strategythat utilizes postings and “light” term vectors (i.e. just the 
> terms) for highlighting multi-term queries (wildcards) without resorting to 
> analysis. Phrase highlighting and wildcard highlighting can both be disabled 
> if you’d rather highlight a little faster albeit not as accurately reflecting 
> the query.
> We’ve benchmarked an earlier version of this highlighter comparing it to the 
> other highlighters and the results were exciting! It’s tempting to share 
> those results but it’s definitely due for another benchmark, so we’ll work on 
> that. Performance was the main motivator for creating the UnifiedHighlighter, 
> as the standard Highlighter (the only one meeting Bloomberg Law’s accuracy 
> requirements) wasn’t fast enough, even with term vectors along with several 
> improvements we contributed back, and even after we forked it to highlight in 
> multiple threads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-7438) UnifiedHighlighter

2016-09-26 Thread Ryan Pedela (JIRA)

[
https://issues.apache.org/jira/browse/LUCENE-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524080#comment-15524080
]

Ryan Pedela edited comment on LUCENE-7438 at 9/26/16 8:27 PM:
--

I am very happy to see this. I use Elasticsearch, and I currently use the
[experimental highlighter
plugin|https://github.com/wikimedia/search-highlighter] for three reasons.

1. It uses either term vectors or postings to increase performance.
2. It has fragment and sentence modes.
3. The sentence mode produces significantly better highlights than the postings
highlighter in my experience.

I would prefer to use an official highlighter and happy to see that the
UnifiedHighlighter will take care of #1 and #2. Now I would like to talk about
#3.

I don't know the specifics of the algorithm, but the experimental highlighter
appears to take proximity and a keyword's document position into account. One
example from memory, I had a medical research paper about warfarin and the
highlight returned by the postings highlighter for the search "warfarin" came
from the references. However the experimental highlighter returned a highlight
near the beginning of the paper and it was a pretty good summary of the paper.

There is also room for improvement for both the experimental and postings
highlighters. They both appear to use the same sentence fragmenter which does
not do a good job with abbreviations and decimal points. Would something like
Stanford CoreNLP help?

Also I would like a highlighter that tries to get as many keywords as possible
into the highlight, at least as a config option. That is hard if only returning
a single sentence or fragment. However I often want three fragments and I would
like the union of the three fragments to contain all the keywords or as many as
possible. For example, I am working on a search engine for SEC filings and a
user searched "BPL hedgings" during a user test. BPL is the stock ticker for
Buckeye Partners, and stock tickers are pretty unique within the SEC filings.
The experimental highlighter returned three fragments with "BPL" but no
"hedgings" (fast vector highlighter produced similar fragments). The user was
very confused because they didn't see the word "hedgings" in the highlight and
thought it wasn't found even though it was. To fix this, I retrieve the top 100
fragments and post-process them to find the best 3 fragments which contain the
most keywords collectively. The post processing is quite naive since it does
not understand proximity, stemming, etc. I would prefer if Lucene or ES did it
because it can be much smarter.

was (Author: rpedela):
I am very happy to see this. I use Elasticsearch, and I currently use the
[experimental highlighter
plugin|https://github.com/wikimedia/search-highlighter] for three reasons.

I would prefer to use an official highlighter and happy to see that the
UnifiedHighlighter will take care of #1 and #2. Now I would like to talk about
#3.

Also I would like a highlighter that tries to get as many keywords as possible
into the highlight, at least as a config option. That is hard if only returning
a single sentence or fragment. However I often want three fragments and I would
like the union of the three fragments to contain all the keywords or as many as
possible. For example, I am working on a search engine for SEC filings and a
user searched "BPL hedgings" during a user test. BPL is the stock ticker for
Buckeye Partners, and stock tickers are pretty unique within the SEC filings.
The experimental highlighter returned three fragments with "BPL" but no
"hedgings" (fast vector highlighter produced similar fragments). The user was
very confused because they didn't see the word "hedgings" in the highlight and
thought that keyword wasn't found even though it was. To fix this, I retrieve
the top 100 fragments and post-process them to find the best 3 fragments which
contain the most keywords

[jira] [Commented] (LUCENE-7438) UnifiedHighlighter

2016-09-26 Thread Ryan Pedela (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15524080#comment-15524080
 ] 

Ryan Pedela commented on LUCENE-7438:
-

I am very happy to see this. I use Elasticsearch, and I currently use the 
[experimental highlighter 
plugin|https://github.com/wikimedia/search-highlighter] for three reasons.

1. It uses either term vectors or postings to increase performance.
2. It has fragment and sentence modes.
3. The sentence mode produces significantly better highlights than the postings 
highlighter in my experience.

I would prefer to use an official highlighter and happy to see that the 
UnifiedHighlighter will take care of #1 and #2. Now I would like to talk about 
#3.

I don't know the specifics of the algorithm, but the experimental highlighter 
appears to take proximity and a keyword's document position into account. One 
example from memory, I had a medical research paper about warfarin and the 
highlight returned by the postings highlighter for the search "warfarin" came 
from the references. However the experimental highlighter returned a highlight 
near the beginning of the paper and it was a pretty good summary of the paper.

There is also room for improvement for both the experimental and postings 
highlighters. They both appear to use the same sentence fragmenter which does 
not do a good job with abbreviations and decimal points. Would something like 
Stanford CoreNLP help?

Also I would like a highlighter that tries to get as many keywords as possible 
into the highlight, at least as a config option. That is hard if only returning 
a single sentence or fragment. However I often want three fragments and I would 
like the union of the three fragments to contain all the keywords or as many as 
possible. For example, I am working on a search engine for SEC filings and a 
user searched "BPL hedgings" during a user test. BPL is the stock ticker for 
Buckeye Partners, and stock tickers are pretty unique within the SEC filings. 
The experimental highlighter returned three fragments with "BPL" but no 
"hedgings" (fast vector highlighter produced similar fragments). The user was 
very confused because they didn't see the word "hedgings" in the highlight and 
thought that keyword wasn't found even though it was. To fix this, I retrieve 
the top 100 fragments and post-process them to find the best 3 fragments which 
contain the most keywords collectively. The post processing is quite naive 
since it does not understand proximity, stemming, etc. I would prefer if Lucene 
or ES did it because it can be much smarter.

> UnifiedHighlighter
> --
>
> Key: LUCENE-7438
> URL: https://issues.apache.org/jira/browse/LUCENE-7438
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: modules/highlighter
>Affects Versions: 6.2
>Reporter: Timothy M. Rodriguez
>Assignee: David Smiley
> Attachments: LUCENE_7438_UH_benchmark.patch
>
>
> The UnifiedHighlighter is an evolution of the PostingsHighlighter that is 
> able to highlight using offsets in either postings, term vectors, or from 
> analysis (a TokenStream). Lucene’s existing highlighters are mostly 
> demarcated along offset source lines, whereas here it is unified -- hence 
> this proposed name. In this highlighter, the offset source strategy is 
> separated from the core highlighting functionalty. The UnifiedHighlighter 
> further improves on the PostingsHighlighter’s design by supporting accurate 
> phrase highlighting using an approach similar to the standard highlighter’s 
> WeightedSpanTermExtractor. The next major improvement is a hybrid offset 
> source strategythat utilizes postings and “light” term vectors (i.e. just the 
> terms) for highlighting multi-term queries (wildcards) without resorting to 
> analysis. Phrase highlighting and wildcard highlighting can both be disabled 
> if you’d rather highlight a little faster albeit not as accurately reflecting 
> the query.
> We’ve benchmarked an earlier version of this highlighter comparing it to the 
> other highlighters and the results were exciting! It’s tempting to share 
> those results but it’s definitely due for another benchmark, so we’ll work on 
> that. Performance was the main motivator for creating the UnifiedHighlighter, 
> as the standard Highlighter (the only one meeting Bloomberg Law’s accuracy 
> requirements) wasn’t fast enough, even with term vectors along with several 
> improvements we contributed back, and even after we forked it to highlight in 
> multiple threads.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9566) Can we avoid doing recovery when collections are first created?


 [ 
https://issues.apache.org/jira/browse/SOLR-9566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated SOLR-9566:

Attachment: SOLR-9566.patch

Here's a patch adding a 'newCollection' parameter to the core create command.  
I haven't run it through a full test yet, but DeleteReplicaTest went from 54 
seconds to 20 seconds, which is a very nice speedup.

[~markrmil...@gmail.com] [~yo...@apache.org] is this proposal sane?

> Can we avoid doing recovery when collections are first created?
> ---
>
> Key: SOLR-9566
> URL: https://issues.apache.org/jira/browse/SOLR-9566
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
> Attachments: SOLR-9566.patch
>
>
> When a core starts up as part of a collection, and it's not a shard leader, 
> it goes into recovery, part of which involves a 7 second wait (set in 
> SOLR-7141) to ensure that updates being sent from the leader don't get missed 
> in between buffering and replication.
> This has the unfortunate side-effect of adding a 7-second pause to collection 
> creation whenever the replication factor is 2 or more, which slows down tests 
> massively - for example, DeleteReplicaTest takes about 54 seconds to execute 
> on my machine, 28 seconds of which is just pauses - over 50% of execution 
> time.  It's not actually possible to add documents to a collection before the 
> creation request has returned, so the recovery stage here isn't strictly 
> speaking necessary.  
> I think we could try adding a parameter to a CoreAdmin create request that 
> says the core is being created as part of a new collection, so it doesn't 
> need to try and recover from it's leader when it starts up.  Does this sound 
> sensible, or am I missing something here?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_102) - Build # 17907 - Failure!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17907/
Java: 64bit/jdk1.8.0_102 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  
org.apache.solr.handler.component.DistributedFacetPivotLongTailTest.test

Error Message:
IOException occured when talking to server at: 
https://127.0.0.1:37236//collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: IOException occured when 
talking to server at: https://127.0.0.1:37236//collection1
at 
__randomizedtesting.SeedInfo.seed([2714036F2A3C998F:AF403CB584C0F477]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:622)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
at 
org.apache.solr.handler.component.DistributedFacetPivotLongTailTest.test(DistributedFacetPivotLongTailTest.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Created] (SOLR-9569) Cleanup sample configuration with best practices

Noble Paul created SOLR-9569:


 Summary: Cleanup sample configuration with best practices
 Key: SOLR-9569
 URL: https://issues.apache.org/jira/browse/SOLR-9569
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul
Assignee: Noble Paul


* Move most commonly used paths to ImplicitPlugins
* move their request parameters to params.json
** To make this easier to understand enhance the solconfig API to expand show 
the params used for each requesthandler inline
* Remove {{path}} usage in {{initParams}}

Today we use the solrconfig.xml as a place to document things. As we move more 
stuff off of solrconfig.xml let's point it to a ref guide page to discuss about 
all the requesthandlers available by default and their configuration



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_102) - Build # 6140 - Failure!

2016-09-26 Thread ASF subversion and git services (JIRA)

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6140/
Java: 32bit/jdk1.8.0_102 -client -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.handler.component.StatsComponentTest.testPercentiles

Error Message:
Unable to parse stats.field local params: percentiles==stat_f due to: empty 
String

Stack Trace:
org.apache.solr.common.SolrException: Unable to parse stats.field local params: 
percentiles==stat_f due to: empty String
at 
__randomizedtesting.SeedInfo.seed([59BC6C0D6E4011A8:6D7BCC392824989C]:0)
at 
org.apache.solr.handler.component.StatsField$Stat$1.parseParams(StatsField.java:110)
at 
org.apache.solr.handler.component.StatsField.populateStatsSets(StatsField.java:533)
at 
org.apache.solr.handler.component.StatsField.(StatsField.java:288)
at 
org.apache.solr.handler.component.StatsInfo.(StatsComponent.java:188)
at 
org.apache.solr.handler.component.StatsComponent.prepare(StatsComponent.java:47)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:268)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2106)
at 
org.apache.solr.util.TestHarness.queryAndResponse(TestHarness.java:329)
at 
org.apache.solr.handler.component.StatsComponentTest.testPercentiles(StatsComponentTest.java:1925)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523679#comment-15523679
 ] 

ASF subversion and git services commented on SOLR-9557:
---

Commit 208e2bc66f9b931feb0a168f0ef62a7bf27a9764 in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=208e2bc ]

SOLR-9557: optimize splitsmart


> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523673#comment-15523673
 ] 

ASF subversion and git services commented on SOLR-9557:
---

Commit 36cd9092922b8d03797dfcbf8bbff6efc9037f51 in lucene-solr's branch 
refs/heads/branch_6x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=36cd909 ]

SOLR-9557: optimize splitsmart


> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523670#comment-15523670
 ] 

Alan Woodward commented on SOLR-9557:
-

Thanks!

> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment


[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523631#comment-15523631
 ] 

Noble Paul commented on SOLR-9506:
--

the cumulative numDocs will be same anyway

I guess it can be reproduced as follows

# take a 2 replica shard
# index and commit multiple times
# delete one doc and commit
# bring down replica
# optimize leader
# bring up replica

I guess this will lead to a full replication





> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Linux (32bit/jdk1.8.0_102) - Build # 1808 - Failure!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1808/
Java: 32bit/jdk1.8.0_102 -server -XX:+UseSerialGC

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
Expected 2 of 3 replicas to be active but only found 1; 
[core_node2:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:45465","node_name":"127.0.0.1:45465_","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/32)={   
"replicationFactor":"3",   "shards":{"shard1":{   
"range":"8000-7fff",   "state":"active",   "replicas":{ 
"core_node1":{   "state":"down",   
"base_url":"http://127.0.0.1:40681;,   
"core":"c8n_1x3_lf_shard1_replica3",   "node_name":"127.0.0.1:40681_"}, 
"core_node2":{   "core":"c8n_1x3_lf_shard1_replica2",   
"base_url":"http://127.0.0.1:45465;,   "node_name":"127.0.0.1:45465_",  
 "state":"active",   "leader":"true"}, "core_node3":{   
"core":"c8n_1x3_lf_shard1_replica1",   
"base_url":"http://127.0.0.1:39762;,   "node_name":"127.0.0.1:39762_",  
 "state":"down",   "router":{"name":"compositeId"},   
"maxShardsPerNode":"1",   "autoAddReplicas":"false"}

Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 
1; 
[core_node2:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:45465","node_name":"127.0.0.1:45465_","state":"active","leader":"true"}];
 clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/32)={
  "replicationFactor":"3",
  "shards":{"shard1":{
  "range":"8000-7fff",
  "state":"active",
  "replicas":{
"core_node1":{
  "state":"down",
  "base_url":"http://127.0.0.1:40681;,
  "core":"c8n_1x3_lf_shard1_replica3",
  "node_name":"127.0.0.1:40681_"},
"core_node2":{
  "core":"c8n_1x3_lf_shard1_replica2",
  "base_url":"http://127.0.0.1:45465;,
  "node_name":"127.0.0.1:45465_",
  "state":"active",
  "leader":"true"},
"core_node3":{
  "core":"c8n_1x3_lf_shard1_replica1",
  "base_url":"http://127.0.0.1:39762;,
  "node_name":"127.0.0.1:39762_",
  "state":"down",
  "router":{"name":"compositeId"},
  "maxShardsPerNode":"1",
  "autoAddReplicas":"false"}
at 
__randomizedtesting.SeedInfo.seed([F10367AA37FB38BA:7957587099075542]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:168)
at 
org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment


[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523564#comment-15523564
 ] 

Pushkar Raste commented on SOLR-9506:
-

I think what [~ichattopadhyaya] is hinting at, is that if {{numDocs}} account 
only for live (active) docs, then once documents are deleted in a segment, 
{{numDocs}} in the cached fingerprint might be wrong. 

Surprising, following test cases passed with my POC
1. {{PeerSyncTest}}
2. {{PeerSyncReplicationTest}}
3. {{SyncSliceTest}}

In the worst case, we can atleast parallalize fingerprint computation. 

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523566#comment-15523566
 ] 

Pushkar Raste commented on SOLR-9506:
-

Adding [~ysee...@gmail.com] in the loop

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523531#comment-15523531
 ] 

ASF subversion and git services commented on SOLR-9557:
---

Commit 3f35cadd44a0515018bed079124731bc159d54d0 in lucene-solr's branch 
refs/heads/branch_6x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3f35cad ]

SOLR-9557: reverting an optimization


> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523526#comment-15523526
 ] 

ASF subversion and git services commented on SOLR-9557:
---

Commit f6622ef3748587a1636aeab046d4597cb475a18a in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f6622ef ]

SOLR-9557: reverting an optimization


> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523521#comment-15523521
 ] 

Noble Paul commented on SOLR-9557:
--

yes. The optimization I did for StrUtils.splitSmart failed the test. Strange

> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523494#comment-15523494
 ] 

Noble Paul commented on SOLR-9557:
--

I shall check this out. But this has nothing to do with localparams

> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-9506) cache IndexFingerprint for each segment


[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523490#comment-15523490
 ] 

Noble Paul edited comment on SOLR-9506 at 9/26/16 4:15 PM:
---

no. segments don't change. you mean fingerprints can change?




was (Author: noble.paul):
no. segments don't change


> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment


[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523490#comment-15523490
 ] 

Noble Paul commented on SOLR-9506:
--

no. segments don't change


> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment


[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523485#comment-15523485
 ] 

Ishan Chattopadhyaya commented on SOLR-9506:


We should keep in mind that previously written segments can change if there are 
deletes. Maybe we should recompute the per-segment fingerprints upon deletion 
in that segment.

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523487#comment-15523487
 ] 

Alan Woodward commented on SOLR-9557:
-

I think this has broken 
org.apache.solr.handler.component.StatsComponentTest.testPercentiles?  
Something to do with empty localparams no longer being reported as having null 
values, I think...


> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9560) Solr should check max open files and other ulimits and refuse to start if they are set too low


[ 
https://issues.apache.org/jira/browse/SOLR-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523464#comment-15523464
 ] 

Jan Høydahl commented on SOLR-9560:
---

Perhaps we could have a {{bin/solr start -prod}} switch which turns on a bunch 
of best-practice settings, such as enforcing file-handles, requiring 64-bit 
Java, disabling stream handler, logging at WARN level, refusing to run as root 
user, increasing log4j.appender.file.MaxFileSize, refusing to start with 
default SSL/BasicAuth passwords etc... I've seen many misconfigured 
prod-systems running OOTB settings :(

> Solr should check max open files and other ulimits and refuse to start if 
> they are set too low
> --
>
> Key: SOLR-9560
> URL: https://issues.apache.org/jira/browse/SOLR-9560
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Shalin Shekhar Mangar
> Fix For: 6.3, master (7.0)
>
>
> Solr should check max open files and other ulimits and refuse to start if 
> they are set too low. Specifically:
> # max open files should be at least 32768
> # max memory size and virtual memory should both be unlimited



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (SOLR-9568) case-insensitive sorting not working as intended.


 [ 
https://issues.apache.org/jira/browse/SOLR-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

satish chennupati reopened SOLR-9568:
-

i have updated the bug with Solr version and the exact query that get posted to 
the Solr engine.

> case-insensitive sorting not working as intended.
> -
>
> Key: SOLR-9568
> URL: https://issues.apache.org/jira/browse/SOLR-9568
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 4.10.3
>Reporter: satish chennupati
>  Labels: dynamic, sorting
>
> Hi,
> Right now when I perform a sort I  am getting result as following:
> "[\"18 hotfix3\"]"
> "[\"Godzilla\"]"
> "[\"Godzilla, King of the Monsters!\"]"
> "[\"Harry Potter and the Sorcerers Stone\"]"
> "[\"How to Train Your Dragon\"]"
> "[\"Jurassic Park\"]"
> "[\"My Big Fat Greek Wedding\"]"
> "[\"National Treasure\"]"
> "[\"Palmer\"]"
> "[\"Patch Adams\"]"
> "[\"Rajan\"]"
> "[\"Sanity\"]"
> "[\"Stardust\"]"
> "[\"Superman\"]"
> "[\"The Amazing Spider-Man 2\"]"
> "[\"The Godfather\"]"
> "[\"The Lord of the Rings: The Fellowship of the Ring\"]"
> "[\"The Matrix\"]"
> "[\"V for Vendetta\"]"
> "[\"abcdefgh\"]"
> "[\"autoui1466571231695\"]"
> "[\"autoui1466605339320\"]"
> "[\"name\"]"
> "[\"test\"]"
> "[\"test2\"]"
> The field type has been defined as follows :
> sortMissingLast="true">
> 
> 
> 
> 
> 
> 
> 
>  pattern="(\d+)" replacement="0$1" replace="all"/>
>  pattern="0*([0-9]{6,})" replacement="$1" 
> replace="all" />
> 
> 
> 
> And for sorting purpose we have a dynamic field in place that used the above 
> field type
> 
>  stored="false" type="SortTextField"/>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9568) case-insensitive sorting not working as intended.


 [ 
https://issues.apache.org/jira/browse/SOLR-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

satish chennupati updated SOLR-9568:

Affects Version/s: 4.10.3

> case-insensitive sorting not working as intended.
> -
>
> Key: SOLR-9568
> URL: https://issues.apache.org/jira/browse/SOLR-9568
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 4.10.3
>Reporter: satish chennupati
>  Labels: dynamic, sorting
>
> Hi,
> Right now when I perform a sort I  am getting result as following:
> "[\"18 hotfix3\"]"
> "[\"Godzilla\"]"
> "[\"Godzilla, King of the Monsters!\"]"
> "[\"Harry Potter and the Sorcerers Stone\"]"
> "[\"How to Train Your Dragon\"]"
> "[\"Jurassic Park\"]"
> "[\"My Big Fat Greek Wedding\"]"
> "[\"National Treasure\"]"
> "[\"Palmer\"]"
> "[\"Patch Adams\"]"
> "[\"Rajan\"]"
> "[\"Sanity\"]"
> "[\"Stardust\"]"
> "[\"Superman\"]"
> "[\"The Amazing Spider-Man 2\"]"
> "[\"The Godfather\"]"
> "[\"The Lord of the Rings: The Fellowship of the Ring\"]"
> "[\"The Matrix\"]"
> "[\"V for Vendetta\"]"
> "[\"abcdefgh\"]"
> "[\"autoui1466571231695\"]"
> "[\"autoui1466605339320\"]"
> "[\"name\"]"
> "[\"test\"]"
> "[\"test2\"]"
> The field type has been defined as follows :
> sortMissingLast="true">
> 
> 
> 
> 
> 
> 
> 
>  pattern="(\d+)" replacement="0$1" replace="all"/>
>  pattern="0*([0-9]{6,})" replacement="$1" 
> replace="all" />
> 
> 
> 
> And for sorting purpose we have a dynamic field in place that used the above 
> field type
> 
>  stored="false" type="SortTextField"/>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9568) case-insensitive sorting not working as intended.


[ 
https://issues.apache.org/jira/browse/SOLR-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523440#comment-15523440
 ] 

satish chennupati commented on SOLR-9568:
-

hi [~erickerickson] i already posted by issue on the solr distribution list few 
days back.

the solr version i am using is : 4.10

the exact query that get posted to solr is : 
q=*:*=bp_urn:"urn:cms:bp:Movie"=state:*+AND+!state:deleted=sort_str_attr_name+asc

> case-insensitive sorting not working as intended.
> -
>
> Key: SOLR-9568
> URL: https://issues.apache.org/jira/browse/SOLR-9568
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: satish chennupati
>  Labels: dynamic, sorting
>
> Hi,
> Right now when I perform a sort I  am getting result as following:
> "[\"18 hotfix3\"]"
> "[\"Godzilla\"]"
> "[\"Godzilla, King of the Monsters!\"]"
> "[\"Harry Potter and the Sorcerers Stone\"]"
> "[\"How to Train Your Dragon\"]"
> "[\"Jurassic Park\"]"
> "[\"My Big Fat Greek Wedding\"]"
> "[\"National Treasure\"]"
> "[\"Palmer\"]"
> "[\"Patch Adams\"]"
> "[\"Rajan\"]"
> "[\"Sanity\"]"
> "[\"Stardust\"]"
> "[\"Superman\"]"
> "[\"The Amazing Spider-Man 2\"]"
> "[\"The Godfather\"]"
> "[\"The Lord of the Rings: The Fellowship of the Ring\"]"
> "[\"The Matrix\"]"
> "[\"V for Vendetta\"]"
> "[\"abcdefgh\"]"
> "[\"autoui1466571231695\"]"
> "[\"autoui1466605339320\"]"
> "[\"name\"]"
> "[\"test\"]"
> "[\"test2\"]"
> The field type has been defined as follows :
> sortMissingLast="true">
> 
> 
> 
> 
> 
> 
> 
>  pattern="(\d+)" replacement="0$1" replace="all"/>
>  pattern="0*([0-9]{6,})" replacement="$1" 
> replace="all" />
> 
> 
> 
> And for sorting purpose we have a dynamic field in place that used the above 
> field type
> 
>  stored="false" type="SortTextField"/>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+136) - Build # 17906 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/17906/
Java: 64bit/jdk-9-ea+136 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.handler.component.StatsComponentTest.testPercentiles

Error Message:
Unable to parse stats.field local params: percentiles==stat_f due to: empty 
String

Stack Trace:
org.apache.solr.common.SolrException: Unable to parse stats.field local params: 
percentiles==stat_f due to: empty String
at 
__randomizedtesting.SeedInfo.seed([7D4164EBA2A220BC:4986C4DFE4C6A988]:0)
at 
org.apache.solr.handler.component.StatsField$Stat$1.parseParams(StatsField.java:110)
at 
org.apache.solr.handler.component.StatsField.populateStatsSets(StatsField.java:533)
at 
org.apache.solr.handler.component.StatsField.(StatsField.java:288)
at 
org.apache.solr.handler.component.StatsInfo.(StatsComponent.java:188)
at 
org.apache.solr.handler.component.StatsComponent.prepare(StatsComponent.java:47)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:268)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2106)
at 
org.apache.solr.util.TestHarness.queryAndResponse(TestHarness.java:329)
at 
org.apache.solr.handler.component.StatsComponentTest.testPercentiles(StatsComponentTest.java:1925)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-ea/Native 
Method)
at 
jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-ea/NativeMethodAccessorImpl.java:62)
at 
jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-ea/DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(java.base@9-ea/Method.java:535)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at

[jira] [Resolved] (SOLR-9568) case-insensitive sorting not working as intended.

2016-09-26 Thread Erick Erickson (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Erick Erickson resolved SOLR-9568.
--
Resolution: Cannot Reproduce

Please raise issues like on the user's list first before raising a JIRA. If the 
consensus is that it's a code problem, _then_ raise a JIRA.

When you do, please specify what version of Solr you're using. Also show the 
exact query you use.

This works for me on Solr 6.x. Did you complete re-index after changing field 
definitions?

> case-insensitive sorting not working as intended.
> -
>
> Key: SOLR-9568
> URL: https://issues.apache.org/jira/browse/SOLR-9568
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: satish chennupati
>  Labels: dynamic, sorting
>
> Hi,
> Right now when I perform a sort I  am getting result as following:
> "[\"18 hotfix3\"]"
> "[\"Godzilla\"]"
> "[\"Godzilla, King of the Monsters!\"]"
> "[\"Harry Potter and the Sorcerers Stone\"]"
> "[\"How to Train Your Dragon\"]"
> "[\"Jurassic Park\"]"
> "[\"My Big Fat Greek Wedding\"]"
> "[\"National Treasure\"]"
> "[\"Palmer\"]"
> "[\"Patch Adams\"]"
> "[\"Rajan\"]"
> "[\"Sanity\"]"
> "[\"Stardust\"]"
> "[\"Superman\"]"
> "[\"The Amazing Spider-Man 2\"]"
> "[\"The Godfather\"]"
> "[\"The Lord of the Rings: The Fellowship of the Ring\"]"
> "[\"The Matrix\"]"
> "[\"V for Vendetta\"]"
> "[\"abcdefgh\"]"
> "[\"autoui1466571231695\"]"
> "[\"autoui1466605339320\"]"
> "[\"name\"]"
> "[\"test\"]"
> "[\"test2\"]"
> The field type has been defined as follows :
> sortMissingLast="true">
> 
> 
> 
> 
> 
> 
> 
>  pattern="(\d+)" replacement="0$1" replace="all"/>
>  pattern="0*([0-9]{6,})" replacement="$1" 
> replace="all" />
> 
> 
> 
> And for sorting purpose we have a dynamic field in place that used the above 
> field type
> 
>  stored="false" type="SortTextField"/>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Solaris (64bit/jdk1.8.0) - Build # 413 - Failure!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Solaris/413/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, 
NRTCachingDirectory, NRTCachingDirectory] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
  at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:372)  
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254) 
 at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) 
 at 
org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
  at java.lang.Thread.run(Thread.java:745)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
  at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:372)  
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254) 
 at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) 
 at 
org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
  at java.lang.Thread.run(Thread.java:745)  
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
  at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
  at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:372)  
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254) 
 at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397) 
 at 
org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
  at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 3 object(s) that were not 
released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:372)
at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254)
at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at 
org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:372)
at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254)
at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at 
org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at 
org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:372)
at 
org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254)
at 
org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
at 
org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:279)
at java.lang.Thread.run(Thread.java:745)


at __randomizedtesting.SeedInfo.seed([2FD8E12FA5588382]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:261)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at

[jira] [Issue Comment Deleted] (SOLR-9506) cache IndexFingerprint for each segment


 [ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pushkar Raste updated SOLR-9506:

Comment: was deleted

(was: In short you are suggesting that when we cache fingerprint for individual 
segments, we keep a list of version numbers in those segments around? That 
would be billions of {{Long}} values cached, which might be counter-productive,)

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7467) Caused by: java.lang.IllegalArgumentException: position increments (and gaps) must be >= 0 (got 65248) for field 'tf_attachments_field_library_attachments'

2016-09-26 Thread adeppa (JIRA)

adeppa created LUCENE-7467:
--

 Summary: Caused by: java.lang.IllegalArgumentException: position 
increments (and gaps) must be >= 0 (got 65248) for field 
'tf_attachments_field_library_attachments'
 Key: LUCENE-7467
 URL: https://issues.apache.org/jira/browse/LUCENE-7467
 Project: Lucene - Core
  Issue Type: Bug
  Components: core/codecs, core/other, core/store, modules/analysis
Affects Versions: 5.1
 Environment: Mac 
Reporter: adeppa


I was try to indexing the large file like PDF,PPT,PPTX and XLXS,XLX,
actual token count are 65248 but this error is coming from the 
DefaultIndexingChain class while executing the public void 
invert(IndexableField field, boolean first) throws IOException, 
AbortingException in side this method we are checking the one condition like 
*if (invertState.position < invertState.lastPosition)* here 
invertState.position value become negative, when it increases int.MAX_VALUE,


org.apache.solr.common.SolrException: Exception writing document id 
dc65t0-marketing_site-141457 to the index; possible analysis error.
at 
org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:167)
at 
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
at 
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:955)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1110)
at 
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:706)
at 
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
at 
org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:250)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177)
at 
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
at 
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
at 
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:528)
at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1099)
at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:672)
at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1520)
at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1476)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at 
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: position increments (and gaps) 
must be >= 0 (got 65248) for field 'tf_attachments_field_library_attachments'
at 
org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:631)
at 
org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:344)
at 
org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:300)
at

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation


[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523315#comment-15523315
 ] 

Michael McCandless commented on LUCENE-7465:


bq.  default to current java regexp impl;

But I think that would mean this new impl would very rarely be used.

I think it's better to give it a separate factory so it has more visibility?  
If it really does work better for users over time, word will spread, new blog 
posts/docs written, etc.

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9520) Kerberos delegation support with CloudSolrClient


[ 
https://issues.apache.org/jira/browse/SOLR-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523297#comment-15523297
 ] 

Ishan Chattopadhyaya commented on SOLR-9520:


Had an offline discussion with Noble, and he just pointed out to me that the 
CSC should take in an LBHSC builder, and the LBHSC should take in the HSC 
builder. This will ensure that the CSC and LBHSC can remain oblivious of the 
delegation token, only the builders will have that information. I'm working on 
that change.

> Kerberos delegation support with CloudSolrClient
> 
>
> Key: SOLR-9520
> URL: https://issues.apache.org/jira/browse/SOLR-9520
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
> Attachments: SOLR-9520.patch
>
>
> HSC support is available, but we need to add support to CSC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9568) case-insensitive sorting not working as intended.


[ 
https://issues.apache.org/jira/browse/SOLR-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523270#comment-15523270
 ] 

satish chennupati commented on SOLR-9568:
-

update :

i also tried the following

   







 





   



 
 

> case-insensitive sorting not working as intended.
> -
>
> Key: SOLR-9568
> URL: https://issues.apache.org/jira/browse/SOLR-9568
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: satish chennupati
>  Labels: dynamic, sorting
>
> Hi,
> Right now when I perform a sort I  am getting result as following:
> "[\"18 hotfix3\"]"
> "[\"Godzilla\"]"
> "[\"Godzilla, King of the Monsters!\"]"
> "[\"Harry Potter and the Sorcerers Stone\"]"
> "[\"How to Train Your Dragon\"]"
> "[\"Jurassic Park\"]"
> "[\"My Big Fat Greek Wedding\"]"
> "[\"National Treasure\"]"
> "[\"Palmer\"]"
> "[\"Patch Adams\"]"
> "[\"Rajan\"]"
> "[\"Sanity\"]"
> "[\"Stardust\"]"
> "[\"Superman\"]"
> "[\"The Amazing Spider-Man 2\"]"
> "[\"The Godfather\"]"
> "[\"The Lord of the Rings: The Fellowship of the Ring\"]"
> "[\"The Matrix\"]"
> "[\"V for Vendetta\"]"
> "[\"abcdefgh\"]"
> "[\"autoui1466571231695\"]"
> "[\"autoui1466605339320\"]"
> "[\"name\"]"
> "[\"test\"]"
> "[\"test2\"]"
> The field type has been defined as follows :
> sortMissingLast="true">
> 
> 
> 
> 
> 
> 
> 
>  pattern="(\d+)" replacement="0$1" replace="all"/>
>  pattern="0*([0-9]{6,})" replacement="$1" 
> replace="all" />
> 
> 
> 
> And for sorting purpose we have a dynamic field in place that used the above 
> field type
> 
>  stored="false" type="SortTextField"/>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment


[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523273#comment-15523273
 ] 

Pushkar Raste commented on SOLR-9506:
-

In short you are suggesting that when we cache fingerprint for individual 
segments, we keep a list of version numbers in those segments around? That 
would be billions of {{Long}} values cached, which might be counter-productive,

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9568) case-insensitive sorting not working as intended.


[ 
https://issues.apache.org/jira/browse/SOLR-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523272#comment-15523272
 ] 

satish chennupati commented on SOLR-9568:
-

update :

i also tried the following

   







 





   



 
 

> case-insensitive sorting not working as intended.
> -
>
> Key: SOLR-9568
> URL: https://issues.apache.org/jira/browse/SOLR-9568
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: satish chennupati
>  Labels: dynamic, sorting
>
> Hi,
> Right now when I perform a sort I  am getting result as following:
> "[\"18 hotfix3\"]"
> "[\"Godzilla\"]"
> "[\"Godzilla, King of the Monsters!\"]"
> "[\"Harry Potter and the Sorcerers Stone\"]"
> "[\"How to Train Your Dragon\"]"
> "[\"Jurassic Park\"]"
> "[\"My Big Fat Greek Wedding\"]"
> "[\"National Treasure\"]"
> "[\"Palmer\"]"
> "[\"Patch Adams\"]"
> "[\"Rajan\"]"
> "[\"Sanity\"]"
> "[\"Stardust\"]"
> "[\"Superman\"]"
> "[\"The Amazing Spider-Man 2\"]"
> "[\"The Godfather\"]"
> "[\"The Lord of the Rings: The Fellowship of the Ring\"]"
> "[\"The Matrix\"]"
> "[\"V for Vendetta\"]"
> "[\"abcdefgh\"]"
> "[\"autoui1466571231695\"]"
> "[\"autoui1466605339320\"]"
> "[\"name\"]"
> "[\"test\"]"
> "[\"test2\"]"
> The field type has been defined as follows :
> sortMissingLast="true">
> 
> 
> 
> 
> 
> 
> 
>  pattern="(\d+)" replacement="0$1" replace="all"/>
>  pattern="0*([0-9]{6,})" replacement="$1" 
> replace="all" />
> 
> 
> 
> And for sorting purpose we have a dynamic field in place that used the above 
> field type
> 
>  stored="false" type="SortTextField"/>



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9568) case-insensitive sorting not working as intended.

satish chennupati created SOLR-9568:
---

 Summary: case-insensitive sorting not working as intended.
 Key: SOLR-9568
 URL: https://issues.apache.org/jira/browse/SOLR-9568
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: satish chennupati


Hi,

Right now when I perform a sort I  am getting result as following:


"[\"18 hotfix3\"]"
"[\"Godzilla\"]"
"[\"Godzilla, King of the Monsters!\"]"
"[\"Harry Potter and the Sorcerers Stone\"]"
"[\"How to Train Your Dragon\"]"
"[\"Jurassic Park\"]"
"[\"My Big Fat Greek Wedding\"]"
"[\"National Treasure\"]"
"[\"Palmer\"]"
"[\"Patch Adams\"]"
"[\"Rajan\"]"
"[\"Sanity\"]"
"[\"Stardust\"]"
"[\"Superman\"]"
"[\"The Amazing Spider-Man 2\"]"
"[\"The Godfather\"]"
"[\"The Lord of the Rings: The Fellowship of the Ring\"]"
"[\"The Matrix\"]"
"[\"V for Vendetta\"]"
"[\"abcdefgh\"]"
"[\"autoui1466571231695\"]"
"[\"autoui1466605339320\"]"
"[\"name\"]"
"[\"test\"]"
"[\"test2\"]"

The field type has been defined as follows :

   














And for sorting purpose we have a dynamic field in place that used the above 
field type






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9506) cache IndexFingerprint for each segment


[ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523257#comment-15523257
 ] 

Noble Paul commented on SOLR-9506:
--

[~praste] I've attached a sample program which computes versionsHash for leader 
and replica using the above example

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9506) cache IndexFingerprint for each segment


 [ 
https://issues.apache.org/jira/browse/SOLR-9506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9506:
-
Attachment: SOLR-9506_POC.patch

> cache IndexFingerprint for each segment
> ---
>
> Key: SOLR-9506
> URL: https://issues.apache.org/jira/browse/SOLR-9506
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
> Attachments: SOLR-9506_POC.patch
>
>
> The IndexFingerprint is cached per index searcher. it is quite useless during 
> high throughput indexing. If the fingerprint is cached per segment it will 
> make it vastly more efficient to compute the fingerprint



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9567) factor out ReRankCollector from ReRankQParserPlugin


 [ 
https://issues.apache.org/jira/browse/SOLR-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke updated SOLR-9567:
--
Attachment: SOLR-9567.patch

oops, unintentionally uploaded only half the patch, here's all of it.

> factor out ReRankCollector from ReRankQParserPlugin
> ---
>
> Key: SOLR-9567
> URL: https://issues.apache.org/jira/browse/SOLR-9567
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9567.patch, SOLR-9567.patch
>
>
> Factor out the collector used by ReRankQParserPlugin so that similar 
> reranking queries (e.g. SOLR-8542) can use the same collector.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9543) reduce code duplication in ReRankQParserPlugin.ReRankCollector.topDocs


 [ 
https://issues.apache.org/jira/browse/SOLR-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke resolved SOLR-9543.
---
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.x

Thanks for the reviews.

> reduce code duplication in ReRankQParserPlugin.ReRankCollector.topDocs
> --
>
> Key: SOLR-9543
> URL: https://issues.apache.org/jira/browse/SOLR-9543
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Fix For: 6.x, master (7.0)
>
> Attachments: SOLR-9543-reduce-part1.patch, 
> SOLR-9543-reduce-part2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9567) factor out ReRankCollector from ReRankQParserPlugin


 [ 
https://issues.apache.org/jira/browse/SOLR-9567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Christine Poerschke updated SOLR-9567:
--
Attachment: SOLR-9567.patch

> factor out ReRankCollector from ReRankQParserPlugin
> ---
>
> Key: SOLR-9567
> URL: https://issues.apache.org/jira/browse/SOLR-9567
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9567.patch
>
>
> Factor out the collector used by ReRankQParserPlugin so that similar 
> reranking queries (e.g. SOLR-8542) can use the same collector.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9567) factor out ReRankCollector from ReRankQParserPlugin

Christine Poerschke created SOLR-9567:
-

 Summary: factor out ReRankCollector from ReRankQParserPlugin
 Key: SOLR-9567
 URL: https://issues.apache.org/jira/browse/SOLR-9567
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Christine Poerschke
Assignee: Christine Poerschke
Priority: Minor


Factor out the collector used by ReRankQParserPlugin so that similar reranking 
queries (e.g. SOLR-8542) can use the same collector.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-6677) Reduce logging during startup and shutdown

2016-09-26 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-6677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-6677:
--
Attachment: SOLR-6677-part3.patch

New patch with some extra INFO->DEBUG conversions related to startup. One of 
the changes is to log this line only when there is actually a change in number 
of nodes:
{noformat}
o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
{noformat}

Do you see a need to log it even if there is no change?

> Reduce logging during startup and shutdown
> --
>
> Key: SOLR-6677
> URL: https://issues.apache.org/jira/browse/SOLR-6677
> Project: Solr
>  Issue Type: Bug
>Reporter: Noble Paul
>Assignee: Jan Høydahl
> Attachments: SOLR-6677-part-2.patch, SOLR-6677-part3.patch, 
> SOLR-6677.patch, SOLR-6677.patch
>
>
> most of what is printed is neither helpful nor useful. It's just noise



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9543) reduce code duplication in ReRankQParserPlugin.ReRankCollector.topDocs


[ 
https://issues.apache.org/jira/browse/SOLR-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523130#comment-15523130
 ] 

ASF subversion and git services commented on SOLR-9543:
---

Commit 4534c7dd1a4e6c9dcc7182eb7d5efc6250135384 in lucene-solr's branch 
refs/heads/branch_6x from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4534c7d ]

SOLR-9543: reduce code duplication in 
ReRankQParserPlugin.ReRankCollector.topDocs (part 2 of 2)


> reduce code duplication in ReRankQParserPlugin.ReRankCollector.topDocs
> --
>
> Key: SOLR-9543
> URL: https://issues.apache.org/jira/browse/SOLR-9543
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9543-reduce-part1.patch, 
> SOLR-9543-reduce-part2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Linux (32bit/jdk1.8.0_102) - Build # 1807 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1807/
Java: 32bit/jdk1.8.0_102 -client -XX:+UseSerialGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CleanupOldIndexTest

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([F645C96A274416ED]:0)


FAILED:  org.apache.solr.cloud.CleanupOldIndexTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([F645C96A274416ED]:0)




Build Log:
[...truncated 12794 lines...]
   [junit4] Suite: org.apache.solr.cloud.CleanupOldIndexTest
   [junit4]   2> Creating dataDir: 
/home/jenkins/workspace/Lucene-Solr-6.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CleanupOldIndexTest_F645C96A274416ED-001/init-core-data-001
   [junit4]   2> 1447903 INFO  
(SUITE-CleanupOldIndexTest-seed#[F645C96A274416ED]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: 
@org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1447905 INFO  
(SUITE-CleanupOldIndexTest-seed#[F645C96A274416ED]-worker) [] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1447905 INFO  (Thread-2833) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1447906 INFO  (Thread-2833) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1448006 INFO  
(SUITE-CleanupOldIndexTest-seed#[F645C96A274416ED]-worker) [] 
o.a.s.c.ZkTestServer start zk server on port:46267
   [junit4]   2> 1448013 INFO  
(SUITE-CleanupOldIndexTest-seed#[F645C96A274416ED]-worker) [] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1448026 INFO  (jetty-launcher-2061-thread-2) [] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1448026 INFO  (jetty-launcher-2061-thread-1) [] 
o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1448027 INFO  (jetty-launcher-2061-thread-1) [] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@d2b776{/solr,null,AVAILABLE}
   [junit4]   2> 1448027 INFO  (jetty-launcher-2061-thread-2) [] 
o.e.j.s.h.ContextHandler Started 
o.e.j.s.ServletContextHandler@25efe{/solr,null,AVAILABLE}
   [junit4]   2> 1448031 INFO  (jetty-launcher-2061-thread-1) [] 
o.e.j.s.ServerConnector Started ServerConnector@100c91{SSL,[ssl, 
http/1.1]}{127.0.0.1:39209}
   [junit4]   2> 1448031 INFO  (jetty-launcher-2061-thread-2) [] 
o.e.j.s.ServerConnector Started ServerConnector@19c9dfb{SSL,[ssl, 
http/1.1]}{127.0.0.1:41375}
   [junit4]   2> 1448031 INFO  (jetty-launcher-2061-thread-1) [] 
o.e.j.s.Server Started @1449499ms
   [junit4]   2> 1448032 INFO  (jetty-launcher-2061-thread-2) [] 
o.e.j.s.Server Started @1449500ms
   [junit4]   2> 1448032 INFO  (jetty-launcher-2061-thread-1) [] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=39209}
   [junit4]   2> 1448032 INFO  (jetty-launcher-2061-thread-2) [] 
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, 
hostPort=41375}
   [junit4]   2> 1448042 INFO  (jetty-launcher-2061-thread-2) [] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1448042 INFO  (jetty-launcher-2061-thread-1) [] 
o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1448042 INFO  (jetty-launcher-2061-thread-1) [] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1448042 INFO  (jetty-launcher-2061-thread-2) [] 
o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1448072 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [] 
o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from 
client sessionid 0x157664274a30002, likely client has closed socket
   [junit4]   2>at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2>at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2>at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1448074 INFO  (jetty-launcher-2061-thread-1) [] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=34=45000=true
   [junit4]   2> 1448074 INFO  (jetty-launcher-2061-thread-2) [] 
o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
socketTimeout=34=45000=true
   [junit4]   2> 1448075 INFO  (jetty-launcher-2061-thread-1) [] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46267/solr
   [junit4]   2> 1448075 INFO  (jetty-launcher-2061-thread-2) [] 
o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46267/solr
   [junit4]   2> 1448076

[jira] [Updated] (LUCENE-5317) Concordance/Key Word In Context (KWIC) capability


 [ 
https://issues.apache.org/jira/browse/LUCENE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Allison updated LUCENE-5317:

Summary: Concordance/Key Word In Context (KWIC) capability  (was: 
Concordance capability)

> Concordance/Key Word In Context (KWIC) capability
> -
>
> Key: LUCENE-5317
> URL: https://issues.apache.org/jira/browse/LUCENE-5317
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: core/search
>Affects Versions: 4.5
>Reporter: Tim Allison
>  Labels: patch
> Attachments: LUCENE-5317.patch, LUCENE-5317.patch, 
> concordance_v1.patch.gz, lucene5317v1.patch, lucene5317v2.patch
>
>
> This patch enables a Lucene-powered concordance search capability.
> Concordances are extremely useful for linguists, lawyers and other analysts 
> performing analytic search vs. traditional snippeting/document retrieval 
> tasks.  By "analytic search," I mean that the user wants to browse every time 
> a term appears (or at least the topn)  in a subset of documents and see the 
> words before and after.  
> Concordance technology is far simpler and less interesting than IR relevance 
> models/methods, but it can be extremely useful for some use cases.
> Traditional concordance sort orders are available (sort on words before the 
> target, words after, target then words before and target then words after).
> Under the hood, this is running SpanQuery's getSpans() and reanalyzing to 
> obtain character offsets.  There is plenty of room for optimizations and 
> refactoring.
> Many thanks to my colleague, Jason Robinson, for input on the design of this 
> patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 868 - Failure!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/868/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.handler.component.StatsComponentTest.testPercentiles

Error Message:
Unable to parse stats.field local params: percentiles==stat_f due to: empty 
String

Stack Trace:
org.apache.solr.common.SolrException: Unable to parse stats.field local params: 
percentiles==stat_f due to: empty String
at 
__randomizedtesting.SeedInfo.seed([1D82EE3F298AC2F7:29454E0B6FEE4BC3]:0)
at 
org.apache.solr.handler.component.StatsField$Stat$1.parseParams(StatsField.java:110)
at 
org.apache.solr.handler.component.StatsField.populateStatsSets(StatsField.java:533)
at 
org.apache.solr.handler.component.StatsField.(StatsField.java:288)
at 
org.apache.solr.handler.component.StatsInfo.(StatsComponent.java:188)
at 
org.apache.solr.handler.component.StatsComponent.prepare(StatsComponent.java:47)
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:268)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2106)
at 
org.apache.solr.util.TestHarness.queryAndResponse(TestHarness.java:329)
at 
org.apache.solr.handler.component.StatsComponentTest.testPercentiles(StatsComponentTest.java:1925)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at

[jira] [Comment Edited] (LUCENE-5317) Concordance capability

[
https://issues.apache.org/jira/browse/LUCENE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523067#comment-15523067
]

Tim Allison edited comment on LUCENE-5317 at 9/26/16 1:40 PM:
--

I received a personal email asking for some more background on this capability.
Here goes (apologies for some repetition with the issue description)...

For an example of concordance output, see these
[slides|https://github.com/tballison/share/blob/master/slides/TextProcessingAndAdvancedSearch_tallison_MITRE_201510_final_abbrev.pdf].
Slides 23 and 24 for LUCENE-5317 and slides 25-28 for LUCENE-5318.

The notion is that you present every time the term appears in the central
column with {{x}} number of words to the left and right. The user can sort on
words before the target term to see what modifies it, or the user can sort on
words after the target term to see what it modifies, or the user can sort on
order of appearance within the documents to effectively read everything in
their docs that matters to them.

By {{target term}}, of course, I mean any term/phrase that can be represented
by a SpanQuery.

This kind of view of the data is extremely helpful to linguists and
philologists to understand how words are being used. It also has practical
applications for anyone doing "analytic" search, that is, they want to see
every time a term/phrase appears -- lawyers, patent examiners, etc.

This view of the data is fundamentally different from snippets, which typically
show the three or so best chunks where the search terms appear, and they're
typically ordered _per document_. Snippets allow the user to determine if a
document is relevant, then the user has to open the document. Snippets are
great if users are seeking the best document to answer their information need.

For "analytic searchers", however, with concordance results, the user can be
saved the step of having to open the document; they can see _every time_ their
term/phrase appears. Also, for "analytic searchers", if their documents are
lengthy, the concordance allows them to see the potentially hundreds of times
that their term/phrase appears in each document instead of the three or so
snippets they might see with traditional search engines.

"But you can increase the number of snippets to whatever you want..." Yes, you
can, but the layout of the concordance allows you to see patterns across
documents very easily. Again, the results are sorted by words to the left or
right, not by which document the target appeared in.

This [link|https://wmtang.org/corpus-linguistics/corpus-linguistics] shows some
output from a concordancer (AntConc). Wikipedia's best description is under
key word in context ([KWIC|https://en.wikipedia.org/wiki/Key_Word_in_Context]).
If you're into tree-ware,
[Oakes|https://global.oup.com/academic/product/statistics-for-corpus-linguistics-9780748608171?cc=us=en;]
has a great introduction to concordances among many other useful topics!

was (Author: talli...@mitre.org):
I received a personal email asking for some more background on this capability.
Here goes (apologies for some repetition with the issue description)...

By {{target term}}, of course, I mean any term/phrase that can be represented
by a SpanQuery.

This view of the data is fundamentally different from snippets, which typically
show the three or so best chunks where the search terms appear. Snippets allow
the user to determine if a document is relevant, then the user has to open the
document. Snippets are great if the user is seeking the best document to
answer the information need. For "analytic searchers", however, with
concordance results, the user can be saved the step of having to open the
document; they can see _every time_ their term/phrase appears. Also, for
"analytic searchers", if their documents are lengthy, the concordance allows
them to see the potentially hundreds of

[jira] [Comment Edited] (LUCENE-5317) Concordance capability

[
https://issues.apache.org/jira/browse/LUCENE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523067#comment-15523067
]

Tim Allison edited comment on LUCENE-5317 at 9/26/16 1:38 PM:
--

I received a personal email asking for some more background on this capability.
Here goes (apologies for some repetition with the issue description)...

By {{target term}}, of course, I mean any term/phrase that can be represented
by a SpanQuery.

was (Author: talli...@mitre.org):
I received a personal email asking for some more background on this capability.
Here goes (apologies for some repetition with the issue description)...

By {{target term}}, of course, I mean any term/phrase that can be represented
by a SpanQuery.

[jira] [Commented] (LUCENE-5317) Concordance capability

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523067#comment-15523067
 ] 

Tim Allison commented on LUCENE-5317:
-

I received a personal email asking for some more background on this capability. 
 Here goes (apologies for some repetition with the issue description)...

For an example of concordance output, see these 
[slides|https://github.com/tballison/share/blob/master/slides/TextProcessingAndAdvancedSearch_tallison_MITRE_201510_final_abbrev.pdf].
  Slides 23 and 24 for LUCENE-5317 and slides 25-28 for LUCENE-5318.

The notion is that you present every time the term appears in the central 
column with {{x}} number of words to the left and right.  The user can sort on 
words before the target term to see what modifies it, or the user can sort on 
words after the target term to see what it modifies, or the user can sort on 
order of appearance.

 By {{target term}}, of course, I mean any term/phrase that can be represented 
by a SpanQuery.

This kind of view of the data is extremely helpful to linguists and 
philologists to understand how words are being used.  It also has practical 
applications for anyone doing "analytic" search, that is, they want to see 
every time a term/phrase appears -- lawyers, patent examiners, etc.

This view of the data is fundamentally different from snippets, which typically 
show the three or so best chunks where the search terms appear.  Snippets allow 
the user to determine if a document is relevant, then the user has to open the 
document.  Snippets are great if the user is seeking the best document to 
answer the information need.  For "analytic searchers", however, with 
concordance results, the user can be saved the step of having to open the 
document; they can see _every time_ their term/phrase appears.

This [link|https://wmtang.org/corpus-linguistics/corpus-linguistics] shows some 
output from a concordancer (AntConc).  Wikipedia's best description is under 
key word in context ([KWIC|https://en.wikipedia.org/wiki/Key_Word_in_Context]). 
If you're into tree-ware, 
[Oakes|https://global.oup.com/academic/product/statistics-for-corpus-linguistics-9780748608171?cc=us=en;]
 has a great introduction to concordances among many other useful topics!

> Concordance capability
> --
>
> Key: LUCENE-5317
> URL: https://issues.apache.org/jira/browse/LUCENE-5317
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: core/search
>Affects Versions: 4.5
>Reporter: Tim Allison
>  Labels: patch
> Attachments: LUCENE-5317.patch, LUCENE-5317.patch, 
> concordance_v1.patch.gz, lucene5317v1.patch, lucene5317v2.patch
>
>
> This patch enables a Lucene-powered concordance search capability.
> Concordances are extremely useful for linguists, lawyers and other analysts 
> performing analytic search vs. traditional snippeting/document retrieval 
> tasks.  By "analytic search," I mean that the user wants to browse every time 
> a term appears (or at least the topn)  in a subset of documents and see the 
> words before and after.  
> Concordance technology is far simpler and less interesting than IR relevance 
> models/methods, but it can be extremely useful for some use cases.
> Traditional concordance sort orders are available (sort on words before the 
> target, words after, target then words before and target then words after).
> Under the hood, this is running SpanQuery's getSpans() and reanalyzing to 
> obtain character offsets.  There is plenty of room for optimizations and 
> refactoring.
> Many thanks to my colleague, Jason Robinson, for input on the design of this 
> patch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9543) reduce code duplication in ReRankQParserPlugin.ReRankCollector.topDocs


[ 
https://issues.apache.org/jira/browse/SOLR-9543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523058#comment-15523058
 ] 

ASF subversion and git services commented on SOLR-9543:
---

Commit a8f4ef02edf83043fcf04db2a8c42832892aaa78 in lucene-solr's branch 
refs/heads/master from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a8f4ef0 ]

SOLR-9543: reduce code duplication in 
ReRankQParserPlugin.ReRankCollector.topDocs (part 2 of 2)


> reduce code duplication in ReRankQParserPlugin.ReRankCollector.topDocs
> --
>
> Key: SOLR-9543
> URL: https://issues.apache.org/jira/browse/SOLR-9543
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9543-reduce-part1.patch, 
> SOLR-9543-reduce-part2.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9538) Relocate {JSON,Smile}WriterTest and TestBinaryResponseWriter to org.apache.solr.response

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523059#comment-15523059
 ] 

ASF subversion and git services commented on SOLR-9538:
---

Commit 7a6567e4a66451a2b4b7cba5e6b8373ad200ea8e in lucene-solr's branch 
refs/heads/master from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7a6567e ]

TestSmileRequest to import response.SmileWriterTest instead of (deprecated) 
request.SmileWriterTest predecessor. Then remove (three) deprecated/relocated 
tests. (Follow-ons from SOLR-9538 itself.)


> Relocate {JSON,Smile}WriterTest and TestBinaryResponseWriter to 
> org.apache.solr.response
> 
>
> Key: SOLR-9538
> URL: https://issues.apache.org/jira/browse/SOLR-9538
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Jonny Marks
>Assignee: Christine Poerschke
>Priority: Minor
> Fix For: 6.x, master (7.0)
>
> Attachments: SOLR-9538.patch
>
>
> JSONWriterTest, SmileWriterTest and TestBinaryResponseWriter are currently in 
> the org.apache.solr.request package but the classes they test 
> (JSONResponseWriter, SmileWriter, BinaryResponseWriter) are in the 
> org.apache.solr.response package. 
> This patch moves the tests to the response package.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation


[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523032#comment-15523032
 ] 

David Smiley commented on LUCENE-7465:
--

bq. I agree this would be nice, but my worry about taking that approach is 
which one we default to? Maybe if we make it a required param? But then how to 
implement back compat?

Not a required param; default to current java regexp impl; no back-compat 
worry.  It's the most flexible and so I think makes the best default.

bq. I think such auto-detection (looking at the user's pattern and picking the 
engine) is dangerous.

Ok.

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation


[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523014#comment-15523014
 ] 

Michael McCandless commented on LUCENE-7465:


Maybe you could share just the regexp :)

But, if you do repeat the test w/ Lucene, then try to use this patch if 
possible (just the {{XXXRunAutomaton}} changes), because it optimizes for code 
points < 256.  Or, if your data is all non-ascii, then don't bother using this 
patch :)

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation

2016-09-26 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15523000#comment-15523000
 ] 

Dawid Weiss commented on LUCENE-7465:
-

I'll try to repeat the experiment with Lucene's regexp when I have a spare 
moment. The benchmarks (or rather: data) cannot be shared, unfortunately, but 
it involved regexps with hundreds of alternatives and globs. Definitely not 
something that people can edit by hand.

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Issue Comment Deleted] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation

2016-09-26 Thread Dawid Weiss (JIRA)


 [ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Weiss updated LUCENE-7465:

Comment: was deleted

(was: I'll try to repeat the experiment with Lucene's regexp when I have a 
spare moment. The benchmarks (or rather: data) cannot be shared, unfortunately, 
but it involved regexps with hundreds of alternatives and globs. Definitely not 
something that people can edit by hand.)

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation

2016-09-26 Thread Dawid Weiss (JIRA)


[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522997#comment-15522997
 ] 

Dawid Weiss commented on LUCENE-7465:
-

I'll try to repeat the experiment with Lucene's regexp when I have a spare 
moment. The benchmarks (or rather: data) cannot be shared, unfortunately, but 
it involved regexps with hundreds of alternatives and globs. Definitely not 
something that people can edit by hand.

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation


[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522992#comment-15522992
 ] 

Michael McCandless commented on LUCENE-7465:


bq. Instead of adding another factory, what about adding an implementation hint 
parameter to PatternTokenizerFactory? e.g. method="lucene" or method="simple". 

I agree this would be nice, but my worry about taking that approach is which 
one we default to?  Maybe if we make it a required param?  But then how to 
implement back compat?

bq. Then I wonder if we might detect circumstances in which this new 
implementation is preferable? 

I think such auto-detection (looking at the user's pattern and picking the 
engine) is a dangerous.  Maybe a user is debugging a tricky regexp, and adding 
one new character causes us to pick a different engine or something.  I think 
for now it should be a conscious choice?

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9562) Minimize queried collections for time series alias


[ 
https://issues.apache.org/jira/browse/SOLR-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522986#comment-15522986
 ] 

David Smiley commented on SOLR-9562:


Thanks for contributing. I'm missing something... why is this metadata on a 
Collection Alias?  What do Collection Aliases logically have to do with this 
feature?  Wouldn't associating with the Shard be better, assuming a design in 
which there is one Collection & manual sharding?

BTW I consider SimpleDateFormat and friends a dead API with the advent of Java 
8's new time API: 
https://docs.oracle.com/javase/8/docs/api/java/time/package-summary.html

> Minimize queried collections for time series alias
> --
>
> Key: SOLR-9562
> URL: https://issues.apache.org/jira/browse/SOLR-9562
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Eungsop Yoo
>Priority: Minor
> Attachments: SOLR-9562.patch
>
>
> For indexing time series data(such as large log data), we can create a new 
> collection regularly(hourly, daily, etc.) with a write alias and create a 
> read alias for all of those collections. But all of the collections of the 
> read alias are queried even if we search over very narrow time window. In 
> this case, the docs to be queried may be stored in very small portion of 
> collections. So we don't need to do that.
> I suggest this patch for read alias to minimize queried collections. Three 
> parameters for CREATEALIAS action are added.
> || Key || Type || Required || Default || Description ||
> | timeField | string | No | | The time field name for time series data. It 
> should be date type. |
> | dateTimeFormat | string | No | | The format of timestamp for collection 
> creation. Every collection should has a suffix(start with "_") with this 
> format. 
> Ex. dateTimeFormat: MMdd, collectionName: col_20160927
> See 
> [SimpleDateFormat|https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html].
>  |
> | timeZone | string | No | | The time zone information for dateTimeFormat 
> parameter.
> Ex. GMT+9. 
> See 
> [SimpleDateFormat|https://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html].
>  |
> And then when we query with filter query like this "timeField:\[fromTime TO 
> toTime\]", only the collections have the docs for a given time range will be 
> queried.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9566) Can we avoid doing recovery when collections are first created?

Alan Woodward created SOLR-9566:
---

 Summary: Can we avoid doing recovery when collections are first 
created?
 Key: SOLR-9566
 URL: https://issues.apache.org/jira/browse/SOLR-9566
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Alan Woodward


When a core starts up as part of a collection, and it's not a shard leader, it 
goes into recovery, part of which involves a 7 second wait (set in SOLR-7141) 
to ensure that updates being sent from the leader don't get missed in between 
buffering and replication.

This has the unfortunate side-effect of adding a 7-second pause to collection 
creation whenever the replication factor is 2 or more, which slows down tests 
massively - for example, DeleteReplicaTest takes about 54 seconds to execute on 
my machine, 28 seconds of which is just pauses - over 50% of execution time.  
It's not actually possible to add documents to a collection before the creation 
request has returned, so the recovery stage here isn't strictly speaking 
necessary.  

I think we could try adding a parameter to a CoreAdmin create request that says 
the core is being created as part of a new collection, so it doesn't need to 
try and recover from it's leader when it starts up.  Does this sound sensible, 
or am I missing something here?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation


[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522972#comment-15522972
 ] 

Michael McCandless commented on LUCENE-7465:


[~dawid.weiss] is this a benchmark I could try to run?  My regexp was 
admittedly trivial so it would be nice to have a beefier real-world regexp to 
play with ;)

The bench is also trivial (I pushed it to luceneutil).

When you tested dk.brics, did you call the {{RunAutomaton.setAlphabet}}?  This 
should be a biggish speedup, especially if your regexp has many unique 
character start/end ranges.  In Lucene's fork of dk.briks we automatically do 
that in the utf8 case, and with this patch, also for the first 256 unicode 
characters in the full character case.

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9560) Solr should check max open files and other ulimits and refuse to start if they are set too low


[ 
https://issues.apache.org/jira/browse/SOLR-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522957#comment-15522957
 ] 

David Smiley commented on SOLR-9560:


Good idea Noble; I like that better.

> Solr should check max open files and other ulimits and refuse to start if 
> they are set too low
> --
>
> Key: SOLR-9560
> URL: https://issues.apache.org/jira/browse/SOLR-9560
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Shalin Shekhar Mangar
> Fix For: 6.3, master (7.0)
>
>
> Solr should check max open files and other ulimits and refuse to start if 
> they are set too low. Specifically:
> # max open files should be at least 32768
> # max memory size and virtual memory should both be unlimited



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7465) Add a PatternTokenizer that uses Lucene's RegExp implementation


[ 
https://issues.apache.org/jira/browse/LUCENE-7465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522947#comment-15522947
 ] 

David Smiley commented on LUCENE-7465:
--

Instead of adding another factory, what about adding an implementation hint 
parameter to PatternTokenizerFactory?  e.g. {{method="lucene"}} or 
{{method="simple"}}.  Then I wonder if we might detect circumstances in which 
this new implementation is preferable?  The motivation for one factory is 
similar to the WhitespaceTokenizer "rule" param.  People know they have a 
regexp and want to tokenize on it... they could easily overlook a different 
factory than the one that is already named well and may appear in 
blogs/docs/examples.

> Add a PatternTokenizer that uses Lucene's RegExp implementation
> ---
>
> Key: LUCENE-7465
> URL: https://issues.apache.org/jira/browse/LUCENE-7465
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Michael McCandless
>Assignee: Michael McCandless
> Fix For: master (7.0), 6.3
>
> Attachments: LUCENE-7465.patch, LUCENE-7465.patch
>
>
> I think there are some nice benefits to a version of PatternTokenizer that 
> uses Lucene's RegExp impl instead of the JDK's:
>   * Lucene's RegExp is compiled to a DFA up front, so if a "too hard" RegExp 
> is attempted the user discovers it up front instead of later on when a 
> "lucky" document arrives
>   * It processes the incoming characters as a stream, only pulling 128 
> characters at a time, vs the existing {{PatternTokenizer}} which currently 
> reads the entire string up front (this has caused heap problems in the past)
>   * It should be fast.
> I named it {{SimplePatternTokenizer}}, and it still needs a factory and 
> improved tests, but I think it's otherwise close.
> It currently does not take a {{group}} parameter because Lucene's RegExps 
> don't yet implement sub group capture.  I think we could add that at some 
> point, but it's a bit tricky.
> This doesn't even have group=-1 support (like String.split) ... I think if we 
> did that we should maybe name it differently 
> ({{SimplePatternSplitTokenizer}}?).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9518) Kerberos Delegation Tokens doesn't work without a chrooted ZK


 [ 
https://issues.apache.org/jira/browse/SOLR-9518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-9518:
---
Attachment: SOLR-9518.patch

> Kerberos Delegation Tokens doesn't work without a chrooted ZK
> -
>
> Key: SOLR-9518
> URL: https://issues.apache.org/jira/browse/SOLR-9518
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
> Attachments: SOLR-9518.patch, SOLR-9518.patch, SOLR-9518.patch
>
>
> Starting up Solr 6.2.0 (with delegation tokens enabled) that doesn't have a 
> chrooted ZK, I see the following in the startup logs:
> {code}
> 2016-09-15 07:08:22.453 ERROR (main) [   ] o.a.s.s.SolrDispatchFilter Could 
> not start Solr. Check solr/home property and the logs
> 2016-09-15 07:08:22.477 ERROR (main) [   ] o.a.s.c.SolrCore 
> null:java.lang.StringIndexOutOfBoundsException: String index out of range: -1
> at java.lang.String.substring(String.java:1927)
> at 
> org.apache.solr.security.KerberosPlugin.init(KerberosPlugin.java:138)
> at 
> org.apache.solr.core.CoreContainer.initializeAuthenticationPlugin(CoreContainer.java:316)
> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:442)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:158)
> at 
> org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:134)
> at 
> org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
> at 
> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:856)
> at 
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:348)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1379)
> at 
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1341)
> at 
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772)
> at 
> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
> at 
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:517)
> at 
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at 
> org.eclipse.jetty.deploy.bindings.StandardStarter.processBinding(StandardStarter.java:41)
> at 
> org.eclipse.jetty.deploy.AppLifeCycle.runBindings(AppLifeCycle.java:188)
> at 
> org.eclipse.jetty.deploy.DeploymentManager.requestAppGoal(DeploymentManager.java:499)
> at 
> org.eclipse.jetty.deploy.DeploymentManager.addApp(DeploymentManager.java:147)
> at 
> org.eclipse.jetty.deploy.providers.ScanningAppProvider.fileAdded(ScanningAppProvider.java:180)
> at 
> org.eclipse.jetty.deploy.providers.WebAppProvider.fileAdded(WebAppProvider.java:458)
> at 
> org.eclipse.jetty.deploy.providers.ScanningAppProvider$1.fileAdded(ScanningAppProvider.java:64)
> at org.eclipse.jetty.util.Scanner.reportAddition(Scanner.java:610)
> at org.eclipse.jetty.util.Scanner.reportDifferences(Scanner.java:529)
> {code}
> To me, it seems that adding a check for the presence of a chrooted ZK, and, 
> calculating the relative ZK path only if it exists should suffice. I'll add a 
> patch for this shortly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9520) Kerberos delegation support with CloudSolrClient

2016-09-26 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-9520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ishan Chattopadhyaya updated SOLR-9520:
---
Attachment: SOLR-9520.patch

Attached a patch that adds support for delegation tokens for CSC. As of this 
patch, if a delegation token has changed, a new CSC instance must be created. 
[~gchanan], can you please review?

> Kerberos delegation support with CloudSolrClient
> 
>
> Key: SOLR-9520
> URL: https://issues.apache.org/jira/browse/SOLR-9520
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Ishan Chattopadhyaya
> Attachments: SOLR-9520.patch
>
>
> HSC support is available, but we need to add support to CSC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9319) DELETEREPLICA should be able to accept just count and remove replicas intelligenty


[ 
https://issues.apache.org/jira/browse/SOLR-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522877#comment-15522877
 ] 

ASF subversion and git services commented on SOLR-9319:
---

Commit 11a98a89fd63271f969ae1072159c410ad6417cc in lucene-solr's branch 
refs/heads/branch_6x from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=11a98a8 ]

SOLR-9132: Cut over DeleteReplica tests

Also fixes some bugs in CollectionAdminRequest.DeleteReplica from SOLR-9319


> DELETEREPLICA should be able to accept  just count and remove replicas 
> intelligenty
> ---
>
> Key: SOLR-9319
> URL: https://issues.apache.org/jira/browse/SOLR-9319
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
> Attachments: DeleteReplicaPatch.jpg, Delete_Replica_count_1.jpg, 
> Delete_Replica_invalid.jpg, Delete_Replica_with_only_1replica.jpg, 
> Delete_replica_count2.jpg, Delte_replica_after.jpg, Delte_replica_before.jpg, 
> Old_deletereplica_api_works.jpg, SOLR-9310.patch, SOLR-9319.patch, 
> SOLR-9319.patch, SOLR-9319.patch, SOLR-9319.patch, Screen Shot 2016-08-26 at 
> 12.59.16 PM.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9132) Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522876#comment-15522876
 ] 

ASF subversion and git services commented on SOLR-9132:
---

Commit 11a98a89fd63271f969ae1072159c410ad6417cc in lucene-solr's branch 
refs/heads/branch_6x from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=11a98a8 ]

SOLR-9132: Cut over DeleteReplica tests

Also fixes some bugs in CollectionAdminRequest.DeleteReplica from SOLR-9319


> Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase
> -
>
> Key: SOLR-9132
> URL: https://issues.apache.org/jira/browse/SOLR-9132
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9132-deletereplicas.patch, SOLR-9132.patch
>
>
> We need to remove AbstractDistribZkTestCase if we want to move away from 
> legacy cloud configurations.  This issue is for migrating tests to 
> SolrCloudTestCase instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9319) DELETEREPLICA should be able to accept just count and remove replicas intelligenty

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522879#comment-15522879
 ] 

ASF subversion and git services commented on SOLR-9319:
---

Commit be4233cb567a8e65ef67d0fc1c8e47936e155a23 in lucene-solr's branch 
refs/heads/master from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=be4233c ]

SOLR-9132: Cut over DeleteReplica tests

Also fixes some bugs in CollectionAdminRequest.DeleteReplica from SOLR-9319


> DELETEREPLICA should be able to accept  just count and remove replicas 
> intelligenty
> ---
>
> Key: SOLR-9319
> URL: https://issues.apache.org/jira/browse/SOLR-9319
> Project: Solr
>  Issue Type: Sub-task
>  Components: SolrCloud
>Reporter: Noble Paul
> Fix For: 6.1
>
> Attachments: DeleteReplicaPatch.jpg, Delete_Replica_count_1.jpg, 
> Delete_Replica_invalid.jpg, Delete_Replica_with_only_1replica.jpg, 
> Delete_replica_count2.jpg, Delte_replica_after.jpg, Delte_replica_before.jpg, 
> Old_deletereplica_api_works.jpg, SOLR-9310.patch, SOLR-9319.patch, 
> SOLR-9319.patch, SOLR-9319.patch, SOLR-9319.patch, Screen Shot 2016-08-26 at 
> 12.59.16 PM.png
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9132) Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase

2016-09-26 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/SOLR-9132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522878#comment-15522878
 ] 

ASF subversion and git services commented on SOLR-9132:
---

Commit be4233cb567a8e65ef67d0fc1c8e47936e155a23 in lucene-solr's branch 
refs/heads/master from [~romseygeek]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=be4233c ]

SOLR-9132: Cut over DeleteReplica tests

Also fixes some bugs in CollectionAdminRequest.DeleteReplica from SOLR-9319


> Cut over AbstractDistribZkTestCase tests to SolrCloudTestCase
> -
>
> Key: SOLR-9132
> URL: https://issues.apache.org/jira/browse/SOLR-9132
> Project: Solr
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
> Attachments: SOLR-9132-deletereplicas.patch, SOLR-9132.patch
>
>
> We need to remove AbstractDistribZkTestCase if we want to move away from 
> legacy cloud configurations.  This issue is for migrating tests to 
> SolrCloudTestCase instead.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9565) Make every UpdateRequestProcessor available implicitly

2016-09-26 Thread ASF subversion and git services (JIRA)

Noble Paul created SOLR-9565:


 Summary: Make every UpdateRequestProcessor available implicitly
 Key: SOLR-9565
 URL: https://issues.apache.org/jira/browse/SOLR-9565
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Noble Paul


Now that we can 'construct' the URP chains through request parameters, we 
should make all the URPs available automatically. The next challenge is to make 
them read the configuration from request parameters as well

to access {{HTMLStripFieldUpdateProcessorFactory}} the parameter could be 
{{processor=HTMLStripField}} (The UpdateProcessorFactory part is automatically 
appended )

The next step is to make the URPs accept request parameters instead of just 
configuration parameters e.g: 
{{processor=HTMLStripField=}}







--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Solr configuration format fracturing

2016-09-26 Thread Noble Paul

We had a very large monolithic solrconfig.xml which users used to
(they still do) hand edit. This ws fine in standalone mode and in
solrcloud it's unwieldy. Imagine you making a single typo and blowing
up an entire sollection with hundreds of nodes. When we change a
single parameter, it resulted in reload of all these cores which would
lead to performance degradation across the cluster. The other problem
is that very little of solrconfig.xml was really 'user  code' it was
mostly system code. It was 95% dictated by solr (and never changed).
So these solutions were important as we moved from the simple
standlone server to Solr cloud.

Now, the question is how do we mitigate this problem and educate users
with the new 'way of life'

We must remove more and more stuff from solrconfig.xml. The strategy
is as follows. Every component should be available implicitly and it
should be accessible through request parameter flags. The rest should
move to the configoverlay.json.


The bigger issue is , how do we make users comfortable with it. It
requires a lot more documentation and many more user-friendly APIs



On Mon, Sep 26, 2016 at 1:26 PM, Alexandre Rafalovitch
 wrote:
> I think each individual change mostly made sense. What I think becomes
> a problem is expecting the user to understand the end result when they
> are not aware of what different things have and where they live.
>
> solrconfig.xml is a very large beast already. But, until recently,
> there was a sense that one could read and/or grep for whatever one did
> not understand (request handler name, parameter name, etc) in one of
> those two files (solrconfig.xml, schema.xml) and have the answer.
>
> Now, we have 3 files and two API calls for solrconfig.xml, two names
> for schema (yes, schema.xml given all _other_ tutorials on the web)
> and whatever the magic _rest_managed.json file is for meta-managing
> managed components. Plus, the implicit handlers, and now implicit
> initParam names for implicit handlers :-) And, still dark magic to me,
> the order of invocation between parameter information passed through
> the request handler, initParams, params.json, request params directly,
> request params mentioning params groups, etc (ok, I am rambling here).
>
> I think magic is not bad (that's a recent change of mind frankly), if
> that significantly simplifies workflow and still allows for easy
> debugging. But I am not sure the current state - taken all together -
> achieves either of two purposes.
>
> I think looking at all these different thinks together, understanding
> the use cases and perhaps slimming down to a unified forward-back path
> would be best. So, a single format (JSON if it works,
> JSON+comments/quotes that Solr actually supports, or whatever). But
> then the inputs and outputs should be consistent so that the same
> tooling could be used for everything.
>
> Too much to dream about?
>
> Regards,
>Alex.
> P.s. Oh, and minimum solrconfig.xml requires luceneMatchVersion to be
> present. And possibly a /select handler. But yes, a valid core
> configuration can be very very small.
>
> 
> Newsletter and resources for Solr beginners and intermediates:
> http://www.solr-start.com/
>
>
> On 26 September 2016 at 14:10, Noble Paul  wrote:
>> As the main culprit here let me put forward my few cents
>>
>> * The intent is to go away from config files. Users should use APIs to
>> read and edit configurations. Editing configuration is both error
>> prone and old school
>> * I prefer to store everything is JSON. json is modern and simple.
>> * Even today it is totally possible to put an empty solrconfig.xml and
>> move everything to configoverlay.json. We may not need to wait for a
>> big release to do that
>> * schema is still in xml . there is significant effort involved in
>> changing it over
>> * There is a reason why the /config output has solrconfig.xml and
>> overlay.json merged because they don't make sense individually. params
>> are not relevant unless there is a reference to it in the config. If a
>> requesthandler refers to a paramset, we probably should show it inline
>> * Yes , you can't round-trip the output of config. I thought of adding
>> a wt which emits a "round-trippable" JSON (like  the SHOW CREATE TABLE
>> functionality in RDBMS)
>>
>> On Mon, Sep 26, 2016 at 5:35 AM, Alexandre Rafalovitch
>>  wrote:
>>> Did you know about configoverlay.json?
>>>
>>> +1 to the discussion.
>>>
>>> Additional fuel to the fire is that /config endpoint will return
>>> solrconfig.xml + overlay.json merged, but not params.json. Confusing.
>>>
>>> Additionally, /config output is JSON but not one that can round-trip AFAIK.
>>>
>>> Regards,
>>> Alex
>>>
>>>
>>> On 26 Sep 2016 12:42 AM, "Shawn Heisey"  wrote:

 There seems to be some fracturing in the format of various Solr
 configs.  Most of the config uses XML, but some new features in the last

Re: Solr configuration format fracturing

2016-09-26 Thread Noble Paul

I agree in principle that there should be a way to download,store and
restore configuration.
Isn't it better achieved through the configset API?

like

bin/solr save-config gettingstarted 

bin/solr restore-config gettingstarted.zip gettingstarted

The problem with saving  a single file is that most of them are
interdependent and saving a single file alone will break stuff.

Users still can download and save a single file from the
/admin/zookeeper path. I know it is not very friendly today but we can
make it better.


On Mon, Sep 26, 2016 at 1:19 PM, Jan Høydahl  wrote:
> I agree that configuring Solr through the APIs is a better option than
> uploading huge XMLs, but we should strive for copy/paste-ability wherever
> possible. Ideally it should be possible to store your config/schema
> information
> in JSON format in GIT and easily POST it after creating a new collection.
> The exact same JSON should also be possible to paste into the Admin UI
> in order to reproduce the same config/schema. And if you do a GET request
> to fetch config/schema, the resulting JSON should ideally be in a format
> that
> can be again checked into GIT and work when re-creating the same collection.
>
> Am I right that the bulk API style of embedding the verb into the data
> object
> itself, e.g. "add-field-type”, is the main source of the difference here, or
> is it more
> complex than that? Looks to me that the JSON object after “add-field-type”
> and
> “replace-field-type” is exactly the same as the one you get from a call to
> GET /solr/collection/schema/fieldtypes/name?omitHeader=true except from
> the latter being wrapped in a “fieldType” object.
>
> I’m not totally against the “add-field-type” bulk style API, but perhaps we
> should also have a GET/POST /solr/collection/schema endpoint which
> supports roundtrip of the whole schema in JSON format?
>
> --
> Jan Høydahl, search solution architect
> Cominvent AS - www.cominvent.com
>
> 26. sep. 2016 kl. 09.10 skrev Noble Paul :
>
> As the main culprit here let me put forward my few cents
>
> * The intent is to go away from config files. Users should use APIs to
> read and edit configurations. Editing configuration is both error
> prone and old school
> * I prefer to store everything is JSON. json is modern and simple.
> * Even today it is totally possible to put an empty solrconfig.xml and
> move everything to configoverlay.json. We may not need to wait for a
> big release to do that
> * schema is still in xml . there is significant effort involved in
> changing it over
> * There is a reason why the /config output has solrconfig.xml and
> overlay.json merged because they don't make sense individually. params
> are not relevant unless there is a reference to it in the config. If a
> requesthandler refers to a paramset, we probably should show it inline
> * Yes , you can't round-trip the output of config. I thought of adding
> a wt which emits a "round-trippable" JSON (like  the SHOW CREATE TABLE
> functionality in RDBMS)
>
> On Mon, Sep 26, 2016 at 5:35 AM, Alexandre Rafalovitch
>  wrote:
>
> Did you know about configoverlay.json?
>
> +1 to the discussion.
>
> Additional fuel to the fire is that /config endpoint will return
> solrconfig.xml + overlay.json merged, but not params.json. Confusing.
>
> Additionally, /config output is JSON but not one that can round-trip AFAIK.
>
> Regards,
>Alex
>
>
> On 26 Sep 2016 12:42 AM, "Shawn Heisey"  wrote:
>
>
> There seems to be some fracturing in the format of various Solr
> configs.  Most of the config uses XML, but some new features in the last
> few years are using JSON, particularly where SolrCloud and Zookeeper are
> concerned.  When notifications about SOLR-9557 came through, it revealed
> that there is a config file sitting next to solrconfig.xml named
> "params.json" that Solr will use.  I wasn't aware of this until reading
> that issue.
>
> This leads me to suggest something rather drastic for 7.0:  Consolidate
> all configuration formats and agree to consistent format usage unless
> there is another major discussion and agreement to change formats.
>
> I did consider starting this discussion in Jira, but it's fairly major,
> so the dev list seemed like the right place to start.
>
> Comments from some new users have come my way along the lines of "XML is
> so 90's ... get with the times!"  Image problems like that can be fatal
> to a software project, even if there's no technical problem.
>
> The likely winner in the format discussion is pure unmodified JSON, but
> I'm not going to make any assumptions.  SOLR-8029 has some format
> discussions that may be relevant here.
>
> IMHO, in order to make the idea successful, Solr 7.0 will need to
> automatically convert most configs on startup from the old format to the
> new format without user intervention.  If there's something that we find
> we can't convert automatically, that should result

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522787#comment-15522787
 ] 

ASF subversion and git services commented on SOLR-9557:
---

Commit 691472c0f0898ab99d16158ce5a6def60f754206 in lucene-solr's branch 
refs/heads/master from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=691472c ]

SOLR-9557: Every implicit requesthandler now has a default 'useParams' attribute


> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9557) Every implicit requesthandler should have a default useParams

2016-09-26 Thread ASF subversion and git services (JIRA)


 [ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul resolved SOLR-9557.
--
   Resolution: Fixed
Fix Version/s: master (7.0)
   6.3

> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522784#comment-15522784
 ] 

ASF subversion and git services commented on SOLR-9557:
---

Commit ecdfe97d34f6e808efc8e8efed6ce1306a4ee8e5 in lucene-solr's branch 
refs/heads/branch_6x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ecdfe97 ]

SOLR-9557: Every implicit requesthandler now has a default 'useParams' attribute


> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9557) Every implicit requesthandler should have a default useParams


[ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522780#comment-15522780
 ] 

Noble Paul commented on SOLR-9557:
--

it is done with an explicit attribute in the configuration. We could change it 
in all our solrconfig.xml files. But they are ugly

> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9557) Every implicit requesthandler should have a default useParams


 [ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9557:
-
Issue Type: Improvement  (was: Bug)

> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9557) Every implicit requesthandler should have a default useParams


 [ 
https://issues.apache.org/jira/browse/SOLR-9557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Noble Paul updated SOLR-9557:
-
Priority: Minor  (was: Major)

> Every implicit requesthandler should have a default useParams 
> --
>
> Key: SOLR-9557
> URL: https://issues.apache.org/jira/browse/SOLR-9557
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Minor
> Attachments: SOLR-9557.patch
>
>
> If a user needs to set any request parameters on an implicitly defined 
> requesthandler , he has to define an {{}} section in 
> solrconfig.xml.  We should define a default paramset with all implicit 
> handlers
> example : the {{/update/json/docs}} can have a default 
> {{useParams="_UPDATE_JSON_DOCS"}}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9560) Solr should check max open files and other ulimits and refuse to start if they are set too low


[ 
https://issues.apache.org/jira/browse/SOLR-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15522777#comment-15522777
 ] 

Noble Paul commented on SOLR-9560:
--

We should just show a splash screen when we start solr with suboptimal 
environment settings. If we fail to start this will be a bad experience for the 
casual user who downloads and plays with Solr on his laptop

{code}
***
*The Max fopen files is too low. set it bigger*
***
{code}

> Solr should check max open files and other ulimits and refuse to start if 
> they are set too low
> --
>
> Key: SOLR-9560
> URL: https://issues.apache.org/jira/browse/SOLR-9560
> Project: Solr
>  Issue Type: Wish
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Shalin Shekhar Mangar
> Fix For: 6.3, master (7.0)
>
>
> Solr should check max open files and other ulimits and refuse to start if 
> they are set too low. Specifically:
> # max open files should be at least 32768
> # max memory size and virtual memory should both be unlimited



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9548) solr.log should start with informative welcome message


 [ 
https://issues.apache.org/jira/browse/SOLR-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jan Høydahl updated SOLR-9548:
--
Attachment: SOLR-9548-detailversion.patch

So the attached patch SOLR-9548-detailversion.patch is the best I could do for 
now. At least now it is evident from the log file by what exact version this 
log was generated.

> solr.log should start with informative welcome message
> --
>
> Key: SOLR-9548
> URL: https://issues.apache.org/jira/browse/SOLR-9548
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Server
>Reporter: Jan Høydahl
>Assignee: Jan Høydahl
>  Labels: logging
> Fix For: 6.3, master (7.0)
>
> Attachments: SOLR-9548-detailversion.patch, SOLR-9548.patch
>
>
> When starting Solr, the first log line should be more informative, such as
> {code}
> Welcome to Apache Solr™ version 7.0.0, running in standalone mode on port 
> 8983 from folder /Users/janhoy/git/lucene-solr/solr
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9563) Collection creation can fail if an ADDREPLICA request arrives at a node before its local ZK state has updated with the new collection data


 [ 
https://issues.apache.org/jira/browse/SOLR-9563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated SOLR-9563:

Summary: Collection creation can fail if an ADDREPLICA request arrives at a 
node before its local ZK state has updated with the new collection data  (was: 
Collection creation can fail is an ADDREPLICA request arrives at a node before 
its local ZK state has updated with the new collection data)

> Collection creation can fail if an ADDREPLICA request arrives at a node 
> before its local ZK state has updated with the new collection data
> --
>
> Key: SOLR-9563
> URL: https://issues.apache.org/jira/browse/SOLR-9563
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
> Attachments: SOLR-9563.patch
>
>
> See https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1792/ for an 
> example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9563) Collection creation can fail is an ADDREPLICA request arrives at a node before its local ZK state has updated with the new collection data


 [ 
https://issues.apache.org/jira/browse/SOLR-9563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated SOLR-9563:

Attachment: SOLR-9563.patch

Patch, moving ZkController.checkStateInZk() to use 
zkStateReader.waitForState(), which should deal with this problem.

> Collection creation can fail is an ADDREPLICA request arrives at a node 
> before its local ZK state has updated with the new collection data
> --
>
> Key: SOLR-9563
> URL: https://issues.apache.org/jira/browse/SOLR-9563
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Alan Woodward
> Attachments: SOLR-9563.patch
>
>
> See https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/1792/ for an 
> example.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9563) Collection creation can fail is an ADDREPLICA request arrives at a node before its local ZK state has updated with the new collection data