[jira] [Created] (SOLR-9904) JSON API facet incorrect counts for specific values when exclusion applied

[Eduard Zlotnik (JIRA)]

Eduard Zlotnik created SOLR-9904:


 Summary: JSON API facet incorrect counts for specific values when 
exclusion applied
 Key: SOLR-9904
 URL: https://issues.apache.org/jira/browse/SOLR-9904
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Facet Module
Affects Versions: 6.3, 6.2.1
Reporter: Eduard Zlotnik






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9902) StandardDirectoryFactory should use Files API for it's move implementation.

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784562#comment-15784562
 ] 

David Smiley commented on SOLR-9902:


Why does your patch do this:
{code}FileSystems.getDefault().getPath(path1.toString(), fileName){code}
Instead of simply:
{code}path1.resolve(fileName){code}

> StandardDirectoryFactory should use Files API for it's move implementation.
> ---
>
> Key: SOLR-9902
> URL: https://issues.apache.org/jira/browse/SOLR-9902
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9902.patch
>
>
> It's done in a platform independent way as opposed to the old File API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-master - Build # 1573 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1573/

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
  at 
org.apache.solr.update.HdfsTransactionLog.(HdfsTransactionLog.java:130)  
at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)  at 
org.apache.solr.update.UpdateHandler.(UpdateHandler.java:137)  at 
org.apache.solr.update.UpdateHandler.(UpdateHandler.java:94)  at 
org.apache.solr.update.DirectUpdateHandler2.(DirectUpdateHandler2.java:102)
  at sun.reflect.GeneratedConstructorAccessor184.newInstance(Unknown Source)  
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)  at 
org.apache.solr.core.SolrCore.createInstance(SolrCore.java:747)  at 
org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:809)  at 
org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1060)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:925)  at 
org.apache.solr.core.SolrCore.(SolrCore.java:817)  at 
org.apache.solr.core.CoreContainer.create(CoreContainer.java:906)  at 
org.apache.solr.core.CoreContainer.lambda$load$3(CoreContainer.java:551)  at 
java.util.concurrent.FutureTask.run(FutureTask.java:266)  at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
 at java.lang.Thread.run(Thread.java:745)  

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not 
released!!! [HdfsTransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:43)
at 
org.apache.solr.update.HdfsTransactionLog.(HdfsTransactionLog.java:130)
at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:202)
at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:137)
at org.apache.solr.update.UpdateHandler.(UpdateHandler.java:94)
at 
org.apache.solr.update.DirectUpdateHandler2.(DirectUpdateHandler2.java:102)
at sun.reflect.GeneratedConstructorAccessor184.newInstance(Unknown 
Source)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:747)
at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:809)
at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1060)
at org.apache.solr.core.SolrCore.(SolrCore.java:925)
at org.apache.solr.core.SolrCore.(SolrCore.java:817)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:906)
at 
org.apache.solr.core.CoreContainer.lambda$load$3(CoreContainer.java:551)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)


at __randomizedtesting.SeedInfo.seed([2BBACA02CF2AA95D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at 
org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:266)
at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_112) - Build # 18643 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18643/
Java: 64bit/jdk1.8.0_112 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
[SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:37875//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:37875//collection1/] expected:<0> but was:<2>

Stack Trace:
java.lang.AssertionError: [SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:37875//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:37875//collection1/] expected:<0> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([A4BF0FC2EDC58CE7:2CEB30184339E11F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Updated] (SOLR-9900) ReversedWildcardFilterFactory yields false positive hits for range query

[Yonik Seeley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yonik Seeley updated SOLR-9900:
---
Attachment: SOLR-9900.patch

Attached is a simple fix for the query parser.
It doesn't really fix everything though, because FieldType.getRangeQuery() will 
still be broken.

> ReversedWildcardFilterFactory yields false positive hits for range query
> 
>
> Key: SOLR-9900
> URL: https://issues.apache.org/jira/browse/SOLR-9900
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Affects Versions: 1.4
>Reporter: Mikhail Khludnev
>Priority: Minor
> Attachments: SOLR-1321-range-q-false-positive.patch, SOLR-9900.patch
>
>
> Range query yields false positives when ReversedWildcardFilterFactory is on. 
> I'm not sure if it's worth to bother.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9877) Use instrumented http client

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784471#comment-15784471
 ] 

ASF subversion and git services commented on SOLR-9877:
---

Commit 662be93ed11abebaff1d13711f3bffca478ba61e in lucene-solr's branch 
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=662be93 ]

SOLR-9877: Null check for metric registry before attempting to use it


> Use instrumented http client
> 
>
> Key: SOLR-9877
> URL: https://issues.apache.org/jira/browse/SOLR-9877
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9877.patch, SOLR-9877.patch, solr-http-metrics.png
>
>
> Use instrumented equivalents of PooledHttpClientConnectionManager and others 
> from metrics-httpclient library.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1041 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1041/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  org.apache.solr.cloud.PeerSyncReplicationTest.test

Error Message:
expected:<152> but was:<138>

Stack Trace:
java.lang.AssertionError: expected:<152> but was:<138>
at 
__randomizedtesting.SeedInfo.seed([8D39E122DB993B15:56DDEF8756556ED]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:280)
at 
org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:244)
at 
org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:130)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 

[jira] [Commented] (SOLR-9897) Solr UnifiedHighlighter should support hl.requireFieldMatch=false

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784436#comment-15784436
 ] 

ASF subversion and git services commented on SOLR-9897:
---

Commit 529c60d914a6c511813445d97d748ca3b4692fb5 in lucene-solr's branch 
refs/heads/branch_6x from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=529c60d ]

SOLR-9897: Add hl.requireFieldMatch=false support when using the 
UnifiedHighlighter

(cherry picked from commit 20362de)


> Solr UnifiedHighlighter should support hl.requireFieldMatch=false
> -
>
> Key: SOLR-9897
> URL: https://issues.apache.org/jira/browse/SOLR-9897
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: SOLR-9897.patch
>
>
> Lucene's UnifiedHighlighter recently gained the ability for it to support 
> Solr's hl.requireFieldMatch option. -- LUCENE-7575 "fieldMatcher" option.  We 
> can finally enable support for this flag in Solr.  Though I don't intend to 
> add support for a regexp or glob at this time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9897) Solr UnifiedHighlighter should support hl.requireFieldMatch=false

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley resolved SOLR-9897.

Resolution: Fixed

> Solr UnifiedHighlighter should support hl.requireFieldMatch=false
> -
>
> Key: SOLR-9897
> URL: https://issues.apache.org/jira/browse/SOLR-9897
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: SOLR-9897.patch
>
>
> Lucene's UnifiedHighlighter recently gained the ability for it to support 
> Solr's hl.requireFieldMatch option. -- LUCENE-7575 "fieldMatcher" option.  We 
> can finally enable support for this flag in Solr.  Though I don't intend to 
> add support for a regexp or glob at this time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9897) Solr UnifiedHighlighter should support hl.requireFieldMatch=false

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784433#comment-15784433
 ] 

ASF subversion and git services commented on SOLR-9897:
---

Commit 20362deb7e6814c1922163595e7edeb652d3ce37 in lucene-solr's branch 
refs/heads/master from [~dsmiley]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=20362de ]

SOLR-9897: Add hl.requireFieldMatch=false support when using the 
UnifiedHighlighter


> Solr UnifiedHighlighter should support hl.requireFieldMatch=false
> -
>
> Key: SOLR-9897
> URL: https://issues.apache.org/jira/browse/SOLR-9897
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: highlighter
>Reporter: David Smiley
>Assignee: David Smiley
> Fix For: 6.4
>
> Attachments: SOLR-9897.patch
>
>
> Lucene's UnifiedHighlighter recently gained the ability for it to support 
> Solr's hl.requireFieldMatch option. -- LUCENE-7575 "fieldMatcher" option.  We 
> can finally enable support for this flag in Solr.  Though I don't intend to 
> add support for a regexp or glob at this time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 659 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/659/

No tests ran.

Build Log:
[...truncated 41962 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist
 [copy] Copying 476 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 260 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.05 sec (3.9 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-7.0.0-src.tgz...
   [smoker] 30.5 MB in 0.04 sec (800.9 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.0.0.tgz...
   [smoker] 64.9 MB in 0.08 sec (820.6 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-7.0.0.zip...
   [smoker] 75.8 MB in 0.09 sec (803.8 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-7.0.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6156 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.0.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6156 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-7.0.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 215 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] success!
   [smoker] 
   [smoker] Test Solr...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.00 sec (71.9 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download solr-7.0.0-src.tgz...
   [smoker] 40.0 MB in 0.05 sec (785.3 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-7.0.0.tgz...
   [smoker] 140.4 MB in 0.18 sec (760.3 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-7.0.0.zip...
   [smoker] 149.9 MB in 0.19 sec (792.4 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack solr-7.0.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] unpack lucene-7.0.0.tgz...
   [smoker]   **WARNING**: skipping check of 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
   [smoker]   **WARNING**: skipping check of 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
   [smoker] copying unpacked distribution for Java 8 ...
   [smoker] test solr example w/ Java 8...
   [smoker]   start Solr instance 
(log=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0-java8/solr-example.log)...
   [smoker] No process found for Solr node running on port 8983
   [smoker]   Running techproducts example on port 8983 from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0-java8
   [smoker] Creating Solr home directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/build/smokeTestRelease/tmp/unpack/solr-7.0.0-java8/example/techproducts/solr
   [smoker] 
   [smoker] Starting up Solr on port 8983 using command:
   [smoker] bin/solr start -p 8983 -s "example/techproducts/solr"
   [smoker] 
   [smoker] Waiting up to 180 seconds to see Solr running on port 8983 [|]  
 [/]   [-]   [\]  
   [smoker] Started Solr server on port 8983 (pid=20583). Happy searching!
   [smoker] 
  

[JENKINS-EA] Lucene-Solr-6.x-Linux (32bit/jdk-9-ea+147) - Build # 2536 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2536/
Java: 32bit/jdk-9-ea+147 -client -XX:+UseParallelGC

10 tests failed.
FAILED:  
org.apache.lucene.codecs.asserting.TestAssertingNormsFormat.testRamBytesUsed

Error Message:
Actual RAM usage 248, but got 5169, -1984.2741935483873% error

Stack Trace:
java.lang.AssertionError: Actual RAM usage 248, but got 5169, 
-1984.2741935483873% error
at 
__randomizedtesting.SeedInfo.seed([F162A7FB50E4201C:3C1B5BB9A9B3F4A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:279)
at 
org.apache.lucene.index.BaseNormsFormatTestCase.testRamBytesUsed(BaseNormsFormatTestCase.java:46)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
org.apache.lucene.codecs.blocktreeords.TestOrdsBlockTree.testRamBytesUsed

Error Message:
Actual RAM usage 272, but got 1959, -620.2205882352941% error

Stack Trace:
java.lang.AssertionError: Actual RAM usage 272, but got 1959, 
-620.2205882352941% error
at 
__randomizedtesting.SeedInfo.seed([150A9541F87165D0:E7A98701320E7A86]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 

[JENKINS] Lucene-Solr-Tests-6.x - Build # 632 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/632/

1 tests failed.
FAILED:  org.apache.solr.update.AutoCommitTest.testMaxTime

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
at 
__randomizedtesting.SeedInfo.seed([EB9A53509FF82A3D:716E2EB20162B601]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:818)
at 
org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: 
xpath=//result[@numFound=1]
xml response was: 

00


request was:q=id:529=standard=0=20=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:811)
... 40 more




Build Log:
[...truncated 11795 lines...]
   [junit4] Suite: org.apache.solr.update.AutoCommitTest
   [junit4]   2> Creating dataDir: 

[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-9-ea+147) - Build # 18642 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18642/
Java: 64bit/jdk-9-ea+147 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
[SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:33641//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:33641//collection1/] expected:<0> but was:<2>

Stack Trace:
java.lang.AssertionError: [SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:33641//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:33641//collection1/] expected:<0> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([D5F1C38D5C3C547B:5DA5FC57F2C03983]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:165)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Resolved] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9

[Uwe Schindler (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uwe Schindler resolved LUCENE-7595.
---
   Resolution: Fixed
Fix Version/s: 6.4
   master (7.0)
   6.x

> RAMUsageTester in test-framework and static field checker no longer works 
> with Java 9
> -
>
> Key: LUCENE-7595
> URL: https://issues.apache.org/jira/browse/LUCENE-7595
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Fix For: 6.x, master (7.0), 6.4
>
> Attachments: LUCENE-7595.patch
>
>
> Lucene/Solr tests have a special rule that records memory usage in static 
> fields before and after test, so we can detect memory leaks. This check dives 
> into JDK classes (like java.lang.String to detect their size). As Java 9 
> build 148 completely forbids setAccessible on any runtime class, we have to 
> change or disable this check:
> - As first step I will only add the rule to LTC, if we not have Java 8
> - As a second step we might investigate how to improve this
> [~rcmuir] had some ideas for the 2nd point:
> - Don't dive into classes from JDK modules and instead "estimate" the size 
> for some special cases (like Strings)
> - Disallow any static field in tests that is not final (constant) and points 
> to an Object except: Strings and native (wrapper) types.
> In addition we also have RAMUsageTester, that has similar problems and is 
> used to compare estimations of Lucene's calculations of 
> Codec/IndexWriter/IndexReader memory usage with reality. We should simply 
> disable those tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784166#comment-15784166
 ] 

ASF subversion and git services commented on LUCENE-7595:
-

Commit ea7db0a176979559e874b292522fa7006b578882 in lucene-solr's branch 
refs/heads/branch_6x from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ea7db0a ]

LUCENE-7595: Improve RAMUsageTester in test-framework to estimate memory usage 
of runtime classes and work with Java 9 EA (b148+). Disable static field heap 
usage checker in LuceneTestCase


> RAMUsageTester in test-framework and static field checker no longer works 
> with Java 9
> -
>
> Key: LUCENE-7595
> URL: https://issues.apache.org/jira/browse/LUCENE-7595
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Attachments: LUCENE-7595.patch
>
>
> Lucene/Solr tests have a special rule that records memory usage in static 
> fields before and after test, so we can detect memory leaks. This check dives 
> into JDK classes (like java.lang.String to detect their size). As Java 9 
> build 148 completely forbids setAccessible on any runtime class, we have to 
> change or disable this check:
> - As first step I will only add the rule to LTC, if we not have Java 8
> - As a second step we might investigate how to improve this
> [~rcmuir] had some ideas for the 2nd point:
> - Don't dive into classes from JDK modules and instead "estimate" the size 
> for some special cases (like Strings)
> - Disallow any static field in tests that is not final (constant) and points 
> to an Object except: Strings and native (wrapper) types.
> In addition we also have RAMUsageTester, that has similar problems and is 
> used to compare estimations of Lucene's calculations of 
> Codec/IndexWriter/IndexReader memory usage with reality. We should simply 
> disable those tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3741 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3741/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
[SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:55779//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:55779//collection1/] expected:<0> but was:<2>

Stack Trace:
java.lang.AssertionError: [SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:55779//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:55779//collection1/] expected:<0> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([732A66A9C69055C3:FB7E5973686C383B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (LUCENE-7595) RAMUsageTester in test-framework and static field checker no longer works with Java 9

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784137#comment-15784137
 ] 

ASF subversion and git services commented on LUCENE-7595:
-

Commit f29d2b5668296dfcdb8d650305449674faa29847 in lucene-solr's branch 
refs/heads/master from [~thetaphi]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f29d2b5 ]

LUCENE-7595: Improve RAMUsageTester in test-framework to estimate memory usage 
of runtime classes and work with Java 9 EA (b148+). Disable static field heap 
usage checker in LuceneTestCase


> RAMUsageTester in test-framework and static field checker no longer works 
> with Java 9
> -
>
> Key: LUCENE-7595
> URL: https://issues.apache.org/jira/browse/LUCENE-7595
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: general/test
>Reporter: Uwe Schindler
>Assignee: Uwe Schindler
>  Labels: Java9
> Attachments: LUCENE-7595.patch
>
>
> Lucene/Solr tests have a special rule that records memory usage in static 
> fields before and after test, so we can detect memory leaks. This check dives 
> into JDK classes (like java.lang.String to detect their size). As Java 9 
> build 148 completely forbids setAccessible on any runtime class, we have to 
> change or disable this check:
> - As first step I will only add the rule to LTC, if we not have Java 8
> - As a second step we might investigate how to improve this
> [~rcmuir] had some ideas for the 2nd point:
> - Don't dive into classes from JDK modules and instead "estimate" the size 
> for some special cases (like Strings)
> - Disallow any static field in tests that is not final (constant) and points 
> to an Object except: Strings and native (wrapper) types.
> In addition we also have RAMUsageTester, that has similar problems and is 
> used to compare estimations of Lucene's calculations of 
> Codec/IndexWriter/IndexReader memory usage with reality. We should simply 
> disable those tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-6.x - Build # 220 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-6.x/220/

No tests ran.

Build Log:
[...truncated 41934 lines...]
prepare-release-no-sign:
[mkdir] Created dir: 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/dist
 [copy] Copying 476 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/dist/lucene
 [copy] Copying 260 files to 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/dist/solr
   [smoker] Java 1.8 JAVA_HOME=/home/jenkins/tools/java/latest1.8
   [smoker] NOTE: output encoding is UTF-8
   [smoker] 
   [smoker] Load release URL 
"file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/dist/"...
   [smoker] 
   [smoker] Test Lucene...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.05 sec (4.2 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download lucene-6.4.0-src.tgz...
   [smoker] 30.5 MB in 0.04 sec (859.6 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-6.4.0.tgz...
   [smoker] 65.0 MB in 0.07 sec (884.5 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download lucene-6.4.0.zip...
   [smoker] 75.8 MB in 0.09 sec (872.9 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack lucene-6.4.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6176 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-6.4.0.zip...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] test demo with 1.8...
   [smoker]   got 6176 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] check Lucene's javadoc JAR
   [smoker]   unpack lucene-6.4.0-src.tgz...
   [smoker] make sure no JARs/WARs in src dist...
   [smoker] run "ant validate"
   [smoker] run tests w/ Java 8 and testArgs='-Dtests.slow=false'...
   [smoker] test demo with 1.8...
   [smoker]   got 229 hits for query "lucene"
   [smoker] checkindex with 1.8...
   [smoker] generate javadocs w/ Java 8...
   [smoker] 
   [smoker] Crawl/parse...
   [smoker] 
   [smoker] Verify...
   [smoker]   confirm all releases have coverage in TestBackwardsCompatibility
   [smoker] find all past Lucene releases...
   [smoker] run TestBackwardsCompatibility..
   [smoker] success!
   [smoker] 
   [smoker] Test Solr...
   [smoker]   test basics...
   [smoker]   get KEYS
   [smoker] 0.2 MB in 0.00 sec (86.9 MB/sec)
   [smoker]   check changes HTML...
   [smoker]   download solr-6.4.0-src.tgz...
   [smoker] 40.1 MB in 0.05 sec (783.8 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-6.4.0.tgz...
   [smoker] 140.2 MB in 0.18 sec (781.0 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   download solr-6.4.0.zip...
   [smoker] 149.7 MB in 0.19 sec (794.9 MB/sec)
   [smoker] verify md5/sha1 digests
   [smoker]   unpack solr-6.4.0.tgz...
   [smoker] verify JAR metadata/identity/no javax.* or java.* classes...
   [smoker] unpack lucene-6.4.0.tgz...
   [smoker]   **WARNING**: skipping check of 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.4.0/contrib/dataimporthandler-extras/lib/javax.mail-1.5.1.jar:
 it has javax.* classes
   [smoker]   **WARNING**: skipping check of 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.4.0/contrib/dataimporthandler-extras/lib/activation-1.1.1.jar:
 it has javax.* classes
   [smoker] copying unpacked distribution for Java 8 ...
   [smoker] test solr example w/ Java 8...
   [smoker]   start Solr instance 
(log=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.4.0-java8/solr-example.log)...
   [smoker] No process found for Solr node running on port 8983
   [smoker]   Running techproducts example on port 8983 from 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.4.0-java8
   [smoker] Creating Solr home directory 
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-6.x/lucene/build/smokeTestRelease/tmp/unpack/solr-6.4.0-java8/example/techproducts/solr
   [smoker] 
   [smoker] Starting up Solr on port 8983 using command:
   [smoker] bin/solr start -p 8983 -s "example/techproducts/solr"
   [smoker] 
   [smoker] Waiting up to 180 seconds to see Solr running on port 8983 [|]  
 [/]   [-]   [\]  
   [smoker] Started Solr server on port 8983 (pid=24223). Happy searching!
   [smoker] 
   [smoker] 
   

[jira] [Commented] (SOLR-9859) replication.properties cannot be updated after being written and neither replication.properties or index.properties are durable in the face of a crash

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784031#comment-15784031
 ] 

ASF subversion and git services commented on SOLR-9859:
---

Commit 96ed221fb6924dd167591004a5eaf70d53f92e4f in lucene-solr's branch 
refs/heads/master from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=96ed221 ]

SOLR-9859: replication.properties cannot be updated after being written and 
neither eplication.properties or ndex.properties are durable in the face of a 
crash.


> replication.properties cannot be updated after being written and neither 
> replication.properties or index.properties are durable in the face of a crash
> --
>
> Key: SOLR-9859
> URL: https://issues.apache.org/jira/browse/SOLR-9859
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.5.3, 6.3
>Reporter: Pushkar Raste
>Assignee: Mark Miller
>Priority: Minor
> Attachments: SOLR-9859.patch, SOLR-9859.patch, SOLR-9859.patch, 
> SOLR-9859.patch, SOLR-9859.patch, SOLR-9859.patch
>
>
> If a shard recovers via replication (vs PeerSync) a file named 
> {{replication.properties}} gets created. If the same shard recovers once more 
> via replication, IndexFetcher fails to write latest replication information 
> as it tries to create {{replication.properties}} but as file already exists. 
> Here is the stack trace I saw 
> {code}
> java.nio.file.FileAlreadyExistsException: 
> \shard-3-001\cores\collection1\data\replication.properties
>   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
>   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>   at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(Unknown Source)
>   at java.nio.file.spi.FileSystemProvider.newOutputStream(Unknown Source)
>   at java.nio.file.Files.newOutputStream(Unknown Source)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:413)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:409)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.solr.handler.IndexFetcher.logReplicationTimeAndConfFiles(IndexFetcher.java:689)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:501)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:265)
>   at 
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:157)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
>   at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:222)
>   at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>   at java.util.concurrent.FutureTask.run(Unknown Source)
>   at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$0(ExecutorUtil.java:229)
>   at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>   at java.lang.Thread.run(Unknown Source)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9899) StandardDirectoryFactory should use optimizations for all FilterDirectorys not just NRTCachingDirectory.

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15784032#comment-15784032
 ] 

ASF subversion and git services commented on SOLR-9899:
---

Commit 262049fc8f60a166f0eed0aef5d7ddd1e7c90bc7 in lucene-solr's branch 
refs/heads/master from markrmiller
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=262049f ]

SOLR-9899: StandardDirectoryFactory should use optimizations for all 
FilterDirectorys not just NRTCachingDirectory.


> StandardDirectoryFactory should use optimizations for all FilterDirectorys 
> not just NRTCachingDirectory.
> 
>
> Key: SOLR-9899
> URL: https://issues.apache.org/jira/browse/SOLR-9899
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9899.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (SOLR-9846) OverseerAutoReplicaFailoverThread can take too long to stop and leak out of unit tests.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller reopened SOLR-9846:
---

I'm still seeing this.

> OverseerAutoReplicaFailoverThread can take too long to stop and leak out of 
> unit tests.
> ---
>
> Key: SOLR-9846
> URL: https://issues.apache.org/jira/browse/SOLR-9846
> Project: Solr
>  Issue Type: Test
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master (7.0), 6.4
>
>
> We should interrupt it on close.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Windows (32bit/jdk1.8.0_112) - Build # 653 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Windows/653/
Java: 32bit/jdk1.8.0_112 -server -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=15416, 
name=SocketProxy-Response-64748:65291, state=RUNNABLE, 
group=TGRP-HttpPartitionTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught 
exception in thread: Thread[id=15416, name=SocketProxy-Response-64748:65291, 
state=RUNNABLE, group=TGRP-HttpPartitionTest]
at 
__randomizedtesting.SeedInfo.seed([896805E993417965:13C3A333DBD149D]:0)
Caused by: java.lang.RuntimeException: java.net.SocketException: Socket is 
closed
at __randomizedtesting.SeedInfo.seed([896805E993417965]:0)
at 
org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:347)
Caused by: java.net.SocketException: Socket is closed
at java.net.Socket.setSoTimeout(Socket.java:1137)
at 
org.apache.solr.cloud.SocketProxy$Bridge$Pump.run(SocketProxy.java:344)




Build Log:
[...truncated 12259 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_896805E993417965-001\init-core-data-001
   [junit4]   2> 1810895 INFO  
(SUITE-HttpPartitionTest-seed#[896805E993417965]-worker) [] 
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: 
@org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1810895 INFO  
(SUITE-HttpPartitionTest-seed#[896805E993417965]-worker) [] 
o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1810897 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1810898 INFO  (Thread-3983) [] o.a.s.c.ZkTestServer client 
port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1810898 INFO  (Thread-3983) [] o.a.s.c.ZkTestServer 
Starting server
   [junit4]   2> 1810997 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.ZkTestServer start zk server on port:64723
   [junit4]   2> 1811015 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml
 to /configs/conf1/solrconfig.xml
   [junit4]   2> 1811018 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml
 to /configs/conf1/schema.xml
   [junit4]   2> 1811020 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml
 to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1811022 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt
 to /configs/conf1/stopwords.txt
   [junit4]   2> 1811024 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt
 to /configs/conf1/protwords.txt
   [junit4]   2> 1811026 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml
 to /configs/conf1/currency.xml
   [junit4]   2> 1811029 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml
 to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1811031 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json
 to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1811033 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 
C:\Users\jenkins\workspace\Lucene-Solr-6.x-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt
 to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1811035 INFO  
(TEST-HttpPartitionTest.test-seed#[896805E993417965]) [] 
o.a.s.c.AbstractZkTestCase put 

[JENKINS] Lucene-Solr-Tests-master - Build # 1572 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/1572/

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestRandomFlRTGCloud

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.TestRandomFlRTGCloud: 
1) Thread[id=2416, 
name=OverseerHdfsCoreFailoverThread-97187435524784137-127.0.0.1:56271_solr-n_01,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at 
java.lang.Thread.sleep(Native Method) at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
 at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.TestRandomFlRTGCloud: 
   1) Thread[id=2416, 
name=OverseerHdfsCoreFailoverThread-97187435524784137-127.0.0.1:56271_solr-n_01,
 state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at 
org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([CE87EAB584CAE4F0]:0)


FAILED:  org.apache.solr.metrics.reporters.SolrGangliaReporterTest.testReporter

Error Message:


Stack Trace:
java.util.ConcurrentModificationException
at 
__randomizedtesting.SeedInfo.seed([CE87EAB584CAE4F0:9163C782EFC677B5]:0)
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
at java.util.ArrayList$Itr.next(ArrayList.java:851)
at 
org.apache.solr.metrics.reporters.SolrGangliaReporterTest.testReporter(SolrGangliaReporterTest.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Created] (SOLR-9903) Stop interrupting the update executor on shutdown as it can cause graceful shutdowns to put replicas into Leader Initiated Recovery.

[Mark Miller (JIRA)]

Mark Miller created SOLR-9903:
-

 Summary: Stop interrupting the update executor on shutdown as it 
can cause graceful shutdowns to put replicas into Leader Initiated Recovery.
 Key: SOLR-9903
 URL: https://issues.apache.org/jira/browse/SOLR-9903
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Mark Miller
Assignee: Mark Miller


I also want this change in SOLR-9824. There is a related JIRA somewhere that 
points this out, though that JIRA may be broader in scope.

I think there are many good reasons not to interrupt after some additional 
experience.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_112) - Build # 18641 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18641/
Java: 32bit/jdk1.8.0_112 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
[SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:46742/_/collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:46742/_/collection1/] expected:<0> but was:<2>

Stack Trace:
java.lang.AssertionError: [SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:46742/_/collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:46742/_/collection1/] expected:<0> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([7B6E335E2C2CC9D1:F33A0C8482D0A429]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-9877) Use instrumented http client

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783839#comment-15783839
 ] 

Mark Miller commented on SOLR-9877:
---

SolrCmdDistributorTest seems to be exposing a race of some kind or something. 
It's fails on 99% of my local runs.

InstrumentedHttpRequestExecutor hits an NPE in Timer timer(HttpRequest 
request), I'd guess because the metricsRegistry has not been assigned yet. 

> Use instrumented http client
> 
>
> Key: SOLR-9877
> URL: https://issues.apache.org/jira/browse/SOLR-9877
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9877.patch, SOLR-9877.patch, solr-http-metrics.png
>
>
> Use instrumented equivalents of PooledHttpClientConnectionManager and others 
> from metrics-httpclient library.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9901) Implement move in HdfsDirectoryFactory.

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783836#comment-15783836
 ] 

Mike Drob commented on SOLR-9901:
-

LGTM.



> Implement move in HdfsDirectoryFactory.
> ---
>
> Key: SOLR-9901
> URL: https://issues.apache.org/jira/browse/SOLR-9901
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9901.patch
>
>
> Without this, you can end up with things like a 0 bytes segment file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9902) StandardDirectoryFactory should use Files API for it's move implementation.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-9902:
--
Attachment: SOLR-9902.patch

> StandardDirectoryFactory should use Files API for it's move implementation.
> ---
>
> Key: SOLR-9902
> URL: https://issues.apache.org/jira/browse/SOLR-9902
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9902.patch
>
>
> It's done in a platform independent way as opposed to the old File API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9902) StandardDirectoryFactory should use Files API for it's move implementation.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-9902:
--
Description: It's done in a platform independent way as opposed to the old 
File API.

> StandardDirectoryFactory should use Files API for it's move implementation.
> ---
>
> Key: SOLR-9902
> URL: https://issues.apache.org/jira/browse/SOLR-9902
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master (7.0), 6.4
>
>
> It's done in a platform independent way as opposed to the old File API.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_112) - Build # 6318 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6318/
Java: 32bit/jdk1.8.0_112 -client -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
[SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:55690//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:55690//collection1/] expected:<0> but was:<2>

Stack Trace:
java.lang.AssertionError: [SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:55690//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:55690//collection1/] expected:<0> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([84B6221F98333750:CE21DC536CF5AA8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Updated] (SOLR-9901) Implement move in HdfsDirectoryFactory.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-9901:
--
Attachment: SOLR-9901.patch

> Implement move in HdfsDirectoryFactory.
> ---
>
> Key: SOLR-9901
> URL: https://issues.apache.org/jira/browse/SOLR-9901
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9901.patch
>
>
> Without this, you can end up with things like a 0 bytes segment file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9899) StandardDirectoryFactory should use optimizations for all FilterDirectorys not just NRTCachingDirectory.

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-9899:
--
Attachment: SOLR-9899.patch

> StandardDirectoryFactory should use optimizations for all FilterDirectorys 
> not just NRTCachingDirectory.
> 
>
> Key: SOLR-9899
> URL: https://issues.apache.org/jira/browse/SOLR-9899
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9899.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-6.x-Linux (32bit/jdk1.8.0_112) - Build # 2534 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2534/
Java: 32bit/jdk1.8.0_112 -server -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit

Error Message:
expected:<1> but was:<2>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([12DC17AF670681F2:EB9184005B73CC78]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Created] (SOLR-9902) StandardDirectoryFactory should use Files API for it's move implementation.

[Mark Miller (JIRA)]

Mark Miller created SOLR-9902:
-

 Summary: StandardDirectoryFactory should use Files API for it's 
move implementation.
 Key: SOLR-9902
 URL: https://issues.apache.org/jira/browse/SOLR-9902
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: master (7.0), 6.4






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9901) Implement move in HdfsDirectoryFactory.

[Mark Miller (JIRA)]

Mark Miller created SOLR-9901:
-

 Summary: Implement move in HdfsDirectoryFactory.
 Key: SOLR-9901
 URL: https://issues.apache.org/jira/browse/SOLR-9901
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: master (7.0), 6.4


Without this, you can end up with things like a 0 bytes segment file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9900) ReversedWildcardFilterFactory yields false positive hits for range query

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9900:
---
Description: Range query yields false positives when 
ReversedWildcardFilterFactory is on. I'm not sure if it's worth to bother.  
(was: Range query yields false positives when ReversedWildcardFilterFactory is 
on)

> ReversedWildcardFilterFactory yields false positive hits for range query
> 
>
> Key: SOLR-9900
> URL: https://issues.apache.org/jira/browse/SOLR-9900
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Affects Versions: 1.4
>Reporter: Mikhail Khludnev
>Priority: Minor
> Attachments: SOLR-1321-range-q-false-positive.patch
>
>
> Range query yields false positives when ReversedWildcardFilterFactory is on. 
> I'm not sure if it's worth to bother.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9900) ReversedWildcardFilterFactory yields false positive hits for range query

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9900:
---
Attachment: SOLR-1321-range-q-false-positive.patch

[^SOLR-1321-range-q-false-positive.patch] proofs false positive on range query. 

> ReversedWildcardFilterFactory yields false positive hits for range query
> 
>
> Key: SOLR-9900
> URL: https://issues.apache.org/jira/browse/SOLR-9900
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Schema and Analysis
>Affects Versions: 1.4
>Reporter: Mikhail Khludnev
>Priority: Minor
> Attachments: SOLR-1321-range-q-false-positive.patch
>
>
> Range query yields false positives when ReversedWildcardFilterFactory is on



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9900) ReversedWildcardFilterFactory yields false positive hits for range query

[Mikhail Khludnev (JIRA)]

Mikhail Khludnev created SOLR-9900:
--

 Summary: ReversedWildcardFilterFactory yields false positive hits 
for range query
 Key: SOLR-9900
 URL: https://issues.apache.org/jira/browse/SOLR-9900
 Project: Solr
  Issue Type: Bug
  Security Level: Public (Default Security Level. Issues are Public)
  Components: Schema and Analysis
Affects Versions: 1.4
Reporter: Mikhail Khludnev
Priority: Minor


Range query yields false positives when ReversedWildcardFilterFactory is on



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-9899) StandardDirectoryFactory should use optimizations for all FilterDirectorys not just NRTCachingDirectory.

[Mark Miller (JIRA)]

Mark Miller created SOLR-9899:
-

 Summary: StandardDirectoryFactory should use optimizations for all 
FilterDirectorys not just NRTCachingDirectory.
 Key: SOLR-9899
 URL: https://issues.apache.org/jira/browse/SOLR-9899
 Project: Solr
  Issue Type: Improvement
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Mark Miller
Assignee: Mark Miller
 Fix For: master (7.0), 6.4






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 240 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/240/

6 tests failed.
FAILED:  org.apache.lucene.search.TestFuzzyQuery.testRandom

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([B9AA01A4CB8C1B16]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestFuzzyQuery

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([B9AA01A4CB8C1B16]:0)


FAILED:  org.apache.solr.handler.extraction.TestExtractionDateUtil.testParseDate

Error Message:
Incorrect parsed timestamp: 1226583351000 != 1226579751000 (Thu Nov 13 04:35:51 
AKST 2008)

Stack Trace:
java.lang.AssertionError: Incorrect parsed timestamp: 1226583351000 != 
1226579751000 (Thu Nov 13 04:35:51 AKST 2008)
at 
__randomizedtesting.SeedInfo.seed([3632FEAD465B2693:7C2B86983DF25126]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.handler.extraction.TestExtractionDateUtil.assertParsedDate(TestExtractionDateUtil.java:59)
at 
org.apache.solr.handler.extraction.TestExtractionDateUtil.testParseDate(TestExtractionDateUtil.java:54)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-9836) Add more graceful recovery steps when failing to create SolrCore

[Mike Drob (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783628#comment-15783628
 ] 

Mike Drob commented on SOLR-9836:
-

bq. Have you run the full test suite as well yet?
I'm getting some errors related to backup/restore operations that go away if I 
modify those tests to {{useFactory("solr.StandardDirectoryFactory")}} instead 
of RAM/Mock. Will look into this further and then post a new patch when I 
figure it out.

> Add more graceful recovery steps when failing to create SolrCore
> 
>
> Key: SOLR-9836
> URL: https://issues.apache.org/jira/browse/SOLR-9836
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Reporter: Mike Drob
> Attachments: SOLR-9836.patch, SOLR-9836.patch, SOLR-9836.patch
>
>
> I have seen several cases where there is a zero-length segments_n file. We 
> haven't identified the root cause of these issues (possibly a poorly timed 
> crash during replication?) but if there is another node available then Solr 
> should be able to recover from this situation. Currently, we log and give up 
> on loading that core, leaving the user to manually intervene.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-master-Linux (32bit/jdk-9-ea+147) - Build # 18640 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18640/
Java: 32bit/jdk-9-ea+147 -server -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
[SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:35034//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:35034//collection1/] expected:<0> but was:<2>

Stack Trace:
java.lang.AssertionError: [SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:35034//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:35034//collection1/] expected:<0> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([69E88EDD692A6BD1:E1BCB107C7D60629]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:165)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-9824) Documents indexed in bulk are replicated using too many HTTP requests

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783578#comment-15783578
 ] 

Mark Miller commented on SOLR-9824:
---

I'm getting pretty comfortable. I spent all the time on SOLR-2646 mostly for 
this issue, so that I could easily use CUSS against a SolrCloud cluster and 
check performance and results and logging under many different cases. Varied 
queue size, random pauses between updates, varied thread counts, etc, etc.

Given I have not touched this in a while, I'll probably do another battery of 
tests before committing it though.

> Documents indexed in bulk are replicated using too many HTTP requests
> -
>
> Key: SOLR-9824
> URL: https://issues.apache.org/jira/browse/SOLR-9824
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.3
>Reporter: David Smiley
>Assignee: Mark Miller
> Attachments: SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch, 
> SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch
>
>
> This takes awhile to explain; bear with me. While working on bulk indexing 
> small documents, I looked at the logs of my SolrCloud nodes.  I noticed that 
> shards would see an /update log message every ~6ms which is *way* too much.  
> These are requests from one shard (that isn't a leader/replica for these docs 
> but the recipient from my client) to the target shard leader (no additional 
> replicas).  One might ask why I'm not sending docs to the right shard in the 
> first place; I have a reason but it's besides the point -- there's a real 
> Solr perf problem here and this probably applies equally to 
> replicationFactor>1 situations too.  I could turn off the logs but that would 
> hide useful stuff, and it's disconcerting to me that so many short-lived HTTP 
> requests are happening, somehow at the bequest of DistributedUpdateProcessor. 
>  After lots of analysis and debugging and hair pulling, I finally figured it 
> out.  
> In SOLR-7333 ([~tpot]) introduced an optimization called 
> {{UpdateRequest.isLastDocInBatch()}} in which ConcurrentUpdateSolrClient will 
> poll with a '0' timeout to the internal queue, so that it can close the 
> connection without it hanging around any longer than needed.  This part makes 
> sense to me.  Currently the only spot that has the smarts to set this flag is 
> {{JavaBinUpdateRequestCodec.unmarshal.readOuterMostDocIterator()}} at the 
> last document.  So if a shard received docs in a javabin stream (but not 
> other formats) one would expect the _last_ document to have this flag.  
> There's even a test.  Docs without this flag get the default poll time; for 
> javabin it's 25ms.  Okay.
> I _suspect_ that if someone used CloudSolrClient or HttpSolrClient to send 
> javabin data in a batch, the intended efficiencies of SOLR-7333 would apply.  
> I didn't try. In my case, I'm using ConcurrentUpdateSolrClient (and BTW 
> DistributedUpdateProcessor uses CUSC too).  CUSC uses the RequestWriter 
> (defaulting to javabin) to send each document separately without any leading 
> marker or trailing marker.  For the XML format by comparison, there is a 
> leading and trailing marker ( ... ).  Since there's no outer 
> container for the javabin unmarshalling to detect the last document, it marks 
> _every_ document as {{req.lastDocInBatch()}}!  Ouch!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-7605) AnalyzingInfixSuggesterTest.testRandomNRT() failures: FixedBitSet cannot be cast to SimpleTextBits

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-7605.
--
   Resolution: Fixed
Fix Version/s: master (7.0)

Thanks [~jim.ferenczi]!

> AnalyzingInfixSuggesterTest.testRandomNRT() failures: FixedBitSet cannot be 
> cast to SimpleTextBits
> --
>
> Key: LUCENE-7605
> URL: https://issues.apache.org/jira/browse/LUCENE-7605
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/codecs, modules/suggest
>Reporter: Steve Rowe
> Fix For: master (7.0)
>
> Attachments: LUCENE-7605.patch
>
>
> While beasting for LUCENE-7564, I found these reproducing master seeds, which 
> trigger casting failures when SimpleText codec is randomly chosen:
> {noformat}
>[junit4] Suite: 
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=AnalyzingInfixSuggesterTest -Dtests.method=testRandomNRT 
> -Dtests.seed=E4D62AFBD9F5FF7F -Dtests.slow=true -Dtests.locale=es-BO 
> -Dtests.timezone=Africa/Asmara -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.18s | AnalyzingInfixSuggesterTest.testRandomNRT <<<
>[junit4]> Throwable #1: java.lang.ClassCastException: 
> org.apache.lucene.util.FixedBitSet cannot be cast to 
> org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat$SimpleTextBits
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([E4D62AFBD9F5FF7F:40F82446812A23C3]:0)
>[junit4]>at 
> org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.writeLiveDocs(SimpleTextLiveDocsFormat.java:111)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:570)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:481)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:539)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:653)
>[junit4]>at 
> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3368)
>[junit4]>at 
> org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3343)
>[junit4]>at 
> org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1134)
>[junit4]>at 
> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1179)
>[junit4]>at 
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.close(AnalyzingInfixSuggester.java:843)
>[junit4]>at 
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest.testRandomNRT(AnalyzingInfixSuggesterTest.java:629)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
>[junit4]   2> NOTE: leaving temporary files on disk at: 
> /home/sarowe/git/lucene-solr-3/lucene/build/suggest/test/J0/temp/lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest_E4D62AFBD9F5FF7F-003
>[junit4]   2> NOTE: test params are: codec=SimpleText, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=es-BO, timezone=Africa/Asmara
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_77 (64-bit)/cpus=16,threads=1,free=435092720,total=514850816
>[junit4]   2> NOTE: All tests run in this JVM: 
> [AnalyzingInfixSuggesterTest]
>[junit4] Completed [1/1 (1!)] in 1.65s, 22 tests, 1 error <<< FAILURES!
> {noformat}
> {noformat}
>[junit4] Suite: 
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=AnalyzingInfixSuggesterTest -Dtests.method=testRandomNRT 
> -Dtests.seed=C9621DF023FA10B9 -Dtests.slow=true -Dtests.locale=ar-SA 
> -Dtests.timezone=Etc/Zulu -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.11s | AnalyzingInfixSuggesterTest.testRandomNRT <<<
>[junit4]> Throwable #1: java.lang.ClassCastException: 
> org.apache.lucene.util.FixedBitSet cannot be cast to 
> org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat$SimpleTextBits
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C9621DF023FA10B9:6D4C134D7B25CC05]:0)
>[junit4]>at 
> org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.writeLiveDocs(SimpleTextLiveDocsFormat.java:111)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:570)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:481)
>[junit4]>at 
> 

[jira] [Updated] (LUCENE-7564) AnalyzingInfixSuggester should close its IndexWriter by default at the end of build()

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe updated LUCENE-7564:
---
Attachment: LUCENE-7564-fix-random-NRT-failures.patch

After several thousand beasting iterations, I still haven't been able to 
reproduce the {{testRandomNRT()}} failures.

I'm attaching a patch that I think will address the problem, which AFAICT is 
that under multi-threaded use, in {{lookup()}}, the suggester’s {{searcherMgr}} 
data member could be re-assigned in the middle of a searcher acquire/release 
cycle due to the changes introduced in LUCENE-7564 - as a result these could be 
imbalanced, with {{acquire()}} called on one instance and {{release()}} called 
on a different one.  The patch takes a local {{SearcherManager}} reference and 
calls {{acquire()}}/{{release()}} though it in every place these are called in 
{{AnalyzingInfixSuggester}} (rather than calling through the {{searcherMgr}} 
data member).

I'm beasting {{AnalyzingInfixSuggester}} with this patch, and I'll commit later 
today if there are no failures.

> AnalyzingInfixSuggester should close its IndexWriter by default at the end of 
> build()
> -
>
> Key: LUCENE-7564
> URL: https://issues.apache.org/jira/browse/LUCENE-7564
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: master (7.0), 6.4
>
> Attachments: LUCENE-7564-fix-random-NRT-failures.patch, 
> LUCENE-7564.patch, LUCENE-7564.patch
>
>
> From SOLR-6246, where AnalyzingInfixSuggester's write lock on its index is 
> causing trouble when reloading a Solr core:
> [~gsingers] wrote:
> bq. One suggestion that might minimize the impact: close the writer after 
> build
> [~varunthacker] wrote:
> {quote}
> This is what I am thinking -
> Create a Lucene issue in which {{AnalyzingInfixSuggester#build}} closes the 
> writer by default at the end.
> The {{add}} and {{update}} methods call {{ensureOpen}} and those who do 
> frequent real time updates directly via lucene won't see any slowdowns.
> [~mikemccand] - Would this approach have any major drawback from Lucene's 
> perspective? Else I can go ahead an tackle this in a Lucene issue
> {quote}
> [~mikemccand] wrote:
> bq. Fixing {{AnalyzingInfixSuggester}} to close the writer at the end of 
> build seems reasonable?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7605) AnalyzingInfixSuggesterTest.testRandomNRT() failures: FixedBitSet cannot be cast to SimpleTextBits

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783525#comment-15783525
 ] 

ASF subversion and git services commented on LUCENE-7605:
-

Commit dc6dcdda8078eb9f100fd6c66b5d488d057b019b in lucene-solr's branch 
refs/heads/master from [~jpountz]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=dc6dcdd ]

LUCENE-7605: Use codec-specific impl of live docs when sorting.


> AnalyzingInfixSuggesterTest.testRandomNRT() failures: FixedBitSet cannot be 
> cast to SimpleTextBits
> --
>
> Key: LUCENE-7605
> URL: https://issues.apache.org/jira/browse/LUCENE-7605
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/codecs, modules/suggest
>Reporter: Steve Rowe
> Attachments: LUCENE-7605.patch
>
>
> While beasting for LUCENE-7564, I found these reproducing master seeds, which 
> trigger casting failures when SimpleText codec is randomly chosen:
> {noformat}
>[junit4] Suite: 
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=AnalyzingInfixSuggesterTest -Dtests.method=testRandomNRT 
> -Dtests.seed=E4D62AFBD9F5FF7F -Dtests.slow=true -Dtests.locale=es-BO 
> -Dtests.timezone=Africa/Asmara -Dtests.asserts=true 
> -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.18s | AnalyzingInfixSuggesterTest.testRandomNRT <<<
>[junit4]> Throwable #1: java.lang.ClassCastException: 
> org.apache.lucene.util.FixedBitSet cannot be cast to 
> org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat$SimpleTextBits
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([E4D62AFBD9F5FF7F:40F82446812A23C3]:0)
>[junit4]>at 
> org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.writeLiveDocs(SimpleTextLiveDocsFormat.java:111)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriterPerThread.sealFlushedSegment(DocumentsWriterPerThread.java:570)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:481)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:539)
>[junit4]>at 
> org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:653)
>[junit4]>at 
> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3368)
>[junit4]>at 
> org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3343)
>[junit4]>at 
> org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1134)
>[junit4]>at 
> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1179)
>[junit4]>at 
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.close(AnalyzingInfixSuggester.java:843)
>[junit4]>at 
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest.testRandomNRT(AnalyzingInfixSuggesterTest.java:629)
>[junit4]>at java.lang.Thread.run(Thread.java:745)
>[junit4]   2> NOTE: leaving temporary files on disk at: 
> /home/sarowe/git/lucene-solr-3/lucene/build/suggest/test/J0/temp/lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest_E4D62AFBD9F5FF7F-003
>[junit4]   2> NOTE: test params are: codec=SimpleText, 
> sim=RandomSimilarity(queryNorm=true): {}, locale=es-BO, timezone=Africa/Asmara
>[junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 
> 1.8.0_77 (64-bit)/cpus=16,threads=1,free=435092720,total=514850816
>[junit4]   2> NOTE: All tests run in this JVM: 
> [AnalyzingInfixSuggesterTest]
>[junit4] Completed [1/1 (1!)] in 1.65s, 22 tests, 1 error <<< FAILURES!
> {noformat}
> {noformat}
>[junit4] Suite: 
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest
>[junit4]   2> NOTE: reproduce with: ant test  
> -Dtestcase=AnalyzingInfixSuggesterTest -Dtests.method=testRandomNRT 
> -Dtests.seed=C9621DF023FA10B9 -Dtests.slow=true -Dtests.locale=ar-SA 
> -Dtests.timezone=Etc/Zulu -Dtests.asserts=true -Dtests.file.encoding=UTF-8
>[junit4] ERROR   0.11s | AnalyzingInfixSuggesterTest.testRandomNRT <<<
>[junit4]> Throwable #1: java.lang.ClassCastException: 
> org.apache.lucene.util.FixedBitSet cannot be cast to 
> org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat$SimpleTextBits
>[junit4]>at 
> __randomizedtesting.SeedInfo.seed([C9621DF023FA10B9:6D4C134D7B25CC05]:0)
>[junit4]>at 
> org.apache.lucene.codecs.simpletext.SimpleTextLiveDocsFormat.writeLiveDocs(SimpleTextLiveDocsFormat.java:111)
>[junit4]>at 
> 

[jira] [Commented] (LUCENE-5325) Move ValueSource and FunctionValues under core/

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783518#comment-15783518
 ] 

Adrien Grand commented on LUCENE-5325:
--

Thanks, it looks great. Some minor comments:
 - Can you make the docs of advanceExact more explicit about the fact it cannot 
go backwards?
 - Can you remove the {{@Test}} annotations from the test case?
 - The {{// force in order}} comment is outdated, scoring always goes in order 
since 5.0. I think we can remove all this testing on BooleanQuery since it is 
now no different from a TermQuery for sorting.

Otherwise +1

> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch, 
> LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs 
> which exist under the queries/ module. That causes any module which wants to 
> depend on these APIs (but not necessarily on any of their actual 
> implementations!), to depend on the queries/ module. If we move these APIs 
> under core/, we can eliminate these dependencies and add some mock impls for 
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the 
> expressions/ on queries, the grouping/ dependency on queries, the spatial/ 
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Reopened] (LUCENE-7564) AnalyzingInfixSuggester should close its IndexWriter by default at the end of build()

[Steve Rowe (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Rowe reopened LUCENE-7564:


Reopening to address random NRT Jenkins failures.

> AnalyzingInfixSuggester should close its IndexWriter by default at the end of 
> build()
> -
>
> Key: LUCENE-7564
> URL: https://issues.apache.org/jira/browse/LUCENE-7564
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Steve Rowe
>Assignee: Steve Rowe
> Fix For: master (7.0), 6.4
>
> Attachments: LUCENE-7564.patch, LUCENE-7564.patch
>
>
> From SOLR-6246, where AnalyzingInfixSuggester's write lock on its index is 
> causing trouble when reloading a Solr core:
> [~gsingers] wrote:
> bq. One suggestion that might minimize the impact: close the writer after 
> build
> [~varunthacker] wrote:
> {quote}
> This is what I am thinking -
> Create a Lucene issue in which {{AnalyzingInfixSuggester#build}} closes the 
> writer by default at the end.
> The {{add}} and {{update}} methods call {{ensureOpen}} and those who do 
> frequent real time updates directly via lucene won't see any slowdowns.
> [~mikemccand] - Would this approach have any major drawback from Lucene's 
> perspective? Else I can go ahead an tackle this in a Lucene issue
> {quote}
> [~mikemccand] wrote:
> bq. Fixing {{AnalyzingInfixSuggester}} to close the writer at the end of 
> build seems reasonable?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7607) LeafFieldComparator.setScorer() should throw IOException

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783492#comment-15783492
 ] 

Adrien Grand commented on LUCENE-7607:
--

+1

> LeafFieldComparator.setScorer() should throw IOException
> 
>
> Key: LUCENE-7607
> URL: https://issues.apache.org/jira/browse/LUCENE-7607
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Attachments: LUCENE-7607.patch
>
>
> Spinoff of LUCENE-5325.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7603) Support Graph Token Streams in QueryBuilder

[Matt Weber (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783462#comment-15783462
 ] 

Matt Weber commented on LUCENE-7603:


Thanks for reviewing [~mikemccand]!  I have added the missing ASF header and 
moved the test into the proper package.  I have also backported this to 6x and 
opened a new PR for that.  The only difference in the 6x backport is disabling 
coord on the rewritten boolean query.  Some of the tests are slightly different 
as well due to the fact that splitOnWhitespace defaults to true in 6x.

Please let me know if you need me to change anything!

> Support Graph Token Streams in QueryBuilder
> ---
>
> Key: LUCENE-7603
> URL: https://issues.apache.org/jira/browse/LUCENE-7603
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/queryparser, core/search
>Reporter: Matt Weber
>
> With [LUCENE-6664|https://issues.apache.org/jira/browse/LUCENE-6664] we can 
> use multi-term synonyms query time.  A "graph token stream" will be created 
> which which is nothing more than using the position length attribute on 
> stacked tokens to indicate how many positions a token should span.  Currently 
> the position length attribute on tokens is ignored during query parsing.  
> This issue will add support for handling these graph token streams inside the 
> QueryBuilder utility class used by query parsers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7603) Support Graph Token Streams in QueryBuilder

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783452#comment-15783452
 ] 

ASF GitHub Bot commented on LUCENE-7603:


GitHub user mattweber opened a pull request:

https://github.com/apache/lucene-solr/pull/130

LUCENE-7603: branch_6x Support Graph Token Streams in QueryBuilder

Adds support for handling graph token streams inside the
QueryBuilder util class used by query parsers.  This is a backport to 
`branch_6x`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattweber/lucene-solr LUCENE-7603_6x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #130


commit 7ab1638a3da002113e939b316d04ef9e58e38a0b
Author: Matt Weber 
Date:   2016-12-26T15:50:58Z

Support Graph Token Streams in QueryBuilder

Adds support for handling graph token streams inside the
QueryBuilder util class used by query parsers.




> Support Graph Token Streams in QueryBuilder
> ---
>
> Key: LUCENE-7603
> URL: https://issues.apache.org/jira/browse/LUCENE-7603
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/queryparser, core/search
>Reporter: Matt Weber
>
> With [LUCENE-6664|https://issues.apache.org/jira/browse/LUCENE-6664] we can 
> use multi-term synonyms query time.  A "graph token stream" will be created 
> which which is nothing more than using the position length attribute on 
> stacked tokens to indicate how many positions a token should span.  Currently 
> the position length attribute on tokens is ignored during query parsing.  
> This issue will add support for handling these graph token streams inside the 
> QueryBuilder utility class used by query parsers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #130: LUCENE-7603: branch_6x Support Graph Token St...

[mattweber]

GitHub user mattweber opened a pull request:

https://github.com/apache/lucene-solr/pull/130

LUCENE-7603: branch_6x Support Graph Token Streams in QueryBuilder

Adds support for handling graph token streams inside the
QueryBuilder util class used by query parsers.  This is a backport to 
`branch_6x`.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mattweber/lucene-solr LUCENE-7603_6x

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/lucene-solr/pull/130.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #130


commit 7ab1638a3da002113e939b316d04ef9e58e38a0b
Author: Matt Weber 
Date:   2016-12-26T15:50:58Z

Support Graph Token Streams in QueryBuilder

Adds support for handling graph token streams inside the
QueryBuilder util class used by query parsers.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1040 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1040/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
[SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:35024//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:35024//collection1/] expected:<0> but was:<2>

Stack Trace:
java.lang.AssertionError: [SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: https://127.0.0.1:35024//collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: https://127.0.0.1:35024//collection1/] expected:<0> but was:<2>
at 
__randomizedtesting.SeedInfo.seed([8A8070009CF638F5:2D44FDA320A550D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (LUCENE-7603) Support Graph Token Streams in QueryBuilder

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783425#comment-15783425
 ] 

ASF GitHub Bot commented on LUCENE-7603:


Github user mattweber commented on the issue:

https://github.com/apache/lucene-solr/pull/129
  
Rebased against master, added missing ASF header.


> Support Graph Token Streams in QueryBuilder
> ---
>
> Key: LUCENE-7603
> URL: https://issues.apache.org/jira/browse/LUCENE-7603
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/queryparser, core/search
>Reporter: Matt Weber
>
> With [LUCENE-6664|https://issues.apache.org/jira/browse/LUCENE-6664] we can 
> use multi-term synonyms query time.  A "graph token stream" will be created 
> which which is nothing more than using the position length attribute on 
> stacked tokens to indicate how many positions a token should span.  Currently 
> the position length attribute on tokens is ignored during query parsing.  
> This issue will add support for handling these graph token streams inside the 
> QueryBuilder utility class used by query parsers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr issue #129: LUCENE-7603: Support Graph Token Streams in QueryBui...

[mattweber]

Github user mattweber commented on the issue:

https://github.com/apache/lucene-solr/pull/129
  
Rebased against master, added missing ASF header.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7607) LeafFieldComparator.setScorer() should throw IOException

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-7607:
--
Attachment: LUCENE-7607.patch

Trivial patch.  Literally no other changes need to be made anywhere in the 
codebase other than adding the throws statement to the method declaration in 
LeafFieldComparator.

> LeafFieldComparator.setScorer() should throw IOException
> 
>
> Key: LUCENE-7607
> URL: https://issues.apache.org/jira/browse/LUCENE-7607
> Project: Lucene - Core
>  Issue Type: Improvement
>Reporter: Alan Woodward
>Assignee: Alan Woodward
>Priority: Minor
> Attachments: LUCENE-7607.patch
>
>
> Spinoff of LUCENE-5325.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (LUCENE-7607) LeafFieldComparator.setScorer() should throw IOException

[Alan Woodward (JIRA)]

Alan Woodward created LUCENE-7607:
-

 Summary: LeafFieldComparator.setScorer() should throw IOException
 Key: LUCENE-7607
 URL: https://issues.apache.org/jira/browse/LUCENE-7607
 Project: Lucene - Core
  Issue Type: Improvement
Reporter: Alan Woodward
Assignee: Alan Woodward
Priority: Minor


Spinoff of LUCENE-5325.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-5325) Move ValueSource and FunctionValues under core/

[Alan Woodward (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Woodward updated LUCENE-5325:
--
Attachment: LUCENE-5325.patch

Thanks for the review, Adrien.  Here's an updated patch:

* advanceTo renamed to advanceExact
* fromIntField delegates to fromLongField in both Long and DoubleValuesSource
* I ended up removing the automatic wrapping of doubles to longs in 
LongValuesSource entirely, in favour of calling 
DoubleValuesSource.toLongValuesSource().  The javadocs on the latter now 
explicitly say that the conversion is done via casting.
* Javadocs generally are more explanatory, particularly around scores
* getValues() now takes a DoubleValues object rather than a Scorer - this ended 
up being a very nice change, and will allow us to remove several of the 
FakeScorer implementations that the current ValueSource API forces us to use in 
things like expressions or facets.

I'll open a separate issue for adding the throws clause for 
FieldComparator.setScorer().

> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch, 
> LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs 
> which exist under the queries/ module. That causes any module which wants to 
> depend on these APIs (but not necessarily on any of their actual 
> implementations!), to depend on the queries/ module. If we move these APIs 
> under core/, we can eliminate these dependencies and add some mock impls for 
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the 
> expressions/ on queries, the grouping/ dependency on queries, the spatial/ 
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_112) - Build # 18639 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18639/
Java: 64bit/jdk1.8.0_112 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
[SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: http://127.0.0.1:44504/iop/zl/collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: http://127.0.0.1:44504/iop/zl/collection1/] expected:<0> but 
was:<2>

Stack Trace:
java.lang.AssertionError: [SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=add{,id=(null)}; node=StdNode: http://127.0.0.1:44504/iop/zl/collection1/, 
SolrCmdDistributor$Error: statusCode=-1; 
exception=java.lang.NullPointerException; req=SolrCmdDistributor$Req: 
cmd=commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false};
 node=StdNode: http://127.0.0.1:44504/iop/zl/collection1/] expected:<0> but 
was:<2>
at 
__randomizedtesting.SeedInfo.seed([BF5DB1A8DBA00E9:83A1E4C023466D11]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:165)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Commented] (SOLR-9824) Documents indexed in bulk are replicated using too many HTTP requests

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783139#comment-15783139
 ] 

David Smiley commented on SOLR-9824:


Ok.  If you feel sufficiently comfortable with this patch to commit it then go 
for it.  I didn't review the details of this patch iteration because it's a 
PITA without a tool like GitHub.  It's an improvement... time will tell if 
there are bugs.

BTW I noticed a minor typo... the test uses the word "negligent" when I think 
"negligible" is intended.

Thanks for working on this issue Mark.  The 6.4 release is going to be awesome.

> Documents indexed in bulk are replicated using too many HTTP requests
> -
>
> Key: SOLR-9824
> URL: https://issues.apache.org/jira/browse/SOLR-9824
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.3
>Reporter: David Smiley
>Assignee: Mark Miller
> Attachments: SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch, 
> SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch
>
>
> This takes awhile to explain; bear with me. While working on bulk indexing 
> small documents, I looked at the logs of my SolrCloud nodes.  I noticed that 
> shards would see an /update log message every ~6ms which is *way* too much.  
> These are requests from one shard (that isn't a leader/replica for these docs 
> but the recipient from my client) to the target shard leader (no additional 
> replicas).  One might ask why I'm not sending docs to the right shard in the 
> first place; I have a reason but it's besides the point -- there's a real 
> Solr perf problem here and this probably applies equally to 
> replicationFactor>1 situations too.  I could turn off the logs but that would 
> hide useful stuff, and it's disconcerting to me that so many short-lived HTTP 
> requests are happening, somehow at the bequest of DistributedUpdateProcessor. 
>  After lots of analysis and debugging and hair pulling, I finally figured it 
> out.  
> In SOLR-7333 ([~tpot]) introduced an optimization called 
> {{UpdateRequest.isLastDocInBatch()}} in which ConcurrentUpdateSolrClient will 
> poll with a '0' timeout to the internal queue, so that it can close the 
> connection without it hanging around any longer than needed.  This part makes 
> sense to me.  Currently the only spot that has the smarts to set this flag is 
> {{JavaBinUpdateRequestCodec.unmarshal.readOuterMostDocIterator()}} at the 
> last document.  So if a shard received docs in a javabin stream (but not 
> other formats) one would expect the _last_ document to have this flag.  
> There's even a test.  Docs without this flag get the default poll time; for 
> javabin it's 25ms.  Okay.
> I _suspect_ that if someone used CloudSolrClient or HttpSolrClient to send 
> javabin data in a batch, the intended efficiencies of SOLR-7333 would apply.  
> I didn't try. In my case, I'm using ConcurrentUpdateSolrClient (and BTW 
> DistributedUpdateProcessor uses CUSC too).  CUSC uses the RequestWriter 
> (defaulting to javabin) to send each document separately without any leading 
> marker or trailing marker.  For the XML format by comparison, there is a 
> leading and trailing marker ( ... ).  Since there's no outer 
> container for the javabin unmarshalling to detect the last document, it marks 
> _every_ document as {{req.lastDocInBatch()}}!  Ouch!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9620) {!join score=.. fromIndex=..} throws "undefined field" for numeric field if from and to schemas are different

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev reassigned SOLR-9620:
--

Assignee: Mikhail Khludnev

> {!join score=.. fromIndex=..} throws "undefined field" for numeric field if 
> from and to schemas are different 
> --
>
> Key: SOLR-9620
> URL: https://issues.apache.org/jira/browse/SOLR-9620
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.3
>Reporter: Mikhail Khludnev
>Assignee: Mikhail Khludnev
> Attachments: SOLR-9620.patch
>
>
> numeric "from" field is mistakenly looked in "to" schema. see 
> org.apache.solr.search.join.ScoreJoinQParserPlugin.parse()
> {code}
>private Query createQuery(final String fromField, final String 
> fromQueryStr,
>  String fromIndex, final String toField, 
> final ScoreMode scoreMode,
>  boolean byPassShortCircutCheck) throws 
> SyntaxError {
> +FieldType.LegacyNumericType fromNumericType = 
> req.getSchema().getField(fromField).getType().getNumericType();
> +FieldType.LegacyNumericType toNumericType = 
> req.getSchema().getField(toField).getType().getNumericType();
> {code}
> it's in branch_6x only. Users who are faced this are advised to just declare 
> "from" field in "to" schema. It should work. Take care.  
> one line fix and improvements for TestCrossCoreJoin.java are quite welcome. 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9620) {!join score=.. fromIndex=..} throws "undefined field" for numeric field if from and to schemas are different

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9620:
---
Attachment: SOLR-9620.patch

> {!join score=.. fromIndex=..} throws "undefined field" for numeric field if 
> from and to schemas are different 
> --
>
> Key: SOLR-9620
> URL: https://issues.apache.org/jira/browse/SOLR-9620
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.3
>Reporter: Mikhail Khludnev
> Attachments: SOLR-9620.patch
>
>
> numeric "from" field is mistakenly looked in "to" schema. see 
> org.apache.solr.search.join.ScoreJoinQParserPlugin.parse()
> {code}
>private Query createQuery(final String fromField, final String 
> fromQueryStr,
>  String fromIndex, final String toField, 
> final ScoreMode scoreMode,
>  boolean byPassShortCircutCheck) throws 
> SyntaxError {
> +FieldType.LegacyNumericType fromNumericType = 
> req.getSchema().getField(fromField).getType().getNumericType();
> +FieldType.LegacyNumericType toNumericType = 
> req.getSchema().getField(toField).getType().getNumericType();
> {code}
> it's in branch_6x only. Users who are faced this are advised to just declare 
> "from" field in "to" schema. It should work. Take care.  
> one line fix and improvements for TestCrossCoreJoin.java are quite welcome. 
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-9856) Collect metrics for shard replication and tlog replay on replicas

[Andrzej Bialecki (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrzej Bialecki  reassigned SOLR-9856:
---

Assignee: Andrzej Bialecki 

> Collect metrics for shard replication and tlog replay on replicas
> -
>
> Key: SOLR-9856
> URL: https://issues.apache.org/jira/browse/SOLR-9856
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Affects Versions: master (7.0)
>Reporter: Andrzej Bialecki 
>Assignee: Andrzej Bialecki 
>Priority: Minor
> Fix For: master (7.0), 6.4
>
>
> Using API from SOLR-4735 add metrics for tracking outgoing replication from 
> leader to shard replicas, and for tracking transaction log processing on 
> replicas.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: Write buffering updates to temporary files?

[David Smiley]

Sure!

> On Dec 28, 2016, at 9:59 AM, Đạt Cao Mạnh  wrote:
> 
> Do you think we should raise an issue for this problem?
> 
> On Wed, Dec 28, 2016 at 12:58 PM David Smiley  > wrote:
> I agree that using a single tlog for 2 purposes is confusing.  Perhaps a 
> separate tlog purely for buffering purposes during recovery/peer-sync etc. be 
> clearer?  You said "another separate file" but I imagine we can use our tlog 
> code but another instance pointed to another directory.
> 
> FWIW I actually question if buffering should happen at all due to the 
> complexity it brings (e.g. SOLR-8030) vs blocking then failing if blocked for 
> too long... but I guess that ship has sailed.
> 
> On Tue, Dec 27, 2016 at 5:52 PM Đạt Cao Mạnh  > wrote:
> Currently, we write buffering logs to current tlog and not apply that updates 
> to index. Then we rely on replay log to apply that updates to index. But at 
> the same time there are some updates also write to current tlog and applied 
> to the index. 
> 
> For example, during peersync, if new updates come to replica we will end up 
> with this tlog
> tlog : old1, new1, new2, old2, new3, old3
> old updates belong to peersync, and these updates are applied to the index.
> new updates belong to buffering updates, and these updates are not applied to 
> the index.
> 
> But writing all the updates to same current tlog make code base very complex. 
> Should we write buffering updates to another temporary file?
> 
> By doing this, it will help our code base simpler. It also makes replica 
> recovery for SOLR-9835 more easier. Because after peersync success we can 
> copy new updates from temporary file to current tlog, for example
> tlog : old1, old2, old3
> temporary tlog : new1, new2, new3
> -->
> tlog : old1, old2, old3, new1, new2, new3
> 
> Note that in  SOLR-9835 we can not rely on fingerprint for peersync because 
> updates are not applied to replicas.
> -- 
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley 
>  | Book: 
> http://www.solrenterprisesearchserver.com 
> 

[jira] [Commented] (LUCENE-5325) Move ValueSource and FunctionValues under core/

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783071#comment-15783071
 ] 

Adrien Grand commented on LUCENE-5325:
--

I am +1 in general, I think it's great that we are getting a cleaner values 
source API! I left some comments about the current patch below:

 - Should we rename advanceTo to advanceExact to be more consistent with the 
doc values API? These methods seem to have the same contract.
 - It seems to be that {{fromIntField}} could just delegate to 
{{fromLongField}} all the time?
 - Can we use {{LongUnaryOperator.identity()}} rather than {{(v) -> v}}
 - {{// TODO make setScorer throw IOException?}} +1
 - {{LongValuesSource.fromDoubleField}} looks a bit trappy to me since it 
implicitly casts. Should we instead expect callers to call 
{{DoubleValuesSource.fromDoubleField}} and then do the casting explicitly using 
{{toLongValuesSource}}?
 - Maybe we should not have the {{DoubleValues getValues(LeafReaderContext)}} 
method (without a Scorer). I'm wondering it might be better to require callers 
to pass null themselves when they know scores are not needed.

About the way we have access to scores, I'm wondering that things might be a 
bit cleaner if we avoided mixing the values source and Scorer APIs by replacing 
{{LongValues getValues(LeafReaderContext ctx, Scorer scorer)}} with 
{{LongValues getValues(LeafReaderContext ctx, DoubleValues scores)}} in 
addition to something like a {{DoubleValues fromScorer(Scorer scorer)}} method 
on {{DoubleValuesSource}}?

In general I think we should also better document the contract of these APIs, 
like what the expectations are for the scorer/scores when {{needsScores}} 
returns {{false}} and what {{toDoubleValues}} and {{toLongValues}} do since not 
all doubles can be represented as a long and vice-versa.

> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch, 
> LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs 
> which exist under the queries/ module. That causes any module which wants to 
> depend on these APIs (but not necessarily on any of their actual 
> implementations!), to depend on the queries/ module. If we move these APIs 
> under core/, we can eliminate these dependencies and add some mock impls for 
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the 
> expressions/ on queries, the grouping/ dependency on queries, the spatial/ 
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk1.8.0) - Build # 3740 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/3740/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.metrics.reporters.SolrGangliaReporterTest.testReporter

Error Message:


Stack Trace:
java.util.ConcurrentModificationException
at 
__randomizedtesting.SeedInfo.seed([5FBC51305B77BE1F:587C07307B2D5A]:0)
at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
at java.util.ArrayList$Itr.next(ArrayList.java:851)
at 
org.apache.solr.metrics.reporters.SolrGangliaReporterTest.testReporter(SolrGangliaReporterTest.java:76)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([5FBC51305B77BE1F:D7E86EEAF58BD3E7]:0)
at 

Re: [JENKINS] Lucene-Solr-Tests-6.x - Build # 628 - Still Unstable

[Adrien Grand]

+1 kudos Steve!

Le mer. 28 déc. 2016 à 10:28, Dawid Weiss  a écrit :

> Tricky indeed, kudos Steve.
>
> D.
>
> On Wed, Dec 28, 2016 at 10:20 AM, Michael McCandless
>  wrote:
> > Phew, tricky!  The patch looks good Steve.  Thanks for digging and
> > beasting and finding a separate crab!
> >
> > Mike McCandless
> >
> > http://blog.mikemccandless.com
> >
> >
> > On Tue, Dec 27, 2016 at 10:00 PM, Steve Rowe  wrote:
> >> Yes, I’ve posted a couple of them on the issue, but they don’t
> reproduce.
> >>
> >> My beasting so far (couple thousand iterations) hasn’t yet reproduced.
> Beasting did find 
> though.
> >>
> >> I have an idea about what might cause the problem: under multi-threaded
> use, in lookup(), the suggester’s SearcherManager could be re-assigned in
> the middle of an acquire/release cycle as a result of <
> https://issues.apache.org/jira/browse/LUCENE-7564>.  I’m also beasting a
> change to take a local reference and use it for the whole cycle.[1]
> >>
> >> --
> >> Steve
> >> www.lucidworks.com
> >>
> >> [1] Patch to hold SearcherManager constant in
> AnalyzingInfixSuggester.lookup():
> >>
> >> ——
> >> diff --git
> a/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java
> b/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java
> >> index 1e98361..c63726f 100644
> >> ---
> a/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java
> >> +++
> b/lucene/suggest/src/java/org/apache/lucene/search/suggest/analyzing/AnalyzingInfixSuggester.java
> >> @@ -642,7 +642,8 @@ public class AnalyzingInfixSuggester extends Lookup
> implements Closeable {
> >>  // only retrieve the first num hits now:
> >>  Collector c2 = new EarlyTerminatingSortingCollector(c, SORT, num);
> >>  List results = null;
> >> -IndexSearcher searcher = searcherMgr.acquire();
> >> +SearcherManager mgr = searcherMgr; // use local reference to
> perform acquire & release on same SearcherManager
> >> +IndexSearcher searcher = mgr.acquire();
> >>  try {
> >>//System.out.println("got searcher=" + searcher);
> >>searcher.search(finalQuery, c2);
> >> @@ -653,7 +654,7 @@ public class AnalyzingInfixSuggester extends Lookup
> implements Closeable {
> >>// hits = searcher.search(query, null, num, SORT);
> >>results = createResults(searcher, hits, num, key, doHighlight,
> matchedTokens, prefixToken);
> >>  } finally {
> >> -  searcherMgr.release(searcher);
> >> +  mgr.release(searcher);
> >>  }
> >>
> >>  //System.out.println((System.currentTimeMillis() - t0) + " msec
> for infix suggest");
> >> ——
> >>
> >>
> >>
> >>> On Dec 26, 2016, at 5:56 PM, Michael McCandless <
> luc...@mikemccandless.com> wrote:
> >>>
> >>> I think (not sure) these failures starting happening after
> >>> https://issues.apache.org/jira/browse/LUCENE-7564 ?
> >>>
> >>> Mike McCandless
> >>>
> >>> http://blog.mikemccandless.com
> >>>
> >>>
> >>> On Mon, Dec 26, 2016 at 4:29 PM, Apache Jenkins Server
> >>>  wrote:
>  Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.x/628/
> 
>  1 tests failed.
>  FAILED:
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest.testRandomNRT
> 
>  Error Message:
>  Captured an uncaught exception in thread: Thread[id=29,
> name=Thread-11, state=RUNNABLE, group=TGRP-AnalyzingInfixSuggesterTest]
> 
>  Stack Trace:
>  com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured
> an uncaught exception in thread: Thread[id=29, name=Thread-11,
> state=RUNNABLE, group=TGRP-AnalyzingInfixSuggesterTest]
> at
> __randomizedtesting.SeedInfo.seed([608DC176393E1CC5:C4A3CFCB61E1C079]:0)
>  Caused by: org.apache.lucene.store.AlreadyClosedException: this
> ReferenceManager is closed
> at __randomizedtesting.SeedInfo.seed([608DC176393E1CC5]:0)
> at
> org.apache.lucene.search.ReferenceManager.acquire(ReferenceManager.java:98)
> at
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:645)
> at
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggester.lookup(AnalyzingInfixSuggester.java:457)
> at
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest$LookupThread.run(AnalyzingInfixSuggesterTest.java:533)
> 
> 
> 
> 
>  Build Log:
>  [...truncated 9362 lines...]
>    [junit4] Suite:
> org.apache.lucene.search.suggest.analyzing.AnalyzingInfixSuggesterTest
>    [junit4]   2> ??? 26, 2016 1:30:30 ??
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
>    [junit4]   2> WARNING: Uncaught exception in thread:
> 

Re: Write buffering updates to temporary files?

[Đạt Cao Mạnh]

Do you think we should raise an issue for this problem?

On Wed, Dec 28, 2016 at 12:58 PM David Smiley 
wrote:

> I agree that using a single tlog for 2 purposes is confusing.  Perhaps a
> separate tlog purely for buffering purposes during recovery/peer-sync etc.
> be clearer?  You said "another separate file" but I imagine we can use our
> tlog code but another instance pointed to another directory.
>
> FWIW I actually question if buffering should happen at all due to the
> complexity it brings (e.g. SOLR-8030) vs blocking then failing if blocked
> for too long... but I guess that ship has sailed.
>
> On Tue, Dec 27, 2016 at 5:52 PM Đạt Cao Mạnh 
> wrote:
>
> Currently, we write buffering logs to current tlog and not apply that
> updates to index. Then we rely on replay log to apply that updates to
> index. But at the same time there are some updates also write to current
> tlog and applied to the index.
>
> For example, during peersync, if new updates come to replica we will end
> up with this tlog
> tlog : old1, new1, new2, old2, new3, old3
> old updates belong to peersync, and these updates are applied to the index.
> new updates belong to buffering updates, and these updates are not applied
> to the index.
>
> But writing all the updates to same current tlog make code base very
> complex. Should we write buffering updates to another temporary file?
>
> By doing this, it will help our code base simpler. It also makes replica
> recovery for SOLR-9835 more easier. Because after peersync success we can
> copy new updates from temporary file to current tlog, for example
> tlog : old1, old2, old3
> temporary tlog : new1, new2, new3
> -->
> tlog : old1, old2, old3, new1, new2, new3
>
> Note that in  SOLR-9835 we can not rely on fingerprint for peersync
> because updates are not applied to replicas.
>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
> http://www.solrenterprisesearchserver.com
>

[jira] [Commented] (SOLR-9442) Add json.nl=arrnvp (array of NamedValuePair) style in JSONResponseWriter

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783042#comment-15783042
 ] 

ASF subversion and git services commented on SOLR-9442:
---

Commit 73e50ceceb836421a176fe0fa2098a20b3c8a304 in lucene-solr's branch 
refs/heads/branch_6x from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=73e50ce ]

SOLR-9787, SOLR-9442: Replace json.nl=arrnvp with json.nl=arrntv (array of Name 
Type Value) style in JSONResponseWriter


> Add json.nl=arrnvp (array of NamedValuePair) style in JSONResponseWriter
> 
>
> Key: SOLR-9442
> URL: https://issues.apache.org/jira/browse/SOLR-9442
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers
>Reporter: Jonny Marks
>Assignee: Christine Poerschke
>Priority: Minor
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9442-arrntv.patch, SOLR-9442.patch, 
> SOLR-9442.patch, SOLR-9442.patch
>
>
> The JSONResponseWriter class currently supports several styles of NamedList 
> output format, documented on the wiki at http://wiki.apache.org/solr/SolJSON 
> and in the code at 
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/response/JSONResponseWriter.java#L71-L76.
> For example the 'arrmap' style:
> {code}NamedList("a"=1,"b"=2,null=3) => [{"a":1},{"b":2},3]
> NamedList("a"=1,"bar”=“foo",null=3.4f) => [{"a":1},{"bar”:”foo"},{3.4}]{code}
> This patch creates a new style ‘arrnvp’ which is an array of named value 
> pairs. For example:
> {code}NamedList("a"=1,"b"=2,null=3) => 
> [{"name":"a","int":1},{"name":"b","int":2},{"int":3}]
> NamedList("a"=1,"bar”=“foo",null=3.4f) => 
> [{"name":"a","int":1},{"name":"b","str":"foo"},{"float":3.4}]{code}
> This style maintains the type information of the values, similar to the xml 
> format:
> {code:xml}
>   1
>   foo
>   3.4
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9787) Replace json.nl=arrnvp with json.nl=arrntv (array of Name Type Value) style in JSONResponseWriter

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15783041#comment-15783041
 ] 

ASF subversion and git services commented on SOLR-9787:
---

Commit 73e50ceceb836421a176fe0fa2098a20b3c8a304 in lucene-solr's branch 
refs/heads/branch_6x from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=73e50ce ]

SOLR-9787, SOLR-9442: Replace json.nl=arrnvp with json.nl=arrntv (array of Name 
Type Value) style in JSONResponseWriter


> Replace json.nl=arrnvp with json.nl=arrntv (array of Name Type Value) style 
> in JSONResponseWriter
> -
>
> Key: SOLR-9787
> URL: https://issues.apache.org/jira/browse/SOLR-9787
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9787.patch, SOLR-9787.patch
>
>
> This follows on from and builds upon SOLR-9442's addition of json.nl=arrnvp 
> style. See 
> https://issues.apache.org/jira/browse/SOLR-9442?focusedCommentId=15664719=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15664719
>  onwards for background info.
> Example:
> {code}
> NamedList("a"=1,"bar”=“foo",null=3.4f,null=null)
> =>
> [
>   { "name":"a",   "type":"int",   "value":1 },
>   { "name":"bar", "type":"str",   "value":"foo" },
>   { "name":null,  "type":"float", "value":3.4   },
>   { "name":null,  "type":"null",  "value":null  }
> ]
> {code}
> This style maintains the type information of the values, similar to the xml 
> format:
> {code}
> 
>   1
>   foo
>   3.4
>   
> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS-EA] Lucene-Solr-6.x-Linux (64bit/jdk-9-ea+147) - Build # 2532 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/2532/
Java: 64bit/jdk-9-ea+147 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI

Error Message:
Error from server at https://127.0.0.1:41295/solr/awhollynewcollection_0: 
Expected mime type application/octet-stream but got text/html.   
 
Error 510HTTP ERROR: 510 Problem 
accessing /solr/awhollynewcollection_0/select. Reason: 
{metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg={awhollynewcollection_0:5},code=510}
 http://eclipse.org/jetty;>Powered by Jetty:// 
9.3.14.v20161028   

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:41295/solr/awhollynewcollection_0: Expected 
mime type application/octet-stream but got text/html. 


Error 510 


HTTP ERROR: 510
Problem accessing /solr/awhollynewcollection_0/select. Reason:

{metadata={error-class=org.apache.solr.common.SolrException,root-error-class=org.apache.solr.common.SolrException},msg={awhollynewcollection_0:5},code=510}
http://eclipse.org/jetty;>Powered by Jetty:// 
9.3.14.v20161028



at 
__randomizedtesting.SeedInfo.seed([31FBC57DD31D8ED4:798EB1C9D52EA141]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:578)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:435)
at 
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:387)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1344)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1095)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1198)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1198)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1198)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1198)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1198)
at 
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1037)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at 
org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:516)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:538)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 

[jira] [Commented] (SOLR-9241) Rebalance API for SolrCloud

[Yago Riveiro (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782991#comment-15782991
 ] 

Yago Riveiro commented on SOLR-9241:


Issue SOLR-9322, the RESHARD command.

> Rebalance API for SolrCloud
> ---
>
> Key: SOLR-9241
> URL: https://issues.apache.org/jira/browse/SOLR-9241
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 6.1
> Environment: Ubuntu, Mac OsX
>Reporter: Nitin Sharma
>  Labels: Cluster, SolrCloud
> Fix For: 6.1
>
> Attachments: Redistribute_After.jpeg, Redistribute_Before.jpeg, 
> Redistribute_call.jpeg, Replace_After.jpeg, Replace_Before.jpeg, 
> Replace_Call.jpeg, SOLR-9241-4.6.patch, SOLR-9241-6.1.patch
>
>   Original Estimate: 2,016h
>  Remaining Estimate: 2,016h
>
> This is the v1 of the patch for Solrcloud Rebalance api (as described in 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/) , built at 
> Bloomreach by Nitin Sharma and Suruchi Shah. The goal of the API  is to 
> provide a zero downtime mechanism to perform data manipulation and  efficient 
> core allocation in solrcloud. This API was envisioned to be the base layer 
> that enables Solrcloud to be an auto scaling platform. (and work in unison 
> with other complementing monitoring and scaling features).
> Patch Status:
> ===
> The patch is work in progress and incremental. We have done a few rounds of 
> code clean up. We wanted to get the patch going first to get initial feed 
> back.  We will continue to work on making it more open source friendly and 
> easily testable.
>  Deployment Status:
> 
> The platform is deployed in production at bloomreach and has been battle 
> tested for large scale load. (millions of documents and hundreds of 
> collections).
>  Internals:
> =
> The internals of the API and performance : 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/
> It is built on top of the admin collections API as an action (with various 
> flavors). At a high level, the rebalance api provides 2 constructs:
> Scaling Strategy:  Decides how to move the data.  Every flavor has multiple 
> options which can be reviewed in the api spec.
> Re-distribute  - Move around data in the cluster based on capacity/allocation.
> Auto Shard  - Dynamically shard a collection to any size.
> Smart Merge - Distributed Mode - Helps merging data from a larger shard setup 
> into smaller one.  (the source should be divisible by destination)
> Scale up -  Add replicas on the fly
> Scale Down - Remove replicas on the fly
> Allocation Strategy:  Decides where to put the data.  (Nodes with least 
> cores, Nodes that do not have this collection etc). Custom implementations 
> can be built on top as well. One other example is Availability Zone aware. 
> Distribute data such that every replica is placed on different availability 
> zone to support HA.
>  Detailed API Spec:
> 
>   https://github.com/bloomreach/solrcloud-rebalance-api
>  Contributors:
> =
>   Nitin Sharma
>   Suruchi Shah
>  Questions/Comments:
> =
>   You can reach me at nitin.sha...@bloomreach.com



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9241) Rebalance API for SolrCloud

[Noble Paul (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782978#comment-15782978
 ] 

Noble Paul commented on SOLR-9241:
--

This has many sub issues. Which one are you specifically interested in?

> Rebalance API for SolrCloud
> ---
>
> Key: SOLR-9241
> URL: https://issues.apache.org/jira/browse/SOLR-9241
> Project: Solr
>  Issue Type: New Feature
>  Components: SolrCloud
>Affects Versions: 6.1
> Environment: Ubuntu, Mac OsX
>Reporter: Nitin Sharma
>  Labels: Cluster, SolrCloud
> Fix For: 6.1
>
> Attachments: Redistribute_After.jpeg, Redistribute_Before.jpeg, 
> Redistribute_call.jpeg, Replace_After.jpeg, Replace_Before.jpeg, 
> Replace_Call.jpeg, SOLR-9241-4.6.patch, SOLR-9241-6.1.patch
>
>   Original Estimate: 2,016h
>  Remaining Estimate: 2,016h
>
> This is the v1 of the patch for Solrcloud Rebalance api (as described in 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/) , built at 
> Bloomreach by Nitin Sharma and Suruchi Shah. The goal of the API  is to 
> provide a zero downtime mechanism to perform data manipulation and  efficient 
> core allocation in solrcloud. This API was envisioned to be the base layer 
> that enables Solrcloud to be an auto scaling platform. (and work in unison 
> with other complementing monitoring and scaling features).
> Patch Status:
> ===
> The patch is work in progress and incremental. We have done a few rounds of 
> code clean up. We wanted to get the patch going first to get initial feed 
> back.  We will continue to work on making it more open source friendly and 
> easily testable.
>  Deployment Status:
> 
> The platform is deployed in production at bloomreach and has been battle 
> tested for large scale load. (millions of documents and hundreds of 
> collections).
>  Internals:
> =
> The internals of the API and performance : 
> http://engineering.bloomreach.com/solrcloud-rebalance-api/
> It is built on top of the admin collections API as an action (with various 
> flavors). At a high level, the rebalance api provides 2 constructs:
> Scaling Strategy:  Decides how to move the data.  Every flavor has multiple 
> options which can be reviewed in the api spec.
> Re-distribute  - Move around data in the cluster based on capacity/allocation.
> Auto Shard  - Dynamically shard a collection to any size.
> Smart Merge - Distributed Mode - Helps merging data from a larger shard setup 
> into smaller one.  (the source should be divisible by destination)
> Scale up -  Add replicas on the fly
> Scale Down - Remove replicas on the fly
> Allocation Strategy:  Decides where to put the data.  (Nodes with least 
> cores, Nodes that do not have this collection etc). Custom implementations 
> can be built on top as well. One other example is Availability Zone aware. 
> Distribute data such that every replica is placed on different availability 
> zone to support HA.
>  Detailed API Spec:
> 
>   https://github.com/bloomreach/solrcloud-rebalance-api
>  Contributors:
> =
>   Nitin Sharma
>   Suruchi Shah
>  Questions/Comments:
> =
>   You can reach me at nitin.sha...@bloomreach.com



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9898) Documentation for metrics collection and /admin/metrics

[Cassandra Targett (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cassandra Targett updated SOLR-9898:

Component/s: documentation

> Documentation for metrics collection and /admin/metrics
> ---
>
> Key: SOLR-9898
> URL: https://issues.apache.org/jira/browse/SOLR-9898
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: documentation
>Affects Versions: master (7.0), 6.4
>Reporter: Andrzej Bialecki 
>
> Draft documentation follows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9898) Documentation for metrics collection and /admin/metrics

[Andrzej Bialecki (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782953#comment-15782953
 ] 

Andrzej Bialecki  commented on SOLR-9898:
-

h2. Reporters
Reporter configurations are specified in {{solr.xml}} file in 
{{}} sections, for example:
{code}

 
  
graphite-server

60
  
  
300
example
updatesLogger
QUERYHANDLER./update

 
...

{code}
Reporter plugins use the following attributes:
* *name* - (required) unique name of the reporter plugin
* *class* - (required) fully-qualified implementation class of the plugin, must 
extend {{SolrMetricReporter}}
* *group* - (optional) one or more of the predefined groups (see above)
* *registry* - (optional) one or more of valid fully-qualified registry names

If both {{group}} and {{registry}} attributes are specified only the {{group}} 
attribute is considered. If neither attribute is specified then the plugin will 
be used for all groups and registries. Multiple group or registry names can be 
specified, separated by comma and/or space.

Additionally, several implementation-specific initialization arguments can be 
specified in nested elements. There are some arguments that are common to 
SLF4J, Ganglia and Graphite reporters:
* *period* - (optional int) period in seconds between reports. Default value is 
60.
* *prefix* - (optional str) prefix to be added to metric names, may be helpful 
in logical grouping of related Solr instances, eg. machine name or cluster 
name. Default is empty string, ie. just the registry name and metric name will 
be used to form a fully-qualified metric name.
* *filter* - (optional str) if not empty then only metric names that start with 
this value will be reported. Default is no filtering, ie. all metrics from 
selected registry will be reported.

Reporters are instantiated for every group and registry that they were 
configured for, at the time when the respective components are initialized (eg. 
on JVM startup or SolrCore load). When reporters are created their 
configuration is validated (and eg. necessary connections are established). 
Uncaught errors at this initialization stage cause the reporter to be discarded 
from the running configuration. Reporters are closed when the corresponding 
component is being closed (eg. on SolrCore close, or JVM shutdown) but metrics 
that they reported are still maintained in respective registries, as explained 
in the previous section.

The following sections provide information on implementation-specific 
arguments. All implementation classes provided with Solr can be found under 
{{org.apache.solr.metrics.reporters}}.

h3. JMX reporter ({{org.apache.solr.metrics.reporters.SolrJmxReporter}})
* *domain* - (optional str) JMX domain name. If not specified then registry 
name will be used.
* *serviceUrl* - (optional str) service URL for a JMX server. If not specified 
then the default platform MBean server will be used.
* *agentId* - (optional str) agent ID for a JMX server. Note: either 
{{serviceUrl}} or {{agentId}} can be specified but not both - if both are 
specified then the default MBean server will be used.

Object names created by this reporter are hierarchical, dot-separated but also 
properly structured to form corresponding hierarchies in eg. JConsole. This 
hierarchy consists of the following elements in the top-down order:
* registry name (eg. {{solr.core.collection1.shard1.replica1}}. Dot-separated 
registry names are also split into ObjectName hierarchy levels, so that metrics 
for this registry will be shown under 
{{/solr/core/collection1/shard1/replica1}} in JConsole, with each domain part 
being assigned to {{dom1, dom2, ... domN}} property.
* reporter name (the value of reporter's {{name}} attribute)
* category, scope and name for request handlers
* or additional {{name1, name2, ... nameN}} elements for metrics from other 
components.

h3. SLF4J reporter ({{org.apache.solr.metrics.reporters.SolrSlf4jReporter}})
(See also common arguments above)
* *logger* - (optional str) name of the logger to use. Default is empty, in 
which case the group or registry name will be used if specified in the plugin 
configuration.

Users can specify logger name (and the corresponding logger configuration in 
eg. Log4j configuration) to output metrics-related logging to separate file(s), 
which can then be processed by external applications. Each log line produced by 
this reporter consists of configuration-specific fields, and a message that 
follows this format:
{code}
type=COUNTER, name={}, count={}

type=GAUGE, name={}, value={}

type=TIMER, name={}, count={}, min={}, max={}, mean={}, stddev={}, median={}, 
p75={}, p95={}, p98={}, p99={}, p999={}, mean_rate={}, m1={}, m5={}, m15={}, 
rate_unit={}, duration_unit={}

type=METER, name={}, count={}, mean_rate={}, m1={}, m5={}, m15={}, rate_unit={}

type=HISTOGRAM, name={}, count={}, min={}, max={}, 

[jira] [Comment Edited] (SOLR-9636) Add support for javabin for /stream, /sql internode communication

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15781496#comment-15781496
 ] 

Joel Bernstein edited comment on SOLR-9636 at 12/28/16 2:05 PM:


I finally had the time to test out the javabin writer with the /export handler 
and streaming stack. My initial findings are really good. Here is a summary:

1) Currently testing must be done on branch_6x. There is a bug in master which 
breaks the /export handler. I haven't gotten to the bottom of it yet but I'm 
pretty sure it was introduced with the new docValues iterator API which is only 
in master. I will open a ticket for this bug shortly and see if I can fix the 
problem.

But testing in branch_6x is better anyway as it won't be testing the new 
docValues iterator API performance at the same time as the javabin /export 
performance.

2) For my test I worked on a single Solr instance (6g heap) with a single data 
shard (collection1) loaded with 10,000,000 small documents. I also created a 
worker collection with 8 shards (collection2). Then I ran the following 
expression with and without the javabin writer.
{code}
parallel(collection2, workers=8, sort="test_s desc", 
 rollup(over="test_s", sum(price_f),
search(collection1, 
   q=*:*,
   fl="test_s, price_f", 
   sort="test_s desc", 
   qt="/export", 
   wt="javabin", 
   partitionKeys=test_s)))
{code}

Notice that there are 8 parallel workers (collection2) partitioning the stream 
from a single data shard (collection1). This is how you achieve maximum 
throughput from a single node.

3) Throughput numbers were fairly impressive with this test expression:

* With json writer: 1,200,000 Tuples per second.
* With javabin writer: 1,400,000 Tuples per second.

So Javabin gives a significant throughput boost. 

4) Javabin also produced a much smaller output, roughly 50% smaller then json.

5) I also reviewed the code and it looks really nice. Big improvement as far 
cleaning up the integration with Solr. 

6) The core export/sort algorithm looked to be untouched, which was nice 
because there was a lot of hardening on that in the past. My biggest concern 
going into this ticket was that refactoring would cause a change in the 
export/sort algorithm and we'd have go through the hardening all over again. 
But that was not the case.

Very nice work [~noble.paul]! Big improvements and so far I haven't found any 
functional problems. I will continue testing.





was (Author: joel.bernstein):
I finally had the time to test out the javabin writer with the /export handler 
and streaming stack. My initial findings are really good. Here is a summary:

1) Currently testing must be done on branch_6x. There is a bug in master which 
breaks the /export handler. I haven't gotten to the bottom of it yet but I'm 
pretty sure it was introduced with the new docValues iterator API which is only 
in master. I will open a ticket for this bug shortly and see if I can fix the 
problem.

But testing in branch_6x is better anyway as it won't be testing the new 
docValues iterator API performance at the same time as the javabin /export 
performance.

2) For my test I worked on a single Solr instance with a single data shard 
(collection1) loaded with 10,000,000 small documents. I also created a worker 
collection with 5 shards (collection2). Then I ran the following expression 
with and without the javabin writer.
{code}
parallel(collection2, workers=5, sort="test_s desc", 
 rollup(over="test_s", sum(price_f),
search(collection1, 
   q=*:*,
   fl="test_s, price_f", 
   sort="test_s desc", 
   qt="/export", 
   wt="javabin", 
   partitionKeys=test_s)))
{code}

Notice that there are five parallel workers (collection2) partitioning the 
stream from a single data shard (collection1). This is how you achieve maximum 
throughput from a single node.

3) Throughput numbers were fairly impressive with this test expression:

* With json writer: 900,000 Tuples per second.
* With javabin writer: 1,100,000 Tuples per second.

So Javabin gives a significant throughput boost. It's also nice to have an 
example of 1 million+ documents per second from a single node.

4) Javabin also produced a much smaller output, roughly 50% smaller then json.

5) I also reviewed the code and it looks really nice. Big improvement as far 
cleaning up the integration with Solr. 

6) The core export/sort algorithm looked to be untouched, which was nice 
because there was a lot of hardening on that in the past. My biggest concern 
going into this ticket was that refactoring would cause a change in the 
export/sort algorithm and we'd have go through 

[jira] [Commented] (SOLR-9636) Add support for javabin for /stream, /sql internode communication

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782945#comment-15782945
 ] 

Joel Bernstein commented on SOLR-9636:
--

After increasing the heap size for the test Solr instance to 6g I saw a large 
boost in throughput. I'll update the numbers above.

> Add support for javabin for /stream, /sql internode communication
> -
>
> Key: SOLR-9636
> URL: https://issues.apache.org/jira/browse/SOLR-9636
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Noble Paul
>Assignee: Noble Paul
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9636.patch
>
>
> currently it uses json, which is verbose and slow



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9644) MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts properly

[Ere Maijala (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782944#comment-15782944
 ] 

Ere Maijala commented on SOLR-9644:
---

Thanks! Note that the commits from 17th should apply to 6x branch too (or at 
least they did at the time).

> MoreLikeThis parsers SimpleMLTQParser and CloudMLTQParser don't handle boosts 
> properly
> --
>
> Key: SOLR-9644
> URL: https://issues.apache.org/jira/browse/SOLR-9644
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: MoreLikeThis
>Affects Versions: 6.2.1
>Reporter: Ere Maijala
>  Labels: patch
>
> It seems SimpleMLTQParser and CloudMLTQParser should be able to handle boost 
> parameters, but it's not working properly. I'll make a pull request to add 
> tests and fix both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9787) Replace json.nl=arrnvp with json.nl=arrntv (array of Name Type Value) style in JSONResponseWriter

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782938#comment-15782938
 ] 

ASF subversion and git services commented on SOLR-9787:
---

Commit e4ef4239f1b23afb116868e8528f1cd947287bd9 in lucene-solr's branch 
refs/heads/master from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e4ef423 ]

SOLR-9787, SOLR-9442: Replace json.nl=arrnvp with json.nl=arrntv (array of Name 
Type Value) style in JSONResponseWriter


> Replace json.nl=arrnvp with json.nl=arrntv (array of Name Type Value) style 
> in JSONResponseWriter
> -
>
> Key: SOLR-9787
> URL: https://issues.apache.org/jira/browse/SOLR-9787
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-9787.patch, SOLR-9787.patch
>
>
> This follows on from and builds upon SOLR-9442's addition of json.nl=arrnvp 
> style. See 
> https://issues.apache.org/jira/browse/SOLR-9442?focusedCommentId=15664719=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15664719
>  onwards for background info.
> Example:
> {code}
> NamedList("a"=1,"bar”=“foo",null=3.4f,null=null)
> =>
> [
>   { "name":"a",   "type":"int",   "value":1 },
>   { "name":"bar", "type":"str",   "value":"foo" },
>   { "name":null,  "type":"float", "value":3.4   },
>   { "name":null,  "type":"null",  "value":null  }
> ]
> {code}
> This style maintains the type information of the values, similar to the xml 
> format:
> {code}
> 
>   1
>   foo
>   3.4
>   
> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9442) Add json.nl=arrnvp (array of NamedValuePair) style in JSONResponseWriter

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782939#comment-15782939
 ] 

ASF subversion and git services commented on SOLR-9442:
---

Commit e4ef4239f1b23afb116868e8528f1cd947287bd9 in lucene-solr's branch 
refs/heads/master from [~cpoerschke]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e4ef423 ]

SOLR-9787, SOLR-9442: Replace json.nl=arrnvp with json.nl=arrntv (array of Name 
Type Value) style in JSONResponseWriter


> Add json.nl=arrnvp (array of NamedValuePair) style in JSONResponseWriter
> 
>
> Key: SOLR-9442
> URL: https://issues.apache.org/jira/browse/SOLR-9442
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Response Writers
>Reporter: Jonny Marks
>Assignee: Christine Poerschke
>Priority: Minor
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9442-arrntv.patch, SOLR-9442.patch, 
> SOLR-9442.patch, SOLR-9442.patch
>
>
> The JSONResponseWriter class currently supports several styles of NamedList 
> output format, documented on the wiki at http://wiki.apache.org/solr/SolJSON 
> and in the code at 
> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/response/JSONResponseWriter.java#L71-L76.
> For example the 'arrmap' style:
> {code}NamedList("a"=1,"b"=2,null=3) => [{"a":1},{"b":2},3]
> NamedList("a"=1,"bar”=“foo",null=3.4f) => [{"a":1},{"bar”:”foo"},{3.4}]{code}
> This patch creates a new style ‘arrnvp’ which is an array of named value 
> pairs. For example:
> {code}NamedList("a"=1,"b"=2,null=3) => 
> [{"name":"a","int":1},{"name":"b","int":2},{"int":3}]
> NamedList("a"=1,"bar”=“foo",null=3.4f) => 
> [{"name":"a","int":1},{"name":"b","str":"foo"},{"float":3.4}]{code}
> This style maintains the type information of the values, similar to the xml 
> format:
> {code:xml}
>   1
>   foo
>   3.4
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-5325) Move ValueSource and FunctionValues under core/

[Alan Woodward (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782914#comment-15782914
 ] 

Alan Woodward commented on LUCENE-5325:
---

Any comments on the patch?  I'll commit in the next couple of days if 
everybody's happy.

> Move ValueSource and FunctionValues under core/
> ---
>
> Key: LUCENE-5325
> URL: https://issues.apache.org/jira/browse/LUCENE-5325
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/search
>Reporter: Shai Erera
> Attachments: LUCENE-5325.patch, LUCENE-5325.patch, LUCENE-5325.patch, 
> LUCENE-5325.patch, LUCENE-5325.patch
>
>
> Spinoff from LUCENE-5298: ValueSource and FunctionValues are abstract APIs 
> which exist under the queries/ module. That causes any module which wants to 
> depend on these APIs (but not necessarily on any of their actual 
> implementations!), to depend on the queries/ module. If we move these APIs 
> under core/, we can eliminate these dependencies and add some mock impls for 
> testing purposes.
> Quoting Robert from LUCENE-5298:
> {quote}
> we should eliminate the suggest/ dependencies on expressions and queries, the 
> expressions/ on queries, the grouping/ dependency on queries, the spatial/ 
> dependency on queries, its a mess.
> {quote}
> To add to that list, facet/ should not depend on queries too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-9580) Exception while updating statistics

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller resolved SOLR-9580.
---
Resolution: Duplicate
  Assignee: Mark Miller

I have a patch for this in SOLR-9859.

> Exception while updating statistics
> ---
>
> Key: SOLR-9580
> URL: https://issues.apache.org/jira/browse/SOLR-9580
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 6.2
>Reporter: Chris de Kok
>Assignee: Mark Miller
>
> The replication throws a warning after the 2nd time the replicaiton occurs 
> complaining about that te replication.properties already exists.
> WARN true
> IndexFetcher
> Exception while updating statistics
> java.nio.file.FileAlreadyExistsException: 
> /var/local/solr/cores/data/replication.properties
>   at 
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:88)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
>   at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
>   at 
> sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
>   at 
> java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)
>   at java.nio.file.Files.newOutputStream(Files.java:216)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:413)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:409)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.lucene.store.NRTCachingDirectory.createOutput(NRTCachingDirectory.java:157)
>   at 
> org.apache.solr.handler.IndexFetcher.logReplicationTimeAndConfFiles(IndexFetcher.java:681)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:493)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:254)
>   at 
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
>   at 
> org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$2(ReplicationHandler.java:1145)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>   at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_112) - Build # 18638 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18638/
Java: 64bit/jdk1.8.0_112 -XX:-UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
1 thread leaked from SUITE scope at 
org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) 
Thread[id=14136, name=jetty-launcher-2764-thread-2-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] 
at sun.misc.Unsafe.park(Native Method) at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) 
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
 at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
 at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)  
   at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
 at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)  
   at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
 at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
 at 
org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
 at 
org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
 at 
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
 at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)   
  at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE 
scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   1) Thread[id=14136, name=jetty-launcher-2764-thread-2-EventThread, 
state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at 
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at 
org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at 
org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at 
org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at 
org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at 
org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at 
org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at 
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
at __randomizedtesting.SeedInfo.seed([8AC2CD1E048F9288]:0)


FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([8AC2CD1E048F9288:296F2C4AA73FF70]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 

[jira] [Updated] (SOLR-9859) replication.properties cannot be updated after being written and neither replication.properties or index.properties are durable in the face of a crash

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-9859:
--
Attachment: SOLR-9859.patch

Added a test for not being able to write replication statistics after the first 
replication.

> replication.properties cannot be updated after being written and neither 
> replication.properties or index.properties are durable in the face of a crash
> --
>
> Key: SOLR-9859
> URL: https://issues.apache.org/jira/browse/SOLR-9859
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: 5.5.3, 6.3
>Reporter: Pushkar Raste
>Assignee: Mark Miller
>Priority: Minor
> Attachments: SOLR-9859.patch, SOLR-9859.patch, SOLR-9859.patch, 
> SOLR-9859.patch, SOLR-9859.patch, SOLR-9859.patch
>
>
> If a shard recovers via replication (vs PeerSync) a file named 
> {{replication.properties}} gets created. If the same shard recovers once more 
> via replication, IndexFetcher fails to write latest replication information 
> as it tries to create {{replication.properties}} but as file already exists. 
> Here is the stack trace I saw 
> {code}
> java.nio.file.FileAlreadyExistsException: 
> \shard-3-001\cores\collection1\data\replication.properties
>   at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
>   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>   at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
>   at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(Unknown Source)
>   at java.nio.file.spi.FileSystemProvider.newOutputStream(Unknown Source)
>   at java.nio.file.Files.newOutputStream(Unknown Source)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:413)
>   at 
> org.apache.lucene.store.FSDirectory$FSIndexOutput.(FSDirectory.java:409)
>   at 
> org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:253)
>   at 
> org.apache.solr.handler.IndexFetcher.logReplicationTimeAndConfFiles(IndexFetcher.java:689)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:501)
>   at 
> org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:265)
>   at 
> org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:397)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:157)
>   at 
> org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
>   at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:222)
>   at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>   at java.util.concurrent.FutureTask.run(Unknown Source)
>   at 
> org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$0(ExecutorUtil.java:229)
>   at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>   at java.lang.Thread.run(Unknown Source)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9877) Use instrumented http client

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782886#comment-15782886
 ] 

ASF subversion and git services commented on SOLR-9877:
---

Commit c2292faaf1f4993bf1cec666f4286ac71f786506 in lucene-solr's branch 
refs/heads/master from [~shalinmangar]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c2292fa ]

SOLR-9877: Remove assertion because many tests use UpdateShardHandler without 
metrics


> Use instrumented http client
> 
>
> Key: SOLR-9877
> URL: https://issues.apache.org/jira/browse/SOLR-9877
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9877.patch, SOLR-9877.patch, solr-http-metrics.png
>
>
> Use instrumented equivalents of PooledHttpClientConnectionManager and others 
> from metrics-httpclient library.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1195 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1195/

7 tests failed.
FAILED:  
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete 
@source_collection:shard1
at 
__randomizedtesting.SeedInfo.seed([EECAAFE38978E3BD:B2D748624758578B]:0)
at 
org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:795)
at 
org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:235)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Updated] (SOLR-9824) Documents indexed in bulk are replicated using too many HTTP requests

[Mark Miller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mark Miller updated SOLR-9824:
--
Attachment: SOLR-9824.patch

Here is my latest patch. Other then the couple changes from your comments, I 
think it has some other minor changes (I think I noticed tighter 
synchronization blocks), but I can't remember as I made those tweaks a while 
back.

> Documents indexed in bulk are replicated using too many HTTP requests
> -
>
> Key: SOLR-9824
> URL: https://issues.apache.org/jira/browse/SOLR-9824
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.3
>Reporter: David Smiley
>Assignee: Mark Miller
> Attachments: SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch, 
> SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch
>
>
> This takes awhile to explain; bear with me. While working on bulk indexing 
> small documents, I looked at the logs of my SolrCloud nodes.  I noticed that 
> shards would see an /update log message every ~6ms which is *way* too much.  
> These are requests from one shard (that isn't a leader/replica for these docs 
> but the recipient from my client) to the target shard leader (no additional 
> replicas).  One might ask why I'm not sending docs to the right shard in the 
> first place; I have a reason but it's besides the point -- there's a real 
> Solr perf problem here and this probably applies equally to 
> replicationFactor>1 situations too.  I could turn off the logs but that would 
> hide useful stuff, and it's disconcerting to me that so many short-lived HTTP 
> requests are happening, somehow at the bequest of DistributedUpdateProcessor. 
>  After lots of analysis and debugging and hair pulling, I finally figured it 
> out.  
> In SOLR-7333 ([~tpot]) introduced an optimization called 
> {{UpdateRequest.isLastDocInBatch()}} in which ConcurrentUpdateSolrClient will 
> poll with a '0' timeout to the internal queue, so that it can close the 
> connection without it hanging around any longer than needed.  This part makes 
> sense to me.  Currently the only spot that has the smarts to set this flag is 
> {{JavaBinUpdateRequestCodec.unmarshal.readOuterMostDocIterator()}} at the 
> last document.  So if a shard received docs in a javabin stream (but not 
> other formats) one would expect the _last_ document to have this flag.  
> There's even a test.  Docs without this flag get the default poll time; for 
> javabin it's 25ms.  Okay.
> I _suspect_ that if someone used CloudSolrClient or HttpSolrClient to send 
> javabin data in a batch, the intended efficiencies of SOLR-7333 would apply.  
> I didn't try. In my case, I'm using ConcurrentUpdateSolrClient (and BTW 
> DistributedUpdateProcessor uses CUSC too).  CUSC uses the RequestWriter 
> (defaulting to javabin) to send each document separately without any leading 
> marker or trailing marker.  For the XML format by comparison, there is a 
> leading and trailing marker ( ... ).  Since there's no outer 
> container for the javabin unmarshalling to detect the last document, it marks 
> _every_ document as {{req.lastDocInBatch()}}!  Ouch!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9824) Documents indexed in bulk are replicated using too many HTTP requests

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782818#comment-15782818
 ] 

Mark Miller commented on SOLR-9824:
---

bq. put that into an else branch. 

I'll do that.

bq. there's a race due to inPoll just being a volatile variable and so it might 
be false and we might not interrupt when we actually wanted to, or vice 
versa... but I suppose it may not be a big issue since the queue is poll'ed 
with timeouts that don't take forever. Adding comments to this effect would be 
good.

Yeah, I don't think it's an issue. Distributed updates does use a very large 
timeout, but our use of blockUntilFinished will loop and interrupt again. We 
should not technically need this right now, but I like that it makes it safe 
for future code additions. For standard use it's really just a best effort to 
cut off any wait. I've done a lot of extensive testing with various update 
rates and update threads and such and have not seen an issue yet.

bq. CUSC

Yonik did almost a rewrite of it not too long ago to fix some bugs, and I don't 
have much appetite to rework it. There are tons of subtle things that can go 
wrong. It's complex, but I think they way it was written, it kind of is what it 
is. I think if we want a simpler model, we should probably create a new class 
with a different streaming design.

I think the queue synchronize is really simple, and runners as well. That is 
fairly simple multithreaded code. I think the complication is in other parts of 
the design myself.

This class is a bit advanced for sure though. You have to be willing to spend 
some time to have confidence changing it.


> Documents indexed in bulk are replicated using too many HTTP requests
> -
>
> Key: SOLR-9824
> URL: https://issues.apache.org/jira/browse/SOLR-9824
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 6.3
>Reporter: David Smiley
>Assignee: Mark Miller
> Attachments: SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch, 
> SOLR-9824.patch, SOLR-9824.patch, SOLR-9824.patch
>
>
> This takes awhile to explain; bear with me. While working on bulk indexing 
> small documents, I looked at the logs of my SolrCloud nodes.  I noticed that 
> shards would see an /update log message every ~6ms which is *way* too much.  
> These are requests from one shard (that isn't a leader/replica for these docs 
> but the recipient from my client) to the target shard leader (no additional 
> replicas).  One might ask why I'm not sending docs to the right shard in the 
> first place; I have a reason but it's besides the point -- there's a real 
> Solr perf problem here and this probably applies equally to 
> replicationFactor>1 situations too.  I could turn off the logs but that would 
> hide useful stuff, and it's disconcerting to me that so many short-lived HTTP 
> requests are happening, somehow at the bequest of DistributedUpdateProcessor. 
>  After lots of analysis and debugging and hair pulling, I finally figured it 
> out.  
> In SOLR-7333 ([~tpot]) introduced an optimization called 
> {{UpdateRequest.isLastDocInBatch()}} in which ConcurrentUpdateSolrClient will 
> poll with a '0' timeout to the internal queue, so that it can close the 
> connection without it hanging around any longer than needed.  This part makes 
> sense to me.  Currently the only spot that has the smarts to set this flag is 
> {{JavaBinUpdateRequestCodec.unmarshal.readOuterMostDocIterator()}} at the 
> last document.  So if a shard received docs in a javabin stream (but not 
> other formats) one would expect the _last_ document to have this flag.  
> There's even a test.  Docs without this flag get the default poll time; for 
> javabin it's 25ms.  Okay.
> I _suspect_ that if someone used CloudSolrClient or HttpSolrClient to send 
> javabin data in a batch, the intended efficiencies of SOLR-7333 would apply.  
> I didn't try. In my case, I'm using ConcurrentUpdateSolrClient (and BTW 
> DistributedUpdateProcessor uses CUSC too).  CUSC uses the RequestWriter 
> (defaulting to javabin) to send each document separately without any leading 
> marker or trailing marker.  For the XML format by comparison, there is a 
> leading and trailing marker ( ... ).  Since there's no outer 
> container for the javabin unmarshalling to detect the last document, it marks 
> _every_ document as {{req.lastDocInBatch()}}!  Ouch!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9898) Documentation for metrics collection and /admin/metrics

[Andrzej Bialecki (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782804#comment-15782804
 ] 

Andrzej Bialecki  commented on SOLR-9898:
-

h1. Overview
Solr 6.4 adds a developer API and instrumentation for the collection of 
detailed performance-oriented metrics throughout the life-cycle of Solr service 
and its various components. Internally it uses [Dropwizard Metrics 
API|http://metrics.dropwizard.io], which uses the following classes of meters 
to measure events:
* *counters* - simply count events. They provide a single long value, e.g. the 
number of requests.
* *meters* - additionally compute rates of events. Provide a count (as above) 
and 1-, 5-, and 15-minute exponentially decaying rates, similarly to the Unix 
system load average.
* *histograms* - calculate approximate distribution of events according to 
their values. Provide the following approximate statistics, with a similar 
exponential decay as above: mean (arithmetic average), median, maximum, 
minimum, standard deviation, and 75-th, 95-th, 98-th, 99-th and 999-th 
percentiles. 
* *timers* - measure the number and duration of events. They provide a count 
and histogram of timings.
* *gauges* - offer instantaneous reading of a current value, e.g. current queue 
depth, current number of active connections, free heap size.

Group of related metrics with unique names is managed in a *metric registry*. 
Solr maintains several such registries, each corresponding to a high-level 
group such as: {{jvm, jetty, http, node, core}} (see below). Metrics are 
maintained and accumulated through all life-cycles of components since the 
start of the process until its shutdown - e.g. metrics for a particular 
SolrCore are tracked through possibly several load / unload / rename 
operations, and deleted only when a core is explicitly deleted. However, 
metrics are not persisted across process restarts - restarting Solr will 
discard all collected metrics.

For each group (and/or for each registry) there can be several *reporters* - 
components responsible for communication of metrics from selected registries to 
external systems. Currently implemented reporters support emitting metrics via 
JMX, Ganglia, Graphite and SLF4J. There is also a dedicated {{/admin/metrics}} 
handler that can be queried to report all or a subset of the current metrics 
from multiple registries.

h2. Metric groups
These are the major groups of metrics that are collected:

h3. JVM level ({{solr.jvm}} registry):
* direct and mapped buffer pools
* class loading / unloading
* OS memory, CPU time, file descriptors, swap, system load
* GC count and time
* heap, non-heap memory and GC pools
* number of threads, their states and deadlocks

h3. Node / CoreContainer level ({{solr.node}} registry):
* handler requests (count, timing): collections, info, admin, configSets, etc.
* number of cores (loaded, lazy, unloaded)

h3. Core (SolrCore) level ({{solr.core}} registries, one for 
each core):
* all common RequestHandler-s report: request timers / counters, timeouts, 
errors.
* index-level events (in progress - SOLR-9854): meters for minor / major 
merges, number of merged docs, number of deleted docs, gauges for currently 
running merges and their size.
* directory-level IO: total read / write meters, histograms for read / write 
operations and their size, optionally split per index file (eg. field data, 
term dictionary, docValues, etc) (SOLR-9854 in progress)
* shard replication and transaction log replay on replicas (TBD, SOLR-9856)
* TBD: caches, update handler details, and other relevant SolrInfoMBean-s

h3. HTTP level ({{solr.http}} registry):
* open / available / pending connections for shard handler and update handler

h3. Jetty level ({{solr.jetty}} registry):
* threads and pools,
* connection and request timers,
* meters for responses by HTTP class (1xx, 2xx, etc)

h3. Shard leader (TBD)
* aggregated metrics from each replica (SOLR-9857)

h3. Overseer (TBD)
* aggregated metrics from shard leaders and cluster nodes (SOLR-9858)


> Documentation for metrics collection and /admin/metrics
> ---
>
> Key: SOLR-9898
> URL: https://issues.apache.org/jira/browse/SOLR-9898
> Project: Solr
>  Issue Type: Task
>  Security Level: Public(Default Security Level. Issues are Public) 
>Affects Versions: master (7.0), 6.4
>Reporter: Andrzej Bialecki 
>
> Draft documentation follows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_112) - Build # 6317 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6317/
Java: 32bit/jdk1.8.0_112 -server -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([F47CDCE53A3C7B21:7C28E33F94C016D9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Commented] (SOLR-9877) Use instrumented http client

[Shalin Shekhar Mangar (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782724#comment-15782724
 ] 

Shalin Shekhar Mangar commented on SOLR-9877:
-

Yes, I'll fix, thanks!

> Use instrumented http client
> 
>
> Key: SOLR-9877
> URL: https://issues.apache.org/jira/browse/SOLR-9877
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9877.patch, SOLR-9877.patch, solr-http-metrics.png
>
>
> Use instrumented equivalents of PooledHttpClientConnectionManager and others 
> from metrics-httpclient library.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-9877) Use instrumented http client

[Mikhail Khludnev (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782691#comment-15782691
 ] 

Mikhail Khludnev commented on SOLR-9877:


Hello, 

I notice reproducing failure 
https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18637/testReport/
{code}
   [junit4]   2> NOTE: reproduce with: ant test  
-Dtestcase=SolrCmdDistributorTest -Dtests.method=test 
-Dtests.seed=234F7262BA762194 -Dtests.multiplier=3 -Dtests.slow=true 
-Dtests.locale=ki-KE -Dtests.timezone=Etc/GMT+4 -Dtests.asserts=true 
-Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 7.82s J1 | SolrCmdDistributorTest.test <<<
   [junit4]> Throwable #1: java.lang.AssertionError: expected:<1> but 
was:<0>
   [junit4]>at 
__randomizedtesting.SeedInfo.seed([234F7262BA762194:AB1B4DB8148A4C6C]:0)
   [junit4]>at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:169)
{code}
since https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18631/changes
It appears with 
{code}
   [junit4]   2> 1122459 ERROR (updateExecutor-1740-thread-1) [] 
o.a.s.u.StreamingSolrClients error
   [junit4]   2> java.lang.AssertionError
   [junit4]   2>at 
org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:56)
   [junit4]   2>at 
org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:271)
   [junit4]   2>at 
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
   [junit4]   2>at 
org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
   [junit4]   2>at 
org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]   2>at 
org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
   [junit4]   2>at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
   [junit4]   2>at 
org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
   [junit4]   2>at 
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.sendUpdateStream(ConcurrentUpdateSolrClient.java:323)
   [junit4]   2>at 
org.apache.solr.client.solrj.impl.ConcurrentUpdateSolrClient$Runner.run(ConcurrentUpdateSolrClient.java:186)
   [junit4]   2>at 
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2>at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1161)
   [junit4]   2>at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
   [junit4]   2>at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> 1122461 INFO  (qtp979734836-10317) [] o.a.s.c.S.Request 
[collection1]  webapp=/_z path=/select params={q=*:*=javabin=2} 
hits=0 status=0 QTime=0
{code}

Can it be related to this jira? 

> Use instrumented http client
> 
>
> Key: SOLR-9877
> URL: https://issues.apache.org/jira/browse/SOLR-9877
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: metrics
>Reporter: Shalin Shekhar Mangar
>Assignee: Shalin Shekhar Mangar
>Priority: Minor
> Fix For: master (7.0), 6.4
>
> Attachments: SOLR-9877.patch, SOLR-9877.patch, solr-http-metrics.png
>
>
> Use instrumented equivalents of PooledHttpClientConnectionManager and others 
> from metrics-httpclient library.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7588) A parallel DrillSideways implementation

[Emmanuel Keller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated LUCENE-7588:

Attachment: LUCENE-7588.patch

> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
> Attachments: LUCENE-7588.patch
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-7588) A parallel DrillSideways implementation

[Emmanuel Keller (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Emmanuel Keller updated LUCENE-7588:

Attachment: (was: LUCENE-7588.patch)

> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7588) A parallel DrillSideways implementation

[Emmanuel Keller (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782658#comment-15782658
 ] 

Emmanuel Keller commented on LUCENE-7588:
-

One using the search method while providing a CollectorManager expects to be 
able to extract the result.

{code:java}
 public  ConcurrentDrillSidewaysResult search(final DrillDownQuery 
query,
  final CollectorManager hitCollectorManager) throws 
IOException {}
{code}

In the meantime, I forgot to expose the new result class 
"ConcurrentDrillSidewaysResult".
It is fixed now, I submit the new patch.

> A parallel DrillSideways implementation
> ---
>
> Key: LUCENE-7588
> URL: https://issues.apache.org/jira/browse/LUCENE-7588
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: master (7.0), 6.3.1
>Reporter: Emmanuel Keller
>Priority: Minor
>  Labels: facet, faceting
> Fix For: master (7.0), 6.3.1
>
> Attachments: LUCENE-7588.patch
>
>
> Currently DrillSideways implementation is based on the single threaded 
> IndexSearcher.search(Query query, Collector results).
> On large document set, the single threaded collection can be really slow.
> The ParallelDrillSideways implementation could:
> 1. Use the CollectionManager based method IndexSearcher.search(Query query, 
> CollectorManager collectorManager)  to get the benefits of multithreading on 
> index segments,
> 2. Compute each DrillSideway subquery on a single thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-9835) Create another replication mode for SolrCloud

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-9835:
---
Attachment: SOLR-9835.patch

Updated patch
- Leader Recovery
- Replica recovery will always do replication ( skip peersync ) because I found 
it very complex to fix all the bugs caused by peersync in this mode. We can 
support peersync for this mode in future. 

> Create another replication mode for SolrCloud
> -
>
> Key: SOLR-9835
> URL: https://issues.apache.org/jira/browse/SOLR-9835
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Shalin Shekhar Mangar
> Attachments: SOLR-9835.patch, SOLR-9835.patch, SOLR-9835.patch
>
>
> The current replication mechanism of SolrCloud is called state machine, which 
> replicas start in same initial state and for each input, the input is 
> distributed across replicas so all replicas will end up with same next state. 
> But this type of replication have some drawbacks
> - The commit (which costly) have to run on all replicas
> - Slow recovery, because if replica miss more than N updates on its down 
> time, the replica have to download entire index from its leader.
> So we create create another replication mode for SolrCloud called state 
> transfer, which acts like master/slave replication. In basically
> - Leader distribute the update to other replicas, but the leader only apply 
> the update to IW, other replicas just store the update to UpdateLog (act like 
> replication).
> - Replicas frequently polling the latest segments from leader.
> Pros:
> - Lightweight for indexing, because only leader are running the commit, 
> updates.
> - Very fast recovery, replicas just have to download the missing segments.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_112) - Build # 18637 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/18637/
Java: 64bit/jdk1.8.0_112 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.update.SolrCmdDistributorTest.test

Error Message:
expected:<1> but was:<0>

Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at 
__randomizedtesting.SeedInfo.seed([1966177507B2CFEA:913228AFA94EA212]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at 
org.apache.solr.update.SolrCmdDistributorTest.test(SolrCmdDistributorTest.java:169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:811)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:462)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Created] (SOLR-9898) Documentation for metrics collection and /admin/metrics

[Andrzej Bialecki (JIRA)]

Andrzej Bialecki  created SOLR-9898:
---

 Summary: Documentation for metrics collection and /admin/metrics
 Key: SOLR-9898
 URL: https://issues.apache.org/jira/browse/SOLR-9898
 Project: Solr
  Issue Type: Task
  Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: master (7.0), 6.4
Reporter: Andrzej Bialecki 


Draft documentation follows.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-7603) Support Graph Token Streams in QueryBuilder

[Michael McCandless (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-7603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15782609#comment-15782609
 ] 

Michael McCandless commented on LUCENE-7603:


This change looks great; I think it's ready!  The new 
{{TestGraphTokenStreamFiniteStrings}} is just missing the copyright header; 
I'll fix that before pushing.

The gist of the change is when query parsing detects that the analyzer produced 
a graph (any token with {{PositionLengthAttribute}} > 1), e.g. because 
{{SynonymGraphFilter}} matched or inserted a multi-token synonym, then it 
creates a {{GraphQuery}} which just a wrapper around sub-queries that traverse 
each path of the graph.

At search time, this query is currently rewritten to {{BooleanQuery}} with one 
clause for each path, but that is maybe something we can improve in the future, 
e.g. if it's a phrase query we could use {{TermAutomatonQuery}} ... but we 
should tackle that separately.

At long last, this (along with using {{SynonymGraphFilter}} at search time) 
finally fixes the long-standing bugs around multi-token synonyms, e.g. 
LUCENE-4499, LUCENE-1622, 
https://lucidworks.com/blog/2014/07/12/solution-for-multi-term-synonyms-in-lucenesolr-using-the-auto-phrasing-tokenfilter
 ...

This will also be useful for other tokenizers/token filters as well, e.g. I'm 
working on having {{WordDelimiterFilter}} set position length correctly and 
Kuromoji ({{JapaneseTokenizer}}) already produces graph tokens.

> Support Graph Token Streams in QueryBuilder
> ---
>
> Key: LUCENE-7603
> URL: https://issues.apache.org/jira/browse/LUCENE-7603
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/queryparser, core/search
>Reporter: Matt Weber
>
> With [LUCENE-6664|https://issues.apache.org/jira/browse/LUCENE-6664] we can 
> use multi-term synonyms query time.  A "graph token stream" will be created 
> which which is nothing more than using the position length attribute on 
> stacked tokens to indicate how many positions a token should span.  Currently 
> the position length attribute on tokens is ignored during query parsing.  
> This issue will add support for handling these graph token streams inside the 
> QueryBuilder utility class used by query parsers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org