[jira] [Commented] (SOLR-6734) Standalone solr as *two* applications -- Solr and a controlling agent
[
https://issues.apache.org/jira/browse/SOLR-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505714#comment-16505714
]
Shawn Heisey commented on SOLR-6734:
Putting a bit of code in a comment so that I don't lose track of it.
{code:java}
package foo;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicLong;
public class Main {
private static final AtomicInteger threadCounter = new AtomicInteger(0);
public static void main(String[] args) throws InterruptedException {
Thread gce = new GCExerciser();
/*
* To see the difference made by the GC exerciser, run the program once
as-is,
* and then again with the following start() call commented.
*/
gce.start();
Thread pd = new PauseDetector();
pd.start();
}
/**
* Background thread to allocate a byte array every 10 milliseconds. This
serves
* to create garbage, so that there will be GC pauses.
*/
public static class GCExerciser extends Thread {
@Override
public void run() {
Thread.currentThread().setName("GCExerciser" +
threadCounter.incrementAndGet());
final long maxHeap = Runtime.getRuntime().maxMemory();
final long thirtyOneGig = 31L * 1024 * 1024 * 1024;
final int size;
if (maxHeap > thirtyOneGig) {
size = Integer.MAX_VALUE - 1;
} else {
size = (int) (maxHeap / 16);
}
byte[] b = null;
while (true) {
try {
Thread.sleep(10);
} catch (InterruptedException e) {
//
}
b = new byte[size];
@SuppressWarnings("unused")
int i = b.length;
}
}
}
public static class PauseDetector extends Thread {
private static final AtomicLong accumulatedNanos = new AtomicLong(0);
private static final int sleepMillis = 500;
private static final long sleepNanos =
TimeUnit.NANOSECONDS.convert(sleepMillis, TimeUnit.MILLISECONDS);
private static final long fiveMinuteNanos = TimeUnit.NANOSECONDS.convert(5,
TimeUnit.MINUTES);
@Override
public void run() {
Thread.currentThread().setName("PauseDetector" +
threadCounter.incrementAndGet());
Thread.currentThread().setPriority(Thread.MIN_PRIORITY);
long fiveMinuteStart = System.nanoTime();
while (true) {
long sleepStart = System.nanoTime();
try {
Thread.sleep(sleepMillis);
} catch (InterruptedException e) {
//
}
long sleepStop = System.nanoTime();
long sleepElapsed = sleepStop - sleepStart;
/*
* If elapsed time is more than the half second sleep, add the
difference to the
* accumulation counter.
*/
if (sleepElapsed > sleepNanos) {
accumulatedNanos.addAndGet(sleepElapsed - sleepNanos);
}
/*
* If total elapsed time is five minutes or longer, output the
accumulated time
* and reset everything.
*/
if ((sleepStop - fiveMinuteStart) >= fiveMinuteNanos) {
System.out.printf("Accumulated pause estimate for five minutes: %d
ms\n",
TimeUnit.MILLISECONDS.convert(accumulatedNanos.get(),
TimeUnit.NANOSECONDS));
fiveMinuteStart = System.nanoTime();
accumulatedNanos.set(0);
}
}
}
}
}
{code}
The PauseDetector thread in that code approximates what the jHiccup tool from
Azul does.
https://www.azul.com/products/open-source-tools/jhiccup-performance-tool/
With a version of this thread in Solr and any other services, it would be very
easy to detect and log severe application pauses, which typically are caused by
GC.
> Standalone solr as *two* applications -- Solr and a controlling agent
> -
>
> Key: SOLR-6734
> URL: https://issues.apache.org/jira/browse/SOLR-6734
> Project: Solr
> Issue Type: Sub-task
>Reporter: Shawn Heisey
>Priority: Major
>
> In a message to the dev list outlining reasons to switch from a webapp to a
> standalone app, Mark Miller included the idea of making Solr into two
> applications, rather than just one. There would be Solr itself, and an agent
> to control Solr.
> http://mail-archives.apache.org/mod_mbox/lucene-dev/201305.mbox/%3C807476C6-E4C3-4E7E-9F67-2BECB63990DE%40gmail.com%3E
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-BadApples-7.x-Linux (64bit/jdk-9.0.4) - Build # 47 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-BadApples-7.x-Linux/47/ Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseParallelGC 4 tests failed. FAILED: org.apache.lucene.analysis.core.TestRandomChains.testRandomChains Error Message: some thread(s) failed Stack Trace: java.lang.RuntimeException: some thread(s) failed at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:583) at org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:861) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.base/java.lang.Thread.run(Thread.java:844) FAILED: org.apache.lucene.analysis.core.TestRandomChains.testRandomChains Error Message: some thread(s) failed Stack Trace: java.lang.RuntimeException: some thread(s) failed at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:583) at org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:861) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at
[jira] [Commented] (SOLR-12361) Change _childDocuments to Map
[ https://issues.apache.org/jira/browse/SOLR-12361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505685#comment-16505685 ] David Smiley commented on SOLR-12361: - I took your PR and ran with it a bit further; patch attached. I had a stupid bug that took some time to debug so I wasn't ready till now. I'm running tests but so far so good. * Moved/added javadocs to SolrDocumentBase instead of subclass impls RE child docs. I added a bit more info to them and made getChildDocumentCount deprecated. * IngoreLargeDocumentProcessorFactory: ensured the value side of estimate(map) uses the recursive estimate() instead of the non-recursive primitiveEstimate(). AddUpdateCommand: * pending nocommit that getLuceneDocument(inPlaceUpdate) is in fact always == isInPlaceUpdate() so this parameter is not needed and I think we can get rid of this overloaded variant. * rename: getLuceneDocsIfNested (to balance with getLuceneDocument). Added javadocs. * this method now returns Iterable and null if it turns out there is no nesting. This solves some awkwardness in exposing SolrInputDocument type to the UpdateHandler which we don't want. * rename: recUnwrap... to flatten * moved isUsableForChildDocs check to getLuceneDocsIfNested where (to me) it seems nicer) DirectUpdateHandler2: * updateDocOrDocValues now has a simpler method contract; it's caller doesn't need to pass the "updateTerm" as it did originally, and improved from the last patch we return void not the idTerm. doNormalUpdate had "cmd.updateTerm" (signature dedupe) logic that needed the idTerm & updateTerm which was why earlier I thought to return it but after some examination I think it's better if this logic was absorbed into updateDocOrDocValues. I inlined updateDocument() into updateDocOrDocValues -- ultimately trying to make the code path and scope of idTerm/updateTerm more clear and I'm pretty happy with it. Note "inline" is a verb used in coding (and can be found in IDE's for example); you earlier mistook my suggestion as a suggestion to rename a method. Looking again at SolrTestCaseJ4.compareSolrInputDocument... maybe this logic ought to go on SolrInputDocument or better SolrDocumentBase as equals()? If it were moved there, you wouldn't have even needed to make the change you did, and thus it'd be simpler and more obvious -- why should one have to know SolrTestCaseJ4.compareSolrInputDocument exists. Heck, I can argue that SolrDocumentBase doesn't meet the contract of the Map it implements since a Map's equals() demands it actually works! (hashcode too) > Change _childDocuments to Map > - > > Key: SOLR-12361 > URL: https://issues.apache.org/jira/browse/SOLR-12361 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: mosh >Priority: Major > Attachments: SOLR-12361.patch, SOLR-12361.patch, SOLR-12361.patch > > Time Spent: 10h > Remaining Estimate: 0h > > During the discussion on SOLR-12298, there was a proposal to change > _childDocuments in SolrDocumentBase to a Map, to incorporate the relationship > between the parent and its child documents. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12361) Change _childDocuments to Map
[ https://issues.apache.org/jira/browse/SOLR-12361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated SOLR-12361: Attachment: SOLR-12361.patch > Change _childDocuments to Map > - > > Key: SOLR-12361 > URL: https://issues.apache.org/jira/browse/SOLR-12361 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: mosh >Priority: Major > Attachments: SOLR-12361.patch, SOLR-12361.patch, SOLR-12361.patch > > Time Spent: 10h > Remaining Estimate: 0h > > During the discussion on SOLR-12298, there was a proposal to change > _childDocuments in SolrDocumentBase to a Map, to incorporate the relationship > between the parent and its child documents. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-7.4 - Build # 1 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.4/1/ 1 tests failed. FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode Error Message: trigger did not fire Stack Trace: java.lang.AssertionError: trigger did not fire at __randomizedtesting.SeedInfo.seed([ECD59DA2A605B6E4:4B3A8001694839FC]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:300) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) Build Log: [...truncated 23 lines...] ERROR: Error cloning remote repo 'origin' hudson.plugins.git.GitException: Command "git fetch --tags --progress git://git.apache.org/lucene-solr.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: stdout: stderr: remote: Counting objects: 138796 remote: Counting objects: 309319 remote:
[jira] [Commented] (SOLR-12449) Response /autoscaling/diagnostics shows improper json
[
https://issues.apache.org/jira/browse/SOLR-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505632#comment-16505632
]
ASF subversion and git services commented on SOLR-12449:
Commit 63c55d864390733db71a72180ce274db92b5f0ce in lucene-solr's branch
refs/heads/branch_7x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=63c55d8 ]
SOLR-12449: Response /autoscaling/diagnostics shows improper json
> Response /autoscaling/diagnostics shows improper json
> -
>
> Key: SOLR-12449
> URL: https://issues.apache.org/jira/browse/SOLR-12449
> Project: Solr
> Issue Type: Sub-task
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Noble Paul
>Assignee: Noble Paul
>Priority: Major
> Fix For: master (8.0), 7.5
>
> Attachments: SOLR-12449.patch
>
>
> the value for the key "replica" is a serialized json itself
> {{"replica":"org.apache.solr.client.solrj.cloud.autoscaling.ReplicaCount:}}{{
> {\n \"NRT\":2,\n \"PULL\":0,\n \"TLOG\":0,\n \"count\":2}
> }}
> "
> {code:java}
> {
> "violations":[{
> "collection":"c1",
> "shard":"s1",
> "tagKey":"8983",
> "violation":{
>
> "replica":"org.apache.solr.client.solrj.cloud.autoscaling.ReplicaCount:{\n
> \"NRT\":2,\n \"PULL\":0,\n \"TLOG\":0,\n \"count\":2}",
> "delta":-2},
> "clause":{
> "replica":0,
> "shard":"#EACH",
> "port":"8983",
> "collection":"c1"}},
> {
> "collection":"c1",
> "shard":"s2",
> "tagKey":"8983",
> "violation":{
>
> "replica":"org.apache.solr.client.solrj.cloud.autoscaling.ReplicaCount:{\n
> \"NRT\":2,\n \"PULL\":0,\n \"TLOG\":0,\n \"count\":2}",
> "delta":-2},
> "clause":{
> "replica":0,
> "shard":"#EACH",
> "port":"8983",
> "collection":"c1"}}]}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12142) EmbeddedSolrServer should use req.getContentWriter
[ https://issues.apache.org/jira/browse/SOLR-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505611#comment-16505611 ] ASF subversion and git services commented on SOLR-12142: Commit fcb791fce1b47c5c806a5933f428e90ae9bfc1e2 in lucene-solr's branch refs/heads/branch_7_4 from [~noble.paul] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=fcb791f ] SOLR-12142: EmbeddedSolrServer should use req.getContentWriter > EmbeddedSolrServer should use req.getContentWriter > --- > > Key: SOLR-12142 > URL: https://issues.apache.org/jira/browse/SOLR-12142 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Assignee: Noble Paul >Priority: Major > Fix For: 7.4 > > Attachments: SOLR-12142.patch, SOLR-12142.patch > > > In SOLR-11380, SolrRequest.getContentWriter was introduced as a replacement > for getContentStreams. However, EmbeddedSolrServer still calls > getContentStreams, and so clients who need to send POST data to it cannot yet > switch from the Deprecated API to the new API. The SolrTextTagger is an > example of a project where one would want to do this. > It seems EmbeddedSolrServer ought to check for getContentWriter and if > present then convert it into a ContentStream somehow. For the time being, > ESS needs to call both since both APIs exist. > CC [~noble.paul] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 78 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/78/
8 tests failed.
FAILED: org.apache.solr.cloud.TestPullReplica.testCreateDelete
{seed=[151641B485075F8D:E06B0F873F7A7BD]}
Error Message:
Error from server at https://127.0.0.1:33658/solr: create the collection time
out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at https://127.0.0.1:33658/solr: create the collection time out:180s
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at
org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at
org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at
org.apache.solr.cloud.TestPullReplica.testCreateDelete(TestPullReplica.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1561 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1561/ 3 tests failed. FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test Error Message: Java heap space Stack Trace: java.lang.OutOfMemoryError: Java heap space at __randomizedtesting.SeedInfo.seed([DE164F63F47C248D:564270B95A804975]:0) at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57) at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:795) at java.nio.charset.Charset.encode(Charset.java:843) at java.nio.charset.Charset.encode(Charset.java:863) at org.apache.http.client.utils.URLEncodedUtils.urlEncode(URLEncodedUtils.java:532) at org.apache.http.client.utils.URLEncodedUtils.encodeFormFields(URLEncodedUtils.java:652) at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:404) at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:382) at org.apache.http.client.entity.UrlEncodedFormEntity.(UrlEncodedFormEntity.java:75) at org.apache.solr.client.solrj.impl.HttpSolrClient.fillContentStream(HttpSolrClient.java:513) at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:420) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483) at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413) at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974) at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990) at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:228) at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:167) at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:668) at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode Error Message: trigger did not fire Stack Trace: java.lang.AssertionError: trigger did not fire at __randomizedtesting.SeedInfo.seed([DE164F63F47C248D:79F952C03B31AB95]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testAddNode(TestLargeCluster.java:300) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at
[JENKINS-EA] Lucene-Solr-master-Linux (64bit/jdk-11-ea+14) - Build # 22200 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22200/ Java: 64bit/jdk-11-ea+14 -XX:-UseCompressedOops -XX:+UseSerialGC 1 tests failed. FAILED: org.apache.solr.update.MaxSizeAutoCommitTest.deleteTest Error Message: Tlog size exceeds the max size bound. Tlog path: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.update.MaxSizeAutoCommitTest_4EBB4F6B68C693D5-001/init-core-data-001/tlog/tlog.003, tlog size: 1265 Stack Trace: java.lang.AssertionError: Tlog size exceeds the max size bound. Tlog path: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.update.MaxSizeAutoCommitTest_4EBB4F6B68C693D5-001/init-core-data-001/tlog/tlog.003, tlog size: 1265 at __randomizedtesting.SeedInfo.seed([4EBB4F6B68C693D5:5EF5AA941368AA24]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.update.MaxSizeAutoCommitTest.getTlogFileSizes(MaxSizeAutoCommitTest.java:379) at org.apache.solr.update.MaxSizeAutoCommitTest.deleteTest(MaxSizeAutoCommitTest.java:200) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-10.0.1) - Build # 22199 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22199/
Java: 64bit/jdk-10.0.1 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir
Error Message:
Captured an uncaught exception in thread: Thread[id=3530,
name=cdcr-replicator-704-thread-1, state=RUNNABLE,
group=TGRP-CdcrBidirectionalTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught
exception in thread: Thread[id=3530, name=cdcr-replicator-704-thread-1,
state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]
Caused by: java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([70223A3CD9801CCB]:0)
at
org.apache.solr.update.CdcrUpdateLog$CdcrLogReader.forwardSeek(CdcrUpdateLog.java:611)
at org.apache.solr.handler.CdcrReplicator.run(CdcrReplicator.java:125)
at
org.apache.solr.handler.CdcrReplicatorScheduler.lambda$start$0(CdcrReplicatorScheduler.java:81)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 12574 lines...]
[junit4] Suite: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest
[junit4] 2> 148113 INFO
(SUITE-CdcrBidirectionalTest-seed#[70223A3CD9801CCB]-worker) []
o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_70223A3CD9801CCB-001/init-core-data-001
[junit4] 2> 148114 WARN
(SUITE-CdcrBidirectionalTest-seed#[70223A3CD9801CCB]-worker) []
o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=26 numCloses=26
[junit4] 2> 148114 INFO
(SUITE-CdcrBidirectionalTest-seed#[70223A3CD9801CCB]-worker) []
o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 148115 INFO
(SUITE-CdcrBidirectionalTest-seed#[70223A3CD9801CCB]-worker) []
o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via:
@org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0,
clientAuth=0.0/0.0)
[junit4] 2> 148116 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[70223A3CD9801CCB]) []
o.a.s.SolrTestCaseJ4 ###Starting testBiDir
[junit4] 2> 148116 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[70223A3CD9801CCB]) []
o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_70223A3CD9801CCB-001/cdcr-cluster2-001
[junit4] 2> 148116 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[70223A3CD9801CCB]) []
o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 148117 INFO (Thread-2024) [] o.a.s.c.ZkTestServer client
port:0.0.0.0/0.0.0.0:0
[junit4] 2> 148117 INFO (Thread-2024) [] o.a.s.c.ZkTestServer
Starting server
[junit4] 2> 148118 ERROR (Thread-2024) [] o.a.z.s.ZooKeeperServer
ZKShutdownHandler is not registered, so ZooKeeper server won't take any action
on ERROR or SHUTDOWN server state changes
[junit4] 2> 148217 INFO
(TEST-CdcrBidirectionalTest.testBiDir-seed#[70223A3CD9801CCB]) []
o.a.s.c.ZkTestServer start zk server on port:35525
[junit4] 2> 148218 INFO (zkConnectionManagerCallback-404-thread-1) []
o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 148221 INFO (jetty-launcher-401-thread-1) []
o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git:
daa59876e6f384329b122929e70a80934569428c; jvm 10.0.1+10
[junit4] 2> 148236 INFO (jetty-launcher-401-thread-1) []
o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 148236 INFO (jetty-launcher-401-thread-1) []
o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 148236 INFO (jetty-launcher-401-thread-1) []
o.e.j.s.session node0 Scavenging every 60ms
[junit4] 2> 148236 INFO (jetty-launcher-401-thread-1) []
o.e.j.s.h.ContextHandler Started
o.e.j.s.ServletContextHandler@2d6b4c98{/solr,null,AVAILABLE}
[junit4] 2> 148238 INFO (jetty-launcher-401-thread-1) []
o.e.j.s.AbstractConnector Started
ServerConnector@59a7d0a7{HTTP/1.1,[http/1.1]}{127.0.0.1:34067}
[junit4] 2> 148239 INFO (jetty-launcher-401-thread-1) []
o.e.j.s.Server Started @148257ms
[junit4] 2> 148239 INFO (jetty-launcher-401-thread-1) []
o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
hostPort=34067}
[junit4]
[jira] [Commented] (SOLR-5501) Ability to work with cold replicas
[
https://issues.apache.org/jira/browse/SOLR-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505336#comment-16505336
]
Tomás Fernández Löbbe commented on SOLR-5501:
-
Spoke too soon. However, it could be resolved with {{shards.preference}}
(SOLR-11982)?
> Ability to work with cold replicas
> --
>
> Key: SOLR-5501
> URL: https://issues.apache.org/jira/browse/SOLR-5501
> Project: Solr
> Issue Type: Improvement
> Components: SolrCloud
>Affects Versions: 4.5.1
>Reporter: Manuel Lenormand
>Priority: Major
> Labels: performance
> Fix For: 4.9, 6.0
>
> Attachments: 5501.patch, cloud_screenshot.png
>
>
> Following this conversation from the mailing list:
> http://lucene.472066.n3.nabble.com/Proposal-for-new-feature-cold-replicas-brainstorming-td4097501.html
> Should give the ability to use replicas mainly as backup cores and not for
> handling high qps rate.
> This way you would avoid using the caching ressources (solr and OS) used when
> routing a query to a replica.
> With many replicas it's harder hitting the solr cache (same query may hit
> another replica) and having many replicas on the same instance would cause a
> useless competition on the OS memory for caching.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8146) Allowing SolrJ CloudSolrClient to have preferred replica for query/read
[
https://issues.apache.org/jira/browse/SOLR-8146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505333#comment-16505333
]
Tomás Fernández Löbbe commented on SOLR-8146:
-
I just related some related Jiras: SOLR-11982 adds support for indicating
preferred replicas, although this only works on the server side (meaning, when
working with multiple shards). SOLR-12217 is the follow up code to make it work
on the client side too
> Allowing SolrJ CloudSolrClient to have preferred replica for query/read
> ---
>
> Key: SOLR-8146
> URL: https://issues.apache.org/jira/browse/SOLR-8146
> Project: Solr
> Issue Type: New Feature
> Components: clients - java
>Affects Versions: 5.3
>Reporter: Arcadius Ahouansou
>Priority: Major
> Attachments: SOLR-8146.patch, SOLR-8146.patch, SOLR-8146.patch,
> SOLR-8146.patch
>
>
> h2. Backgrouds
> Currently, the CloudSolrClient randomly picks a replica to query.
> This is done by shuffling the list of live URLs to query then, picking the
> first item from the list.
> This ticket is to allow more flexibility and control to some extend which
> URLs will be picked up for queries.
> Note that this is for queries only and would not affect update/delete/admin
> operations.
> h2. Implementation
> The current patch uses regex pattern and moves to the top of the list of URLs
> only those matching the given regex specified by the system property
> {code}solr.preferredQueryNodePattern{code}
> Initially, I thought it may be good to have Solr nodes tagged with a string
> pattern (snitch?) and use that pattern for matching the URLs.
> Any comment, recommendation or feedback would be appreciated.
> h2. Use Cases
> There are many cases where the ability to choose the node where queries go
> can be very handy:
> h3. Special node for manual user queries and analytics
> One may have a SolrCLoud cluster where every node host the same set of
> collections with:
> - multiple large SolrCLoud nodes (L) used for production apps and
> - have 1 small node (S) in the same cluster with less ram/cpu used only for
> manual user queries, data export and other production issue investigation.
> This ticket would allow to configure the applications using SolrJ to query
> only the (L) nodes
> This use case is similar to the one described in SOLR-5501 raised by [~manuel
> lenormand]
> h3. Minimizing network traffic
>
> For simplicity, let's say that we have a SolrSloud cluster deployed on 2 (or
> N) separate racks: rack1 and rack2.
> On each rack, we have a set of SolrCloud VMs as well as a couple of client
> VMs querying solr using SolrJ.
> All solr nodes are identical and have the same number of collections.
> What we would like to achieve is:
> - clients on rack1 will by preference query only SolrCloud nodes on rack1,
> and
> - clients on rack2 will by preference query only SolrCloud nodes on rack2.
> - Cross-rack read will happen if and only if one of the racks has no
> available Solr node to serve a request.
> In other words, we want read operations to be local to a rack whenever
> possible.
> Note that write/update/delete/admin operations should not be affected.
> Note that in our use case, we have a cross DC deployment. So, replace
> rack1/rack2 by DC1/DC2
> Any comment would be very appreciated.
> Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-5501) Ability to work with cold replicas
[ https://issues.apache.org/jira/browse/SOLR-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505327#comment-16505327 ] Tomás Fernández Löbbe commented on SOLR-5501: - This is resolved with Replica Types, right? > Ability to work with cold replicas > -- > > Key: SOLR-5501 > URL: https://issues.apache.org/jira/browse/SOLR-5501 > Project: Solr > Issue Type: Improvement > Components: SolrCloud >Affects Versions: 4.5.1 >Reporter: Manuel Lenormand >Priority: Major > Labels: performance > Fix For: 4.9, 6.0 > > Attachments: 5501.patch, cloud_screenshot.png > > > Following this conversation from the mailing list: > http://lucene.472066.n3.nabble.com/Proposal-for-new-feature-cold-replicas-brainstorming-td4097501.html > Should give the ability to use replicas mainly as backup cores and not for > handling high qps rate. > This way you would avoid using the caching ressources (solr and OS) used when > routing a query to a replica. > With many replicas it's harder hitting the solr cache (same query may hit > another replica) and having many replicas on the same instance would cause a > useless competition on the OS memory for caching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12438) Improve status reporting of metrics history API
[
https://issues.apache.org/jira/browse/SOLR-12438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505295#comment-16505295
]
ASF subversion and git services commented on SOLR-12438:
Commit 87c23545a103d16aff19b3623b3985ae9b63a870 in lucene-solr's branch
refs/heads/branch_7x from Andrzej Bialecki
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=87c2354 ]
SOLR-12438: Improve status reporting of metrics history API.
> Improve status reporting of metrics history API
> ---
>
> Key: SOLR-12438
> URL: https://issues.apache.org/jira/browse/SOLR-12438
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12438.patch, SOLR-12438.patch
>
>
> In an offline conversation with [~janhoy] we identified the following areas
> of improvement to the metrics history API in order to increase its
> user-friendliness and provide more details about its status.
>
> * there are three possible states for the API: inactive (when not in cloud
> mode), in-memory only (when {{.system}} collection doesn’t exist), and
> persisted when it’s both active and persisted in Solr. The
> /admin/metrics/history endpoint should give some hint about this status, such
> as "mode":"memory/index", "active": true|false. Or a separate action=status
> just to poll status? Currently when the API is inactive it simply returns 404
> Not Found.
> * when in "memory" mode a call to /admin/metrics/history on a non-overseer
> node should forward the request to the overseer, so that the client does not
> need to care what mode it is in - kind of like how a query works in
> distributed mode.
> * better documentation for the API behavior in each mode.
> * perhaps if mode=memory, there could also be a "message":"Warning, metrics
> history is not being persisted. Please create the .system collection to start
> persisting history"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12142) EmbeddedSolrServer should use req.getContentWriter
[ https://issues.apache.org/jira/browse/SOLR-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505293#comment-16505293 ] David Smiley commented on SOLR-12142: - needs to go to the branch_7_4 too. > EmbeddedSolrServer should use req.getContentWriter > --- > > Key: SOLR-12142 > URL: https://issues.apache.org/jira/browse/SOLR-12142 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Assignee: Noble Paul >Priority: Major > Fix For: 7.4 > > Attachments: SOLR-12142.patch, SOLR-12142.patch > > > In SOLR-11380, SolrRequest.getContentWriter was introduced as a replacement > for getContentStreams. However, EmbeddedSolrServer still calls > getContentStreams, and so clients who need to send POST data to it cannot yet > switch from the Deprecated API to the new API. The SolrTextTagger is an > example of a project where one would want to do this. > It seems EmbeddedSolrServer ought to check for getContentWriter and if > present then convert it into a ContentStream somehow. For the time being, > ESS needs to call both since both APIs exist. > CC [~noble.paul] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12458) ADLS support for SOLR
[ https://issues.apache.org/jira/browse/SOLR-12458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Wingert updated SOLR-12458: Priority: Minor (was: Major) > ADLS support for SOLR > - > > Key: SOLR-12458 > URL: https://issues.apache.org/jira/browse/SOLR-12458 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: Hadoop Integration >Affects Versions: master (8.0) >Reporter: Mike Wingert >Priority: Minor > Fix For: master (8.0) > > Attachments: SOLR-12458.patch > > > This is to track ADLS support for SOLR. > ADLS is a HDFS like API available in Microsoft Azure. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: ScorerSupplier and cost() avoidance
Got it. I thought you were talking about the class method, not the static function. Le jeu. 7 juin 2018 à 22:34, David Smiley a écrit : > I'll have to look further with the team on what these queries look like in > terms of relative cheapness and throughput. > > RE MinShouldMatchSumScorer.cost() (static method called by > Boolean2ScorerSupplier.computeCost) -- it's pretty easy to see that this > is called with minShouldMatch==0 (or 1). Set a conditional breakpoint in > your IDE and run TestBooleanMinShouldMatch.testRandomQueries > > On Thu, Jun 7, 2018 at 3:43 PM Adrien Grand wrote: > >> I suspect this could only show up as a bottleneck if they run very cheap >> queries (low cost) at a very high throughput? Is it the case? I've seen a >> couple workloads like that in the past and profilers suggested that things >> that usually do not matter were bottleneck like creating scorers or >> deciding whether a query should be cached. But trying to fix it didn't >> really help as there are lots of things that we need to do to decide how to >> run a query that run in O(num_segments * num_clauses) >> >> I'm confused why MinShouldMatchSumScorer would be used when >> minShouldMatch is 0 or 1. DisjunctionSumScorer should be used instead for >> such values of minShouldMatch? >> >> Le jeu. 7 juin 2018 à 19:38, Michael McCandless < >> luc...@mikemccandless.com> a écrit : >> >>> Doesn't BQ rewrite itself if it has only one clause? >>> >>> Or maybe if there were more than one clause, and then all but one of >>> them had null scorers (on SHOULD clauses) you could wind up in that state? >>> >>> Mike McCandless >>> >>> http://blog.mikemccandless.com >>> >>> On Thu, Jun 7, 2018 at 1:21 PM, David Smiley >>> wrote: >>> I'm working with some folks who did some profiling and noticed that ScorerSupplier.cost() can be expensive (as the javadocs say). cost() says only to call it if necessary. Unfortunately, a BooleanQuery is going to call cost() (via BooleanWeight.scorer() even if ultimately no Query in the tree cares what the cost is. I'm not sure if that's a perf bug or not; it's hard to tell. The expensive part of cost() for Boolean2ScorerSupplier is over in MinShouldMatchSumScorer.cost which creates a PriorityQueue every time, even if trivially numScorers == 1. That's a weird case... why do we even need a Boolean2ScorerSupplier around one clause; couldn't that clause be returned from the outer weight, BooleanWeight.scorerSupplier() close to the end as an optimization? I could file an issue. ~ David -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com >>> >>> -- > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: > http://www.solrenterprisesearchserver.com >
Re: ScorerSupplier and cost() avoidance
I'll have to look further with the team on what these queries look like in terms of relative cheapness and throughput. RE MinShouldMatchSumScorer.cost() (static method called by Boolean2ScorerSupplier.computeCost) -- it's pretty easy to see that this is called with minShouldMatch==0 (or 1). Set a conditional breakpoint in your IDE and run TestBooleanMinShouldMatch.testRandomQueries On Thu, Jun 7, 2018 at 3:43 PM Adrien Grand wrote: > I suspect this could only show up as a bottleneck if they run very cheap > queries (low cost) at a very high throughput? Is it the case? I've seen a > couple workloads like that in the past and profilers suggested that things > that usually do not matter were bottleneck like creating scorers or > deciding whether a query should be cached. But trying to fix it didn't > really help as there are lots of things that we need to do to decide how to > run a query that run in O(num_segments * num_clauses) > > I'm confused why MinShouldMatchSumScorer would be used when minShouldMatch > is 0 or 1. DisjunctionSumScorer should be used instead for such values of > minShouldMatch? > > Le jeu. 7 juin 2018 à 19:38, Michael McCandless > a écrit : > >> Doesn't BQ rewrite itself if it has only one clause? >> >> Or maybe if there were more than one clause, and then all but one of them >> had null scorers (on SHOULD clauses) you could wind up in that state? >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> On Thu, Jun 7, 2018 at 1:21 PM, David Smiley >> wrote: >> >>> I'm working with some folks who did some profiling and noticed that >>> ScorerSupplier.cost() can be expensive (as the javadocs say). cost() says >>> only to call it if necessary. Unfortunately, a BooleanQuery is going to >>> call cost() (via BooleanWeight.scorer() even if ultimately no Query in the >>> tree cares what the cost is. I'm not sure if that's a perf bug or not; >>> it's hard to tell. >>> >>> The expensive part of cost() for Boolean2ScorerSupplier is over in >>> MinShouldMatchSumScorer.cost which creates a PriorityQueue every time, even >>> if trivially numScorers == 1. That's a weird case... why do we even need a >>> Boolean2ScorerSupplier around one clause; couldn't that clause be returned >>> from the outer weight, BooleanWeight.scorerSupplier() close to the end as >>> an optimization? I could file an issue. >>> >>> ~ David >>> -- >>> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker >>> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: >>> http://www.solrenterprisesearchserver.com >>> >> >> -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
[jira] [Commented] (SOLR-12438) Improve status reporting of metrics history API
[
https://issues.apache.org/jira/browse/SOLR-12438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505210#comment-16505210
]
ASF subversion and git services commented on SOLR-12438:
Commit 417940cdd76ded0a41906ce794a79f4f09136314 in lucene-solr's branch
refs/heads/master from Andrzej Bialecki
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=417940c ]
SOLR-12438: Improve status reporting of metrics history API.
> Improve status reporting of metrics history API
> ---
>
> Key: SOLR-12438
> URL: https://issues.apache.org/jira/browse/SOLR-12438
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12438.patch, SOLR-12438.patch
>
>
> In an offline conversation with [~janhoy] we identified the following areas
> of improvement to the metrics history API in order to increase its
> user-friendliness and provide more details about its status.
>
> * there are three possible states for the API: inactive (when not in cloud
> mode), in-memory only (when {{.system}} collection doesn’t exist), and
> persisted when it’s both active and persisted in Solr. The
> /admin/metrics/history endpoint should give some hint about this status, such
> as "mode":"memory/index", "active": true|false. Or a separate action=status
> just to poll status? Currently when the API is inactive it simply returns 404
> Not Found.
> * when in "memory" mode a call to /admin/metrics/history on a non-overseer
> node should forward the request to the overseer, so that the client does not
> need to care what mode it is in - kind of like how a query works in
> distributed mode.
> * better documentation for the API behavior in each mode.
> * perhaps if mode=memory, there could also be a "message":"Warning, metrics
> history is not being persisted. Please create the .system collection to start
> persisting history"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12465) Triggers should look at metrics less often
[
https://issues.apache.org/jira/browse/SOLR-12465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505167#comment-16505167
]
Andrzej Bialecki commented on SOLR-12465:
--
It makes sense to me what you mentioned in the other issue - that triggers may
consider a fraction of {{waitFor}} as the basis for how often they check
cluster state. However, this depends on trigger implementation and how likely
it is to miss important state change between samples.
In case of nodeLost / nodeAdded the condition will persist for a relatively
long time, so it makes sense to track it less closely (although we need to
consider the "flaky node" scenario too, see SOLR-11730). Search rate trigger
samples a 1-min average metric, so it's probably also ok if it checks less
frequently.
Triggers should also check that their {{waitFor}} is at least twice as long as
the `triggerScheduleDelaySeconds`.
> Triggers should look at metrics less often
> --
>
> Key: SOLR-12465
> URL: https://issues.apache.org/jira/browse/SOLR-12465
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Varun Thacker
>Priority: Major
>
> As soon as I create a search rate trigger I see that the metrics endpoint is
> being queried every 1 second . As discussed on SOLR-12463 we should change
> the default to be 10 seconds
> Example trigger
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
> "set-trigger": {
> "name" : "search_rate_trigger",
> "event" : "searchRate",
> "collection" : "test_rate_trigger",
> "rate" : 1.0,
> "waitFor" : "1m",
> "enabled" : true,
> "actions" : [
> {
> "name" : "compute_plan",
> "class": "solr.ComputePlanAction"
> },
> {
> "name" : "execute_plan",
> "class": "solr.ExecutePlanAction"
> }
> ]
> }
> }' http://localhost:8983/solr/admin/autoscaling
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: ScorerSupplier and cost() avoidance
I suspect this could only show up as a bottleneck if they run very cheap queries (low cost) at a very high throughput? Is it the case? I've seen a couple workloads like that in the past and profilers suggested that things that usually do not matter were bottleneck like creating scorers or deciding whether a query should be cached. But trying to fix it didn't really help as there are lots of things that we need to do to decide how to run a query that run in O(num_segments * num_clauses) I'm confused why MinShouldMatchSumScorer would be used when minShouldMatch is 0 or 1. DisjunctionSumScorer should be used instead for such values of minShouldMatch? Le jeu. 7 juin 2018 à 19:38, Michael McCandless a écrit : > Doesn't BQ rewrite itself if it has only one clause? > > Or maybe if there were more than one clause, and then all but one of them > had null scorers (on SHOULD clauses) you could wind up in that state? > > Mike McCandless > > http://blog.mikemccandless.com > > On Thu, Jun 7, 2018 at 1:21 PM, David Smiley > wrote: > >> I'm working with some folks who did some profiling and noticed that >> ScorerSupplier.cost() can be expensive (as the javadocs say). cost() says >> only to call it if necessary. Unfortunately, a BooleanQuery is going to >> call cost() (via BooleanWeight.scorer() even if ultimately no Query in the >> tree cares what the cost is. I'm not sure if that's a perf bug or not; >> it's hard to tell. >> >> The expensive part of cost() for Boolean2ScorerSupplier is over in >> MinShouldMatchSumScorer.cost which creates a PriorityQueue every time, even >> if trivially numScorers == 1. That's a weird case... why do we even need a >> Boolean2ScorerSupplier around one clause; couldn't that clause be returned >> from the outer weight, BooleanWeight.scorerSupplier() close to the end as >> an optimization? I could file an issue. >> >> ~ David >> -- >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: >> http://www.solrenterprisesearchserver.com >> > >
[jira] [Commented] (SOLR-12376) New TaggerRequestHandler (aka SolrTextTagger)
[
https://issues.apache.org/jira/browse/SOLR-12376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505143#comment-16505143
]
ASF subversion and git services commented on SOLR-12376:
Commit d7abebd7af83ea08d7bdea94077212ffd7c5efe1 in lucene-solr's branch
refs/heads/master from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d7abebd ]
SOLR-12376: Add some links to other Ref Guide pages; minor format & typo cleanup
> New TaggerRequestHandler (aka SolrTextTagger)
> -
>
> Key: SOLR-12376
> URL: https://issues.apache.org/jira/browse/SOLR-12376
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12376.patch, SOLR-12376.patch, SOLR-12376.patch
>
>
> This issue introduces a new RequestHandler: {{TaggerRequestHandler}}, AKA the
> SolrTextTagger from the OpenSextant project
> [https://github.com/OpenSextant/SolrTextTagger]. It's used for named entity
> recognition (NER) of text past to it. It doesn't do any NLP (outside of
> Lucene text analysis) so it's said to be a "naive tagger", but it's
> definitely useful as-is and a more complete NER or ERD (entity recognition
> and disambiguation) system can be built with this as a key component. The
> SolrTextTagger has been used on queries for query-understanding, and it's
> been used on full-text, and it's been used on dictionaries that number tens
> of millions in size. Since it's small and has been used a bunch (including
> helping win an ERD competition and in [Apache
> Stanbol|https://stanbol.apache.org/]), several people have asked me when or
> why isn't this in Solr yet. So here it is.
> To use it, first you need a collection of documents that have a name-like
> field (short text) indexed with the ConcatenateFilter (LUCENE-8323) at the
> end. We call this the dictionary. Once that's in place, you simply post text
> to a {{TaggerRequestHandler}} and it returns the offset pairs into that text
> for matches in the dictionary along with the uniqueKey of the matching
> documents. It can also return other document data desired. That's the gist;
> I'll add more details on use to the Solr Reference Guide.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12376) New TaggerRequestHandler (aka SolrTextTagger)
[
https://issues.apache.org/jira/browse/SOLR-12376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505147#comment-16505147
]
ASF subversion and git services commented on SOLR-12376:
Commit 7cff08c7a844f26bd292f8408b1aa3c6a8bec86f in lucene-solr's branch
refs/heads/branch_7_4 from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7cff08c ]
SOLR-12376: Add some links to other Ref Guide pages; minor format & typo cleanup
> New TaggerRequestHandler (aka SolrTextTagger)
> -
>
> Key: SOLR-12376
> URL: https://issues.apache.org/jira/browse/SOLR-12376
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12376.patch, SOLR-12376.patch, SOLR-12376.patch
>
>
> This issue introduces a new RequestHandler: {{TaggerRequestHandler}}, AKA the
> SolrTextTagger from the OpenSextant project
> [https://github.com/OpenSextant/SolrTextTagger]. It's used for named entity
> recognition (NER) of text past to it. It doesn't do any NLP (outside of
> Lucene text analysis) so it's said to be a "naive tagger", but it's
> definitely useful as-is and a more complete NER or ERD (entity recognition
> and disambiguation) system can be built with this as a key component. The
> SolrTextTagger has been used on queries for query-understanding, and it's
> been used on full-text, and it's been used on dictionaries that number tens
> of millions in size. Since it's small and has been used a bunch (including
> helping win an ERD competition and in [Apache
> Stanbol|https://stanbol.apache.org/]), several people have asked me when or
> why isn't this in Solr yet. So here it is.
> To use it, first you need a collection of documents that have a name-like
> field (short text) indexed with the ConcatenateFilter (LUCENE-8323) at the
> end. We call this the dictionary. Once that's in place, you simply post text
> to a {{TaggerRequestHandler}} and it returns the offset pairs into that text
> for matches in the dictionary along with the uniqueKey of the matching
> documents. It can also return other document data desired. That's the gist;
> I'll add more details on use to the Solr Reference Guide.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12376) New TaggerRequestHandler (aka SolrTextTagger)
[
https://issues.apache.org/jira/browse/SOLR-12376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505146#comment-16505146
]
ASF subversion and git services commented on SOLR-12376:
Commit c01287d7b34293d9ae7b0abcd1bf66334f9d5138 in lucene-solr's branch
refs/heads/branch_7x from [~ctargett]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c01287d ]
SOLR-12376: Add some links to other Ref Guide pages; minor format & typo cleanup
> New TaggerRequestHandler (aka SolrTextTagger)
> -
>
> Key: SOLR-12376
> URL: https://issues.apache.org/jira/browse/SOLR-12376
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12376.patch, SOLR-12376.patch, SOLR-12376.patch
>
>
> This issue introduces a new RequestHandler: {{TaggerRequestHandler}}, AKA the
> SolrTextTagger from the OpenSextant project
> [https://github.com/OpenSextant/SolrTextTagger]. It's used for named entity
> recognition (NER) of text past to it. It doesn't do any NLP (outside of
> Lucene text analysis) so it's said to be a "naive tagger", but it's
> definitely useful as-is and a more complete NER or ERD (entity recognition
> and disambiguation) system can be built with this as a key component. The
> SolrTextTagger has been used on queries for query-understanding, and it's
> been used on full-text, and it's been used on dictionaries that number tens
> of millions in size. Since it's small and has been used a bunch (including
> helping win an ERD competition and in [Apache
> Stanbol|https://stanbol.apache.org/]), several people have asked me when or
> why isn't this in Solr yet. So here it is.
> To use it, first you need a collection of documents that have a name-like
> field (short text) indexed with the ConcatenateFilter (LUCENE-8323) at the
> end. We call this the dictionary. Once that's in place, you simply post text
> to a {{TaggerRequestHandler}} and it returns the offset pairs into that text
> for matches in the dictionary along with the uniqueKey of the matching
> documents. It can also return other document data desired. That's the gist;
> I'll add more details on use to the Solr Reference Guide.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-repro - Build # 779 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/779/ [...truncated 29 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/237/consoleText [repro] Revision: e691bf734270296ae31cd9d330f3ce0137ec5124 [repro] Ant options: -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 [repro] Repro line: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testSplitIntegration -Dtests.seed=8A744AD1E39180D7 -Dtests.multiplier=2 -Dtests.locale=it-IT -Dtests.timezone=Africa/Maseru -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testMergeIntegration -Dtests.seed=8A744AD1E39180D7 -Dtests.multiplier=2 -Dtests.locale=it-IT -Dtests.timezone=Africa/Maseru -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=TestLargeCluster -Dtests.method=testNodeLost -Dtests.seed=8A744AD1E39180D7 -Dtests.multiplier=2 -Dtests.locale=hr-HR -Dtests.timezone=PST8PDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=TestLargeCluster -Dtests.method=testAddNode -Dtests.seed=8A744AD1E39180D7 -Dtests.multiplier=2 -Dtests.locale=hr-HR -Dtests.timezone=PST8PDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a [repro] git fetch [repro] git checkout e691bf734270296ae31cd9d330f3ce0137ec5124 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestLargeCluster [repro] IndexSizeTriggerTest [repro] ant compile-test [...truncated 3318 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 -Dtests.class="*.TestLargeCluster|*.IndexSizeTriggerTest" -Dtests.showOutput=onerror -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 -Dtests.seed=8A744AD1E39180D7 -Dtests.multiplier=2 -Dtests.locale=hr-HR -Dtests.timezone=PST8PDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 31893 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 4/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster [repro] Re-testing 100% failures at the tip of branch_7x [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestLargeCluster [repro] ant compile-test [...truncated 3318 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestLargeCluster" -Dtests.showOutput=onerror -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 -Dtests.seed=8A744AD1E39180D7 -Dtests.multiplier=2 -Dtests.locale=hr-HR -Dtests.timezone=PST8PDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 24873 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of branch_7x: [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster [repro] Re-testing 100% failures at the tip of branch_7x without a seed [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestLargeCluster [repro] ant compile-test [...truncated 3318 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestLargeCluster" -Dtests.showOutput=onerror -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 -Dtests.multiplier=2 -Dtests.locale=hr-HR -Dtests.timezone=PST8PDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 21000 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of branch_7x without a seed: [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster [repro] git checkout 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 5 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: ScorerSupplier and cost() avoidance
I think that's right Michael -- some null scorers. Started with 3 in one case and wound up with 1. Also, in MinShouldMatchSumScorer.cost, sometimes the size of the PriorityQueue isn't needed at all, which is when minShouldMatch is 0 or 1. In that case, simply sum the costs. On Thu, Jun 7, 2018 at 1:38 PM Michael McCandless wrote: > Doesn't BQ rewrite itself if it has only one clause? > > Or maybe if there were more than one clause, and then all but one of them > had null scorers (on SHOULD clauses) you could wind up in that state? > > Mike McCandless > > http://blog.mikemccandless.com > > On Thu, Jun 7, 2018 at 1:21 PM, David Smiley > wrote: > >> I'm working with some folks who did some profiling and noticed that >> ScorerSupplier.cost() can be expensive (as the javadocs say). cost() says >> only to call it if necessary. Unfortunately, a BooleanQuery is going to >> call cost() (via BooleanWeight.scorer() even if ultimately no Query in the >> tree cares what the cost is. I'm not sure if that's a perf bug or not; >> it's hard to tell. >> >> The expensive part of cost() for Boolean2ScorerSupplier is over in >> MinShouldMatchSumScorer.cost which creates a PriorityQueue every time, even >> if trivially numScorers == 1. That's a weird case... why do we even need a >> Boolean2ScorerSupplier around one clause; couldn't that clause be returned >> from the outer weight, BooleanWeight.scorerSupplier() close to the end as >> an optimization? I could file an issue. >> >> ~ David >> -- >> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker >> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: >> http://www.solrenterprisesearchserver.com >> > > -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
[jira] [Created] (SOLR-12466) Autoscaling properties does not revert back to default when set to null
Varun Thacker created SOLR-12466:
Summary: Autoscaling properties does not revert back to default
when set to null
Key: SOLR-12466
URL: https://issues.apache.org/jira/browse/SOLR-12466
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: AutoScaling
Reporter: Varun Thacker
1. By default the metrics get queried every 1 second
2. Now I change it to 5 second via this API and this works
{code:java}
curl -X POST -H 'Content-type:application/json' --data-binary '{
"set-properties": { "triggerScheduleDelaySeconds": 1 } }'
http://localhost:8983/solr/admin/autoscaling{code}
3. I want to change it back to the default, so I applied null but that didn't
work ( the metrics is still getting queried at 5 second interval and not 1
second). Explicitly setting it to 1 works
{code:java}
curl -X POST -H 'Content-type:application/json' --data-binary '{
"set-properties": { "triggerScheduleDelaySeconds": null } }'
http://localhost:8983/solr/admin/autoscaling{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12465) Triggers should look at metrics less often
[
https://issues.apache.org/jira/browse/SOLR-12465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505137#comment-16505137
]
Varun Thacker commented on SOLR-12465:
--
Logs from the node showing that we query the metrics endpoint every 1 second
{code:java}
INFO - 2018-06-07 15:29:16.243; [ ] org.apache.solr.servlet.HttpSolrCall;
[admin] webapp=null path=/admin/metrics
params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
status=0 QTime=0
INFO - 2018-06-07 15:29:17.253; [ ] org.apache.solr.servlet.HttpSolrCall;
[admin] webapp=null path=/admin/metrics
params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
status=0 QTime=0{code}
> Triggers should look at metrics less often
> --
>
> Key: SOLR-12465
> URL: https://issues.apache.org/jira/browse/SOLR-12465
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Varun Thacker
>Priority: Major
>
> As soon as I create a search rate trigger I see that the metrics endpoint is
> being queried every 1 second . As discussed on SOLR-12463 we should change
> the default to be 10 seconds
> Example trigger
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
> "set-trigger": {
> "name" : "search_rate_trigger",
> "event" : "searchRate",
> "collection" : "test_rate_trigger",
> "rate" : 1.0,
> "waitFor" : "1m",
> "enabled" : true,
> "actions" : [
> {
> "name" : "compute_plan",
> "class": "solr.ComputePlanAction"
> },
> {
> "name" : "execute_plan",
> "class": "solr.ExecutePlanAction"
> }
> ]
> }
> }' http://localhost:8983/solr/admin/autoscaling
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-12463) Search rate trigger should check metrics less often
[
https://issues.apache.org/jira/browse/SOLR-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Thacker resolved SOLR-12463.
--
Resolution: Won't Fix
> Search rate trigger should check metrics less often
> ---
>
> Key: SOLR-12463
> URL: https://issues.apache.org/jira/browse/SOLR-12463
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Varun Thacker
>Priority: Major
>
> I created a search rate trigger by running this command
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
> "set-trigger": {
> "name" : "search_rate_trigger",
> "event" : "searchRate",
> "collection" : "gettingstarted",
> "rate" : 1.0,
> "waitFor" : "1m",
> "enabled" : true,
> "actions" : [
> {
> "name" : "compute_plan",
> "class": "solr.ComputePlanAction"
> },
> {
> "name" : "execute_plan",
> "class": "solr.ExecutePlanAction"
> }
> ]
> }
> }' http://localhost:8983/solr/admin/autoscaling
> {code}
> As soon as I add this by logs start getting spammed with these every 1 second
> {code:java}
> INFO - 2018-06-07 15:29:16.243; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:17.253; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:18.263; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> {code}
> Does 1 second seem to aggressive ?
> Maybe the waitFor could be taken into account and query the metrics at
> waitFor/2 or something ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12465) Triggers should look at metrics less often
Varun Thacker created SOLR-12465:
Summary: Triggers should look at metrics less often
Key: SOLR-12465
URL: https://issues.apache.org/jira/browse/SOLR-12465
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: AutoScaling
Reporter: Varun Thacker
As soon as I create a search rate trigger I see that the metrics endpoint is
being queried every 1 second . As discussed on SOLR-12463 we should change the
default to be 10 seconds
Example trigger
{code:java}
curl -X POST -H 'Content-type:application/json' --data-binary '{
"set-trigger": {
"name" : "search_rate_trigger",
"event" : "searchRate",
"collection" : "test_rate_trigger",
"rate" : 1.0,
"waitFor" : "1m",
"enabled" : true,
"actions" : [
{
"name" : "compute_plan",
"class": "solr.ComputePlanAction"
},
{
"name" : "execute_plan",
"class": "solr.ExecutePlanAction"
}
]
}
}' http://localhost:8983/solr/admin/autoscaling
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12463) Search rate trigger should check metrics less often
[
https://issues.apache.org/jira/browse/SOLR-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505133#comment-16505133
]
Varun Thacker commented on SOLR-12463:
--
Yeah 10 sounds fine to me. I'll file two separate Jiras and close this one
out.
> Search rate trigger should check metrics less often
> ---
>
> Key: SOLR-12463
> URL: https://issues.apache.org/jira/browse/SOLR-12463
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Varun Thacker
>Priority: Major
>
> I created a search rate trigger by running this command
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
> "set-trigger": {
> "name" : "search_rate_trigger",
> "event" : "searchRate",
> "collection" : "gettingstarted",
> "rate" : 1.0,
> "waitFor" : "1m",
> "enabled" : true,
> "actions" : [
> {
> "name" : "compute_plan",
> "class": "solr.ComputePlanAction"
> },
> {
> "name" : "execute_plan",
> "class": "solr.ExecutePlanAction"
> }
> ]
> }
> }' http://localhost:8983/solr/admin/autoscaling
> {code}
> As soon as I add this by logs start getting spammed with these every 1 second
> {code:java}
> INFO - 2018-06-07 15:29:16.243; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:17.253; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:18.263; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> {code}
> Does 1 second seem to aggressive ?
> Maybe the waitFor could be taken into account and query the metrics at
> waitFor/2 or something ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12463) Search rate trigger should check metrics less often
[
https://issues.apache.org/jira/browse/SOLR-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505125#comment-16505125
]
Shalin Shekhar Mangar commented on SOLR-12463:
--
Setting null should reset to the default -- it is a bug.
bq. Also do you think the default of 1 second is too fine grained? Maybe 15
seconds? Just thinking aloud
yeah, once a second was a good default back when we had only node added and
lost triggers. We should bump it up higher now, maybe 10 seconds is okay?
> Search rate trigger should check metrics less often
> ---
>
> Key: SOLR-12463
> URL: https://issues.apache.org/jira/browse/SOLR-12463
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Varun Thacker
>Priority: Major
>
> I created a search rate trigger by running this command
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
> "set-trigger": {
> "name" : "search_rate_trigger",
> "event" : "searchRate",
> "collection" : "gettingstarted",
> "rate" : 1.0,
> "waitFor" : "1m",
> "enabled" : true,
> "actions" : [
> {
> "name" : "compute_plan",
> "class": "solr.ComputePlanAction"
> },
> {
> "name" : "execute_plan",
> "class": "solr.ExecutePlanAction"
> }
> ]
> }
> }' http://localhost:8983/solr/admin/autoscaling
> {code}
> As soon as I add this by logs start getting spammed with these every 1 second
> {code:java}
> INFO - 2018-06-07 15:29:16.243; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:17.253; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:18.263; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> {code}
> Does 1 second seem to aggressive ?
> Maybe the waitFor could be taken into account and query the metrics at
> waitFor/2 or something ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12454) tweak Overseer leadership transition related logging
[
https://issues.apache.org/jira/browse/SOLR-12454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505118#comment-16505118
]
Christine Poerschke commented on SOLR-12454:
Attached revised SOLR-12454.patch and (not intended to be committed) patch with
script to relatively easily explore different overseer scenarios e.g.
spontaneous disappearance (here deliberate deletion) of the
{{/overseer_elect/leader}} node.
> tweak Overseer leadership transition related logging
>
>
> Key: SOLR-12454
> URL: https://issues.apache.org/jira/browse/SOLR-12454
> Project: Solr
> Issue Type: Task
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-12454-overseer-scenario-run.patch,
> SOLR-12454.patch, SOLR-12454.patch, overseer-scenario-run.sh
>
>
> Proposed patch to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12454) tweak Overseer leadership transition related logging
[ https://issues.apache.org/jira/browse/SOLR-12454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke updated SOLR-12454: --- Attachment: SOLR-12454-overseer-scenario-run.patch > tweak Overseer leadership transition related logging > > > Key: SOLR-12454 > URL: https://issues.apache.org/jira/browse/SOLR-12454 > Project: Solr > Issue Type: Task >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-12454-overseer-scenario-run.patch, > SOLR-12454.patch, SOLR-12454.patch, overseer-scenario-run.sh > > > Proposed patch to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12454) tweak Overseer leadership transition related logging
[ https://issues.apache.org/jira/browse/SOLR-12454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke updated SOLR-12454: --- Attachment: SOLR-12454.patch > tweak Overseer leadership transition related logging > > > Key: SOLR-12454 > URL: https://issues.apache.org/jira/browse/SOLR-12454 > Project: Solr > Issue Type: Task >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Attachments: SOLR-12454-overseer-scenario-run.patch, > SOLR-12454.patch, SOLR-12454.patch, overseer-scenario-run.sh > > > Proposed patch to follow. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-repro - Build # 778 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/778/ [...truncated 29 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/73/consoleText [repro] Revision: 7c6d74376a784224963b57cb8380a07279fd7608 [repro] Repro line: ant test -Dtestcase=TestLargeCluster -Dtests.method=testAddNode -Dtests.seed=A8BF159B93290B83 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-SG -Dtests.timezone=Asia/Seoul -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] Repro line: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testSplitIntegration -Dtests.seed=A8BF159B93290B83 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sl -Dtests.timezone=Australia/Queensland -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a [repro] git fetch [repro] git checkout 7c6d74376a784224963b57cb8380a07279fd7608 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] IndexSizeTriggerTest [repro] TestLargeCluster [repro] ant compile-test [...truncated 3300 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 -Dtests.class="*.IndexSizeTriggerTest|*.TestLargeCluster" -Dtests.showOutput=onerror -Dtests.seed=A8BF159B93290B83 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sl -Dtests.timezone=Australia/Queensland -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [...truncated 24727 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 2/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster [repro] Re-testing 100% failures at the tip of master [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestLargeCluster [repro] ant compile-test [...truncated 3300 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestLargeCluster" -Dtests.showOutput=onerror -Dtests.seed=A8BF159B93290B83 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-SG -Dtests.timezone=Asia/Seoul -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [...truncated 20846 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of master: [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster [repro] Re-testing 100% failures at the tip of master without a seed [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestLargeCluster [repro] ant compile-test [...truncated 3300 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestLargeCluster" -Dtests.showOutput=onerror -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-SG -Dtests.timezone=Asia/Seoul -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1 [...truncated 20851 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of master without a seed: [repro] 5/5 failed: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster [repro] git checkout 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 5 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12463) Search rate trigger should check metrics less often
[
https://issues.apache.org/jira/browse/SOLR-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505073#comment-16505073
]
Varun Thacker commented on SOLR-12463:
--
Fair enough. I wasn't aware of this setting so this should work for me.
I noticed one thing that is maybe a bug though?
1. By default the metrics get queried every 1 second
2. Now I change it to 5 second via this API and this works
{code:java}
curl -X POST -H 'Content-type:application/json' --data-binary '{
"set-properties": { "triggerScheduleDelaySeconds": 1 } }'
http://localhost:8983/solr/admin/autoscaling{code}
3. I want to change it back to the default, so I applied null but that didn't
work ( the metrics is still getting queried at 5 second interval and not 1
second). Explicitly setting it to 1 works
{code:java}
curl -X POST -H 'Content-type:application/json' --data-binary '{
"set-properties": { "triggerScheduleDelaySeconds": null } }'
http://localhost:8983/solr/admin/autoscaling{code}
Also do you think the default of 1 second is too fine grained? Maybe 15
seconds? Just thinking aloud
> Search rate trigger should check metrics less often
> ---
>
> Key: SOLR-12463
> URL: https://issues.apache.org/jira/browse/SOLR-12463
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Varun Thacker
>Priority: Major
>
> I created a search rate trigger by running this command
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
> "set-trigger": {
> "name" : "search_rate_trigger",
> "event" : "searchRate",
> "collection" : "gettingstarted",
> "rate" : 1.0,
> "waitFor" : "1m",
> "enabled" : true,
> "actions" : [
> {
> "name" : "compute_plan",
> "class": "solr.ComputePlanAction"
> },
> {
> "name" : "execute_plan",
> "class": "solr.ExecutePlanAction"
> }
> ]
> }
> }' http://localhost:8983/solr/admin/autoscaling
> {code}
> As soon as I add this by logs start getting spammed with these every 1 second
> {code:java}
> INFO - 2018-06-07 15:29:16.243; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:17.253; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:18.263; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> {code}
> Does 1 second seem to aggressive ?
> Maybe the waitFor could be taken into account and query the metrics at
> waitFor/2 or something ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_172) - Build # 22198 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22198/ Java: 64bit/jdk1.8.0_172 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC 4 tests failed. FAILED: org.apache.solr.handler.extraction.TestExtractionDateUtil.testParseDate Error Message: Incorrect parsed timestamp: 1226583351000 != 1226579751000 (Thu Nov 13 04:35:51 AKST 2008) Stack Trace: java.lang.AssertionError: Incorrect parsed timestamp: 1226583351000 != 1226579751000 (Thu Nov 13 04:35:51 AKST 2008) at __randomizedtesting.SeedInfo.seed([598FCB62B41B2A0B:1396B357CFB25DBE]:0) at org.junit.Assert.fail(Assert.java:93) at org.junit.Assert.assertTrue(Assert.java:43) at org.apache.solr.handler.extraction.TestExtractionDateUtil.assertParsedDate(TestExtractionDateUtil.java:59) at org.apache.solr.handler.extraction.TestExtractionDateUtil.testParseDate(TestExtractionDateUtil.java:54) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) FAILED: org.apache.solr.handler.extraction.TestExtractionDateUtil.testParseDate Error Message: Incorrect parsed timestamp: 1226583351000 != 1226579751000 (Thu Nov 13 04:35:51 AKST 2008) Stack Trace: java.lang.AssertionError: Incorrect parsed timestamp: 1226583351000 != 1226579751000 (Thu Nov 13 04:35:51 AKST 2008) at
[jira] [Commented] (PYLUCENE-42) JCC build fails with Python 3.7 (release candidate).
[
https://issues.apache.org/jira/browse/PYLUCENE-42?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505050#comment-16505050
]
Andi Vajda commented on PYLUCENE-42:
Patch applied, thank you for the fix !
> JCC build fails with Python 3.7 (release candidate).
>
>
> Key: PYLUCENE-42
> URL: https://issues.apache.org/jira/browse/PYLUCENE-42
> Project: PyLucene
> Issue Type: Bug
>Reporter: A. Coady
>Priority: Major
> Labels: build
> Attachments: patch.txt
>
>
> Using PyLucene 7.2.0-rc1 and the official docker image {{python:rc}}.
> {code}
> gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall
> -Wstrict-prototypes -fPIC -D_java_generics -DJCC_VER="3.1"
> -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux
> -I_jcc3 -Ijcc3/sources -I/usr/local/include/python3.7m -c
> jcc3/sources/jcc.cpp -o build/temp.linux-x86_64-3.7/jcc3/sources/jcc.o
> -DPYTHON -fno-strict-aliasing -Wno-write-strings
> cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for
> C/ObjC but not for C++
> jcc3/sources/jcc.cpp: In function ‘PyObject* initVM(PyObject*, PyObject*,
> PyObject*)’:
> jcc3/sources/jcc.cpp:486:52: error: invalid conversion from ‘const char*’ to
> ‘char*’ [-fpermissive]
> char *option = PyUnicode_AsUTF8(arg);
> ^
> error: command 'gcc' failed with exit status 1
> The command '/bin/sh -c cd jcc && JCC_JDK=/usr/lib/jvm/default-java
> python setup.py install' returned a non-zero code: 1
> make: *** [rc] Error 1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (PYLUCENE-42) JCC build fails with Python 3.7 (release candidate).
[
https://issues.apache.org/jira/browse/PYLUCENE-42?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andi Vajda resolved PYLUCENE-42.
Resolution: Fixed
> JCC build fails with Python 3.7 (release candidate).
>
>
> Key: PYLUCENE-42
> URL: https://issues.apache.org/jira/browse/PYLUCENE-42
> Project: PyLucene
> Issue Type: Bug
>Reporter: A. Coady
>Priority: Major
> Labels: build
> Attachments: patch.txt
>
>
> Using PyLucene 7.2.0-rc1 and the official docker image {{python:rc}}.
> {code}
> gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall
> -Wstrict-prototypes -fPIC -D_java_generics -DJCC_VER="3.1"
> -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/default-java/include/linux
> -I_jcc3 -Ijcc3/sources -I/usr/local/include/python3.7m -c
> jcc3/sources/jcc.cpp -o build/temp.linux-x86_64-3.7/jcc3/sources/jcc.o
> -DPYTHON -fno-strict-aliasing -Wno-write-strings
> cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for
> C/ObjC but not for C++
> jcc3/sources/jcc.cpp: In function ‘PyObject* initVM(PyObject*, PyObject*,
> PyObject*)’:
> jcc3/sources/jcc.cpp:486:52: error: invalid conversion from ‘const char*’ to
> ‘char*’ [-fpermissive]
> char *option = PyUnicode_AsUTF8(arg);
> ^
> error: command 'gcc' failed with exit status 1
> The command '/bin/sh -c cd jcc && JCC_JDK=/usr/lib/jvm/default-java
> python setup.py install' returned a non-zero code: 1
> make: *** [rc] Error 1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 237 - Still unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/237/ 7 tests failed. FAILED: org.apache.solr.cloud.RollingRestartTest.test Error Message: KeeperErrorCode = Session expired for /overseer_elect/leader Stack Trace: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader at __randomizedtesting.SeedInfo.seed([A505572E8A8C81FF:2D5168F42470EC07]:0) at org.apache.zookeeper.KeeperException.create(KeeperException.java:130) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215) at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:341) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:341) at org.apache.solr.cloud.OverseerTaskProcessor.getLeaderId(OverseerTaskProcessor.java:379) at org.apache.solr.cloud.OverseerTaskProcessor.getLeaderNode(OverseerTaskProcessor.java:370) at org.apache.solr.cloud.RollingRestartTest.waitUntilOverseerDesignateIsLeader(RollingRestartTest.java:137) at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104) at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:52) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[JENKINS] Lucene-Solr-repro - Build # 777 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/777/ [...truncated 29 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1042/consoleText [repro] Revision: 7c6d74376a784224963b57cb8380a07279fd7608 [repro] Ant options: -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 [repro] Repro line: ant test -Dtestcase=IndexSizeTriggerTest -Dtests.method=testMergeIntegration -Dtests.seed=40B79A9D574AC6EF -Dtests.multiplier=2 -Dtests.locale=ar-MA -Dtests.timezone=Europe/Gibraltar -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=TestExecutePlanAction -Dtests.method=testExecute -Dtests.seed=40B79A9D574AC6EF -Dtests.multiplier=2 -Dtests.locale=cs-CZ -Dtests.timezone=Pacific/Truk -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a [repro] git fetch [...truncated 2 lines...] [repro] git checkout 7c6d74376a784224963b57cb8380a07279fd7608 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] IndexSizeTriggerTest [repro] TestExecutePlanAction [repro] ant compile-test [...truncated 3300 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 -Dtests.class="*.IndexSizeTriggerTest|*.TestExecutePlanAction" -Dtests.showOutput=onerror -DsmokeTestRelease.java9=/home/jenkins/tools/java/latest1.9 -Dtests.seed=40B79A9D574AC6EF -Dtests.multiplier=2 -Dtests.locale=ar-MA -Dtests.timezone=Europe/Gibraltar -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 1941 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 0/5 failed: org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction [repro] 1/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest [repro] git checkout 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 5 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-repro - Build # 776 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/776/ [...truncated 58 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-Tests-master/2557/consoleText [repro] Revision: d1631593f3a0ff9f58468ff0f32d7c1e8a404567 [repro] Repro line: ant test -Dtestcase=TestCloudPivotFacet -Dtests.method=test -Dtests.seed=50A3CB9B76501094 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es-PY -Dtests.timezone=Africa/Tunis -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=TestExecutePlanAction -Dtests.method=testExecute -Dtests.seed=50A3CB9B76501094 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=Europe/Ulyanovsk -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] Repro line: ant test -Dtestcase=Tagger2Test -Dtests.method=testStopWords -Dtests.seed=50A3CB9B76501094 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sk-SK -Dtests.timezone=Pacific/Marquesas -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a [repro] git fetch [...truncated 2 lines...] [repro] git checkout d1631593f3a0ff9f58468ff0f32d7c1e8a404567 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] TestExecutePlanAction [repro] Tagger2Test [repro] TestCloudPivotFacet [repro] ant compile-test [...truncated 3300 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 -Dtests.class="*.TestExecutePlanAction|*.Tagger2Test|*.TestCloudPivotFacet" -Dtests.showOutput=onerror -Dtests.seed=50A3CB9B76501094 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=th-TH -Dtests.timezone=Europe/Ulyanovsk -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 5817 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 0/5 failed: org.apache.solr.cloud.TestCloudPivotFacet [repro] 1/5 failed: org.apache.solr.cloud.autoscaling.sim.TestExecutePlanAction [repro] 5/5 failed: org.apache.solr.handler.tagger.Tagger2Test [repro] Re-testing 100% failures at the tip of master [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] Tagger2Test [repro] ant compile-test [...truncated 3300 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.Tagger2Test" -Dtests.showOutput=onerror -Dtests.seed=50A3CB9B76501094 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sk-SK -Dtests.timezone=Pacific/Marquesas -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 802 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of master: [repro] 5/5 failed: org.apache.solr.handler.tagger.Tagger2Test [repro] Re-testing 100% failures at the tip of master without a seed [repro] ant clean [...truncated 8 lines...] [repro] Test suites by module: [repro]solr/core [repro] Tagger2Test [repro] ant compile-test [...truncated 3300 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.Tagger2Test" -Dtests.showOutput=onerror -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sk-SK -Dtests.timezone=Pacific/Marquesas -Dtests.asserts=true -Dtests.file.encoding=UTF-8 [...truncated 801 lines...] [repro] Setting last failure code to 256 [repro] Failures at the tip of master without a seed: [repro] 5/5 failed: org.apache.solr.handler.tagger.Tagger2Test [repro] git checkout 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 6 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505016#comment-16505016 ] Alessandro Benedetti commented on LUCENE-8343: -- In the meantime I attached the patch with the overflow edge case fixed and a better handling of the weight just when it is too small. Happy to discuss the implications of "turning weight=0 into 1" with the community! > BlendedInfixSuggester bad score calculus for certain suggestion weights > --- > > Key: LUCENE-8343 > URL: https://issues.apache.org/jira/browse/LUCENE-8343 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: LUCENE-8343.patch, LUCENE-8343.patch, LUCENE-8343.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the BlendedInfixSuggester return a (long) score to rank the > suggestions. > This score is calculated as a multiplication between : > long *Weight* : the suggestion weight, coming from a document field, it can > be any long value ( including 1, 0,.. ) > double *Coefficient* : 0<=x<=1, calculated based on the position match, > earlier the better > The resulting score is a long, which means that at the moment, any weight<10 > can bring inconsistencies. > *Edge cases* > Weight =1 > Score = 1( if we have a match at the beginning of the suggestion) or 0 ( for > any other match) > Weight =0 > Score = 0 ( independently of the position match coefficient) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Benedetti updated LUCENE-8343: - Attachment: LUCENE-8343.patch > BlendedInfixSuggester bad score calculus for certain suggestion weights > --- > > Key: LUCENE-8343 > URL: https://issues.apache.org/jira/browse/LUCENE-8343 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: LUCENE-8343.patch, LUCENE-8343.patch, LUCENE-8343.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the BlendedInfixSuggester return a (long) score to rank the > suggestions. > This score is calculated as a multiplication between : > long *Weight* : the suggestion weight, coming from a document field, it can > be any long value ( including 1, 0,.. ) > double *Coefficient* : 0<=x<=1, calculated based on the position match, > earlier the better > The resulting score is a long, which means that at the moment, any weight<10 > can bring inconsistencies. > *Edge cases* > Weight =1 > Score = 1( if we have a match at the beginning of the suggestion) or 0 ( for > any other match) > Weight =0 > Score = 0 ( independently of the position match coefficient) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-master - Build # 2558 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2558/ 1 tests failed. FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling Error Message: Both triggers should have fired by now Stack Trace: java.lang.AssertionError: Both triggers should have fired by now at __randomizedtesting.SeedInfo.seed([FF551A54CE11CD1D:477B2711CBB2E8F]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling(TestTriggerIntegration.java:185) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) Build Log: [...truncated 14531 lines...] [junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration [junit4] 2> 3037585 INFO (SUITE-TestTriggerIntegration-seed#[FF551A54CE11CD1D]-worker) [] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null &
[jira] [Comment Edited] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504909#comment-16504909 ] Alessandro Benedetti edited comment on LUCENE-8343 at 6/7/18 5:42 PM: -- Hi [~jpountz], thanks for your time, I can give you a quick explanation here: The (positional) coefficient should be a double 0<=x<=1 calculated with 3 possible formulas from the position of the first matching query term in the suggestion ( linear doesn't respect that constraint and can go negative for postion which are farer than 10 positions from the beginning ) : * *position_linear*: (1 – 0.10*position): Matches to the start will be given a higher score (Default) * *position_reciprocal*: 1/(1+position): Matches to the start will be given a score which decay faster than linear * *position_exponential_reciprocal*: 1/pow(1+position,exponent): Matches to the start will be given a score which decay faster than reciprocal To answer your questions : 1) "turning weight=0 into 1" , so this is an interesting one : You don't want all your weights to be 0 for the BlendedInfixSuggester because you would just flat to 0 the positional score of the suggestion, which is the only reason to use the Blended Infix ( if you are not interested in the positional score for the suggestion, you should use the parent suggester : AnalyzingInfixSuggester) If you don't configure the weight field ( which is not and shouldn't be mandatory) all your weights go to 0s (org.apache.lucene.search.suggest.DocumentDictionary.DocumentInputIterator#getWeight ) and your BlendedInfixSuggester doesn't blend anything anymore scoring each suggestion a constant 0. That was the reason to move the weight 0 to the smallest bigger value ( which in a long data type is 1) . With that fix you limit the ability of a user to move certain suggestions to 0 weight ( they can just drop them to 1 weight) , but you gain a good bug fix for the missing weight field scenario. 2) So the chosen of 10 was completely arbitrary to get at least 10 possible ranked outcomes out of the positional coefficient. You may end up in overflows if : - the weight is already big enough. You are right maybe we can apply that scaling factor only if the weight is small. The overflow according to my analysis can not come from the coefficient, because the edge cases for linear are : 1 - where input position is 0 -2.147483637002E8 - where input position is [Integer.MAX_VALUE|http://java.sun.com/j2se/1.5.0/docs/api/java/lang/Integer.html#MAX_VALUE] ( which is not going to be achievable as Strings full length are maxed by that value) was (Author: alessandro.benedetti): Hi [~jpountz], thanks for your time, I can give you a quick explanation here: The (positional) coefficient should be a double 0<=x<=1 calculated with 3 possible formulas from the position of the first matching query term in the suggestion ( linear doesn't respect that constraint and can go negative for postion which are farer than 10 positions from the beginning ) : * *position_linear*: (1 – 0.10*position): Matches to the start will be given a higher score (Default) * *position_reciprocal*: 1/(1+position): Matches to the start will be given a score which decay faster than linear * *position_exponential_reciprocal*: 1/pow(1+position,exponent): Matches to the start will be given a score which decay faster than reciprocal To answer your questions : 1) "turning weight=0 into 1" , so this is an interesting one : You don't want all your weights to be 0 for the BlendedInfixSuggester because you would just flat to 0 the positional score of the suggestion, which is the only reason to use the Blended Infix ( if you are not interested in the positional score for the suggestion, you should use the parent suggester : AnalyzingInfixSuggester) If you don't configure the weight field ( which is not and shouldn't be mandatory) all your weights go to 0s (org.apache.lucene.search.suggest.DocumentDictionary.DocumentInputIterator#getWeight ) and your BlendedInfixSuggester doesn't blend anything anymore scoring each suggestion a constant 0. That was the reason to move the weight 0 to the smallest bigger value ( which in a long data type is 1) . With that fix you limit the ability of a user to move certain suggestions to 0 weight ( they can just drop them to 1 weight) , but you gain a good bug fix for the missing weight field scenario. 2) So the chosen of 10 was completely arbitrary to get at least 10 possible ranked outcomes out of the positional coefficient. You may end up in overflows if : : - the weight is already big enough. You are right maybe we can apply that scaling factor only if the weight is small. - the linear coefficient goes deep negative ( we can limit the coefficient score to a minimum of 0, which will also give Linear a behaviour similar to its
Re: ScorerSupplier and cost() avoidance
Doesn't BQ rewrite itself if it has only one clause? Or maybe if there were more than one clause, and then all but one of them had null scorers (on SHOULD clauses) you could wind up in that state? Mike McCandless http://blog.mikemccandless.com On Thu, Jun 7, 2018 at 1:21 PM, David Smiley wrote: > I'm working with some folks who did some profiling and noticed that > ScorerSupplier.cost() can be expensive (as the javadocs say). cost() says > only to call it if necessary. Unfortunately, a BooleanQuery is going to > call cost() (via BooleanWeight.scorer() even if ultimately no Query in the > tree cares what the cost is. I'm not sure if that's a perf bug or not; > it's hard to tell. > > The expensive part of cost() for Boolean2ScorerSupplier is over in > MinShouldMatchSumScorer.cost which creates a PriorityQueue every time, even > if trivially numScorers == 1. That's a weird case... why do we even need a > Boolean2ScorerSupplier around one clause; couldn't that clause be returned > from the outer weight, BooleanWeight.scorerSupplier() close to the end as > an optimization? I could file an issue. > > ~ David > -- > Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker > LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www. > solrenterprisesearchserver.com >
ScorerSupplier and cost() avoidance
I'm working with some folks who did some profiling and noticed that ScorerSupplier.cost() can be expensive (as the javadocs say). cost() says only to call it if necessary. Unfortunately, a BooleanQuery is going to call cost() (via BooleanWeight.scorer() even if ultimately no Query in the tree cares what the cost is. I'm not sure if that's a perf bug or not; it's hard to tell. The expensive part of cost() for Boolean2ScorerSupplier is over in MinShouldMatchSumScorer.cost which creates a PriorityQueue every time, even if trivially numScorers == 1. That's a weird case... why do we even need a Boolean2ScorerSupplier around one clause; couldn't that clause be returned from the outer weight, BooleanWeight.scorerSupplier() close to the end as an optimization? I could file an issue. ~ David -- Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.solrenterprisesearchserver.com
[jira] [Commented] (SOLR-12463) Search rate trigger should check metrics less often
[
https://issues.apache.org/jira/browse/SOLR-12463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504967#comment-16504967
]
Shalin Shekhar Mangar commented on SOLR-12463:
--
You can change the trigger delay by setting an autoscaling property called
"triggerScheduleDelaySeconds". However, it changes the delay for all triggers.
We don't support different delays for different triggers today. Doing so seems
complicated to me without much upside.
> Search rate trigger should check metrics less often
> ---
>
> Key: SOLR-12463
> URL: https://issues.apache.org/jira/browse/SOLR-12463
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: AutoScaling
>Reporter: Varun Thacker
>Priority: Major
>
> I created a search rate trigger by running this command
> {code:java}
> curl -X POST -H 'Content-type:application/json' --data-binary '{
> "set-trigger": {
> "name" : "search_rate_trigger",
> "event" : "searchRate",
> "collection" : "gettingstarted",
> "rate" : 1.0,
> "waitFor" : "1m",
> "enabled" : true,
> "actions" : [
> {
> "name" : "compute_plan",
> "class": "solr.ComputePlanAction"
> },
> {
> "name" : "execute_plan",
> "class": "solr.ExecutePlanAction"
> }
> ]
> }
> }' http://localhost:8983/solr/admin/autoscaling
> {code}
> As soon as I add this by logs start getting spammed with these every 1 second
> {code:java}
> INFO - 2018-06-07 15:29:16.243; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:17.253; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> INFO - 2018-06-07 15:29:18.263; [ ] org.apache.solr.servlet.HttpSolrCall;
> [admin] webapp=null path=/admin/metrics
> params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
> status=0 QTime=0
> {code}
> Does 1 second seem to aggressive ?
> Maybe the waitFor could be taken into account and query the metrics at
> waitFor/2 or something ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12458) ADLS support for SOLR
[ https://issues.apache.org/jira/browse/SOLR-12458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504930#comment-16504930 ] Mike Wingert commented on SOLR-12458: - Attaching my first pass. It's based on the HDFS implementation. things seem to be working. I'll be adding tests and documentation. > ADLS support for SOLR > - > > Key: SOLR-12458 > URL: https://issues.apache.org/jira/browse/SOLR-12458 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: Hadoop Integration >Affects Versions: master (8.0) >Reporter: Mike Wingert >Priority: Major > Fix For: master (8.0) > > Attachments: SOLR-12458.patch > > > This is to track ADLS support for SOLR. > ADLS is a HDFS like API available in Microsoft Azure. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12458) ADLS support for SOLR
[ https://issues.apache.org/jira/browse/SOLR-12458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Wingert updated SOLR-12458: Attachment: SOLR-12458.patch > ADLS support for SOLR > - > > Key: SOLR-12458 > URL: https://issues.apache.org/jira/browse/SOLR-12458 > Project: Solr > Issue Type: New Feature > Security Level: Public(Default Security Level. Issues are Public) > Components: Hadoop Integration >Affects Versions: master (8.0) >Reporter: Mike Wingert >Priority: Major > Fix For: master (8.0) > > Attachments: SOLR-12458.patch > > > This is to track ADLS support for SOLR. > ADLS is a HDFS like API available in Microsoft Azure. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12142) EmbeddedSolrServer should use req.getContentWriter
[ https://issues.apache.org/jira/browse/SOLR-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504912#comment-16504912 ] ASF subversion and git services commented on SOLR-12142: Commit 41d04e6e22e1841cb88ba9812d9826939a7c3d81 in lucene-solr's branch refs/heads/branch_7x from [~noble.paul] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=41d04e6 ] SOLR-12142: EmbeddedSolrServer should use req.getContentWriter > EmbeddedSolrServer should use req.getContentWriter > --- > > Key: SOLR-12142 > URL: https://issues.apache.org/jira/browse/SOLR-12142 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Assignee: Noble Paul >Priority: Major > Fix For: 7.4 > > Attachments: SOLR-12142.patch, SOLR-12142.patch > > > In SOLR-11380, SolrRequest.getContentWriter was introduced as a replacement > for getContentStreams. However, EmbeddedSolrServer still calls > getContentStreams, and so clients who need to send POST data to it cannot yet > switch from the Deprecated API to the new API. The SolrTextTagger is an > example of a project where one would want to do this. > It seems EmbeddedSolrServer ought to check for getContentWriter and if > present then convert it into a ContentStream somehow. For the time being, > ESS needs to call both since both APIs exist. > CC [~noble.paul] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504909#comment-16504909 ] Alessandro Benedetti commented on LUCENE-8343: -- Hi [~jpountz], thanks for your time, I can give you a quick explanation here: The (positional) coefficient should be a double 0<=x<=1 calculated with 3 possible formulas from the position of the first matching query term in the suggestion ( linear doesn't respect that constraint and can go negative for postion which are farer than 10 positions from the beginning ) : * *position_linear*: (1 – 0.10*position): Matches to the start will be given a higher score (Default) * *position_reciprocal*: 1/(1+position): Matches to the start will be given a score which decay faster than linear * *position_exponential_reciprocal*: 1/pow(1+position,exponent): Matches to the start will be given a score which decay faster than reciprocal To answer your questions : 1) "turning weight=0 into 1" , so this is an interesting one : You don't want all your weights to be 0 for the BlendedInfixSuggester because you would just flat to 0 the positional score of the suggestion, which is the only reason to use the Blended Infix ( if you are not interested in the positional score for the suggestion, you should use the parent suggester : AnalyzingInfixSuggester) If you don't configure the weight field ( which is not and shouldn't be mandatory) all your weights go to 0s (org.apache.lucene.search.suggest.DocumentDictionary.DocumentInputIterator#getWeight ) and your BlendedInfixSuggester doesn't blend anything anymore scoring each suggestion a constant 0. That was the reason to move the weight 0 to the smallest bigger value ( which in a long data type is 1) . With that fix you limit the ability of a user to move certain suggestions to 0 weight ( they can just drop them to 1 weight) , but you gain a good bug fix for the missing weight field scenario. 2) So the chosen of 10 was completely arbitrary to get at least 10 possible ranked outcomes out of the positional coefficient. You may end up in overflows if : : - the weight is already big enough. You are right maybe we can apply that scaling factor only if the weight is small. - the linear coefficient goes deep negative ( we can limit the coefficient score to a minimum of 0, which will also give Linear a behaviour similar to its siblings blender types) > BlendedInfixSuggester bad score calculus for certain suggestion weights > --- > > Key: LUCENE-8343 > URL: https://issues.apache.org/jira/browse/LUCENE-8343 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: LUCENE-8343.patch, LUCENE-8343.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the BlendedInfixSuggester return a (long) score to rank the > suggestions. > This score is calculated as a multiplication between : > long *Weight* : the suggestion weight, coming from a document field, it can > be any long value ( including 1, 0,.. ) > double *Coefficient* : 0<=x<=1, calculated based on the position match, > earlier the better > The resulting score is a long, which means that at the moment, any weight<10 > can bring inconsistencies. > *Edge cases* > Weight =1 > Score = 1( if we have a match at the beginning of the suggestion) or 0 ( for > any other match) > Weight =0 > Score = 0 ( independently of the position match coefficient) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12142) EmbeddedSolrServer should use req.getContentWriter
[ https://issues.apache.org/jira/browse/SOLR-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504894#comment-16504894 ] ASF subversion and git services commented on SOLR-12142: Commit 9b65d7e1a20c6f1cb4eeb298951e98bef6a6cf8a in lucene-solr's branch refs/heads/master from [~noble.paul] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9b65d7e ] SOLR-12142: EmbeddedSolrServer should use req.getContentWriter > EmbeddedSolrServer should use req.getContentWriter > --- > > Key: SOLR-12142 > URL: https://issues.apache.org/jira/browse/SOLR-12142 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Assignee: Noble Paul >Priority: Major > Fix For: 7.4 > > Attachments: SOLR-12142.patch, SOLR-12142.patch > > > In SOLR-11380, SolrRequest.getContentWriter was introduced as a replacement > for getContentStreams. However, EmbeddedSolrServer still calls > getContentStreams, and so clients who need to send POST data to it cannot yet > switch from the Deprecated API to the new API. The SolrTextTagger is an > example of a project where one would want to do this. > It seems EmbeddedSolrServer ought to check for getContentWriter and if > present then convert it into a ContentStream somehow. For the time being, > ESS needs to call both since both APIs exist. > CC [~noble.paul] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12018) Ref Guide: Comment system is offline
[ https://issues.apache.org/jira/browse/SOLR-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504858#comment-16504858 ] ASF subversion and git services commented on SOLR-12018: Commit 377da9ca94871e651b8251e7029e06791c60c594 in lucene-solr's branch refs/heads/branch_7_4 from [~ctargett] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=377da9c ] SOLR-12018: add committer attribution > Ref Guide: Comment system is offline > > > Key: SOLR-12018 > URL: https://issues.apache.org/jira/browse/SOLR-12018 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: RefGuideCommentsBroken.png, SOLR-12018.patch > > > The Ref Guide uses comments.apache.org to allow user comments. Sometime in > December/early January, it was taken offline. > I filed INFRA-15947 to ask after it's long-term status, and recently got an > answer that it an ETA is mid-March for a permanent INFRA-hosted system. > However, it's of course possible changes in priorities or other factors will > delay that timeline. > Every Ref Guide page currently invites users to leave comments, but since the > whole Comments area is pulled in via JavaScript from a non-existent server, > there's no space to do so (see attached screenshot). While we wait for the > permanent server to be online, we have a couple of options: > # Leave it the way it is and hopefully by mid-March it will be back > # Change the text to tell users it's not working temporarily on all published > versions > # Remove it from all the published versions and put it back when it's back > I'm not a great fan of #2 or #3, because it'd be a bit of work for me to > backport changes to 4 branches and republish every guide just to fix it again > in a month or so. I'm fine with option #1 since I've known about it for about > a month at least and as far as I can tell no one else has noticed. But if > people feel strongly about it now that they know about it, we can figure > something out. > If for some reason it takes longer than mid-March to get it back, or INFRA > chooses to stop supporting it entirely, this issue can morph into what we > should do for an alternative permanent solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504852#comment-16504852 ] Alessandro Benedetti edited comment on LUCENE-8343 at 6/7/18 4:00 PM: -- Hi [~ctargett], I did the change and pushed, so they were just in the Jira associated Pull Request : [GitHub Pull Request #391 |https://github.com/apache/lucene-solr/pull/391]I just uploaded the patch as well. You can take a look now ( I think the Github Pull Request is easier to read, but feel free to use the patch at your convenience)| was (Author: alessandro.benedetti): Hi Cassandra, I did the change and pushed, so they were just in the Jira associated Pull Request : [GitHub Pull Request #391 |https://github.com/apache/lucene-solr/pull/391]I just uploaded the patch as well. You can take a look now ( I think the Github Pull Request is easier to read, but feel free to use the patch at your convenience) > BlendedInfixSuggester bad score calculus for certain suggestion weights > --- > > Key: LUCENE-8343 > URL: https://issues.apache.org/jira/browse/LUCENE-8343 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: LUCENE-8343.patch, LUCENE-8343.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the BlendedInfixSuggester return a (long) score to rank the > suggestions. > This score is calculated as a multiplication between : > long *Weight* : the suggestion weight, coming from a document field, it can > be any long value ( including 1, 0,.. ) > double *Coefficient* : 0<=x<=1, calculated based on the position match, > earlier the better > The resulting score is a long, which means that at the moment, any weight<10 > can bring inconsistencies. > *Edge cases* > Weight =1 > Score = 1( if we have a match at the beginning of the suggestion) or 0 ( for > any other match) > Weight =0 > Score = 0 ( independently of the position match coefficient) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12018) Ref Guide: Comment system is offline
[ https://issues.apache.org/jira/browse/SOLR-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504854#comment-16504854 ] ASF subversion and git services commented on SOLR-12018: Commit 9d48bd9d2a980c99c2277a000c5bd955e2d01c2c in lucene-solr's branch refs/heads/branch_7x from [~ctargett] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9d48bd9 ] SOLR-12018: add committer attribution > Ref Guide: Comment system is offline > > > Key: SOLR-12018 > URL: https://issues.apache.org/jira/browse/SOLR-12018 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: RefGuideCommentsBroken.png, SOLR-12018.patch > > > The Ref Guide uses comments.apache.org to allow user comments. Sometime in > December/early January, it was taken offline. > I filed INFRA-15947 to ask after it's long-term status, and recently got an > answer that it an ETA is mid-March for a permanent INFRA-hosted system. > However, it's of course possible changes in priorities or other factors will > delay that timeline. > Every Ref Guide page currently invites users to leave comments, but since the > whole Comments area is pulled in via JavaScript from a non-existent server, > there's no space to do so (see attached screenshot). While we wait for the > permanent server to be online, we have a couple of options: > # Leave it the way it is and hopefully by mid-March it will be back > # Change the text to tell users it's not working temporarily on all published > versions > # Remove it from all the published versions and put it back when it's back > I'm not a great fan of #2 or #3, because it'd be a bit of work for me to > backport changes to 4 branches and republish every guide just to fix it again > in a month or so. I'm fine with option #1 since I've known about it for about > a month at least and as far as I can tell no one else has noticed. But if > people feel strongly about it now that they know about it, we can figure > something out. > If for some reason it takes longer than mid-March to get it back, or INFRA > chooses to stop supporting it entirely, this issue can morph into what we > should do for an alternative permanent solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504852#comment-16504852 ] Alessandro Benedetti commented on LUCENE-8343: -- Hi Cassandra, I did the change and pushed, so they were just in the Jira associated Pull Request : [GitHub Pull Request #391 |https://github.com/apache/lucene-solr/pull/391]I just uploaded the patch as well. You can take a look now ( I think the Github Pull Request is easier to read, but feel free to use the patch at your convenience) > BlendedInfixSuggester bad score calculus for certain suggestion weights > --- > > Key: LUCENE-8343 > URL: https://issues.apache.org/jira/browse/LUCENE-8343 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: LUCENE-8343.patch, LUCENE-8343.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the BlendedInfixSuggester return a (long) score to rank the > suggestions. > This score is calculated as a multiplication between : > long *Weight* : the suggestion weight, coming from a document field, it can > be any long value ( including 1, 0,.. ) > double *Coefficient* : 0<=x<=1, calculated based on the position match, > earlier the better > The resulting score is a long, which means that at the moment, any weight<10 > can bring inconsistencies. > *Edge cases* > Weight =1 > Score = 1( if we have a match at the beginning of the suggestion) or 0 ( for > any other match) > Weight =0 > Score = 0 ( independently of the position match coefficient) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12018) Ref Guide: Comment system is offline
[ https://issues.apache.org/jira/browse/SOLR-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504851#comment-16504851 ] ASF subversion and git services commented on SOLR-12018: Commit 8cb6e3d3c88182f1fdc3fc4f6de640db13a88745 in lucene-solr's branch refs/heads/master from [~ctargett] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8cb6e3d ] SOLR-12018: add committer attribution > Ref Guide: Comment system is offline > > > Key: SOLR-12018 > URL: https://issues.apache.org/jira/browse/SOLR-12018 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: RefGuideCommentsBroken.png, SOLR-12018.patch > > > The Ref Guide uses comments.apache.org to allow user comments. Sometime in > December/early January, it was taken offline. > I filed INFRA-15947 to ask after it's long-term status, and recently got an > answer that it an ETA is mid-March for a permanent INFRA-hosted system. > However, it's of course possible changes in priorities or other factors will > delay that timeline. > Every Ref Guide page currently invites users to leave comments, but since the > whole Comments area is pulled in via JavaScript from a non-existent server, > there's no space to do so (see attached screenshot). While we wait for the > permanent server to be online, we have a couple of options: > # Leave it the way it is and hopefully by mid-March it will be back > # Change the text to tell users it's not working temporarily on all published > versions > # Remove it from all the published versions and put it back when it's back > I'm not a great fan of #2 or #3, because it'd be a bit of work for me to > backport changes to 4 branches and republish every guide just to fix it again > in a month or so. I'm fine with option #1 since I've known about it for about > a month at least and as far as I can tell no one else has noticed. But if > people feel strongly about it now that they know about it, we can figure > something out. > If for some reason it takes longer than mid-March to get it back, or INFRA > chooses to stop supporting it entirely, this issue can morph into what we > should do for an alternative permanent solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Benedetti updated LUCENE-8343: - Attachment: LUCENE-8343.patch > BlendedInfixSuggester bad score calculus for certain suggestion weights > --- > > Key: LUCENE-8343 > URL: https://issues.apache.org/jira/browse/LUCENE-8343 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: LUCENE-8343.patch, LUCENE-8343.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the BlendedInfixSuggester return a (long) score to rank the > suggestions. > This score is calculated as a multiplication between : > long *Weight* : the suggestion weight, coming from a document field, it can > be any long value ( including 1, 0,.. ) > double *Coefficient* : 0<=x<=1, calculated based on the position match, > earlier the better > The resulting score is a long, which means that at the moment, any weight<10 > can bring inconsistencies. > *Edge cases* > Weight =1 > Score = 1( if we have a match at the beginning of the suggestion) or 0 ( for > any other match) > Weight =0 > Score = 0 ( independently of the position match coefficient) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12454) tweak Overseer leadership transition related logging
[
https://issues.apache.org/jira/browse/SOLR-12454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504841#comment-16504841
]
Christine Poerschke commented on SOLR-12454:
bq. ... {{Overseer (id=null) closing}} logging emitted at the _start_ of
apparently _every_ solr node seems confusing?
Okay, so this is at the start of _almost_ every solr node excluding the node
that wins the overseer election. SOLR-12464 opened for this specifically
separately for clarity.
> tweak Overseer leadership transition related logging
>
>
> Key: SOLR-12454
> URL: https://issues.apache.org/jira/browse/SOLR-12454
> Project: Solr
> Issue Type: Task
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-12454.patch, overseer-scenario-run.sh
>
>
> Proposed patch to follow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504840#comment-16504840 ] Adrien Grand commented on LUCENE-8343: -- I'm not very familiar with this suggester so I might be missing obvious things, but I don't understand why turning weight=0 into 1 is right, or why it's better to multiply the coefficient by 10. It might even introduce overflows? > BlendedInfixSuggester bad score calculus for certain suggestion weights > --- > > Key: LUCENE-8343 > URL: https://issues.apache.org/jira/browse/LUCENE-8343 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: LUCENE-8343.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the BlendedInfixSuggester return a (long) score to rank the > suggestions. > This score is calculated as a multiplication between : > long *Weight* : the suggestion weight, coming from a document field, it can > be any long value ( including 1, 0,.. ) > double *Coefficient* : 0<=x<=1, calculated based on the position match, > earlier the better > The resulting score is a long, which means that at the moment, any weight<10 > can bring inconsistencies. > *Edge cases* > Weight =1 > Score = 1( if we have a match at the beginning of the suggestion) or 0 ( for > any other match) > Weight =0 > Score = 0 ( independently of the position match coefficient) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12464) reduce Overseer.close() logging (for non-Overseer leaders)
[
https://issues.apache.org/jira/browse/SOLR-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke updated SOLR-12464:
---
Attachment: SOLR-12464.patch
> reduce Overseer.close() logging (for non-Overseer leaders)
> --
>
> Key: SOLR-12464
> URL: https://issues.apache.org/jira/browse/SOLR-12464
> Project: Solr
> Issue Type: Task
> Components: logging
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-12464-ea.patch, SOLR-12464.patch
>
>
> ZkController.init creates an Overseer object:
> * If that object becomes overseer leader then its start method is called.
> * If the object does not become overseer leader then its close method is
> called and that currently emits mildly confusing {{Overseer (id=null)
> closing}} info logging -- confusing especially since this happens at node
> startup.
> ZkController.init creates an Overseer object:
> * If that object becomes overseer leader then its start method is called and
> (if assertions are enabled) ObjectReleaseTracker.track is called.
> * If the object does not become overseer leader then its close method is
> called and (if assertions are enabled) ObjectReleaseTracker.release is called
> despite ObjectReleaseTracker.track not having been called previously.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12464) reduce Overseer.close() logging (for non-Overseer leaders)
[
https://issues.apache.org/jira/browse/SOLR-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504836#comment-16504836
]
Christine Poerschke commented on SOLR-12464:
Attached proposed SOLR-12464.patch file.
> reduce Overseer.close() logging (for non-Overseer leaders)
> --
>
> Key: SOLR-12464
> URL: https://issues.apache.org/jira/browse/SOLR-12464
> Project: Solr
> Issue Type: Task
> Components: logging
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-12464-ea.patch, SOLR-12464.patch
>
>
> ZkController.init creates an Overseer object:
> * If that object becomes overseer leader then its start method is called.
> * If the object does not become overseer leader then its close method is
> called and that currently emits mildly confusing {{Overseer (id=null)
> closing}} info logging -- confusing especially since this happens at node
> startup.
> ZkController.init creates an Overseer object:
> * If that object becomes overseer leader then its start method is called and
> (if assertions are enabled) ObjectReleaseTracker.track is called.
> * If the object does not become overseer leader then its close method is
> called and (if assertions are enabled) ObjectReleaseTracker.release is called
> despite ObjectReleaseTracker.track not having been called previously.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12464) reduce Overseer.close() logging (for non-Overseer leaders)
[
https://issues.apache.org/jira/browse/SOLR-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504833#comment-16504833
]
Christine Poerschke commented on SOLR-12464:
The attached demo SOLR-12464-ea.patch in conjunction with the SOLR-12462.patch
shows the 'release of untracked object' behaviour when starting/stopping a
cloud (with external zookeeper, have not tried with embedded zookeeper).
SOLR-12464-ea.patch is _not_ intended to be committed.
> reduce Overseer.close() logging (for non-Overseer leaders)
> --
>
> Key: SOLR-12464
> URL: https://issues.apache.org/jira/browse/SOLR-12464
> Project: Solr
> Issue Type: Task
> Components: logging
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-12464-ea.patch
>
>
> ZkController.init creates an Overseer object:
> * If that object becomes overseer leader then its start method is called.
> * If the object does not become overseer leader then its close method is
> called and that currently emits mildly confusing {{Overseer (id=null)
> closing}} info logging -- confusing especially since this happens at node
> startup.
> ZkController.init creates an Overseer object:
> * If that object becomes overseer leader then its start method is called and
> (if assertions are enabled) ObjectReleaseTracker.track is called.
> * If the object does not become overseer leader then its close method is
> called and (if assertions are enabled) ObjectReleaseTracker.release is called
> despite ObjectReleaseTracker.track not having been called previously.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12464) reduce Overseer.close() logging (for non-Overseer leaders)
[
https://issues.apache.org/jira/browse/SOLR-12464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke updated SOLR-12464:
---
Attachment: SOLR-12464-ea.patch
> reduce Overseer.close() logging (for non-Overseer leaders)
> --
>
> Key: SOLR-12464
> URL: https://issues.apache.org/jira/browse/SOLR-12464
> Project: Solr
> Issue Type: Task
> Components: logging
>Reporter: Christine Poerschke
>Assignee: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-12464-ea.patch
>
>
> ZkController.init creates an Overseer object:
> * If that object becomes overseer leader then its start method is called.
> * If the object does not become overseer leader then its close method is
> called and that currently emits mildly confusing {{Overseer (id=null)
> closing}} info logging -- confusing especially since this happens at node
> startup.
> ZkController.init creates an Overseer object:
> * If that object becomes overseer leader then its start method is called and
> (if assertions are enabled) ObjectReleaseTracker.track is called.
> * If the object does not become overseer leader then its close method is
> called and (if assertions are enabled) ObjectReleaseTracker.release is called
> despite ObjectReleaseTracker.track not having been called previously.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12464) reduce Overseer.close() logging (for non-Overseer leaders)
Christine Poerschke created SOLR-12464:
--
Summary: reduce Overseer.close() logging (for non-Overseer leaders)
Key: SOLR-12464
URL: https://issues.apache.org/jira/browse/SOLR-12464
Project: Solr
Issue Type: Task
Components: logging
Reporter: Christine Poerschke
Assignee: Christine Poerschke
ZkController.init creates an Overseer object:
* If that object becomes overseer leader then its start method is called.
* If the object does not become overseer leader then its close method is called
and that currently emits mildly confusing {{Overseer (id=null) closing}} info
logging -- confusing especially since this happens at node startup.
ZkController.init creates an Overseer object:
* If that object becomes overseer leader then its start method is called and
(if assertions are enabled) ObjectReleaseTracker.track is called.
* If the object does not become overseer leader then its close method is called
and (if assertions are enabled) ObjectReleaseTracker.release is called despite
ObjectReleaseTracker.track not having been called previously.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12462) ObjectReleaseTracker silently 'releases' untracked objects
[ https://issues.apache.org/jira/browse/SOLR-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504822#comment-16504822 ] Christine Poerschke commented on SOLR-12462: Attached proposed patch with logging at info level on the basis that current usage mostly (but not exclusively) happens only if assertions are enabled. Happy to log at debug instead of info level if folks prefer that. > ObjectReleaseTracker silently 'releases' untracked objects > -- > > Key: SOLR-12462 > URL: https://issues.apache.org/jira/browse/SOLR-12462 > Project: Solr > Issue Type: Task > Components: logging >Reporter: Christine Poerschke >Priority: Minor > Attachments: SOLR-12462.patch > > > Proposing to log (at info or debug level) when the 'release' of an untracked > object is attempted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8343) BlendedInfixSuggester bad score calculus for certain suggestion weights
[ https://issues.apache.org/jira/browse/LUCENE-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504819#comment-16504819 ] Cassandra Targett commented on LUCENE-8343: --- [~alessandro.benedetti] - you mentioned on solr-user mailing list that the patch here had Ref Guide changes in it, but I don't see them? > BlendedInfixSuggester bad score calculus for certain suggestion weights > --- > > Key: LUCENE-8343 > URL: https://issues.apache.org/jira/browse/LUCENE-8343 > Project: Lucene - Core > Issue Type: Bug > Components: core/search >Affects Versions: 7.3.1 >Reporter: Alessandro Benedetti >Priority: Major > Attachments: LUCENE-8343.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Currently the BlendedInfixSuggester return a (long) score to rank the > suggestions. > This score is calculated as a multiplication between : > long *Weight* : the suggestion weight, coming from a document field, it can > be any long value ( including 1, 0,.. ) > double *Coefficient* : 0<=x<=1, calculated based on the position match, > earlier the better > The resulting score is a long, which means that at the moment, any weight<10 > can bring inconsistencies. > *Edge cases* > Weight =1 > Score = 1( if we have a match at the beginning of the suggestion) or 0 ( for > any other match) > Weight =0 > Score = 0 ( independently of the position match coefficient) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12462) ObjectReleaseTracker silently 'releases' untracked objects
[ https://issues.apache.org/jira/browse/SOLR-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke updated SOLR-12462: --- Attachment: SOLR-12462.patch > ObjectReleaseTracker silently 'releases' untracked objects > -- > > Key: SOLR-12462 > URL: https://issues.apache.org/jira/browse/SOLR-12462 > Project: Solr > Issue Type: Task > Components: logging >Reporter: Christine Poerschke >Priority: Minor > Attachments: SOLR-12462.patch > > > Proposing to log (at info or debug level) when the 'release' of an untracked > object is attempted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12463) Search rate trigger should check metrics less often
Varun Thacker created SOLR-12463:
Summary: Search rate trigger should check metrics less often
Key: SOLR-12463
URL: https://issues.apache.org/jira/browse/SOLR-12463
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: AutoScaling
Reporter: Varun Thacker
I created a search rate trigger by running this command
{code:java}
curl -X POST -H 'Content-type:application/json' --data-binary '{
"set-trigger": {
"name" : "search_rate_trigger",
"event" : "searchRate",
"collection" : "gettingstarted",
"rate" : 1.0,
"waitFor" : "1m",
"enabled" : true,
"actions" : [
{
"name" : "compute_plan",
"class": "solr.ComputePlanAction"
},
{
"name" : "execute_plan",
"class": "solr.ExecutePlanAction"
}
]
}
}' http://localhost:8983/solr/admin/autoscaling
{code}
As soon as I add this by logs start getting spammed with these every 1 second
{code:java}
INFO - 2018-06-07 15:29:16.243; [ ] org.apache.solr.servlet.HttpSolrCall;
[admin] webapp=null path=/admin/metrics
params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
status=0 QTime=0
INFO - 2018-06-07 15:29:17.253; [ ] org.apache.solr.servlet.HttpSolrCall;
[admin] webapp=null path=/admin/metrics
params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
status=0 QTime=0
INFO - 2018-06-07 15:29:18.263; [ ] org.apache.solr.servlet.HttpSolrCall;
[admin] webapp=null path=/admin/metrics
params={wt=javabin=2=solr.core.gettingstarted.shard2.replica_n4:QUERY./select.requestTimes:1minRate=solr.core.gettingstarted.shard1.replica_n1:QUERY./select.requestTimes:1minRate}
status=0 QTime=0
{code}
Does 1 second seem to aggressive ?
Maybe the waitFor could be taken into account and query the metrics at
waitFor/2 or something ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #395: WIP SOLR-12362: add tests for working relatio...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/395#discussion_r193785664
--- Diff: solr/core/src/java/org/apache/solr/handler/loader/JsonLoader.java
---
@@ -668,7 +682,40 @@ private boolean isChildDoc(SolrInputDocument
extendedMap) {
return
extendedMap.containsKey(req.getSchema().getUniqueKeyField().getName());
}
-private SolrInputDocument generateExtendedValueMap(int ev) throws
IOException {
+private boolean entryIsChildDoc(Object val) {
+ if(val instanceof List) {
+List listVal = (List) val;
+if (listVal.size() == 0) return false;
+return listVal.get(0) instanceof Map;
+ }
+ return val instanceof Map;
+}
+
+private void safeAddValue(SolrInputDocument doc, String fieldName,
Object value) {
--- End diff --
I'm confused why this is necessary. What goes wrong if
doc.addField(name,value) is called? If something goes wrong and this is
necessary, that explanation needs to be in the comments.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #395: WIP SOLR-12362: add tests for working relatio...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/395#discussion_r193783038
--- Diff: solr/core/src/java/org/apache/solr/handler/loader/JsonLoader.java
---
@@ -249,14 +251,27 @@ public void handle(Map record, String
path) {
private SolrInputDocument buildDoc(Map m) {
SolrInputDocument result = new SolrInputDocument();
for (Map.Entry e : m.entrySet()) {
-if (e.getKey() == null) {// special case. JsonRecordReader emits
child docs with null key
+if (entryIsChildDoc(e.getValue())) {// special case.
JsonRecordReader emits child docs with null key
if (e.getValue() instanceof List) {
List value = (List) e.getValue();
for (Object o : value) {
- if (o instanceof Map) result.addChildDocument(buildDoc((Map)
o));
+ if (o instanceof Map) {
+if (anonChildDocFlag) {
+ result.addChildDocument(buildDoc((Map) o));
+} else {
+ if(!result.containsKey(e.getKey())) {
+result.setField(e.getKey(), new ArrayList<>(1));
--- End diff --
This looks unnecessary; SolrInputDocument internally handles ensuring
multiple values can be captures.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #395: WIP SOLR-12362: add tests for working relatio...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/395#discussion_r193786291
--- Diff: solr/core/src/test/org/apache/solr/handler/JsonLoaderTest.java ---
@@ -435,61 +435,80 @@ public void testJsonDocFormat() throws Exception{
public void testFewParentsJsonDoc() throws Exception {
String json = PARENT_TWO_CHILDREN_JSON;
+assertFewParents(json, true);
+tearDown();
--- End diff --
I'm don't think I've ever seen a test do this, and I don't think we should
start now. Simply add two test methods, one that passes true, one that passes
false. test methods need to be named differently of course. This pattern in
testing is pretty common.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #395: WIP SOLR-12362: add tests for working relatio...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/395#discussion_r193789756
--- Diff:
solr/solrj/src/java/org/apache/solr/common/util/JsonRecordReader.java ---
@@ -438,18 +442,19 @@ void walkObject() throws IOException {
}
}
-private void addChildDoc2ParentDoc(Map record,
Map values) {
+private void addChildDoc2ParentDoc(Map record,
Map values, String path) {
+ String trimmedPath = trimPath(path);
--- End diff --
I think "path" should be renamed "key", and the trimPath() should be done
by the caller, and not be called trimPath either for that matter -- it takes
the "leaf" or "suffix" part of the path, or perhaps you have an idea of a
better name.
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] lucene-solr pull request #395: WIP SOLR-12362: add tests for working relatio...
Github user dsmiley commented on a diff in the pull request:
https://github.com/apache/lucene-solr/pull/395#discussion_r193783399
--- Diff: solr/core/src/java/org/apache/solr/handler/loader/JsonLoader.java
---
@@ -249,14 +251,27 @@ public void handle(Map record, String
path) {
private SolrInputDocument buildDoc(Map m) {
SolrInputDocument result = new SolrInputDocument();
for (Map.Entry e : m.entrySet()) {
-if (e.getKey() == null) {// special case. JsonRecordReader emits
child docs with null key
+if (entryIsChildDoc(e.getValue())) {// special case.
JsonRecordReader emits child docs with null key
--- End diff --
the comment is concerning; we don't want JsonRecordReader to do that
anymore?
---
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-12462) ObjectReleaseTracker silently 'releases' untracked objects
Christine Poerschke created SOLR-12462: -- Summary: ObjectReleaseTracker silently 'releases' untracked objects Key: SOLR-12462 URL: https://issues.apache.org/jira/browse/SOLR-12462 Project: Solr Issue Type: Task Components: logging Reporter: Christine Poerschke Proposing to log (at info or debug level) when the 'release' of an untracked object is attempted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12018) Ref Guide: Comment system is offline
[ https://issues.apache.org/jira/browse/SOLR-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cassandra Targett updated SOLR-12018: - Fix Version/s: master (8.0) 7.4 > Ref Guide: Comment system is offline > > > Key: SOLR-12018 > URL: https://issues.apache.org/jira/browse/SOLR-12018 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: RefGuideCommentsBroken.png, SOLR-12018.patch > > > The Ref Guide uses comments.apache.org to allow user comments. Sometime in > December/early January, it was taken offline. > I filed INFRA-15947 to ask after it's long-term status, and recently got an > answer that it an ETA is mid-March for a permanent INFRA-hosted system. > However, it's of course possible changes in priorities or other factors will > delay that timeline. > Every Ref Guide page currently invites users to leave comments, but since the > whole Comments area is pulled in via JavaScript from a non-existent server, > there's no space to do so (see attached screenshot). While we wait for the > permanent server to be online, we have a couple of options: > # Leave it the way it is and hopefully by mid-March it will be back > # Change the text to tell users it's not working temporarily on all published > versions > # Remove it from all the published versions and put it back when it's back > I'm not a great fan of #2 or #3, because it'd be a bit of work for me to > backport changes to 4 branches and republish every guide just to fix it again > in a month or so. I'm fine with option #1 since I've known about it for about > a month at least and as far as I can tell no one else has noticed. But if > people feel strongly about it now that they know about it, we can figure > something out. > If for some reason it takes longer than mid-March to get it back, or INFRA > chooses to stop supporting it entirely, this issue can morph into what we > should do for an alternative permanent solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12018) Ref Guide: Comment system is offline
[ https://issues.apache.org/jira/browse/SOLR-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504814#comment-16504814 ] Cassandra Targett commented on SOLR-12018: -- I've removed it for 7.4 and the future; leaving this open until I have time to deal with older versions that are still broken. > Ref Guide: Comment system is offline > > > Key: SOLR-12018 > URL: https://issues.apache.org/jira/browse/SOLR-12018 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Fix For: 7.4, master (8.0) > > Attachments: RefGuideCommentsBroken.png, SOLR-12018.patch > > > The Ref Guide uses comments.apache.org to allow user comments. Sometime in > December/early January, it was taken offline. > I filed INFRA-15947 to ask after it's long-term status, and recently got an > answer that it an ETA is mid-March for a permanent INFRA-hosted system. > However, it's of course possible changes in priorities or other factors will > delay that timeline. > Every Ref Guide page currently invites users to leave comments, but since the > whole Comments area is pulled in via JavaScript from a non-existent server, > there's no space to do so (see attached screenshot). While we wait for the > permanent server to be online, we have a couple of options: > # Leave it the way it is and hopefully by mid-March it will be back > # Change the text to tell users it's not working temporarily on all published > versions > # Remove it from all the published versions and put it back when it's back > I'm not a great fan of #2 or #3, because it'd be a bit of work for me to > backport changes to 4 branches and republish every guide just to fix it again > in a month or so. I'm fine with option #1 since I've known about it for about > a month at least and as far as I can tell no one else has noticed. But if > people feel strongly about it now that they know about it, we can figure > something out. > If for some reason it takes longer than mid-March to get it back, or INFRA > chooses to stop supporting it entirely, this issue can morph into what we > should do for an alternative permanent solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12018) Ref Guide: Comment system is offline
[ https://issues.apache.org/jira/browse/SOLR-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504809#comment-16504809 ] ASF subversion and git services commented on SOLR-12018: Commit 920cefd2fa06d7b3449d2d7ebcdf5d9830a0ad05 in lucene-solr's branch refs/heads/branch_7_4 from [~ctargett] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=920cefd ] SOLR-12018: Remove comments.apache.org integration for the Ref Guide > Ref Guide: Comment system is offline > > > Key: SOLR-12018 > URL: https://issues.apache.org/jira/browse/SOLR-12018 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Attachments: RefGuideCommentsBroken.png, SOLR-12018.patch > > > The Ref Guide uses comments.apache.org to allow user comments. Sometime in > December/early January, it was taken offline. > I filed INFRA-15947 to ask after it's long-term status, and recently got an > answer that it an ETA is mid-March for a permanent INFRA-hosted system. > However, it's of course possible changes in priorities or other factors will > delay that timeline. > Every Ref Guide page currently invites users to leave comments, but since the > whole Comments area is pulled in via JavaScript from a non-existent server, > there's no space to do so (see attached screenshot). While we wait for the > permanent server to be online, we have a couple of options: > # Leave it the way it is and hopefully by mid-March it will be back > # Change the text to tell users it's not working temporarily on all published > versions > # Remove it from all the published versions and put it back when it's back > I'm not a great fan of #2 or #3, because it'd be a bit of work for me to > backport changes to 4 branches and republish every guide just to fix it again > in a month or so. I'm fine with option #1 since I've known about it for about > a month at least and as far as I can tell no one else has noticed. But if > people feel strongly about it now that they know about it, we can figure > something out. > If for some reason it takes longer than mid-March to get it back, or INFRA > chooses to stop supporting it entirely, this issue can morph into what we > should do for an alternative permanent solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12018) Ref Guide: Comment system is offline
[ https://issues.apache.org/jira/browse/SOLR-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504805#comment-16504805 ] ASF subversion and git services commented on SOLR-12018: Commit b7881e45ddbfddeba81c5f6eca2b97a37dc9b775 in lucene-solr's branch refs/heads/branch_7x from [~ctargett] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b7881e4 ] SOLR-12018: Remove comments.apache.org integration for the Ref Guide > Ref Guide: Comment system is offline > > > Key: SOLR-12018 > URL: https://issues.apache.org/jira/browse/SOLR-12018 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Attachments: RefGuideCommentsBroken.png, SOLR-12018.patch > > > The Ref Guide uses comments.apache.org to allow user comments. Sometime in > December/early January, it was taken offline. > I filed INFRA-15947 to ask after it's long-term status, and recently got an > answer that it an ETA is mid-March for a permanent INFRA-hosted system. > However, it's of course possible changes in priorities or other factors will > delay that timeline. > Every Ref Guide page currently invites users to leave comments, but since the > whole Comments area is pulled in via JavaScript from a non-existent server, > there's no space to do so (see attached screenshot). While we wait for the > permanent server to be online, we have a couple of options: > # Leave it the way it is and hopefully by mid-March it will be back > # Change the text to tell users it's not working temporarily on all published > versions > # Remove it from all the published versions and put it back when it's back > I'm not a great fan of #2 or #3, because it'd be a bit of work for me to > backport changes to 4 branches and republish every guide just to fix it again > in a month or so. I'm fine with option #1 since I've known about it for about > a month at least and as far as I can tell no one else has noticed. But if > people feel strongly about it now that they know about it, we can figure > something out. > If for some reason it takes longer than mid-March to get it back, or INFRA > chooses to stop supporting it entirely, this issue can morph into what we > should do for an alternative permanent solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12018) Ref Guide: Comment system is offline
[ https://issues.apache.org/jira/browse/SOLR-12018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504798#comment-16504798 ] ASF subversion and git services commented on SOLR-12018: Commit 6e880352976e15f876fa2b3c578a563c4d26cd2d in lucene-solr's branch refs/heads/master from [~ctargett] [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6e88035 ] SOLR-12018: Remove comments.apache.org integration for the Ref Guide > Ref Guide: Comment system is offline > > > Key: SOLR-12018 > URL: https://issues.apache.org/jira/browse/SOLR-12018 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) > Components: documentation >Reporter: Cassandra Targett >Assignee: Cassandra Targett >Priority: Major > Attachments: RefGuideCommentsBroken.png, SOLR-12018.patch > > > The Ref Guide uses comments.apache.org to allow user comments. Sometime in > December/early January, it was taken offline. > I filed INFRA-15947 to ask after it's long-term status, and recently got an > answer that it an ETA is mid-March for a permanent INFRA-hosted system. > However, it's of course possible changes in priorities or other factors will > delay that timeline. > Every Ref Guide page currently invites users to leave comments, but since the > whole Comments area is pulled in via JavaScript from a non-existent server, > there's no space to do so (see attached screenshot). While we wait for the > permanent server to be online, we have a couple of options: > # Leave it the way it is and hopefully by mid-March it will be back > # Change the text to tell users it's not working temporarily on all published > versions > # Remove it from all the published versions and put it back when it's back > I'm not a great fan of #2 or #3, because it'd be a bit of work for me to > backport changes to 4 branches and republish every guide just to fix it again > in a month or so. I'm fine with option #1 since I've known about it for about > a month at least and as far as I can tell no one else has noticed. But if > people feel strongly about it now that they know about it, we can figure > something out. > If for some reason it takes longer than mid-March to get it back, or INFRA > chooses to stop supporting it entirely, this issue can morph into what we > should do for an alternative permanent solution. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8326) More Like This Params Refactor
[
https://issues.apache.org/jira/browse/LUCENE-8326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504780#comment-16504780
]
Lucene/Solr QA commented on LUCENE-8326:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 0m 59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 0m 49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 0m 49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
51s{color} | {color:green} classification in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
39s{color} | {color:green} queries in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 78m 46s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.cloud.autoscaling.sim.TestTriggerIntegration |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-8326 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12926768/LUCENE-8326.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 3.13.0-88-generic #135-Ubuntu SMP Wed Jun 8
21:10:42 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-LUCENE-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / a4fa168 |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on April 8 2014 |
| Default Java | 1.8.0_172 |
| unit |
https://builds.apache.org/job/PreCommit-LUCENE-Build/29/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-LUCENE-Build/29/testReport/ |
| modules | C: lucene/classification lucene/queries solr/core U: . |
| Console output |
https://builds.apache.org/job/PreCommit-LUCENE-Build/29/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> More Like This Params Refactor
> --
>
> Key: LUCENE-8326
> URL: https://issues.apache.org/jira/browse/LUCENE-8326
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/query/scoring
>Reporter: Alessandro Benedetti
>Priority: Major
> Attachments: LUCENE-8326.patch, LUCENE-8326.patch, LUCENE-8326.patch,
> LUCENE-8326.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> More Like This ca be refactored to improve the code readability, test
> coverage and maintenance.
> Scope of this Jira issue is to start the More Like This refactor from the
> More Like This Params.
> This Jira will not improve the current More Like This but just keep the same
> functionality with a refactored code.
> Other Jira issues will follow improving the overall code readability, test
> coverage and maintenance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-11984) Update dropwizard-metrics dependency to version 4.0.2
[ https://issues.apache.org/jira/browse/SOLR-11984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Braun resolved SOLR-11984. -- Resolution: Duplicate > Update dropwizard-metrics dependency to version 4.0.2 > - > > Key: SOLR-11984 > URL: https://issues.apache.org/jira/browse/SOLR-11984 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Michael Braun >Priority: Minor > > Change list available at > http://metrics.dropwizard.io/4.0.0/about/release-notes.html . 4.0.2 fixed a > bug introduced by one of these changes (changing Timer.Context into an > interface) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12461) Upgrade Dropwizard Metrics to 4.0.2 release
[
https://issues.apache.org/jira/browse/SOLR-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cassandra Targett updated SOLR-12461:
-
Description:
This version of the library contains several improvements and it's compatible
with Java 9.
However, starting from 4.0.0 metrics-ganglia is no longer available, which
means that if we upgrade we will have to remove the corresponding
{{SolrGangliaReporter}}.
Such change is not back-compatible, so I see the following options:
* wait with the upgrade until 8.0
* upgrade and remove {{SolrGangliaReporter}} and describe this in the release
notes.
Any other suggestions?
> Upgrade Dropwizard Metrics to 4.0.2 release
> ---
>
> Key: SOLR-12461
> URL: https://issues.apache.org/jira/browse/SOLR-12461
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
>
> This version of the library contains several improvements and it's compatible
> with Java 9.
> However, starting from 4.0.0 metrics-ganglia is no longer available, which
> means that if we upgrade we will have to remove the corresponding
> {{SolrGangliaReporter}}.
> Such change is not back-compatible, so I see the following options:
> * wait with the upgrade until 8.0
> * upgrade and remove {{SolrGangliaReporter}} and describe this in the release
> notes.
> Any other suggestions?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12461) Upgrade Dropwizard Metrics to 4.0.2 release
[
https://issues.apache.org/jira/browse/SOLR-12461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Cassandra Targett updated SOLR-12461:
-
Environment: (was: This version of the library contains several
improvements and it's compatible with Java 9.
However, starting from 4.0.0 metrics-ganglia is no longer available, which
means that if we upgrade we will have to remove the corresponding
{{SolrGangliaReporter}}.
Such change is not back-compatible, so I see the following options:
* wait with the upgrade until 8.0
* upgrade and remove {{SolrGangliaReporter}} and describe this in the release
notes.
Any other suggestions?)
> Upgrade Dropwizard Metrics to 4.0.2 release
> ---
>
> Key: SOLR-12461
> URL: https://issues.apache.org/jira/browse/SOLR-12461
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
>Reporter: Andrzej Bialecki
>Assignee: Andrzej Bialecki
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8347) BlendedInfixSuggester to handle multi term matches better
[
https://issues.apache.org/jira/browse/LUCENE-8347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504752#comment-16504752
]
Alessandro Benedetti commented on LUCENE-8347:
--
Added some additional edge cases tests + bugfixes :
- assertThat(responses.get(8).key, is("Bar Fridge Mini"));
Management of shuffled position in the suggestion but all terms match
- term query repetition properly managed
> BlendedInfixSuggester to handle multi term matches better
> -
>
> Key: LUCENE-8347
> URL: https://issues.apache.org/jira/browse/LUCENE-8347
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/search
>Affects Versions: 7.3.1
>Reporter: Alessandro Benedetti
>Priority: Major
> Attachments: LUCENE-8347.patch, LUCENE-8347.patch
>
>
> Currently the blendedInfix suggester considers just the first match position
> when scoring a suggestion.
> From the lucene-dev mailing list :
> "
> If I write more than one term in the query, let's say
>
> "Mini Bar Fridge"
>
> I would expect in the results something like (note that allTermsRequired=true
> and the schema weight field always returns 1000)
>
> - *Mini Bar Fridge* something
> - *Mini Bar Fridge* something else
> - *Mini Bar* something *Fridge*
> - *Mini Bar* something else *Fridge*
> - *Mini* something *Bar Fridge*
> ...
>
> Instead I see this:
>
> - *Mini Bar* something *Fridge*
> - *Mini Bar* something else *Fridge*
> - *Mini Bar Fridge* something
> - *Mini Bar Fridge* something else
> - *Mini* something *Bar Fridge*
> ...
>
> After having a look at the suggester code
> (BlendedInfixSuggester.createCoefficient), I see that the component takes in
> account only one position, which is the lowest position (among the three
> matching terms) within the term vector ("mini" in the example above) so all
> the suggestions above have the same weight
> "
> Scope of this Jira issue is to improve the BlendedInfix to better manage
> those scenarios.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-6734) Standalone solr as *two* applications -- Solr and a controlling agent
[ https://issues.apache.org/jira/browse/SOLR-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504722#comment-16504722 ] Shawn Heisey commented on SOLR-6734: bq. What do you mean by “service” here? Process or OS service/daemon? I'm referring to an OS service. Something in /etc/init.d, systemd, upstart, etc. Which I suppose also equates to a process, and each one would need to write a PID file. (Once the paradigm is nailed, come up with something similar for Windows) The only reason that I propose this for Solr and ZK, rather than having those be daemons under the sole control of the agent service, is to make it straightforward for anyone (the user OR the agent) to start/stop them. If the agent (node controller in your example) were the only service directly controlled by the OS and the others were controlled by the agent (probably by calling scripts, if Java's process control is braindead), I think that would work as well. If there's an open source framework for this that meets all the criteria, I'd be happy to look at it. The criteria I think it needs: 1) Has an acceptable license. 2) Is generic enough for our purposes. 3) Works on both POSIX platforms and Windows with minimal differences between the two. I'm developing a pretty clear mental picture of how all these pieces would work together, monitor each other, and report problems. It's a very 3-foot view, with a bunch of black boxes where the precise implementation hasn't been completely worked out, but the general mechanics seem doable. > Standalone solr as *two* applications -- Solr and a controlling agent > - > > Key: SOLR-6734 > URL: https://issues.apache.org/jira/browse/SOLR-6734 > Project: Solr > Issue Type: Sub-task >Reporter: Shawn Heisey >Priority: Major > > In a message to the dev list outlining reasons to switch from a webapp to a > standalone app, Mark Miller included the idea of making Solr into two > applications, rather than just one. There would be Solr itself, and an agent > to control Solr. > http://mail-archives.apache.org/mod_mbox/lucene-dev/201305.mbox/%3C807476C6-E4C3-4E7E-9F67-2BECB63990DE%40gmail.com%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8164) IndexWriter silently accepts broken payload
[ https://issues.apache.org/jira/browse/LUCENE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504689#comment-16504689 ] Nhat Nguyen commented on LUCENE-8164: - [~rcmuir] We are good now. The test throws an IndexOutOfBoundsException exception as expected. Thanks for your great initial work. > IndexWriter silently accepts broken payload > --- > > Key: LUCENE-8164 > URL: https://issues.apache.org/jira/browse/LUCENE-8164 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Attachments: LUCENE-8164.patch, LUCENE-8164_test.patch > > > IndexWriter seems to be missing bounds checks for payloads completely. > If you pass a "broken" payload (e.g. BytesRef's offset + length is out of > bounds), it will silently index it as if nothing went wrong. What actually > happens? Doesn't matter, we should be getting an exception. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8164) IndexWriter silently accepts broken payload
[ https://issues.apache.org/jira/browse/LUCENE-8164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nhat Nguyen updated LUCENE-8164: Attachment: LUCENE-8164.patch > IndexWriter silently accepts broken payload > --- > > Key: LUCENE-8164 > URL: https://issues.apache.org/jira/browse/LUCENE-8164 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Attachments: LUCENE-8164.patch, LUCENE-8164_test.patch > > > IndexWriter seems to be missing bounds checks for payloads completely. > If you pass a "broken" payload (e.g. BytesRef's offset + length is out of > bounds), it will silently index it as if nothing went wrong. What actually > happens? Doesn't matter, we should be getting an exception. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8165) ban Arrays.copyOfRange with forbidden APIs
[ https://issues.apache.org/jira/browse/LUCENE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504656#comment-16504656 ] Nhat Nguyen commented on LUCENE-8165: - [~rcmuir] I will look at LUCENE-8164 and post the output. > ban Arrays.copyOfRange with forbidden APIs > -- > > Key: LUCENE-8165 > URL: https://issues.apache.org/jira/browse/LUCENE-8165 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Fix For: master (8.0), 7.5 > > Attachments: LUCENE-8165.patch, LUCENE-8165_copy_of.patch, > LUCENE-8165_copy_of_range.patch, LUCENE-8165_start.patch, > LUCENE-8165_start.patch > > > This method is no good, because instead of throwing AIOOBE for bad bounds, it > will silently fill with zeros (essentially silent corruption). Unfortunately > it is used in quite a few places so replacing it with e.g. arrayCopy may > uncover some interesting surprises. > See LUCENE-8164 for motivation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12460) Filtercache getting filled up when domain switches are involved in Json facets
[
https://issues.apache.org/jira/browse/SOLR-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-12460:
Description:
FacetProcessor.java :: {{handleJoinField}} calls
SolrIndexSearcher.getDocSet(..domainquery..) which eventually cache join
queries like:
{code}
key:{ !join from=[vin_l-value] to=[vin_l-value] } { !cache=false }
ConstantScore(BitSetDocTopFilter)
value: org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
{code}
and when we execute the same query again, some of the entries are not getting
used at all. Please note: the filterCache entries are not getting used strictly
when we have more than 1 shard in the collection.
Sample: {{car_stuff}} collection with multiple doc-types: vehicle, claims,
defects - *10 shards*
{code}
http://localhost:8983/solr/car_stuff/query?rows=0=doc_type_s:vehicle AND
v_model_s:model_009={
models:{
type:terms,
field:"v_model_s",
limit:-1,
facet:{
year_per_model:{
type:terms,
field:"v_year_i",
limit:-1,
facet:{
claim_month:{
domain:{
join:{
from:"vin_l",
to:"vin_l"
},
filter:"doc_type_s:claim"
},
type:terms,
field:"claim_opcode_s",
limit:-1
}
}
}
}
}
}
{code}
After executing this query for first time, filterCache for one of the cores
looks like:
{code}
"CACHE.searcher.filterCache":{
"lookups":151,
"hits":49,
"cumulative_evictions":0,
"size":103,
"hitratio":0.32,
"evictions":0,
"cumulative_lookups":151,
"cumulative_hitratio":0.32,
"warmupTime":0,
"inserts":103,
"cumulative_inserts":103,
"cumulative_hits":49}
{code}
2nd time executing same query:
{code}
"CACHE.searcher.filterCache":{
"lookups":303,
"hits":151,
"cumulative_evictions":0,
"size":153,
"hitratio":0.5,
"evictions":0,
"cumulative_lookups":303,
"cumulative_hitratio":0.5,
"warmupTime":0,
"inserts":153,
"cumulative_inserts":153,
"cumulative_hits":151}
{code}
Looking into the entries of the filterCache looks like this:
{code}
.
key: v_year_i:[1977 TO 1977] value: org.apache.solr.search.BitDocSet@1524fa9c
lastAccessed:457,
key: v_model_s:model_003 value: org.apache.solr.search.BitDocSet@61f348dd
lastAccessed:157,
key:{ !join from=[vin_l-value] to=[vin_l-value] } { !cache=false }
ConstantScore(BitSetDocTopFilter)
value: org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
.
{code}
The collection backup is uploaded on the JIRA.
was:
FacetProcessor.java :: {{handleJoinField}} calls
SolrIndexSearcher.getDocSet(..domainquery..) which eventually cache join
queries like:
{code}
key:{ !join from=[vin_l-value] to=[vin_l-value] } { !cache=false }
ConstantScore(BitSetDocTopFilter)
value: org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
{code}
and when we execute the same query again, some of the entries are not getting
used at all. Please note: the filterCache entries are not getting used strictly
when we have more than 1 shard in the collection.
Sample: {{car_stuff}} collection with multiple doc-types: vehicle, claims,
defects - 10 shards
{code}
http://localhost:8983/solr/car_stuff/query?rows=0=doc_type_s:vehicle AND
v_model_s:model_009={
models:{
type:terms,
field:"v_model_s",
limit:-1,
facet:{
year_per_model:{
type:terms,
field:"v_year_i",
limit:-1,
facet:{
claim_month:{
domain:{
join:{
from:"vin_l",
to:"vin_l"
},
filter:"doc_type_s:claim"
},
type:terms,
field:"claim_opcode_s",
limit:-1
}
}
}
}
}
}
{code}
After executing this query for first time, filterCache for one of the cores
looks like:
{code}
"CACHE.searcher.filterCache":{
"lookups":151,
"hits":49,
"cumulative_evictions":0,
"size":103,
"hitratio":0.32,
"evictions":0,
"cumulative_lookups":151,
"cumulative_hitratio":0.32,
"warmupTime":0,
"inserts":103,
"cumulative_inserts":103,
"cumulative_hits":49}
{code}
2nd time executing same query:
{code}
"CACHE.searcher.filterCache":{
"lookups":303,
"hits":151,
[jira] [Commented] (LUCENE-8165) ban Arrays.copyOfRange with forbidden APIs
[ https://issues.apache.org/jira/browse/LUCENE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504650#comment-16504650 ] Robert Muir commented on LUCENE-8165: - I think these commits may also fix LUCENE-8164 too: I haven't yet tried re-running the test yet to see if it now hits exception. > ban Arrays.copyOfRange with forbidden APIs > -- > > Key: LUCENE-8165 > URL: https://issues.apache.org/jira/browse/LUCENE-8165 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Fix For: master (8.0), 7.5 > > Attachments: LUCENE-8165.patch, LUCENE-8165_copy_of.patch, > LUCENE-8165_copy_of_range.patch, LUCENE-8165_start.patch, > LUCENE-8165_start.patch > > > This method is no good, because instead of throwing AIOOBE for bad bounds, it > will silently fill with zeros (essentially silent corruption). Unfortunately > it is used in quite a few places so replacing it with e.g. arrayCopy may > uncover some interesting surprises. > See LUCENE-8164 for motivation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12460) Filtercache getting filled up when domain switches are involved in Json facets
[
https://issues.apache.org/jira/browse/SOLR-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-12460:
Attachment: car_stuff.zip
> Filtercache getting filled up when domain switches are involved in Json facets
> --
>
> Key: SOLR-12460
> URL: https://issues.apache.org/jira/browse/SOLR-12460
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Facet Module
>Affects Versions: master (8.0)
>Reporter: Amrit Sarkar
>Priority: Major
> Attachments: car_stuff.zip
>
>
> FacetProcessor.java :: {{handleJoinField}} calls
> SolrIndexSearcher.getDocSet(..domainquery..) which eventually cache join
> queries like:
> {code}
> key:{ !join from=[vin_l-value] to=[vin_l-value] } { !cache=false }
> ConstantScore(BitSetDocTopFilter)
> value: org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
> {code}
> and when we execute the same query again, some of the entries are not getting
> used at all. Please note: the filterCache entries are not getting used
> strictly when we have more than 1 shard in the collection.
> Sample: {{car_stuff}} collection with multiple doc-types: vehicle, claims,
> defects - 10 shards
> {code}
> http://localhost:8983/solr/car_stuff/query?rows=0=doc_type_s:vehicle AND
> v_model_s:model_009={
>models:{
> type:terms,
> field:"v_model_s",
> limit:-1,
> facet:{
> year_per_model:{
> type:terms,
> field:"v_year_i",
> limit:-1,
> facet:{
>claim_month:{
> domain:{
> join:{
> from:"vin_l",
> to:"vin_l"
> },
> filter:"doc_type_s:claim"
> },
> type:terms,
> field:"claim_opcode_s",
> limit:-1
>}
> }
> }
> }
>}
> }
> {code}
> After executing this query for first time, filterCache for one of the cores
> looks like:
> {code}
> "CACHE.searcher.filterCache":{
> "lookups":151,
> "hits":49,
> "cumulative_evictions":0,
> "size":103,
> "hitratio":0.32,
> "evictions":0,
> "cumulative_lookups":151,
> "cumulative_hitratio":0.32,
> "warmupTime":0,
> "inserts":103,
> "cumulative_inserts":103,
> "cumulative_hits":49}
> {code}
> 2nd time executing same query:
> {code}
> "CACHE.searcher.filterCache":{
> "lookups":303,
> "hits":151,
> "cumulative_evictions":0,
> "size":153,
> "hitratio":0.5,
> "evictions":0,
> "cumulative_lookups":303,
> "cumulative_hitratio":0.5,
> "warmupTime":0,
> "inserts":153,
> "cumulative_inserts":153,
> "cumulative_hits":151}
> {code}
> Looking into the entries of the filterCache looks like this:
> {code}
> .
> key: v_year_i:[1977 TO 1977] value:
> org.apache.solr.search.BitDocSet@1524fa9c lastAccessed:457,
> key: v_model_s:model_003 value: org.apache.solr.search.BitDocSet@61f348dd
> lastAccessed:157,
> key:{ !join from=[vin_l-value] to=[vin_l-value] } { !cache=false }
> ConstantScore(BitSetDocTopFilter)
> value: org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
> .
> {code}
> The collection backup is uploaded on the JIRA.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12460) Filtercache getting filled up when domain switches are involved in Json facets
[
https://issues.apache.org/jira/browse/SOLR-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amrit Sarkar updated SOLR-12460:
Description:
FacetProcessor.java :: {{handleJoinField}} calls
SolrIndexSearcher.getDocSet(..domainquery..) which eventually cache join
queries like:
{code}
key:{ !join from=[vin_l-value] to=[vin_l-value] } { !cache=false }
ConstantScore(BitSetDocTopFilter)
value: org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
{code}
and when we execute the same query again, some of the entries are not getting
used at all. Please note: the filterCache entries are not getting used strictly
when we have more than 1 shard in the collection.
Sample: {{car_stuff}} collection with multiple doc-types: vehicle, claims,
defects - 10 shards
{code}
http://localhost:8983/solr/car_stuff/query?rows=0=doc_type_s:vehicle AND
v_model_s:model_009={
models:{
type:terms,
field:"v_model_s",
limit:-1,
facet:{
year_per_model:{
type:terms,
field:"v_year_i",
limit:-1,
facet:{
claim_month:{
domain:{
join:{
from:"vin_l",
to:"vin_l"
},
filter:"doc_type_s:claim"
},
type:terms,
field:"claim_opcode_s",
limit:-1
}
}
}
}
}
}
{code}
After executing this query for first time, filterCache for one of the cores
looks like:
{code}
"CACHE.searcher.filterCache":{
"lookups":151,
"hits":49,
"cumulative_evictions":0,
"size":103,
"hitratio":0.32,
"evictions":0,
"cumulative_lookups":151,
"cumulative_hitratio":0.32,
"warmupTime":0,
"inserts":103,
"cumulative_inserts":103,
"cumulative_hits":49}
{code}
2nd time executing same query:
{code}
"CACHE.searcher.filterCache":{
"lookups":303,
"hits":151,
"cumulative_evictions":0,
"size":153,
"hitratio":0.5,
"evictions":0,
"cumulative_lookups":303,
"cumulative_hitratio":0.5,
"warmupTime":0,
"inserts":153,
"cumulative_inserts":153,
"cumulative_hits":151}
{code}
Looking into the entries of the filterCache looks like this:
{code}
.
key: v_year_i:[1977 TO 1977] value: org.apache.solr.search.BitDocSet@1524fa9c
lastAccessed:457,
key: v_model_s:model_003 value: org.apache.solr.search.BitDocSet@61f348dd
lastAccessed:157,
key:{ !join from=[vin_l-value] to=[vin_l-value] } { !cache=false }
ConstantScore(BitSetDocTopFilter)
value: org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
.
{code}
The collection backup is uploaded on the JIRA.
was:
FacetProcessor.java :: {{handleJoinField}} calls
SolrIndexSearcher.getDocSet(..domainquery..) which eventually cache join
queries like:
{code}
key:{ !join from=[vin_s-value] to=[vin_s-value] } { !cache=false }
ConstantScore(BitSetDocTopFilter)
value: org.apache.solr.search.SortedIntDocSet@40886054 lastAccessed:824,
{code}
and when we execute the same query again, some of the entries are not getting
used at all. Please note: the filterCache entries are not getting used strictly
when we have more than 1 shard in the collection.
Sample: {{car_stuff}} collection with multiple doc-types: vehicle, claims,
defects - 10 shards
{code}
http://localhost:8983/solr/car_stuff/query?rows=0=doc_type_s:vehicle AND
v_model_s:model_009={
models:{
type:terms,
field:"v_model_s",
limit:-1,
facet:{
year_per_model:{
type:terms,
field:"v_year_i",
limit:-1,
facet:{
claim_month:{
domain:{
join:{
from:"vin_s",
to:"vin_s"
},
filter:"doc_type_s:claim"
},
type:terms,
field:"claim_opcode_s",
limit:-1
}
}
}
}
}
}
{code}
After executing this query for first time, filterCache for one of the cores
looks like:
{code}
"CACHE.searcher.filterCache":{
"lookups":145,
"hits":47,
"cumulative_evictions":0,
"size":99,
"hitratio":0.32,
"evictions":0,
"cumulative_lookups":145,
"cumulative_hitratio":0.32,
"warmupTime":0,
"inserts":99,
"cumulative_inserts":99,
"cumulative_hits":47},
{code}
2nd time executing same query:
{code}
"CACHE.searcher.filterCache":{
"lookups":291,
"hits":145,
"cumulative_evictions":0,
[jira] [Commented] (LUCENE-8165) ban Arrays.copyOfRange with forbidden APIs
[ https://issues.apache.org/jira/browse/LUCENE-8165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504647#comment-16504647 ] Adrien Grand commented on LUCENE-8165: -- You're welcome. I was also wondering about unexpected side effects, hence branching first. Hopefully this will be less risky in the long term than the leniency of copyOf(Range). > ban Arrays.copyOfRange with forbidden APIs > -- > > Key: LUCENE-8165 > URL: https://issues.apache.org/jira/browse/LUCENE-8165 > Project: Lucene - Core > Issue Type: Bug >Reporter: Robert Muir >Priority: Major > Fix For: master (8.0), 7.5 > > Attachments: LUCENE-8165.patch, LUCENE-8165_copy_of.patch, > LUCENE-8165_copy_of_range.patch, LUCENE-8165_start.patch, > LUCENE-8165_start.patch > > > This method is no good, because instead of throwing AIOOBE for bad bounds, it > will silently fill with zeros (essentially silent corruption). Unfortunately > it is used in quite a few places so replacing it with e.g. arrayCopy may > uncover some interesting surprises. > See LUCENE-8164 for motivation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12142) EmbeddedSolrServer should use req.getContentWriter
[ https://issues.apache.org/jira/browse/SOLR-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16504645#comment-16504645 ] David Smiley commented on SOLR-12142: - Looks like we've got some real test failures here in SolrExampleEmbeddedTest.testContentStreamRequest (nice that Yetus helped surface this) > EmbeddedSolrServer should use req.getContentWriter > --- > > Key: SOLR-12142 > URL: https://issues.apache.org/jira/browse/SOLR-12142 > Project: Solr > Issue Type: Bug > Security Level: Public(Default Security Level. Issues are Public) >Reporter: David Smiley >Assignee: Noble Paul >Priority: Major > Fix For: 7.4 > > Attachments: SOLR-12142.patch, SOLR-12142.patch > > > In SOLR-11380, SolrRequest.getContentWriter was introduced as a replacement > for getContentStreams. However, EmbeddedSolrServer still calls > getContentStreams, and so clients who need to send POST data to it cannot yet > switch from the Deprecated API to the new API. The SolrTextTagger is an > example of a project where one would want to do this. > It seems EmbeddedSolrServer ought to check for getContentWriter and if > present then convert it into a ContentStream somehow. For the time being, > ESS needs to call both since both APIs exist. > CC [~noble.paul] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: discuss: stop adding 'via' from CHANGES.txt entries (take two)
I agree with Marks position on this, the information of who committed has significant value. Joel Bernstein http://joelsolr.blogspot.com/ On Wed, Jun 6, 2018 at 10:14 PM, Mark Miller wrote: > I have the same opinion as last time. Taking ownership of actually > committing something to the code base is an important attribution and that > is why it has been included in CHANGES. I don't agree that it takes away > credit at all - via means the commit went through you, which is an accurate > reflection of things. Committing others work is a major contribution and > should be called out, for the positives that it creates as well as the > responsibility for that change you have undertaken by being a very key part > of the via route. > > - Mark > > On Wed, Jun 6, 2018 at 8:10 AM Yonik Seeley wrote: > >> I don't have much of an opinion about "via" one way or the other, >> however I think we should avoid using the mental model of authorship >> for CHANGES.txt. >> We've generally been listing people who made meaningful contributions >> to the patch, including sometimes the person who opened the issue for >> example. >> >> -Yonik >> >> - >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> -- > - Mark > about.me/markrmiller >
