Re: pylucene on Ubuntu 18.04

[Jeff Breidenbach]

And for what it is worth, python-setuptools claims to be version 39.0.1.
I've
probably spent about 10 to 12 hours trying to get something to work. It is
not clear to me if it needs patching to not.

Re: pylucene on Ubuntu 18.04

[Jeff Breidenbach]

To be a little more specific, here's what happens with version 4.9.0
which I've had good luck with in the past. The system contains the
following shared libraries.

/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libjava.so

It looks like LFLAGS in jcc/setup.py should find them. I'm on the
linux2/X86_64 platform, using '/usr/lib/jvm/java-8-openjdk-amd64'
for the JDK. JCC builds and installs without complaint, but fails at
runtime.

 Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 163, in _run_module_as_main
mod_name, _Error)
  File "/usr/lib/python2.7/runpy.py", line 111, in _get_module_details
__import__(mod_name)  # Do not catch exceptions initializing package
  File
"/usr/local/lib/python2.7/dist-packages/JCC-2.20-py2.7-linux-x86_64.egg/jcc/__init__.py",
line 31, in 
from jcc import _jcc
ImportError: libjvm.so: cannot open shared object file: No such file or
directory

[jira] [Commented] (SOLR-12088) Shards with dead replicas cause increased write latency

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493091#comment-16493091
 ] 

Cao Manh Dat commented on SOLR-12088:
-

[~jerry.bao] Since Solr 7.3.1 is released. Can you confirm about the state of 
this issue?

> Shards with dead replicas cause increased write latency
> ---
>
> Key: SOLR-12088
> URL: https://issues.apache.org/jira/browse/SOLR-12088
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrCloud
>Affects Versions: 7.2
>Reporter: Jerry Bao
>Priority: Major
>
> If a collection's shard contains dead replicas, write latency to the 
> collection is increased. For example, if a collection has 10 shards with a 
> replication factor of 3, and one of those shards contains 3 replicas and 3 
> downed replicas, write latency is increased in comparison to a shard that 
> contains only 3 replicas.
> My feeling here is that downed replicas should be completely ignored and not 
> cause issues to other alive replicas in terms of write latency.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

pylucene on Ubuntu 18.04

[Jeff Breidenbach]

I'm having all sorts of trouble getting PyLucene to run on Ubuntu 18.04,
which has openjdk-8, openjdk-11, python 2.7.15. Has anyone had success,
and if so, with which version of pylucene?

Thanks,
Jeff

[JENKINS] Lucene-Solr-Tests-7.x - Build # 627 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/627/

1 tests failed.
FAILED:  
org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure

Error Message:
Did not see a fully active cluster after 30 seconds

Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
at 
__randomizedtesting.SeedInfo.seed([AAED9D4F01206240:22DB3F1CD98F8A52]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:250)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 16302 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> 751313 INFO  

[jira] [Commented] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[Lucene/Solr QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493017#comment-16493017
 ] 

Lucene/Solr QA commented on LUCENE-8334:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  5s{color} 
| {color:red} LUCENE-8334 does not apply to master. Rebase required? Wrong 
Branch? See 
https://wiki.apache.org/lucene-java/HowToContribute#Contributing_your_work for 
help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | LUCENE-8334 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12925386/LUCENE-8334.patch |
| Console output | 
https://builds.apache.org/job/PreCommit-LUCENE-Build/19/console |
| Powered by | Apache Yetus 0.7.0   http://yetus.apache.org |


This message was automatically generated.



> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8334.patch, LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 70 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/70/

3 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic

Error Message:
there should be new MOVERPLICA ops

Stack Trace:
java.lang.AssertionError: there should be new MOVERPLICA ops
at 
__randomizedtesting.SeedInfo.seed([FCCFD4F15C7579A1:5735C9E483A9FF8F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic(TestLargeCluster.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)


FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate

Error Message:
The trigger did not fire at all

Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at 
__randomizedtesting.SeedInfo.seed([FCCFD4F15C7579A1:A187CA7893B3DFEE]:0)
at 

[jira] [Commented] (SOLR-12290) Do not close any servlet streams and improve our servlet stream closing prevention code for users and devs.

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492978#comment-16492978
 ] 

Varun Thacker commented on SOLR-12290:
--

Sounds good! Here's a patch which squashes all the 4 commits into one. I'll run 
tests and precommit before pushing this

> Do not close any servlet streams and improve our servlet stream closing 
> prevention code for users and devs.
> ---
>
> Key: SOLR-12290
> URL: https://issues.apache.org/jira/browse/SOLR-12290
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (8.0)
>
> Attachments: SOLR-12290.patch, SOLR-12290.patch, SOLR-12290.patch, 
> SOLR-12290.patch, SOLR-12290_7x.patch
>
>
> Original Summary:
> When you fetch a file for replication we close the request output stream 
> after writing the file which ruins the connection for reuse.
> We can't close response output streams, we need to reuse these connections. 
> If we do close them, clients are hit with connection problems when they try 
> and reuse the connection from their pool.
> New Summary:
> At some point the above was addressed during refactoring. We should remove 
> these neutered closes and review our close shield code.
> If you are here to track down why this is done:
> Connection reuse requires that we read all streams and do not close them - 
> instead the container itself must manage request and response streams. If we 
> allow them to be closed, not only do we lose some connection reuse, but we 
> can cause spurious client errors that can cause expensive recoveries for no 
> reason. The spec allows us to count on the container to manage streams. It's 
> our job simply to not close them and to always read them fully, from client 
> and server. 
> Java itself can help with always reading the streams fully up to some small 
> default amount of unread stream slack, but that is very dangerous to count 
> on, so we always manually eat up anything on the streams our normal logic 
> ends up not reading for whatever reason.
> We also cannot call abort without ruining the connection or sendError. These 
> should be options of very last resort (requiring a blood sacrifice) or when 
> shutting down.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12290) Do not close any servlet streams and improve our servlet stream closing prevention code for users and devs.

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-12290:
-
Attachment: SOLR-12290_7x.patch

> Do not close any servlet streams and improve our servlet stream closing 
> prevention code for users and devs.
> ---
>
> Key: SOLR-12290
> URL: https://issues.apache.org/jira/browse/SOLR-12290
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (8.0)
>
> Attachments: SOLR-12290.patch, SOLR-12290.patch, SOLR-12290.patch, 
> SOLR-12290.patch, SOLR-12290_7x.patch
>
>
> Original Summary:
> When you fetch a file for replication we close the request output stream 
> after writing the file which ruins the connection for reuse.
> We can't close response output streams, we need to reuse these connections. 
> If we do close them, clients are hit with connection problems when they try 
> and reuse the connection from their pool.
> New Summary:
> At some point the above was addressed during refactoring. We should remove 
> these neutered closes and review our close shield code.
> If you are here to track down why this is done:
> Connection reuse requires that we read all streams and do not close them - 
> instead the container itself must manage request and response streams. If we 
> allow them to be closed, not only do we lose some connection reuse, but we 
> can cause spurious client errors that can cause expensive recoveries for no 
> reason. The spec allows us to count on the container to manage streams. It's 
> our job simply to not close them and to always read them fully, from client 
> and server. 
> Java itself can help with always reading the streams fully up to some small 
> default amount of unread stream slack, but that is very dangerous to count 
> on, so we always manually eat up anything on the streams our normal logic 
> ends up not reading for whatever reason.
> We also cannot call abort without ruining the connection or sendError. These 
> should be options of very last resort (requiring a blood sacrifice) or when 
> shutting down.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11578) Solr 7 Admin UI (Cloud > Graph) should reflect the Replica type to give a more accurate representation of the cluster

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-11578:
-
Attachment: replica_info.png

> Solr 7 Admin UI (Cloud > Graph) should reflect the Replica type to give a 
> more accurate representation of the cluster
> -
>
> Key: SOLR-11578
> URL: https://issues.apache.org/jira/browse/SOLR-11578
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Admin UI
>Affects Versions: 7.0, 7.1
>Reporter: Rohit
>Assignee: Erick Erickson
>Priority: Minor
> Attachments: NRT_Tooltip.png, OnFirefox.png, OnSafari.png, 
> SOLR-11578.patch, SOLR-11578.patch, SOLR-11578.patch, Screen Shot-2.png, 
> Screenshot-1.png, TLOG_Tooltip.png, Updated Graph.png, Updated Legend.png, 
> Updated Radial Graph.png, jquery-ui.min.css, jquery-ui.min.js, 
> jquery-ui.structure.min.css, replica_info.png
>
>
> New replica types were introduced in Solr 7.
> 1. The Solr Admin UI --> Cloud --> Graph mode should be updated to reflect 
> the new replica types (NRT, TLOG, PULL)
> 2. It will give a better overview of the cluster as well as help in 
> troubleshooting and diagnosing issues.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11578) Solr 7 Admin UI (Cloud > Graph) should reflect the Replica type to give a more accurate representation of the cluster

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492971#comment-16492971
 ] 

Varun Thacker commented on SOLR-11578:
--

Thanks Rohit for the patch! Looks great

Here's what I think about it
- Is there a possibility to copy info from the tool-tip? That would be handy 
- I think the info in the tooltip is a little much - See attached screenshot 
(replica_info.png)
>From this I think we should display the following

replica name : core_node19
node_name : 192.168.0.4:8983_solr


Things which I thought were redundant
- state : the green already indicates it's active no?
- type : We know the type because the replica has (N) in the graph view already
- Since we already have node_name base_url seems like not needed?

> Solr 7 Admin UI (Cloud > Graph) should reflect the Replica type to give a 
> more accurate representation of the cluster
> -
>
> Key: SOLR-11578
> URL: https://issues.apache.org/jira/browse/SOLR-11578
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Admin UI
>Affects Versions: 7.0, 7.1
>Reporter: Rohit
>Assignee: Erick Erickson
>Priority: Minor
> Attachments: NRT_Tooltip.png, OnFirefox.png, OnSafari.png, 
> SOLR-11578.patch, SOLR-11578.patch, SOLR-11578.patch, Screen Shot-2.png, 
> Screenshot-1.png, TLOG_Tooltip.png, Updated Graph.png, Updated Legend.png, 
> Updated Radial Graph.png, jquery-ui.min.css, jquery-ui.min.js, 
> jquery-ui.structure.min.css
>
>
> New replica types were introduced in Solr 7.
> 1. The Solr Admin UI --> Cloud --> Graph mode should be updated to reflect 
> the new replica types (NRT, TLOG, PULL)
> 2. It will give a better overview of the cluster as well as help in 
> troubleshooting and diagnosing issues.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 715 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/715/

[...truncated 28 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/69/consoleText

[repro] Revision: 22e333f1dc7bd5f07386df02c2a1d5a55e980e46

[repro] Repro line:  ant test  -Dtestcase=TestSolrConfigHandlerCloud 
-Dtests.method=test -Dtests.seed=338C71662437D38 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ko 
-Dtests.timezone=Asia/Dushanbe -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestSQLHandler -Dtests.method=doTest 
-Dtests.seed=338C71662437D38 -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=uk -Dtests.timezone=America/St_Vincent 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=SolrShardReporterTest 
-Dtests.method=test -Dtests.seed=338C71662437D38 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=fr-LU 
-Dtests.timezone=Pacific/Bougainville -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=SolrShardReporterTest 
-Dtests.seed=338C71662437D38 -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=fr-LU 
-Dtests.timezone=Pacific/Bougainville -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
44015e2acda37d8e0bebf6833d298ae3b263e580
[repro] git fetch
[repro] git checkout 22e333f1dc7bd5f07386df02c2a1d5a55e980e46

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   TestSolrConfigHandlerCloud
[repro]   SolrShardReporterTest
[repro]   TestSQLHandler
[repro] ant compile-test

[...truncated 3316 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 
-Dtests.class="*.TestSolrConfigHandlerCloud|*.SolrShardReporterTest|*.TestSQLHandler"
 -Dtests.showOutput=onerror  -Dtests.seed=338C71662437D38 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ko 
-Dtests.timezone=Asia/Dushanbe -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 1902 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.handler.TestSolrConfigHandlerCloud
[repro]   0/5 failed: 
org.apache.solr.metrics.reporters.solr.SolrShardReporterTest
[repro]   2/5 failed: org.apache.solr.handler.TestSQLHandler
[repro] git checkout 44015e2acda37d8e0bebf6833d298ae3b263e580

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 5 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 229 - Failure

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/229/

1 tests failed.
FAILED:  
org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit

Error Message:
GC overhead limit exceeded

Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at 
org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.(DirectPostingsFormat.java:458)
at 
org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.(DirectPostingsFormat.java:129)
at 
org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:113)
at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.(PerFieldPostingsFormat.java:292)
at 
org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
at 
org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:113)
at org.apache.lucene.index.SegmentReader.(SegmentReader.java:82)
at 
org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:193)
at 
org.apache.lucene.index.ReadersAndUpdates.getReadOnlyClone(ReadersAndUpdates.java:232)
at 
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:105)
at org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:522)
at 
org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:410)
at 
org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:332)
at 
org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit(TestDocTermOrdsUninvertLimit.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)




Build Log:
[...truncated 14220 lines...]
   [junit4] Suite: org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit
   [junit4]   2> May 29, 2018 4:14:09 AM 
com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
 uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge 
Thread #893,5,TGRP-TestDocTermOrdsUninvertLimit]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: 
org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: 
this IndexWriter hit an unrecoverable exception
   [junit4]   2>at 
__randomizedtesting.SeedInfo.seed([AE650DB4572BE565]:0)
   [junit4]   2>at 
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:704)
   [junit4]   2>at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
   [junit4]   2> Caused by: org.apache.lucene.store.AlreadyClosedException: 
refusing to delete any files: this IndexWriter hit an unrecoverable exception
   [junit4]   2>at 
org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)
   [junit4]   2>at 
org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)
   [junit4]   2>at 
org.apache.lucene.index.IndexFileDeleter.deleteNewFiles(IndexFileDeleter.java:664)
   [junit4]   2>at 

[jira] [Commented] (SOLR-12290) Do not close any servlet streams and improve our servlet stream closing prevention code for users and devs.

[Mark Miller (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492948#comment-16492948
 ] 

Mark Miller commented on SOLR-12290:


Yeah, feel free - the reason I’m not very concerned is that the other issue I 
did around around the same time that gives updates and only updates their own 
connection pool should remove any issue this would prob help with. 

> Do not close any servlet streams and improve our servlet stream closing 
> prevention code for users and devs.
> ---
>
> Key: SOLR-12290
> URL: https://issues.apache.org/jira/browse/SOLR-12290
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (8.0)
>
> Attachments: SOLR-12290.patch, SOLR-12290.patch, SOLR-12290.patch, 
> SOLR-12290.patch
>
>
> Original Summary:
> When you fetch a file for replication we close the request output stream 
> after writing the file which ruins the connection for reuse.
> We can't close response output streams, we need to reuse these connections. 
> If we do close them, clients are hit with connection problems when they try 
> and reuse the connection from their pool.
> New Summary:
> At some point the above was addressed during refactoring. We should remove 
> these neutered closes and review our close shield code.
> If you are here to track down why this is done:
> Connection reuse requires that we read all streams and do not close them - 
> instead the container itself must manage request and response streams. If we 
> allow them to be closed, not only do we lose some connection reuse, but we 
> can cause spurious client errors that can cause expensive recoveries for no 
> reason. The spec allows us to count on the container to manage streams. It's 
> our job simply to not close them and to always read them fully, from client 
> and server. 
> Java itself can help with always reading the streams fully up to some small 
> default amount of unread stream slack, but that is very dangerous to count 
> on, so we always manually eat up anything on the streams our normal logic 
> ends up not reading for whatever reason.
> We also cannot call abort without ruining the connection or sendError. These 
> should be options of very last resort (requiring a blood sacrifice) or when 
> shutting down.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8332) New ConcatenateGraphTokenStream (move/rename CompletionTokenStream)

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492904#comment-16492904
 ] 

David Smiley commented on LUCENE-8332:
--

Oh I wanted to mention one thing; perhaps just here though I could put in the 
docs.

An alternative approach to this tagger might be to use the SynonymGraphFilter 
(with other steps/configuration),
 which has a lot of similarities with the Tagger's algorithm.  I've heard of 
others that have done this (Dice.com?), and before I created the tagger I 
thought about this approach too.  There are some issues/barriers to "just" 
using the synonym filter::
* if the filter finds multiple overlapping matches, it only returns one without 
any control over its choice.  (compare to the STT's "overlaps" param with 
several choices and it's pluggable)
* the filter doesn't hold any metadata; it's just a set of names.  Though you 
could use synonyms to map to an ID that you then lookup in something else (e.g. 
some DB or Solr index).
* the synonym filter must re-construct its FST on startup each time; 
customizations are necessary to load an existing one from disk.
* you have to arrange for any text processing/analysis (e.g. tokenization rules 
or phonetic filters) of the dictionary to create synonym entries.  With the STT 
this is all configurable in a standard way like any text field.
* and of course you'd have to glue it all together somehow.

> New ConcatenateGraphTokenStream (move/rename CompletionTokenStream)
> ---
>
> Key: LUCENE-8332
> URL: https://issues.apache.org/jira/browse/LUCENE-8332
> Project: Lucene - Core
>  Issue Type: New Feature
>  Components: modules/analysis
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Lets move and rename the CompletionTokenStream in the suggest module into the 
> analysis module renamed as ConcatenateGraphTokenStream. See comments in 
> LUCENE-8323 leading to this idea. Such a TokenStream (or TokenFilter?) has 
> several uses:
>  * for the suggest module
>  * by the SolrTextTagger for NER/ERD use cases – SOLR-12376
>  * for doing complete match search efficiently
> It will need a factory – a TokenFilterFactory, even though we don't have a 
> TokenFilter based subclass of TokenStream.
> It appears there is no back-compat concern in it suddenly disappearing from 
> the suggest module as it's marked experimental and it only seems to be public 
> now perhaps due to some technicality (it has package level constructors).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8331) MergePolicy simulator utility

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492919#comment-16492919
 ] 

David Smiley commented on LUCENE-8331:
--

Thanks for your input Simon.

bq.  I am not sure it needs to be a commandline util.

How else would something like this be executed?  Maybe I don't understand your 
subsequent recommendation...

bq. I would rather build the individual tools to plug stuff together as an API 
and put most of the utils like creating the simulated segments into the base 
tests class.

I may not be getting your point but I think you're saying you'd like Lucene's 
test infrastructure to have _some_ of the elements of what this test does.  
Sounds good to me.  Nevertheless the outcome of that would be less code in this 
simulator... but somewhere there needs to be a main() to literally run the 
simulation and setup whatever the simulated environment is, and code to track 
some stats of interest.  Right?

Are you basically fine with me committing this?

> MergePolicy simulator utility
> -
>
> Key: LUCENE-8331
> URL: https://issues.apache.org/jira/browse/LUCENE-8331
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Attachments: LUCENE-8331.patch
>
>
> This issue introduces a MergePolicy simulator utility to help evaluate the 
> effectiveness of a MergePolicy.  The simulator does not result in the actual 
> indexing and merging of segments; instead it provides some dummy constructs 
> to MergePolicy to evaluate its decisions.  Therefore you can do simulation 
> runs in little time.
> I'm not sure where it would live.  Perhaps dev-tools, or in tests, or in 
> benchmark?
> I mentioned this recently here:
> https://issues.apache.org/jira/browse/LUCENE-7976?focusedCommentId=16446985=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16446985
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11453) Create separate logger for slow requests

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492915#comment-16492915
 ] 

ASF subversion and git services commented on SOLR-11453:


Commit 44015e2acda37d8e0bebf6833d298ae3b263e580 in lucene-solr's branch 
refs/heads/master from [~varun_saxena]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=44015e2 ]

SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a 
separate file - solr_slow_requests.log


> Create separate logger for slow requests
> 
>
> Key: SOLR-11453
> URL: https://issues.apache.org/jira/browse/SOLR-11453
> Project: Solr
>  Issue Type: Improvement
>  Components: logging
>Reporter: Shawn Heisey
>Assignee: Varun Thacker
>Priority: Minor
> Attachments: SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> slowlog-informational.patch, slowlog-informational.patch, 
> slowlog-informational.patch
>
>
> There is some desire on the mailing list to create a separate logfile for 
> slow queries.  Currently it is not possible to do this cleanly, because the 
> WARN level used by slow query logging within the SolrCore class is also used 
> for other events that SolrCore can log.  Those messages would be out of place 
> in a slow query log.  They should typically stay in main solr logfile.
> I propose creating a custom logger for slow queries, similar to what has been 
> set up for request logging.  In the SolrCore class, which is 
> org.apache.solr.core.SolrCore, there is a special logger at 
> org.apache.solr.core.SolrCore.Request.  This is not a real class, just a 
> logger which makes it possible to handle those log messages differently than 
> the rest of Solr's logging.  I propose setting up another custom logger 
> within SolrCore which could be org.apache.solr.core.SolrCore.SlowRequest.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11453) Create separate logger for slow requests

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492917#comment-16492917
 ] 

ASF subversion and git services commented on SOLR-11453:


Commit 9349a9d6325761147de3232c1f793b08e6082390 in lucene-solr's branch 
refs/heads/branch_7x from [~varun_saxena]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9349a9d ]

SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a 
separate file - solr_slow_requests.log

(cherry picked from commit 44015e2)


> Create separate logger for slow requests
> 
>
> Key: SOLR-11453
> URL: https://issues.apache.org/jira/browse/SOLR-11453
> Project: Solr
>  Issue Type: Improvement
>  Components: logging
>Reporter: Shawn Heisey
>Assignee: Varun Thacker
>Priority: Minor
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> slowlog-informational.patch, slowlog-informational.patch, 
> slowlog-informational.patch
>
>
> There is some desire on the mailing list to create a separate logfile for 
> slow queries.  Currently it is not possible to do this cleanly, because the 
> WARN level used by slow query logging within the SolrCore class is also used 
> for other events that SolrCore can log.  Those messages would be out of place 
> in a slow query log.  They should typically stay in main solr logfile.
> I propose creating a custom logger for slow queries, similar to what has been 
> set up for request logging.  In the SolrCore class, which is 
> org.apache.solr.core.SolrCore, there is a special logger at 
> org.apache.solr.core.SolrCore.Request.  This is not a real class, just a 
> logger which makes it possible to handle those log messages differently than 
> the rest of Solr's logging.  I propose setting up another custom logger 
> within SolrCore which could be org.apache.solr.core.SolrCore.SlowRequest.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-SmokeRelease-7.x - Build # 228 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/228/

No tests ran.

Build Log:
[...truncated 24218 lines...]
[asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: 
invalid part, must have at least one section (e.g., chapter, appendix, etc.)
[asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid 
part, must have at least one section (e.g., chapter, appendix, etc.)
 [java] Processed 2200 links (1755 relative) to 2952 anchors in 228 files
 [echo] Validated Links & Anchors via: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr-ref-guide/bare-bones-html/

-dist-changes:
 [copy] Copying 4 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/changes

-dist-keys:
  [get] Getting: http://home.apache.org/keys/group/lucene.asc
  [get] To: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/KEYS

package:

-unpack-solr-tgz:

-ensure-solr-tgz-exists:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr.tgz.unpacked
[untar] Expanding: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/package/solr-7.4.0.tgz
 into 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/solr/build/solr.tgz.unpacked

generate-maven-artifacts:

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-7.x/lucene/top-level-ivy-settings.xml

[jira] [Assigned] (SOLR-11453) Create separate logger for slow requests

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker reassigned SOLR-11453:


  Assignee: Varun Thacker  (was: Shawn Heisey)
Attachment: SOLR-11453.patch

Final patch.  We log slow queries only to one file  : solr_slow_requests.log as 
a warn 

> Create separate logger for slow requests
> 
>
> Key: SOLR-11453
> URL: https://issues.apache.org/jira/browse/SOLR-11453
> Project: Solr
>  Issue Type: Improvement
>  Components: logging
>Reporter: Shawn Heisey
>Assignee: Varun Thacker
>Priority: Minor
> Attachments: SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> slowlog-informational.patch, slowlog-informational.patch, 
> slowlog-informational.patch
>
>
> There is some desire on the mailing list to create a separate logfile for 
> slow queries.  Currently it is not possible to do this cleanly, because the 
> WARN level used by slow query logging within the SolrCore class is also used 
> for other events that SolrCore can log.  Those messages would be out of place 
> in a slow query log.  They should typically stay in main solr logfile.
> I propose creating a custom logger for slow queries, similar to what has been 
> set up for request logging.  In the SolrCore class, which is 
> org.apache.solr.core.SolrCore, there is a special logger at 
> org.apache.solr.core.SolrCore.Request.  This is not a real class, just a 
> logger which makes it possible to handle those log messages differently than 
> the rest of Solr's logging.  I propose setting up another custom logger 
> within SolrCore which could be org.apache.solr.core.SolrCore.SlowRequest.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12376) New TaggerRequestHandler (aka SolrTextTagger)

[David Smiley (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492897#comment-16492897
 ] 

David Smiley commented on SOLR-12376:
-

Updated patch to use the new ConcatenateGraphFilterFactory (which is a WIP; not 
committed yet LUCENE-8332). CGFF supports synonyms and other filters producing 
stacked tokens at indexing time. This is _very_ useful for the tagger!
* I added a test for this -- testWDF to test that WordDelimiterGraphFilter 
works with catenation options.
* partial tagging (via shingling) is no longer easily supported so I commented 
this out. It has to do with difficulties in configuring the separator char 
(CGFF doesn't have this configurable). This feature is probably dubious any way.

Added docs, which was an amalgamation of the SolrTextTagger's existing README 
and QUICK_START files hand-edited/massaged some. I verified the tutorial 
instructions. I added a bin/post version of sending the CSV.  That was a bit of 
a pain to figure out.

At this point it's ready but pending LUCENE-8332.  

> New TaggerRequestHandler (aka SolrTextTagger)
> -
>
> Key: SOLR-12376
> URL: https://issues.apache.org/jira/browse/SOLR-12376
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12376.patch, SOLR-12376.patch, SOLR-12376.patch
>
>
> This issue introduces a new RequestHandler: {{TaggerRequestHandler}}, AKA the 
> SolrTextTagger from the OpenSextant project 
> [https://github.com/OpenSextant/SolrTextTagger]. It's used for named entity 
> recognition (NER) of text past to it. It doesn't do any NLP (outside of 
> Lucene text analysis) so it's said to be a "naive tagger", but it's 
> definitely useful as-is and a more complete NER or ERD (entity recognition 
> and disambiguation) system can be built with this as a key component. The 
> SolrTextTagger has been used on queries for query-understanding, and it's 
> been used on full-text, and it's been used on dictionaries that number tens 
> of millions in size. Since it's small and has been used a bunch (including 
> helping win an ERD competition and in [Apache 
> Stanbol|https://stanbol.apache.org/]), several people have asked me when or 
> why isn't this in Solr yet. So here it is.
> To use it, first you need a collection of documents that have a name-like 
> field (short text) indexed with the ConcatenateFilter (LUCENE-8323) at the 
> end. We call this the dictionary. Once that's in place, you simply post text 
> to a {{TaggerRequestHandler}} and it returns the offset pairs into that text 
> for matches in the dictionary along with the uniqueKey of the matching 
> documents. It can also return other document data desired. That's the gist; 
> I'll add more details on use to the Solr Reference Guide.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12376) New TaggerRequestHandler (aka SolrTextTagger)

[David Smiley (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Smiley updated SOLR-12376:

Attachment: SOLR-12376.patch

> New TaggerRequestHandler (aka SolrTextTagger)
> -
>
> Key: SOLR-12376
> URL: https://issues.apache.org/jira/browse/SOLR-12376
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: David Smiley
>Assignee: David Smiley
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12376.patch, SOLR-12376.patch, SOLR-12376.patch
>
>
> This issue introduces a new RequestHandler: {{TaggerRequestHandler}}, AKA the 
> SolrTextTagger from the OpenSextant project 
> [https://github.com/OpenSextant/SolrTextTagger]. It's used for named entity 
> recognition (NER) of text past to it. It doesn't do any NLP (outside of 
> Lucene text analysis) so it's said to be a "naive tagger", but it's 
> definitely useful as-is and a more complete NER or ERD (entity recognition 
> and disambiguation) system can be built with this as a key component. The 
> SolrTextTagger has been used on queries for query-understanding, and it's 
> been used on full-text, and it's been used on dictionaries that number tens 
> of millions in size. Since it's small and has been used a bunch (including 
> helping win an ERD competition and in [Apache 
> Stanbol|https://stanbol.apache.org/]), several people have asked me when or 
> why isn't this in Solr yet. So here it is.
> To use it, first you need a collection of documents that have a name-like 
> field (short text) indexed with the ConcatenateFilter (LUCENE-8323) at the 
> end. We call this the dictionary. Once that's in place, you simply post text 
> to a {{TaggerRequestHandler}} and it returns the offset pairs into that text 
> for matches in the dictionary along with the uniqueKey of the matching 
> documents. It can also return other document data desired. That's the gist; 
> I'll add more details on use to the Solr Reference Guide.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (SOLR-10428) CloudSolrClient: Qerying multiple collection aliases leads to SolrException: Collection not found

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492891#comment-16492891
 ] 

Shawn Heisey edited comment on SOLR-10428 at 5/28/18 7:09 PM:
--

The collection parameter that is mentioned on the wiki page you linked is a URL 
parameter.  To use that kind of syntax with SolrJ, you would use this code:

{code:java}
params.set("collection","alias-a,alias-b");
solrClient.query(params, SolrRequest.METHOD.POST)
{code}

The way your code supplies the collection parameter works differently.  I think 
that SolrJ puts the provided collection into the URL path -- if you use , 
SolrJ sends to http://host:port/solr/ as the base URL.  I have not verified 
100% that this is the case, but I think that is how it works.



was (Author: elyograg):
The collection parameter that is mentioned on the wiki page you linked is a URL 
parameter.  To use that kind of syntax with SolrJ, you would use this code:

{code:java}
params.set("collection","alias-a,alias-b");
solrClient.query(params, SolrRequest.METHOD.POST)
{code}

The way your code supplies the collection parameter on the works differently.  
I think that SolrJ puts the provided collection into the URL path -- if you use 
, SolrJ sends to http://host:port/solr/ as the base URL.  I have not 
verified 100% that this is the case, but I think that is how it works.


> CloudSolrClient: Qerying multiple collection aliases leads to SolrException: 
> Collection not found
> -
>
> Key: SOLR-10428
> URL: https://issues.apache.org/jira/browse/SOLR-10428
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: 6.4, 6.4.1, 6.4.2, 6.5, 7.0
>Reporter: Philip Pock
>Priority: Minor
>
> We have multiple collections and an alias is created for each of them. e.g.:
> alias-a -> collection-a, alias-b -> collection-b
> We search in multiple collections by passing the aliases of the collections 
> in the collections parameter.
> {code}solrClient.query("alias-a,alias-b", params, 
> SolrRequest.METHOD.POST){code}
> The client can't find the collection and throws an Exception. Relevant parts 
> of the stacktrace using v6.5.0:
> {noformat}
> org.apache.solr.common.SolrException: Collection not found: collection-a
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1394)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1087)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1057)
>   at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
>   at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
> {noformat}
> Everything works fine with a single alias.
> I think this issue was introduced with SOLR-9784. Please see my comment below.
> {code:title=org.apache.solr.client.solrj.impl.CloudSolrClient }
> Set getCollectionNames(String collection) {
> List rawCollectionsList = StrUtils.splitSmart(collection, ",", 
> true);
> Set collectionNames = new HashSet<>();
> for (String collectionName : rawCollectionsList) {
>   if (stateProvider.getState(collectionName) == null) {
> // I assume that collectionName should be passed to getAlias here
> String alias = stateProvider.getAlias(collection);
> if (alias != null) {
>   List aliasList = StrUtils.splitSmart(alias, ",", true);
>   collectionNames.addAll(aliasList);
>   continue;
> }
>   throw new SolrException(ErrorCode.BAD_REQUEST, "Collection not 
> found: " + collectionName);
> }
>   collectionNames.add(collectionName);
> }
> return collectionNames;
>   }
> {code}
> The suggested change is similar to the previous revision: 
> https://github.com/apache/lucene-solr/commit/5650939a8d41b7bad584947a2c9dcedf3774b8de#diff-c8d54eacd46180b332c86c7ae448abaeL1301



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10428) CloudSolrClient: Qerying multiple collection aliases leads to SolrException: Collection not found

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492891#comment-16492891
 ] 

Shawn Heisey commented on SOLR-10428:
-

The collection parameter that is mentioned on the wiki page you linked is a URL 
parameter.  To use that kind of syntax with SolrJ, you would use this code:

{code:java}
params.set("collection","alias-a,alias-b");
solrClient.query(params, SolrRequest.METHOD.POST)
{code}

The way your code supplies the collection parameter on the works differently.  
I think that SolrJ puts the provided collection into the URL path -- if you use 
, SolrJ sends to http://host:port/solr/ as the base URL.  I have not 
verified 100% that this is the case, but I think that is how it works.


> CloudSolrClient: Qerying multiple collection aliases leads to SolrException: 
> Collection not found
> -
>
> Key: SOLR-10428
> URL: https://issues.apache.org/jira/browse/SOLR-10428
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: 6.4, 6.4.1, 6.4.2, 6.5, 7.0
>Reporter: Philip Pock
>Priority: Minor
>
> We have multiple collections and an alias is created for each of them. e.g.:
> alias-a -> collection-a, alias-b -> collection-b
> We search in multiple collections by passing the aliases of the collections 
> in the collections parameter.
> {code}solrClient.query("alias-a,alias-b", params, 
> SolrRequest.METHOD.POST){code}
> The client can't find the collection and throws an Exception. Relevant parts 
> of the stacktrace using v6.5.0:
> {noformat}
> org.apache.solr.common.SolrException: Collection not found: collection-a
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1394)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1087)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1057)
>   at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
>   at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
> {noformat}
> Everything works fine with a single alias.
> I think this issue was introduced with SOLR-9784. Please see my comment below.
> {code:title=org.apache.solr.client.solrj.impl.CloudSolrClient }
> Set getCollectionNames(String collection) {
> List rawCollectionsList = StrUtils.splitSmart(collection, ",", 
> true);
> Set collectionNames = new HashSet<>();
> for (String collectionName : rawCollectionsList) {
>   if (stateProvider.getState(collectionName) == null) {
> // I assume that collectionName should be passed to getAlias here
> String alias = stateProvider.getAlias(collection);
> if (alias != null) {
>   List aliasList = StrUtils.splitSmart(alias, ",", true);
>   collectionNames.addAll(aliasList);
>   continue;
> }
>   throw new SolrException(ErrorCode.BAD_REQUEST, "Collection not 
> found: " + collectionName);
> }
>   collectionNames.add(collectionName);
> }
> return collectionNames;
>   }
> {code}
> The suggested change is similar to the previous revision: 
> https://github.com/apache/lucene-solr/commit/5650939a8d41b7bad584947a2c9dcedf3774b8de#diff-c8d54eacd46180b332c86c7ae448abaeL1301



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11453) Create separate logger for slow requests

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-11453:
-
Affects Version/s: (was: 7.0.1)

> Create separate logger for slow requests
> 
>
> Key: SOLR-11453
> URL: https://issues.apache.org/jira/browse/SOLR-11453
> Project: Solr
>  Issue Type: Improvement
>  Components: logging
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> SOLR-11453.patch, SOLR-11453.patch, slowlog-informational.patch, 
> slowlog-informational.patch, slowlog-informational.patch
>
>
> There is some desire on the mailing list to create a separate logfile for 
> slow queries.  Currently it is not possible to do this cleanly, because the 
> WARN level used by slow query logging within the SolrCore class is also used 
> for other events that SolrCore can log.  Those messages would be out of place 
> in a slow query log.  They should typically stay in main solr logfile.
> I propose creating a custom logger for slow queries, similar to what has been 
> set up for request logging.  In the SolrCore class, which is 
> org.apache.solr.core.SolrCore, there is a special logger at 
> org.apache.solr.core.SolrCore.Request.  This is not a real class, just a 
> logger which makes it possible to handle those log messages differently than 
> the rest of Solr's logging.  I propose setting up another custom logger 
> within SolrCore which could be org.apache.solr.core.SolrCore.SlowRequest.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-11453) Create separate logger for slow requests

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492888#comment-16492888
 ] 

Varun Thacker commented on SOLR-11453:
--

Updated patch folding in changes from Shawn's patch. Running tests etc.

If everything looks okay I'll commit it later today

> Create separate logger for slow requests
> 
>
> Key: SOLR-11453
> URL: https://issues.apache.org/jira/browse/SOLR-11453
> Project: Solr
>  Issue Type: Improvement
>  Components: logging
>Affects Versions: 7.0.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> SOLR-11453.patch, SOLR-11453.patch, slowlog-informational.patch, 
> slowlog-informational.patch, slowlog-informational.patch
>
>
> There is some desire on the mailing list to create a separate logfile for 
> slow queries.  Currently it is not possible to do this cleanly, because the 
> WARN level used by slow query logging within the SolrCore class is also used 
> for other events that SolrCore can log.  Those messages would be out of place 
> in a slow query log.  They should typically stay in main solr logfile.
> I propose creating a custom logger for slow queries, similar to what has been 
> set up for request logging.  In the SolrCore class, which is 
> org.apache.solr.core.SolrCore, there is a special logger at 
> org.apache.solr.core.SolrCore.Request.  This is not a real class, just a 
> logger which makes it possible to handle those log messages differently than 
> the rest of Solr's logging.  I propose setting up another custom logger 
> within SolrCore which could be org.apache.solr.core.SolrCore.SlowRequest.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-11453) Create separate logger for slow requests

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-11453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker updated SOLR-11453:
-
Attachment: SOLR-11453.patch

> Create separate logger for slow requests
> 
>
> Key: SOLR-11453
> URL: https://issues.apache.org/jira/browse/SOLR-11453
> Project: Solr
>  Issue Type: Improvement
>  Components: logging
>Affects Versions: 7.0.1
>Reporter: Shawn Heisey
>Assignee: Shawn Heisey
>Priority: Minor
> Attachments: SOLR-11453.patch, SOLR-11453.patch, SOLR-11453.patch, 
> SOLR-11453.patch, SOLR-11453.patch, slowlog-informational.patch, 
> slowlog-informational.patch, slowlog-informational.patch
>
>
> There is some desire on the mailing list to create a separate logfile for 
> slow queries.  Currently it is not possible to do this cleanly, because the 
> WARN level used by slow query logging within the SolrCore class is also used 
> for other events that SolrCore can log.  Those messages would be out of place 
> in a slow query log.  They should typically stay in main solr logfile.
> I propose creating a custom logger for slow queries, similar to what has been 
> set up for request logging.  In the SolrCore class, which is 
> org.apache.solr.core.SolrCore, there is a special logger at 
> org.apache.solr.core.SolrCore.Request.  This is not a real class, just a 
> logger which makes it possible to handle those log messages differently than 
> the rest of Solr's logging.  I propose setting up another custom logger 
> within SolrCore which could be org.apache.solr.core.SolrCore.SlowRequest.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12401) Add getValue() and setValue() Stream Evaluators

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492855#comment-16492855
 ] 

ASF subversion and git services commented on SOLR-12401:


Commit 3cac1b2cfe0b3be2f11dee86814b94c125db27b0 in lucene-solr's branch 
refs/heads/branch_7x from Joel
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3cac1b2 ]

SOLR-12401: Add getValue() and setValue() Stream Evaluators


> Add getValue() and setValue() Stream Evaluators
> ---
>
> Key: SOLR-12401
> URL: https://issues.apache.org/jira/browse/SOLR-12401
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Jan Høydahl
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12401.patch
>
>
> We need functions to retrieve a value from a tuple and to set a value in an 
> existing tuple:
> Joel writes in 
> [solr-user|https://lists.apache.org/thread.html/f8fb5ae325b172b8d1729e33445beddcc443f7bbd672760cdd0ed25c@%3Csolr-user.lucene.apache.org%3E]:
> {quote}We can add afunctions called:
>  getValue(tuple, key)
>  setValue(tuple, key, value)
> {quote}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12314) ConcurrentUpdateSolrClient doesn't respect the timeout's defined in the solr.xml file

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492878#comment-16492878
 ] 

ASF subversion and git services commented on SOLR-12314:


Commit 071df6e114c3af51de5545c9363189c8692311c1 in lucene-solr's branch 
refs/heads/master from [~varun_saxena]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=071df6e ]

SOLR-12314: Use http timeout's defined in solr.xml for creating 
ConcurrentUpdateSolrClient during indexing requests between leader and replica


> ConcurrentUpdateSolrClient doesn't respect the timeout's defined in the 
> solr.xml file
> -
>
> Key: SOLR-12314
> URL: https://issues.apache.org/jira/browse/SOLR-12314
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12314.patch, SOLR-12314.patch, SOLR-12314.patch
>
>
> In ConcurrentUpdateSolrClient we create an HttpPost Request which allows you 
> to set a request config. If the request config is not provided httpclient 
> will use the default request config. 
>  
> {code:java}
> org.apache.http.client.config.RequestConfig.Builder requestConfigBuilder = 
> HttpClientUtil.createDefaultRequestConfigBuilder();
> if (soTimeout != null) {
>   requestConfigBuilder.setSocketTimeout(soTimeout);
> }
> if (connectionTimeout != null) {
>   requestConfigBuilder.setConnectTimeout(connectionTimeout);
> }
> method.setConfig(requestConfigBuilder.build());{code}
> While creating the httpclient object we ensure that the default request is 
> set with the properties we care about.  This happens in 
> HttpClientUtils#setupBuilder
> {code:java}
> RequestConfig requestConfig = requestConfigBuilder.build();
> HttpClientBuilder retBuilder = 
> builder.setDefaultRequestConfig(requestConfig);{code}
> So there is no need to set a per request config 
>  
> Here is where the httpclient picks the request config is provided on the 
> request itself : 
> [https://github.com/apache/httpcomponents-client/blob/4.5.3/httpclient/src/main/java/org/apache/http/impl/client/InternalHttpClient.java#L168]
>  
> And if it's not provided it uses the default here : 
> https://github.com/apache/httpcomponents-client/blob/4.5.3/httpclient/src/main/java/org/apache/http/impl/client/InternalHttpClient.java#L148



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk-9) - Build # 674 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/674/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.OverseerTaskQueueTest.testPeekElements

Error Message:


Stack Trace:
java.lang.AssertionError
at 
__randomizedtesting.SeedInfo.seed([47D741316D200685:BAF9FB10BD195298]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at 
org.apache.solr.cloud.DistributedQueueTest.testPeekElements(DistributedQueueTest.java:265)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 1821 lines...]
   [junit4] JVM J1: stderr was not empty, see: 
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/core/test/temp/junit4-J1-20180528_161018_84712372224710800752075.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) 
   [junit4] Java HotSpot(TM) 64-Bit 

[jira] [Commented] (SOLR-12290) Do not close any servlet streams and improve our servlet stream closing prevention code for users and devs.

[Varun Thacker (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492883#comment-16492883
 ] 

Varun Thacker commented on SOLR-12290:
--

Hi Mark,

Is it okay if we backport this to branch_7x  ?

The motivation being that you mentioned on SOLR-1881 ( 
https://issues.apache.org/jira/browse/SOLR-11881?focusedCommentId=16458322=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16458322
 ) that this could be one of the underlying reasons for the Jetty exception. 

> Do not close any servlet streams and improve our servlet stream closing 
> prevention code for users and devs.
> ---
>
> Key: SOLR-12290
> URL: https://issues.apache.org/jira/browse/SOLR-12290
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Mark Miller
>Assignee: Mark Miller
>Priority: Major
> Fix For: master (8.0)
>
> Attachments: SOLR-12290.patch, SOLR-12290.patch, SOLR-12290.patch, 
> SOLR-12290.patch
>
>
> Original Summary:
> When you fetch a file for replication we close the request output stream 
> after writing the file which ruins the connection for reuse.
> We can't close response output streams, we need to reuse these connections. 
> If we do close them, clients are hit with connection problems when they try 
> and reuse the connection from their pool.
> New Summary:
> At some point the above was addressed during refactoring. We should remove 
> these neutered closes and review our close shield code.
> If you are here to track down why this is done:
> Connection reuse requires that we read all streams and do not close them - 
> instead the container itself must manage request and response streams. If we 
> allow them to be closed, not only do we lose some connection reuse, but we 
> can cause spurious client errors that can cause expensive recoveries for no 
> reason. The spec allows us to count on the container to manage streams. It's 
> our job simply to not close them and to always read them fully, from client 
> and server. 
> Java itself can help with always reading the streams fully up to some small 
> default amount of unread stream slack, but that is very dangerous to count 
> on, so we always manually eat up anything on the streams our normal logic 
> ends up not reading for whatever reason.
> We also cannot call abort without ruining the connection or sendError. These 
> should be options of very last resort (requiring a blood sacrifice) or when 
> shutting down.
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12411) Add arima Stream Evaluator

[Joel Bernstein (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492875#comment-16492875
 ] 

Joel Bernstein commented on SOLR-12411:
---

This implementation looks quite promising:

https://github.com/Workday/timeseries-forecast

> Add arima Stream Evaluator
> --
>
> Key: SOLR-12411
> URL: https://issues.apache.org/jira/browse/SOLR-12411
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
>
> This ticket will add support for time series ARIMA modeling to the Math 
> Expression library:
> https://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12314) ConcurrentUpdateSolrClient doesn't respect the timeout's defined in the solr.xml file

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492879#comment-16492879
 ] 

ASF subversion and git services commented on SOLR-12314:


Commit caebf73758250191fa3895b52ebe1f14375892d0 in lucene-solr's branch 
refs/heads/branch_7x from [~varun_saxena]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=caebf73 ]

SOLR-12314: Use http timeout's defined in solr.xml for creating 
ConcurrentUpdateSolrClient during indexing requests between leader and replica

(cherry picked from commit 071df6e)


> ConcurrentUpdateSolrClient doesn't respect the timeout's defined in the 
> solr.xml file
> -
>
> Key: SOLR-12314
> URL: https://issues.apache.org/jira/browse/SOLR-12314
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Major
> Attachments: SOLR-12314.patch, SOLR-12314.patch, SOLR-12314.patch
>
>
> In ConcurrentUpdateSolrClient we create an HttpPost Request which allows you 
> to set a request config. If the request config is not provided httpclient 
> will use the default request config. 
>  
> {code:java}
> org.apache.http.client.config.RequestConfig.Builder requestConfigBuilder = 
> HttpClientUtil.createDefaultRequestConfigBuilder();
> if (soTimeout != null) {
>   requestConfigBuilder.setSocketTimeout(soTimeout);
> }
> if (connectionTimeout != null) {
>   requestConfigBuilder.setConnectTimeout(connectionTimeout);
> }
> method.setConfig(requestConfigBuilder.build());{code}
> While creating the httpclient object we ensure that the default request is 
> set with the properties we care about.  This happens in 
> HttpClientUtils#setupBuilder
> {code:java}
> RequestConfig requestConfig = requestConfigBuilder.build();
> HttpClientBuilder retBuilder = 
> builder.setDefaultRequestConfig(requestConfig);{code}
> So there is no need to set a per request config 
>  
> Here is where the httpclient picks the request config is provided on the 
> request itself : 
> [https://github.com/apache/httpcomponents-client/blob/4.5.3/httpclient/src/main/java/org/apache/http/impl/client/InternalHttpClient.java#L168]
>  
> And if it's not provided it uses the default here : 
> https://github.com/apache/httpcomponents-client/blob/4.5.3/httpclient/src/main/java/org/apache/http/impl/client/InternalHttpClient.java#L148



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Created] (SOLR-12411) Add arima Stream Evaluator

[Joel Bernstein (JIRA)]

Joel Bernstein created SOLR-12411:
-

 Summary: Add arima Stream Evaluator
 Key: SOLR-12411
 URL: https://issues.apache.org/jira/browse/SOLR-12411
 Project: Solr
  Issue Type: New Feature
  Security Level: Public (Default Security Level. Issues are Public)
Reporter: Joel Bernstein


This ticket will add support for time series ARIMA modeling to the Math 
Expression library:

https://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (SOLR-12314) ConcurrentUpdateSolrClient doesn't respect the timeout's defined in the solr.xml file

[Varun Thacker (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Varun Thacker resolved SOLR-12314.
--
   Resolution: Fixed
Fix Version/s: master (8.0)
   7.4

Thanks Mark !

> ConcurrentUpdateSolrClient doesn't respect the timeout's defined in the 
> solr.xml file
> -
>
> Key: SOLR-12314
> URL: https://issues.apache.org/jira/browse/SOLR-12314
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Varun Thacker
>Assignee: Varun Thacker
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-12314.patch, SOLR-12314.patch, SOLR-12314.patch
>
>
> In ConcurrentUpdateSolrClient we create an HttpPost Request which allows you 
> to set a request config. If the request config is not provided httpclient 
> will use the default request config. 
>  
> {code:java}
> org.apache.http.client.config.RequestConfig.Builder requestConfigBuilder = 
> HttpClientUtil.createDefaultRequestConfigBuilder();
> if (soTimeout != null) {
>   requestConfigBuilder.setSocketTimeout(soTimeout);
> }
> if (connectionTimeout != null) {
>   requestConfigBuilder.setConnectTimeout(connectionTimeout);
> }
> method.setConfig(requestConfigBuilder.build());{code}
> While creating the httpclient object we ensure that the default request is 
> set with the properties we care about.  This happens in 
> HttpClientUtils#setupBuilder
> {code:java}
> RequestConfig requestConfig = requestConfigBuilder.build();
> HttpClientBuilder retBuilder = 
> builder.setDefaultRequestConfig(requestConfig);{code}
> So there is no need to set a per request config 
>  
> Here is where the httpclient picks the request config is provided on the 
> request itself : 
> [https://github.com/apache/httpcomponents-client/blob/4.5.3/httpclient/src/main/java/org/apache/http/impl/client/InternalHttpClient.java#L168]
>  
> And if it's not provided it uses the default here : 
> https://github.com/apache/httpcomponents-client/blob/4.5.3/httpclient/src/main/java/org/apache/http/impl/client/InternalHttpClient.java#L148



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Assigned] (SOLR-12411) Add arima Stream Evaluator

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein reassigned SOLR-12411:
-

Assignee: Joel Bernstein

> Add arima Stream Evaluator
> --
>
> Key: SOLR-12411
> URL: https://issues.apache.org/jira/browse/SOLR-12411
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
>
> This ticket will add support for time series ARIMA modeling to the Math 
> Expression library:
> https://en.wikipedia.org/wiki/Autoregressive_integrated_moving_average



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12401) Add getValue() and setValue() Stream Evaluators

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492850#comment-16492850
 ] 

ASF subversion and git services commented on SOLR-12401:


Commit 11cfb864894e99cc5953bae19ab29d07599a4e15 in lucene-solr's branch 
refs/heads/master from Joel
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=11cfb86 ]

SOLR-12401: Add getValue() and setValue() Stream Evaluators


> Add getValue() and setValue() Stream Evaluators
> ---
>
> Key: SOLR-12401
> URL: https://issues.apache.org/jira/browse/SOLR-12401
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Jan Høydahl
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12401.patch
>
>
> We need functions to retrieve a value from a tuple and to set a value in an 
> existing tuple:
> Joel writes in 
> [solr-user|https://lists.apache.org/thread.html/f8fb5ae325b172b8d1729e33445beddcc443f7bbd672760cdd0ed25c@%3Csolr-user.lucene.apache.org%3E]:
> {quote}We can add afunctions called:
>  getValue(tuple, key)
>  setValue(tuple, key, value)
> {quote}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (SOLR-12401) Add getValue() and setValue() Stream Evaluators

[Joel Bernstein (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joel Bernstein updated SOLR-12401:
--
Summary: Add getValue() and setValue() Stream Evaluators  (was: Streaming 
functions getValue() and setValue())

> Add getValue() and setValue() Stream Evaluators
> ---
>
> Key: SOLR-12401
> URL: https://issues.apache.org/jira/browse/SOLR-12401
> Project: Solr
>  Issue Type: New Feature
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: streaming expressions
>Reporter: Jan Høydahl
>Assignee: Joel Bernstein
>Priority: Major
> Fix For: 7.4
>
> Attachments: SOLR-12401.patch
>
>
> We need functions to retrieve a value from a tuple and to set a value in an 
> existing tuple:
> Joel writes in 
> [solr-user|https://lists.apache.org/thread.html/f8fb5ae325b172b8d1729e33445beddcc443f7bbd672760cdd0ed25c@%3Csolr-user.lucene.apache.org%3E]:
> {quote}We can add afunctions called:
>  getValue(tuple, key)
>  setValue(tuple, key, value)
> {quote}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-Tests-master - Build # 2548 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2548/

1 tests failed.
FAILED:  org.apache.solr.handler.component.DistributedTermsComponentTest.test

Error Message:
Error from server at http://127.0.0.1:36885/u_/x/collection1: ERROR: [doc=22] 
Error adding field 'c_t'='snake spider' msg=Multiple values encountered for non 
multiValued copy field text: snake spider

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at http://127.0.0.1:36885/u_/x/collection1: ERROR: [doc=22] Error 
adding field 'c_t'='snake spider' msg=Multiple values encountered for non 
multiValued copy field text: snake spider
at 
__randomizedtesting.SeedInfo.seed([8471559AD33D35B0:C256A407DC15848]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at 
org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
at 
org.apache.solr.BaseDistributedSearchTestCase.indexDoc(BaseDistributedSearchTestCase.java:483)
at 
org.apache.solr.BaseDistributedSearchTestCase.index(BaseDistributedSearchTestCase.java:476)
at 
org.apache.solr.handler.component.DistributedTermsComponentTest.test(DistributedTermsComponentTest.java:38)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsRepeatStatement.callStatement(BaseDistributedSearchTestCase.java:1019)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 

[jira] [Created] (LUCENE-8336) Refresh Snowball stemming module to add Arabic stemmer

[Benzahia Lakhdar (JIRA)]

Benzahia Lakhdar created LUCENE-8336:


 Summary: Refresh Snowball stemming module to add Arabic stemmer
 Key: LUCENE-8336
 URL: https://issues.apache.org/jira/browse/LUCENE-8336
 Project: Lucene - Core
  Issue Type: Improvement
  Components: modules/analysis
Reporter: Benzahia Lakhdar


Snowball has been added new light stemming algorithm for Arabic.
You can test the algorithm from the official website: 
[http://snowballstem.org/demo.html]
see the code source from github repo 
https://github.com/snowballstem/snowball/tree/master/algorithms



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 714 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/714/

[...truncated 50 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-Tests-master/2547/consoleText

[repro] Revision: fd929c1d601bd6e946489f51e7b3c0887a3392a6

[repro] Repro line:  ant test  -Dtestcase=SearchRateTriggerTest 
-Dtests.method=testTrigger -Dtests.seed=CBEF0CD8505CAB6C -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=zh-HK -Dtests.timezone=Indian/Cocos 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=SearchHandlerTest 
-Dtests.method=testRequireZkConnectedDistrib -Dtests.seed=CBEF0CD8505CAB6C 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es-CL 
-Dtests.timezone=Europe/Volgograd -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
f8ae144054b67cc65be655e5fb95391cfab26362
[repro] git fetch

[...truncated 3 lines...]
[repro] git checkout fd929c1d601bd6e946489f51e7b3c0887a3392a6

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   SearchHandlerTest
[repro]   SearchRateTriggerTest
[repro] ant compile-test

[...truncated 3298 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 
-Dtests.class="*.SearchHandlerTest|*.SearchRateTriggerTest" 
-Dtests.showOutput=onerror  -Dtests.seed=CBEF0CD8505CAB6C -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=es-CL -Dtests.timezone=Europe/Volgograd 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 7151 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.handler.SearchHandlerTest
[repro]   0/5 failed: org.apache.solr.handler.component.SearchHandlerTest
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest

[repro] Re-testing 100% failures at the tip of master
[repro] ant clean

[...truncated 8 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   SearchRateTriggerTest
[repro] ant compile-test

[...truncated 3298 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.SearchRateTriggerTest" -Dtests.showOutput=onerror  
-Dtests.seed=CBEF0CD8505CAB6C -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=zh-HK -Dtests.timezone=Indian/Cocos -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[...truncated 7098 lines...]
[repro] Setting last failure code to 256

[repro] Failures at the tip of master:
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest

[repro] Re-testing 100% failures at the tip of master without a seed
[repro] ant clean

[...truncated 8 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   SearchRateTriggerTest
[repro] ant compile-test

[...truncated 3298 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.SearchRateTriggerTest" -Dtests.showOutput=onerror  
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=zh-HK 
-Dtests.timezone=Indian/Cocos -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 7068 lines...]
[repro] Setting last failure code to 256

[repro] Failures at the tip of master without a seed:
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest
[repro] git checkout f8ae144054b67cc65be655e5fb95391cfab26362

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr issue #367: [Docs] Fix incorrect BitUtil.deinterleave() descript...

[jpountz]

Github user jpountz commented on the issue:

https://github.com/apache/lucene-solr/pull/367
  
Thank you @nyurik.


---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[GitHub] lucene-solr pull request #367: [Docs] Fix incorrect BitUtil.deinterleave() d...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/lucene-solr/pull/367


---

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 652 - Still Unstable!

[Adrien Grand]

It doesn't reproduce for me which is not too surprising given it uses
threads. However 1000 iterations with ant beast didn't reproduce either.
Maybe someone more familiar with IndexWriter than me can think about cases
when this could happen?

Le lun. 28 mai 2018 à 01:58, Policeman Jenkins Server 
a écrit :

> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/652/
> Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseParallelGC
>
> 1 tests failed.
> FAILED:
> org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce
>
> Error Message:
> MockDirectoryWrapper: cannot close: there are still 26 open files:
> {_7.cfs=1, _c.cfs=1, _l.tvd=1, _2.cfs=1, _h.cfs=1, _1.cfs=1, _l.fdt=1,
> _l.tvx=1, _6.cfs=1, _d.cfs=1, _l.fdx=1, _k.fdt=1, _k.tvx=1, _0.cfs=1,
> _5.cfs=1, _9.cfs=1, _e.cfs=1, _a.cfs=1, _k.fdx=1, _b.cfs=1, _g.cfs=1,
> _k.tvd=1, _3.cfs=1, _4.cfs=1, _f.cfs=1, _8.cfs=1}
>
> Stack Trace:
> java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are
> still 26 open files: {_7.cfs=1, _c.cfs=1, _l.tvd=1, _2.cfs=1, _h.cfs=1,
> _1.cfs=1, _l.fdt=1, _l.tvx=1, _6.cfs=1, _d.cfs=1, _l.fdx=1, _k.fdt=1,
> _k.tvx=1, _0.cfs=1, _5.cfs=1, _9.cfs=1, _e.cfs=1, _a.cfs=1, _k.fdx=1,
> _b.cfs=1, _g.cfs=1, _k.tvd=1, _3.cfs=1, _4.cfs=1, _f.cfs=1, _8.cfs=1}
> at
> __randomizedtesting.SeedInfo.seed([C30D1A9FD0BCA6E7:945B3D7E750DE3B9]:0)
> at
> org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:841)
> at
> org.apache.lucene.index.TestIndexWriterWithThreads._testMultipleThreadsFailure(TestIndexWriterWithThreads.java:341)
> at
> org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringAbortWithThreadsOnlyOnce(TestIndexWriterWithThreads.java:464)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> 

[jira] [Commented] (LUCENE-8186) CustomAnalyzer with a LowerCaseTokenizerFactory fails to normalize multiterms

[Adrien Grand (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492759#comment-16492759
 ] 

Adrien Grand commented on LUCENE-8186:
--

Thanks [~talli...@apache.org].

> CustomAnalyzer with a LowerCaseTokenizerFactory fails to normalize multiterms 
> --
>
> Key: LUCENE-8186
> URL: https://issues.apache.org/jira/browse/LUCENE-8186
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Tim Allison
>Priority: Minor
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8186.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> While working on SOLR-12034, a unit test that relied on the 
> LowerCaseTokenizerFactory failed.
> After some digging, I was able to replicate this at the Lucene level.
> Unit test:
> {noformat}
>   @Test
>   public void testLCTokenizerFactoryNormalize() throws Exception {
> Analyzer analyzer =  
> CustomAnalyzer.builder().withTokenizer(LowerCaseTokenizerFactory.class).build();
> //fails
> assertEquals(new BytesRef("hello"), analyzer.normalize("f", "Hello"));
> 
> //now try an integration test with the classic query parser
> QueryParser p = new QueryParser("f", analyzer);
> Query q = p.parse("Hello");
> //passes
> assertEquals(new TermQuery(new Term("f", "hello")), q);
> q = p.parse("Hello*");
> //fails
> assertEquals(new PrefixQuery(new Term("f", "hello")), q);
> q = p.parse("Hel*o");
> //fails
> assertEquals(new WildcardQuery(new Term("f", "hel*o")), q);
>   }
> {noformat}
> The problem is that the CustomAnalyzer iterates through the tokenfilters, but 
> does not call the tokenizer, which, in the case of the LowerCaseTokenizer, 
> does the filtering work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-8186) CustomAnalyzer with a LowerCaseTokenizerFactory fails to normalize multiterms

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-8186.
--
   Resolution: Fixed
Fix Version/s: master (8.0)
   7.4

> CustomAnalyzer with a LowerCaseTokenizerFactory fails to normalize multiterms 
> --
>
> Key: LUCENE-8186
> URL: https://issues.apache.org/jira/browse/LUCENE-8186
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Tim Allison
>Priority: Minor
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8186.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> While working on SOLR-12034, a unit test that relied on the 
> LowerCaseTokenizerFactory failed.
> After some digging, I was able to replicate this at the Lucene level.
> Unit test:
> {noformat}
>   @Test
>   public void testLCTokenizerFactoryNormalize() throws Exception {
> Analyzer analyzer =  
> CustomAnalyzer.builder().withTokenizer(LowerCaseTokenizerFactory.class).build();
> //fails
> assertEquals(new BytesRef("hello"), analyzer.normalize("f", "Hello"));
> 
> //now try an integration test with the classic query parser
> QueryParser p = new QueryParser("f", analyzer);
> Query q = p.parse("Hello");
> //passes
> assertEquals(new TermQuery(new Term("f", "hello")), q);
> q = p.parse("Hello*");
> //fails
> assertEquals(new PrefixQuery(new Term("f", "hello")), q);
> q = p.parse("Hel*o");
> //fails
> assertEquals(new WildcardQuery(new Term("f", "hel*o")), q);
>   }
> {noformat}
> The problem is that the CustomAnalyzer iterates through the tokenfilters, but 
> does not call the tokenizer, which, in the case of the LowerCaseTokenizer, 
> does the filtering work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8186) CustomAnalyzer with a LowerCaseTokenizerFactory fails to normalize multiterms

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492756#comment-16492756
 ] 

ASF subversion and git services commented on LUCENE-8186:
-

Commit 9ea8927f1c33edcd041b5b83f22af0e0a473ed54 in lucene-solr's branch 
refs/heads/branch_7x from [~jpountz]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9ea8927 ]

LUCENE-8186: LowerCaseTokenizerFactory now lowercases text in multi-term 
queries.


> CustomAnalyzer with a LowerCaseTokenizerFactory fails to normalize multiterms 
> --
>
> Key: LUCENE-8186
> URL: https://issues.apache.org/jira/browse/LUCENE-8186
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Tim Allison
>Priority: Minor
> Attachments: LUCENE-8186.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> While working on SOLR-12034, a unit test that relied on the 
> LowerCaseTokenizerFactory failed.
> After some digging, I was able to replicate this at the Lucene level.
> Unit test:
> {noformat}
>   @Test
>   public void testLCTokenizerFactoryNormalize() throws Exception {
> Analyzer analyzer =  
> CustomAnalyzer.builder().withTokenizer(LowerCaseTokenizerFactory.class).build();
> //fails
> assertEquals(new BytesRef("hello"), analyzer.normalize("f", "Hello"));
> 
> //now try an integration test with the classic query parser
> QueryParser p = new QueryParser("f", analyzer);
> Query q = p.parse("Hello");
> //passes
> assertEquals(new TermQuery(new Term("f", "hello")), q);
> q = p.parse("Hello*");
> //fails
> assertEquals(new PrefixQuery(new Term("f", "hello")), q);
> q = p.parse("Hel*o");
> //fails
> assertEquals(new WildcardQuery(new Term("f", "hel*o")), q);
>   }
> {noformat}
> The problem is that the CustomAnalyzer iterates through the tokenfilters, but 
> does not call the tokenizer, which, in the case of the LowerCaseTokenizer, 
> does the filtering work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8186) CustomAnalyzer with a LowerCaseTokenizerFactory fails to normalize multiterms

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492757#comment-16492757
 ] 

ASF subversion and git services commented on LUCENE-8186:
-

Commit 1971ef310906239d88602444ae6b74081648f3e4 in lucene-solr's branch 
refs/heads/master from [~jpountz]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1971ef3 ]

LUCENE-8186: LowerCaseTokenizerFactory now lowercases text in multi-term 
queries.


> CustomAnalyzer with a LowerCaseTokenizerFactory fails to normalize multiterms 
> --
>
> Key: LUCENE-8186
> URL: https://issues.apache.org/jira/browse/LUCENE-8186
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Tim Allison
>Priority: Minor
> Attachments: LUCENE-8186.patch
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> While working on SOLR-12034, a unit test that relied on the 
> LowerCaseTokenizerFactory failed.
> After some digging, I was able to replicate this at the Lucene level.
> Unit test:
> {noformat}
>   @Test
>   public void testLCTokenizerFactoryNormalize() throws Exception {
> Analyzer analyzer =  
> CustomAnalyzer.builder().withTokenizer(LowerCaseTokenizerFactory.class).build();
> //fails
> assertEquals(new BytesRef("hello"), analyzer.normalize("f", "Hello"));
> 
> //now try an integration test with the classic query parser
> QueryParser p = new QueryParser("f", analyzer);
> Query q = p.parse("Hello");
> //passes
> assertEquals(new TermQuery(new Term("f", "hello")), q);
> q = p.parse("Hello*");
> //fails
> assertEquals(new PrefixQuery(new Term("f", "hello")), q);
> q = p.parse("Hel*o");
> //fails
> assertEquals(new WildcardQuery(new Term("f", "hel*o")), q);
>   }
> {noformat}
> The problem is that the CustomAnalyzer iterates through the tokenfilters, but 
> does not call the tokenizer, which, in the case of the LowerCaseTokenizer, 
> does the filtering work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492749#comment-16492749
 ] 

Cao Manh Dat commented on SOLR-12338:
-

Attached a patch base on [~dsmiley]'s review.

> Replay buffering tlog in parallel
> -
>
> Key: SOLR-12338
> URL: https://issues.apache.org/jira/browse/SOLR-12338
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-12338.patch, SOLR-12338.patch, SOLR-12338.patch, 
> SOLR-12338.patch
>
>
> Since updates with different id are independent, therefore it is safe to 
> replay them in parallel. This will significantly reduce recovering time of 
> replicas in high load indexing environment. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: lucene-solr:branch_7x: SOLR-12358: Autoscaling suggestions fail randomly with sorting

[Adrien Grand]

Hi Noble,

This commit shouldn't add a 8.0.0 section to the 7.x changelog, can you
fix? It looks like there are other unintended changes to the changelog.

Le lun. 28 mai 2018 à 08:30,  a écrit :

> Repository: lucene-solr
> Updated Branches:
>   refs/heads/branch_7x dc0dc1d6e -> a875300a8
>
>
> SOLR-12358: Autoscaling suggestions fail randomly with sorting
>
>
> Project: http://git-wip-us.apache.org/repos/asf/lucene-solr/repo
> Commit: http://git-wip-us.apache.org/repos/asf/lucene-solr/commit/a875300a
> Tree: http://git-wip-us.apache.org/repos/asf/lucene-solr/tree/a875300a
> Diff: http://git-wip-us.apache.org/repos/asf/lucene-solr/diff/a875300a
>
> Branch: refs/heads/branch_7x
> Commit: a875300a897521bc618d5072b20fcd60c8f13985
> Parents: dc0dc1d
> Author: Noble Paul 
> Authored: Thu May 24 01:26:50 2018 +1000
> Committer: Noble Paul 
> Committed: Mon May 28 16:30:23 2018 +1000
>
> --
>  solr/CHANGES.txt| 130 
>  .../autoscaling/AutoScalingHandlerTest.java |   4 +-
>  .../client/solrj/cloud/autoscaling/Policy.java  |  32 +-
>  .../solrj/cloud/autoscaling/PolicyHelper.java   |   1 +
>  .../solrj/cloud/autoscaling/Preference.java |   3 -
>  .../client/solrj/cloud/autoscaling/Row.java |  10 +-
>  .../solrj/cloud/autoscaling/TestPolicy.java | 310 ++-
>  7 files changed, 422 insertions(+), 68 deletions(-)
> --
>
>
>
> http://git-wip-us.apache.org/repos/asf/lucene-solr/blob/a875300a/solr/CHANGES.txt
> --
> diff --git a/solr/CHANGES.txt b/solr/CHANGES.txt
> index 2537d37..99ff4b8 100644
> --- a/solr/CHANGES.txt
> +++ b/solr/CHANGES.txt
> @@ -16,6 +16,35 @@ In this release, there is an example Solr server
> including a bundled
>  servlet container in the directory named "example".
>  See the Solr tutorial at
> https://lucene.apache.org/solr/guide/solr-tutorial.html
>
> +==  8.0.0 ==
> +
> +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in
> this release.
> +
> +Versions of Major Components
> +-
> +Apache Tika 1.16
> +Carrot2 3.15.0
> +Velocity 1.7 and Velocity Tools 2.0
> +Apache UIMA 2.3.1
> +Apache ZooKeeper 3.4.11
> +Jetty 9.4.10.v20180503
> +
> +Upgrade Notes
> +--
> +
> +* LUCENE-7996: The 'func' query parser now returns scores that are equal
> to 0
> +  when a negative value is produced. This change is due to the fact that
> +  Lucene now requires scores to be positive. (Adrien Grand)
> +
> +* SOLR-11882: SolrMetric registries retained references to SolrCores when
> closed. A
> +  change of SolrMetricMAnager.registerGauge and
> SolrMetricProducer.initializeMetrics
> +  method signatures was required to fix it. Third party components that
> use this API
> +  need to be updated. (Eros Taborelli, Erick Erickson, ab)
> +
> +* LUCENE-8267: Memory codecs have been removed from the codebase
> (MemoryPostings,
> +  MemoryDocValues). If you used postingsFormat="Memory" or
> docValuesFormat="Memory"
> +  switch to "Direct" instead. (Dawid Weiss)
> +
>  ==  7.4.0 ==
>
>  Consult the LUCENE_CHANGES.txt file for additional, low level, changes in
> this release.
> @@ -23,11 +52,11 @@ Consult the LUCENE_CHANGES.txt file for additional,
> low level, changes in this r
>  Versions of Major Components
>  -
>  Apache Tika 1.17
> -Carrot2 3.16.0
> +Carrot2 3.15.0
>  Velocity 1.7 and Velocity Tools 2.0
>  Apache UIMA 2.3.1
>  Apache ZooKeeper 3.4.11
> -Jetty 9.3.20.v20170531
> +Jetty 9.4.10.v20180503
>
>  Upgrade Notes
>  --
> @@ -44,8 +73,6 @@ Upgrade Notes
>  New Features
>  --
>
> -* SOLR-12396: Upgrade Carrot2 to 3.16.0, HPPC to 0.8.1, morfologik to
> 2.1.5. (Dawid Weiss)
> -
>  * SOLR-11200: A new CMS config option 'ioThrottle' to manually
> enable/disable
>ConcurrentMergeSchedule.doAutoIOThrottle. (Amrit Sarkar, Nawab Zada
> Asad iqbal via Dawid Weiss)
>
> @@ -92,29 +119,9 @@ New Features
>  * SOLR-9480: A new 'relatedness()' aggregate function for JSON Faceting
> to enable building Semantic
>Knowledge Graphs. (Trey Grainger, hossman)
>
> -* SOLR-12378: Support missing versionField on indexed docs in
> DocBasedVersionConstraintsURP.
> -  (Oliver Bates, Michael Braun via Mark Miller)
> -
> -* SOLR-12388: Enable a strict ZooKeeper-connected search request mode, in
> which search
> -  requests will fail when the coordinating node can't communicate with
> ZooKeeper,
> -  by setting the "shards.tolerant" param to "requireZkConnected".  (Steve
> Rowe)
> -
> -* SOLR-9685: #Tagging queries in JSON Query DSL, equivalent to
> LocalParams based query/filter
> -  tagging.  Multiple tags are comma separated.
> -  

[jira] [Updated] (SOLR-12338) Replay buffering tlog in parallel

[Cao Manh Dat (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Cao Manh Dat updated SOLR-12338:

Attachment: SOLR-12338.patch

> Replay buffering tlog in parallel
> -
>
> Key: SOLR-12338
> URL: https://issues.apache.org/jira/browse/SOLR-12338
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-12338.patch, SOLR-12338.patch, SOLR-12338.patch, 
> SOLR-12338.patch
>
>
> Since updates with different id are independent, therefore it is safe to 
> replay them in parallel. This will significantly reduce recovering time of 
> replicas in high load indexing environment. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk-9.0.4) - Build # 620 - Still Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/620/
Java: 64bit/jdk-9.0.4 -XX:+UseCompressedOops -XX:+UseG1GC

4 tests failed.
FAILED:  
junit.framework.TestSuite.org.apache.solr.prometheus.exporter.SolrExporterTest

Error Message:
Solr servers failed to register with ZK. Current count: 3; Expected count: 4

Stack Trace:
java.lang.IllegalStateException: Solr servers failed to register with ZK. 
Current count: 3; Expected count: 4
at __randomizedtesting.SeedInfo.seed([37D24C5368643AAF]:0)
at 
org.apache.solr.cloud.MiniSolrCloudCluster.waitForAllNodes(MiniSolrCloudCluster.java:283)
at 
org.apache.solr.cloud.MiniSolrCloudCluster.(MiniSolrCloudCluster.java:263)
at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.build(SolrCloudTestCase.java:198)
at 
org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:190)
at 
org.apache.solr.prometheus.exporter.SolrExporterTestBase.setupCluster(SolrExporterTestBase.java:43)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  
junit.framework.TestSuite.org.apache.solr.prometheus.exporter.SolrExporterTest

Error Message:
ObjectTracker found 26 object(s) that were not released!!! [SolrZkClient, 
SolrZkClient, InternalHttpClient, InternalHttpClient, SolrZkClient, 
SolrZkClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
SolrZkClient, InternalHttpClient, InternalHttpClient, ZkController, 
InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, 
SolrZkClient, SolrZkClient, ZkController, ZkController, InternalHttpClient] 
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: 
org.apache.solr.common.cloud.SolrZkClient  at 
org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
  at org.apache.solr.common.cloud.SolrZkClient.(SolrZkClient.java:185)  
at org.apache.solr.cloud.ZkController.(ZkController.java:323)  at 
org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113)  at 
org.apache.solr.core.CoreContainer.load(CoreContainer.java:521)  at 
org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:268)
  at 
org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:188)  
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)  at 
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741)  
at 
org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477)
  at 

[JENKINS] Lucene-Solr-BadApples-Tests-master - Build # 66 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/66/

4 tests failed.
FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5]:0)


FAILED:  
junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue

Error Message:
Suite timeout exceeded (>= 720 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 720 msec).
at __randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5]:0)


FAILED:  
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling

Error Message:
Both triggers should have fired by now

Stack Trace:
java.lang.AssertionError: Both triggers should have fired by now
at 
__randomizedtesting.SeedInfo.seed([F900E77ED97C0FB5:2224F5B0BD6EC27]:0)
at org.junit.Assert.fail(Assert.java:93)
at 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling(TestTriggerIntegration.java:185)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 

[JENKINS] Lucene-Solr-repro - Build # 712 - Still Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/712/

[...truncated 28 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-master/65/consoleText

[repro] Revision: fd929c1d601bd6e946489f51e7b3c0887a3392a6

[repro] Repro line:  ant test  -Dtestcase=TestRandomChains 
-Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=66E54A55B5221BCF 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=ru-RU -Dtests.timezone=Africa/Djibouti -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=IndexSizeTriggerTest 
-Dtests.method=testTrigger -Dtests.seed=F24B3A2F5BFBCD30 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-MT 
-Dtests.timezone=Africa/Bangui -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=IndexSizeTriggerTest 
-Dtests.method=testSplitIntegration -Dtests.seed=F24B3A2F5BFBCD30 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=en-MT -Dtests.timezone=Africa/Bangui -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestTriggerIntegration 
-Dtests.method=testSearchRate -Dtests.seed=F24B3A2F5BFBCD30 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=es-VE -Dtests.timezone=America/Santo_Domingo 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=MoveReplicaHDFSTest 
-Dtests.method=testFailedMove -Dtests.seed=F24B3A2F5BFBCD30 
-Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true 
-Dtests.locale=lt-LT -Dtests.timezone=EST -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
f8ae144054b67cc65be655e5fb95391cfab26362
[repro] git fetch
[repro] git checkout fd929c1d601bd6e946489f51e7b3c0887a3392a6

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   MoveReplicaHDFSTest
[repro]   IndexSizeTriggerTest
[repro]   TestTriggerIntegration
[repro]lucene/analysis/common
[repro]   TestRandomChains
[repro] ant compile-test

[...truncated 3298 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 
-Dtests.class="*.MoveReplicaHDFSTest|*.IndexSizeTriggerTest|*.TestTriggerIntegration"
 -Dtests.showOutput=onerror  -Dtests.seed=F24B3A2F5BFBCD30 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=lt-LT 
-Dtests.timezone=EST -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 15326 lines...]
[repro] Setting last failure code to 256

[repro] ant compile-test

[...truncated 102 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.TestRandomChains" -Dtests.showOutput=onerror  
-Dtests.seed=66E54A55B5221BCF -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=ru-RU -Dtests.timezone=Africa/Djibouti 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 212 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.MoveReplicaHDFSTest
[repro]   5/5 failed: org.apache.lucene.analysis.core.TestRandomChains
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[repro]   5/5 failed: 
org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration

[repro] Re-testing 100% failures at the tip of master
[repro] ant clean

[...truncated 8 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   IndexSizeTriggerTest
[repro]   TestTriggerIntegration
[repro]lucene/analysis/common
[repro]   TestRandomChains
[repro] ant compile-test

[...truncated 3298 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=10 
-Dtests.class="*.IndexSizeTriggerTest|*.TestTriggerIntegration" 
-Dtests.showOutput=onerror  -Dtests.seed=F24B3A2F5BFBCD30 -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=en-MT 
-Dtests.timezone=Africa/Bangui -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 13156 lines...]
[repro] Setting last failure code to 256

[repro] ant compile-test

[...truncated 102 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 
-Dtests.class="*.TestRandomChains" -Dtests.showOutput=onerror  
-Dtests.seed=66E54A55B5221BCF -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.badapples=true -Dtests.locale=ru-RU -Dtests.timezone=Africa/Djibouti 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 211 lines...]
[repro] Setting last failure code to 256

[repro] Failures at the tip of master:
[repro]   4/5 failed: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest
[repro]   5/5 failed: org.apache.lucene.analysis.core.TestRandomChains
[repro]   5/5 failed: 

[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 1034 - Still Failing

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1034/

No tests ran.

Build Log:
[...truncated 24174 lines...]
[asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: 
invalid part, must have at least one section (e.g., chapter, appendix, etc.)
[asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid 
part, must have at least one section (e.g., chapter, appendix, etc.)
 [java] Processed 2213 links (1767 relative) to 3083 anchors in 245 files
 [echo] Validated Links & Anchors via: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr-ref-guide/bare-bones-html/

-dist-changes:
 [copy] Copying 4 files to 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/changes

-dist-keys:
  [get] Getting: http://home.apache.org/keys/group/lucene.asc
  [get] To: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/KEYS

package:

-unpack-solr-tgz:

-ensure-solr-tgz-exists:
[mkdir] Created dir: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked
[untar] Expanding: 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/solr-8.0.0.tgz
 into 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked

generate-maven-artifacts:

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml

resolve:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file = 

[jira] [Commented] (LUCENE-8324) Unreferenced files of dropped segments should be released

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492689#comment-16492689
 ] 

ASF subversion and git services commented on LUCENE-8324:
-

Commit 9a560563d8f8fb68e71ddba313f974b8bc345643 in lucene-solr's branch 
refs/heads/branch_7x from [~jpountz]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=9a56056 ]

LUCENE-8324: Make test pass with ExtraFS.


> Unreferenced files of dropped segments should be released
> -
>
> Key: LUCENE-8324
> URL: https://issues.apache.org/jira/browse/LUCENE-8324
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.4, master (8.0)
>Reporter: Nhat Nguyen
>Priority: Major
> Attachments: LUCENE-8324.patch, release-files.patch
>
>
> {quote} This has the side-effect that flushed segments that are 100% hard 
> deleted are also
> cleaned up right after they are flushed, previously these segments were 
> sticking
> around for a while until they got picked for a merge or received another 
> delete.{quote}
>  
> Since LUCENE-8253, a fully deleted segment is dropped immediately when it's 
> flushed, however, its files might be kept around even after a commit. In 
> other words, we may have unreferenced files which are retained by Deleter.
> I am not entirely sure if we should fix this but it's nice to have a 
> consistent content between current files and commit points as before.
> I attached a failed test for this.
> /cc [~simonw]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8324) Unreferenced files of dropped segments should be released

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492690#comment-16492690
 ] 

ASF subversion and git services commented on LUCENE-8324:
-

Commit f8ae144054b67cc65be655e5fb95391cfab26362 in lucene-solr's branch 
refs/heads/master from [~jpountz]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f8ae144 ]

LUCENE-8324: Make test pass with ExtraFS.


> Unreferenced files of dropped segments should be released
> -
>
> Key: LUCENE-8324
> URL: https://issues.apache.org/jira/browse/LUCENE-8324
> Project: Lucene - Core
>  Issue Type: Bug
>  Components: core/index
>Affects Versions: 7.4, master (8.0)
>Reporter: Nhat Nguyen
>Priority: Major
> Attachments: LUCENE-8324.patch, release-files.patch
>
>
> {quote} This has the side-effect that flushed segments that are 100% hard 
> deleted are also
> cleaned up right after they are flushed, previously these segments were 
> sticking
> around for a while until they got picked for a merge or received another 
> delete.{quote}
>  
> Since LUCENE-8253, a fully deleted segment is dropped immediately when it's 
> flushed, however, its files might be kept around even after a commit. In 
> other words, we may have unreferenced files which are retained by Deleter.
> I am not entirely sure if we should fix this but it's nice to have a 
> consistent content between current files and commit points as before.
> I attached a failed test for this.
> /cc [~simonw]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Re: [JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_172) - Build # 22107 - Failure!

[Adrien Grand]

This particular seed was setting a maximum number of buffered docs equal to
2, which the test didn't like. I pushed a fix.

Le sam. 26 mai 2018 à 14:40, Policeman Jenkins Server 
a écrit :

> Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22107/
> Java: 32bit/jdk1.8.0_172 -client -XX:+UseSerialGC
>
> 1 tests failed.
> FAILED:  org.apache.lucene.search.TestLRUQueryCache.testBulkScorerLocking
>
> Error Message:
> Java heap space
>
> Stack Trace:
> java.lang.OutOfMemoryError: Java heap space
> at
> __randomizedtesting.SeedInfo.seed([BFC83A820B346010:F23F1820122B36A2]:0)
> at
> org.apache.lucene.search.TestLRUQueryCache$DummyDirectoryReader$1.wrap(TestLRUQueryCache.java:1250)
> at
> org.apache.lucene.index.FilterDirectoryReader$SubReaderWrapper.wrap(FilterDirectoryReader.java:56)
> at
> org.apache.lucene.index.FilterDirectoryReader$SubReaderWrapper.access$000(FilterDirectoryReader.java:51)
> at
> org.apache.lucene.index.FilterDirectoryReader.(FilterDirectoryReader.java:83)
> at
> org.apache.lucene.search.TestLRUQueryCache$DummyDirectoryReader.(TestLRUQueryCache.java:1247)
> at
> org.apache.lucene.search.TestLRUQueryCache.testBulkScorerLocking(TestLRUQueryCache.java:1632)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
>
>
>
>
> Build Log:
> [...truncated 1984 lines...]
>[junit4] Suite: org.apache.lucene.search.TestLRUQueryCache
>[junit4]   2> NOTE: reproduce with: ant test
> -Dtestcase=TestLRUQueryCache -Dtests.method=testBulkScorerLocking
> -Dtests.seed=BFC83A820B346010 -Dtests.multiplier=3 -Dtests.slow=true
> -Dtests.locale=es -Dtests.timezone=Africa/Kigali -Dtests.asserts=true
> -Dtests.file.encoding=ISO-8859-1
>[junit4] ERROR   6940s J1 | TestLRUQueryCache.testBulkScorerLocking <<<
>[junit4]> Throwable #1: java.lang.OutOfMemoryError: Java heap space
>[junit4]>at
> __randomizedtesting.SeedInfo.seed([BFC83A820B346010:F23F1820122B36A2]:0)
>[junit4]>at
> org.apache.lucene.search.TestLRUQueryCache$DummyDirectoryReader$1.wrap(TestLRUQueryCache.java:1250)
>[junit4]>at
> org.apache.lucene.index.FilterDirectoryReader$SubReaderWrapper.wrap(FilterDirectoryReader.java:56)
>

[jira] [Commented] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492615#comment-16492615
 ] 

ASF subversion and git services commented on LUCENE-8334:
-

Commit b918ee8fe69a5cb0323ca8a842149fde55e09411 in lucene-solr's branch 
refs/heads/branch_7x from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b918ee8 ]

LUCENE-8334: Ensure SR#getSementInfo() returns snapshot

The SegmentCommitInfo passed to the segment reader is mutated concurrently.
An instance obtained from SR#getSegmentInfo() might return wrong delete counts
or generation ids. This ensures that the SR will use a clone internally while 
stil
maintaining the original SI since it's needed inside IW for maintainance like
accessing pooled readers etc.


> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8334.patch, LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492611#comment-16492611
 ] 

ASF subversion and git services commented on LUCENE-8334:
-

Commit 0941cae532ddf7b9af3df55c63941f547c769108 in lucene-solr's branch 
refs/heads/master from [~simonw]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0941cae ]

LUCENE-8334: Ensure SR#getSementInfo() returns snapshot

The SegmentCommitInfo passed to the segment reader is mutated concurrently.
An instance obtained from SR#getSegmentInfo() might return wrong delete counts
or generation ids. This ensures that the SR will use a clone internally while 
stil
maintaining the original SI since it's needed inside IW for maintainance like
accessing pooled readers etc.


> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Affects Versions: 7.4, master (8.0)
>Reporter: Simon Willnauer
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: LUCENE-8334.patch, LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492538#comment-16492538
 ] 

Cao Manh Dat commented on SOLR-12338:
-

{quote}
The hot while loop of map.putIfAbsent seems fishy to me. Even if it may be rare 
in practice, I wonder if we can do something simpler? You may get luck with 
map.compute* methods on ConcurrentHashMap which execute the lambda atomically. 
Though I don't know if it's bad to block if we try to acquire a lock within 
there. I see remove() removes the value of the Map but perhaps it the value 
were a mechanism that tracked that there's a producer pending, then we should 
not remove the value from the lock? If we did this, then maybe that would 
simplify add()? I'm not sure.
{quote}
After putting more thought on this, Change the remove method to this one seems 
to solve the problem.
{code}
public void remove(T t) {
  // There can be many threads are waiting for this lock
  map.remove(t).release(Integer.MAX_VALUE);
  sizeLock.release();
}
{code}
In short of the idea of SetBlockingQueue.add(T t) is 
# all participations will try to call {{map.putIfAbsent(t, myLock)}}, 
# only one will win, other participations will have to wait for the lock of the 
winner
# when the winner get removed from the set, it also release + remove its lock
# back to 1.

> Replay buffering tlog in parallel
> -
>
> Key: SOLR-12338
> URL: https://issues.apache.org/jira/browse/SOLR-12338
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-12338.patch, SOLR-12338.patch, SOLR-12338.patch
>
>
> Since updates with different id are independent, therefore it is safe to 
> replay them in parallel. This will significantly reduce recovering time of 
> replicas in high load indexing environment. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492507#comment-16492507
 ] 

Simon Willnauer commented on LUCENE-8334:
-

alright cool, no I just made the ctor pkg private.

> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492502#comment-16492502
 ] 

Dawid Weiss commented on LUCENE-8334:
-

Yeah... I look as much as my time permits, which isn't too much. :( What you 
mention is actually what we do: I wrongly thought you removed the public 
qualifier from the class (as well as the constructor); looking at the diff only 
confused me. LGTM!

> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[Simon Willnauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-8334:

Labels: PatchAvailable  (was: )

> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[Simon Willnauer (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Simon Willnauer updated LUCENE-8334:

Labels:   (was: PatchAvailable)

> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492496#comment-16492496
 ] 

Simon Willnauer commented on LUCENE-8334:
-

[~dweiss] I am glad you checking stuff out here. I did look and I wonder why 
you can't just open the index you need from a directory and then steal the 
SegmentReader from it's leaves? I can totally back this part out and open 
another issue to discuss it, just asking to understand why there is no 
alternative?

> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8162) Make it possible to throttle (Tiered)MergePolicy when commit rate is high

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492469#comment-16492469
 ] 

Tommaso Teofili edited comment on LUCENE-8162 at 5/28/18 10:05 AM:
---

{quote}but many users index at full speed for a long time and suppressing 
merges in that case is dangerous
{quote}
yes, that might make search performance degrade. To mitigate that the proposed 
MP has a maximum number of segments allowed for throttling. So for example if 
the throttling algorithm makes the number of segments go beyond a configurable 
threshold (e.g. 20), the throttling algorithm doesn't kick in in the next merge 
and until the number of segments gets back beyond the threshold (by using 
standard TMP merge algorithm).

I have been trying to use [https://github.com/mikemccand/luceneutil] to make 
some benchmarks. However it seems the tool only creates one index per 
benchmark, if anyone has suggestions about how to benchmark both indexing (time 
and space) and querying performance that'd be great. 


was (Author: teofili):
{quote}but many users index at full speed for a long time and suppressing 
merges in that case is dangerous
{quote}
yes, that might make search performance degrade. To mitigate that the proposed 
MP has a maximum number of segments allowed for throttling. So for example if 
the throttling algorithm makes the number of segments go beyond a configurable 
threshold (e.g. 20), the throttling algorithm doesn't kick in in the next merge 
and until the number of segments gets back beyond the threshold.

I have been trying to use [https://github.com/mikemccand/luceneutil] to make 
some benchmarks. However it seems the tool only creates one index per 
benchmark. 

> Make it possible to throttle (Tiered)MergePolicy when commit rate is high
> -
>
> Key: LUCENE-8162
> URL: https://issues.apache.org/jira/browse/LUCENE-8162
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Tommaso Teofili
>Priority: Major
> Fix For: trunk
>
> Attachments: LUCENE-8162.0.patch
>
>
> As discussed in a recent mailing list thread [1] and observed in a project 
> using Lucene (see OAK-5192 and OAK-6710), it is sometimes helpful to throttle 
> the aggressiveness of (Tiered)MergePolicy when commit rate is high.
> In the case of Apache Jackrabbit Oak a dedicated {{MergePolicy}} was 
> implemented [2].
> That MP doesn't merge in case the number of segments is below a certain 
> threshold (e.g. 30) and commit rate (docs per sec and MB per sec) is high 
> (e.g. above 1000 doc / sec , 5MB / sec).
> In such impl, the commit rate thresholds adapt to average commit rate by 
> means of single exponential smoothing.
> The results in that specific case looked encouraging as it brought a 5% perf 
> improvement in querying and ~10% reduced IO. However Oak has some specifics 
> which might not fit in other scenarios. Anyway it could be interesting to see 
> how this behaves in plain Lucene scenario.
> [1] : [http://markmail.org/message/re3ifmq2664bqfjk]
> [2] : 
> [https://github.com/apache/jackrabbit-oak/blob/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/writer/CommitMitigatingTieredMergePolicy.java]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8335) Do not allow changing soft-deletes field

[Simon Willnauer (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492483#comment-16492483
 ] 

Simon Willnauer commented on LUCENE-8335:
-

> i dont think lucene needs to enforce this. from my perspective its just a 
>docvalues field. given that lucene doesnt even know the difference between a 
>integer amd a rloat field, i dont think it should be tracking expert shit for 
>elasticsearch.

my reasoning here is that in-turn to make this a more widely useable feature 
ie. [~mikemccand] indicated he want's to use it and I suspect the usecase is 
becoming more widely adopted we can be more strict about it and make it a 
non-expert feature. The fact that it's just a DV field is great and under the 
hood not many changes were necessary. The overhead of tracking this is small in 
my opinion and enforcing this would allow us to make the feature much less 
trappy down the road. ie. factory methods can automatically wrap indices that 
have a soft-deletes field, we can track the numSoftDeletes which helps a ton 
with assertions and allows to pull index stats by reading seginfos alone 
without opening a reader. I can work around all these thing and I am not even 
convinced we should do automatically wrapping a reader etc. but we can make 
this feature easy to use with a simple setter on IWC. From a interface 
perspective it's simple and the changes necessary to make it way less trappy 
warrent the change. I wonder what others think.

> Do not allow changing soft-deletes field
> 
>
> Key: LUCENE-8335
> URL: https://issues.apache.org/jira/browse/LUCENE-8335
> Project: Lucene - Core
>  Issue Type: Improvement
>Affects Versions: 7.4, master (8.0)
>Reporter: Nhat Nguyen
>Assignee: Simon Willnauer
>Priority: Minor
> Attachments: LUCENE-8335.patch
>
>
> Today we do not enforce an index to use a single soft-deletes field. A user 
> can create an index with one soft-deletes field then open an IW with another 
> field or add an index with a different soft-deletes field. This should not be 
> allowed and reported the error to users as soon as possible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8162) Make it possible to throttle (Tiered)MergePolicy when commit rate is high

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492469#comment-16492469
 ] 

Tommaso Teofili edited comment on LUCENE-8162 at 5/28/18 10:04 AM:
---

{quote}but many users index at full speed for a long time and suppressing 
merges in that case is dangerous
{quote}
yes, that might make search performance degrade. To mitigate that the proposed 
MP has a maximum number of segments allowed for throttling. So for example if 
the throttling algorithm makes the number of segments go beyond a configurable 
threshold (e.g. 20), the throttling algorithm doesn't kick in in the next merge 
and until the number of segments gets back beyond the threshold.

I have been trying to use [https://github.com/mikemccand/luceneutil] to make 
some benchmarks. However it seems the tool only creates one index per 
benchmark. 


was (Author: teofili):
{quote}but many users index at full speed for a long time and suppressing 
merges in that case is dangerous
{quote}
yes, that might make search degrade. To mitigate that the proposed MP has a 
maximum number of segments allowed for throttling. So for example if the 
throttling algorithm makes the number of segments go beyond a configurable 
threshold (e.g. 20), the throttling algorithm doesn't kick in in the next merge 
and until the number of segments gets back beyond the threshold.

I have been trying to use [https://github.com/mikemccand/luceneutil] to make 
some benchmarks. However it seems the tool only creates one index per 
benchmark. 

> Make it possible to throttle (Tiered)MergePolicy when commit rate is high
> -
>
> Key: LUCENE-8162
> URL: https://issues.apache.org/jira/browse/LUCENE-8162
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Tommaso Teofili
>Priority: Major
> Fix For: trunk
>
> Attachments: LUCENE-8162.0.patch
>
>
> As discussed in a recent mailing list thread [1] and observed in a project 
> using Lucene (see OAK-5192 and OAK-6710), it is sometimes helpful to throttle 
> the aggressiveness of (Tiered)MergePolicy when commit rate is high.
> In the case of Apache Jackrabbit Oak a dedicated {{MergePolicy}} was 
> implemented [2].
> That MP doesn't merge in case the number of segments is below a certain 
> threshold (e.g. 30) and commit rate (docs per sec and MB per sec) is high 
> (e.g. above 1000 doc / sec , 5MB / sec).
> In such impl, the commit rate thresholds adapt to average commit rate by 
> means of single exponential smoothing.
> The results in that specific case looked encouraging as it brought a 5% perf 
> improvement in querying and ~10% reduced IO. However Oak has some specifics 
> which might not fit in other scenarios. Anyway it could be interesting to see 
> how this behaves in plain Lucene scenario.
> [1] : [http://markmail.org/message/re3ifmq2664bqfjk]
> [2] : 
> [https://github.com/apache/jackrabbit-oak/blob/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/writer/CommitMitigatingTieredMergePolicy.java]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8162) Make it possible to throttle (Tiered)MergePolicy when commit rate is high

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492469#comment-16492469
 ] 

Tommaso Teofili commented on LUCENE-8162:
-

{quote}but many users index at full speed for a long time and suppressing 
merges in that case is dangerous
{quote}
yes, that might make search degrade. To mitigate that the proposed MP has a 
maximum number of segments allowed for throttling. So for example if the 
throttling algorithm makes the number of segments go beyond a configurable 
threshold (e.g. 20), the throttling algorithm doesn't kick in in the next merge 
and until the number of segments gets back beyond the threshold.

I have been trying to use [https://github.com/mikemccand/luceneutil] to make 
some benchmarks. However it seems the tool only creates one index per 
benchmark. 

> Make it possible to throttle (Tiered)MergePolicy when commit rate is high
> -
>
> Key: LUCENE-8162
> URL: https://issues.apache.org/jira/browse/LUCENE-8162
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Tommaso Teofili
>Priority: Major
> Fix For: trunk
>
> Attachments: LUCENE-8162.0.patch
>
>
> As discussed in a recent mailing list thread [1] and observed in a project 
> using Lucene (see OAK-5192 and OAK-6710), it is sometimes helpful to throttle 
> the aggressiveness of (Tiered)MergePolicy when commit rate is high.
> In the case of Apache Jackrabbit Oak a dedicated {{MergePolicy}} was 
> implemented [2].
> That MP doesn't merge in case the number of segments is below a certain 
> threshold (e.g. 30) and commit rate (docs per sec and MB per sec) is high 
> (e.g. above 1000 doc / sec , 5MB / sec).
> In such impl, the commit rate thresholds adapt to average commit rate by 
> means of single exponential smoothing.
> The results in that specific case looked encouraging as it brought a 5% perf 
> improvement in querying and ~10% reduced IO. However Oak has some specifics 
> which might not fit in other scenarios. Anyway it could be interesting to see 
> how this behaves in plain Lucene scenario.
> [1] : [http://markmail.org/message/re3ifmq2664bqfjk]
> [2] : 
> [https://github.com/apache/jackrabbit-oak/blob/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/writer/CommitMitigatingTieredMergePolicy.java]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-repro - Build # 711 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-repro/711/

[...truncated 50 lines...]
[repro] Jenkins log URL: 
https://builds.apache.org/job/Lucene-Solr-Tests-7.x/625/consoleText

[repro] Revision: 3c6fb974aefc50f411cba90b2b267804fbb6e1cb

[repro] Repro line:  ant test  -Dtestcase=TestDistribDocBasedVersion 
-Dtests.method=test -Dtests.seed=E5D805CC3EED224C -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=lv-LV -Dtests.timezone=Europe/Sofia 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestCloudConsistency 
-Dtests.method=testOutOfSyncReplicasCannotBecomeLeader 
-Dtests.seed=E5D805CC3EED224C -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=el-GR -Dtests.timezone=Atlantic/Azores -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=TestCloudConsistency 
-Dtests.method=testOutOfSyncReplicasCannotBecomeLeaderAfterRestart 
-Dtests.seed=E5D805CC3EED224C -Dtests.multiplier=2 -Dtests.slow=true 
-Dtests.locale=el-GR -Dtests.timezone=Atlantic/Azores -Dtests.asserts=true 
-Dtests.file.encoding=UTF-8

[repro] Repro line:  ant test  -Dtestcase=SearchRateTriggerTest 
-Dtests.method=testTrigger -Dtests.seed=E5D805CC3EED224C -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=pl -Dtests.timezone=Africa/Tripoli 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] git rev-parse --abbrev-ref HEAD
[repro] git rev-parse HEAD
[repro] Initial local git branch/revision: 
4e12546b02ecfc9b142a026dcaca9996234a409d
[repro] git fetch
[repro] git checkout 3c6fb974aefc50f411cba90b2b267804fbb6e1cb

[...truncated 2 lines...]
[repro] git merge --ff-only

[...truncated 1 lines...]
[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]solr/core
[repro]   SearchRateTriggerTest
[repro]   TestDistribDocBasedVersion
[repro]   TestCloudConsistency
[repro] ant compile-test

[...truncated 3316 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 
-Dtests.class="*.SearchRateTriggerTest|*.TestDistribDocBasedVersion|*.TestCloudConsistency"
 -Dtests.showOutput=onerror  -Dtests.seed=E5D805CC3EED224C -Dtests.multiplier=2 
-Dtests.slow=true -Dtests.locale=pl -Dtests.timezone=Africa/Tripoli 
-Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 5790 lines...]
[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.TestCloudConsistency
[repro]   0/5 failed: org.apache.solr.cloud.TestDistribDocBasedVersion
[repro]   4/5 failed: org.apache.solr.cloud.autoscaling.SearchRateTriggerTest
[repro] git checkout 4e12546b02ecfc9b142a026dcaca9996234a409d

[...truncated 2 lines...]
[repro] Exiting with code 256

[...truncated 6 lines...]

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-12338) Replay buffering tlog in parallel

[Cao Manh Dat (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492410#comment-16492410
 ] 

Cao Manh Dat commented on SOLR-12338:
-

Thanks a lot for your review [~dsmiley], I was too busy recently.
{quote}
- I think the "hash" variable should not be called this to avoid confusion as 
there is no hashing. Maybe just "id" or "lockId"
- Do we still need the Random stuff?
- Maybe rename your "SetBlockingQueue" to "SetSemaphore" or probably better 
"SetLock" as it does not hold anything (Queues hold stuff)
- Can "Semaphore sizeLock" be renamed to "sizeSemaphore" or "sizePermits" is it 
does not extend Lock?
- Can the "closed" state be removed from SetBlockingQueue altogether? It's not 
clear it actually needs to be "closed". It seems wrong; other concurrent 
mechanisms don't have this notion (no Queue, Lock, or Semaphore does, etc.) 
FWIW I stripped this from the class and the test passed.
{quote}
+1

{quote}
Perhaps its better to acquire() the size permit first in add() instead of last 
to prevent lots of producing threads inserting keys into a map only to 
eventually wait. Although it might add annoying try-finally to add() to ensure 
we put the permit back if there's an exception after (e.g. interrupt). Heck; 
maybe that's an issue no matter what the sequence is.
{quote}
I don't think we should do that. {{sizeLock}} kinda like the number of maximum 
threads, if we reached that number, it seems better to let them wait before 
trying to enqueue more tasks.

{quote}
Can the value side of the ConcurrentHashMap be a Lock (I guess ReentrantLock 
impl)? It seems like the most direct concept we want; Semaphore is more than a 
Lock as it tracks permits that we don't need here?
{quote}
We can't. Lock or ReetrantLock only allows us to lock and unlock in the same 
thread. In the OrderedExecutor, we lock first then unlock in the thread of 
delegate executor.

{quote}
The hot while loop of map.putIfAbsent seems fishy to me. Even if it may be rare 
in practice, I wonder if we can do something simpler? You may get luck with 
map.compute* methods on ConcurrentHashMap which execute the lambda atomically. 
Though I don't know if it's bad to block if we try to acquire a lock within 
there. I see remove() removes the value of the Map but perhaps it the value 
were a mechanism that tracked that there's a producer pending, then we should 
not remove the value from the lock? If we did this, then maybe that would 
simplify add()? I'm not sure.
{quote}
I will think more about this.

> Replay buffering tlog in parallel
> -
>
> Key: SOLR-12338
> URL: https://issues.apache.org/jira/browse/SOLR-12338
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>Reporter: Cao Manh Dat
>Assignee: Cao Manh Dat
>Priority: Major
> Attachments: SOLR-12338.patch, SOLR-12338.patch, SOLR-12338.patch
>
>
> Since updates with different id are independent, therefore it is safe to 
> replay them in parallel. This will significantly reduce recovering time of 
> replicas in high load indexing environment. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 69 - Unstable

[Apache Jenkins Server]

Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/69/

5 tests failed.
FAILED:  org.apache.solr.handler.TestSQLHandler.doTest

Error Message:
Error from server at https://127.0.0.1:33032: KeeperErrorCode = NoNode for 
/overseer/collection-queue-work/qnr-12

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error 
from server at https://127.0.0.1:33032: KeeperErrorCode = NoNode for 
/overseer/collection-queue-work/qnr-12
at 
__randomizedtesting.SeedInfo.seed([338C71662437D38:A47C7FB20FF86E81]:0)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at 
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at 
org.apache.solr.cloud.AbstractFullDistribZkTestBase.reloadCollection(AbstractFullDistribZkTestBase.java:2054)
at 
org.apache.solr.handler.TestSQLHandler.testBasicSelect(TestSQLHandler.java:307)
at org.apache.solr.handler.TestSQLHandler.doTest(TestSQLHandler.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at 
org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 

[jira] [Updated] (SOLR-9685) tag a query in JSON syntax

[Mikhail Khludnev (JIRA)]

 [ 
https://issues.apache.org/jira/browse/SOLR-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Khludnev updated SOLR-9685:
---
Fix Version/s: master (8.0)

> tag a query in JSON syntax
> --
>
> Key: SOLR-9685
> URL: https://issues.apache.org/jira/browse/SOLR-9685
> Project: Solr
>  Issue Type: Improvement
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: Facet Module, JSON Request API
>Reporter: Yonik Seeley
>Assignee: Yonik Seeley
>Priority: Major
> Fix For: 7.4, master (8.0)
>
> Attachments: SOLR-9685.patch, SOLR-9685.patch
>
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> There should be a way to tag a query/filter in JSON syntax.
> Perhaps these two forms could be equivalent:
> {code}
> "{!tag=COLOR}color:blue"
> { tagged : { COLOR : "color:blue" }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Resolved] (LUCENE-4892) Create a compressed LiveDocsFormat

[Adrien Grand (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adrien Grand resolved LUCENE-4892.
--
Resolution: Won't Fix

I was hoping that the smaller memory footprint might make it comparable to 
FixedBitSet in speed in spite of the higher overhead, but apparently it is not 
enough:

{noformat}
TaskQPS baseline  StdDev   QPS patch  StdDev
Pct diff
   HighTermMonthSort 1221.82  (8.7%)  738.86  (3.4%)  
-39.5% ( -47% -  -30%)
  IntNRQ  126.95  (4.5%)   78.65  (1.4%)  
-38.0% ( -42% -  -33%)
 Prefix3  464.10  (2.5%)  331.94  (1.4%)  
-28.5% ( -31% -  -25%)
HighTerm  572.63  (1.4%)  416.42  (0.7%)  
-27.3% ( -28% -  -25%)
   HighTermDayOfYearSort  324.51  (1.9%)  241.01  (1.4%)  
-25.7% ( -28% -  -22%)
   OrHighLow  511.62  (1.0%)  387.64  (0.5%)  
-24.2% ( -25% -  -22%)
 MedTerm 1609.19  (1.8%) 1225.70  (0.9%)  
-23.8% ( -26% -  -21%)
  OrHighHigh  144.60  (5.0%)  118.70  (2.9%)  
-17.9% ( -24% -  -10%)
   OrHighMed  421.88  (4.0%)  349.80  (2.7%)  
-17.1% ( -22% -  -10%)
 LowTerm 3987.51  (6.4%) 3503.75  (3.5%)  
-12.1% ( -20% -   -2%)
Wildcard  241.67  (2.9%)  216.89  (1.7%)  
-10.3% ( -14% -   -5%)
  HighPhrase  178.98  (3.9%)  165.85  (3.8%)   
-7.3% ( -14% -0%)
HighSpanNear  144.91  (2.7%)  134.36  (2.6%)   
-7.3% ( -12% -   -2%)
 AndHighHigh  214.80  (1.0%)  199.49  (1.2%)   
-7.1% (  -9% -   -4%)
HighSloppyPhrase  249.43  (3.5%)  232.35  (3.4%)   
-6.8% ( -13% -0%)
  Fuzzy1  159.45  (2.0%)  149.14  (2.6%)   
-6.5% ( -10% -   -1%)
   MedPhrase  175.70  (2.6%)  165.17  (2.5%)   
-6.0% ( -10% -0%)
 MedSpanNear  645.69  (3.0%)  609.97  (2.7%)   
-5.5% ( -10% -0%)
 MedSloppyPhrase  144.37  (2.5%)  136.76  (2.6%)   
-5.3% ( -10% -0%)
   LowPhrase  285.30  (2.5%)  270.93  (2.8%)   
-5.0% ( -10% -0%)
 LowSloppyPhrase  398.34  (1.9%)  383.09  (2.0%)   
-3.8% (  -7% -0%)
 LowSpanNear  517.04  (1.4%)  506.96  (1.4%)   
-1.9% (  -4% -0%)
  AndHighMed 1090.21  (4.2%) 1077.55  (4.1%)   
-1.2% (  -9% -7%)
 Respell  171.53  (2.8%)  171.52  (2.1%)   
-0.0% (  -4% -5%)
  AndHighLow 1346.57  (2.4%) 1367.21  (1.9%)
1.5% (  -2% -5%)
  Fuzzy2   65.94  (8.6%)   67.84 (11.5%)
2.9% ( -15% -   25%)
{noformat}

> Create a compressed LiveDocsFormat
> --
>
> Key: LUCENE-4892
> URL: https://issues.apache.org/jira/browse/LUCENE-4892
> Project: Lucene - Core
>  Issue Type: New Feature
>Reporter: Adrien Grand
>Priority: Trivial
>  Labels: gsoc2014
> Attachments: LUCENE-4892.patch
>
>
> There are lots of use-cases where the number of deleted documents is low. 
> This makes live docs very dense and I think it would be interesting to study 
> the impact of some bitmap compression techniques on performance (intuitively 
> I imagine it will be slower, but since it would make data smaller maybe CPU 
> caches could help us so I'd be curious to see how it would behave). This 
> format would make a good addition to our CheapBastardCodec.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Updated] (LUCENE-8162) Make it possible to throttle (Tiered)MergePolicy when commit rate is high

[Tommaso Teofili (JIRA)]

 [ 
https://issues.apache.org/jira/browse/LUCENE-8162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tommaso Teofili updated LUCENE-8162:

Attachment: LUCENE-8162.0.patch

> Make it possible to throttle (Tiered)MergePolicy when commit rate is high
> -
>
> Key: LUCENE-8162
> URL: https://issues.apache.org/jira/browse/LUCENE-8162
> Project: Lucene - Core
>  Issue Type: Improvement
>  Components: core/index
>Reporter: Tommaso Teofili
>Priority: Major
> Fix For: trunk
>
> Attachments: LUCENE-8162.0.patch
>
>
> As discussed in a recent mailing list thread [1] and observed in a project 
> using Lucene (see OAK-5192 and OAK-6710), it is sometimes helpful to throttle 
> the aggressiveness of (Tiered)MergePolicy when commit rate is high.
> In the case of Apache Jackrabbit Oak a dedicated {{MergePolicy}} was 
> implemented [2].
> That MP doesn't merge in case the number of segments is below a certain 
> threshold (e.g. 30) and commit rate (docs per sec and MB per sec) is high 
> (e.g. above 1000 doc / sec , 5MB / sec).
> In such impl, the commit rate thresholds adapt to average commit rate by 
> means of single exponential smoothing.
> The results in that specific case looked encouraging as it brought a 5% perf 
> improvement in querying and ~10% reduced IO. However Oak has some specifics 
> which might not fit in other scenarios. Anyway it could be interesting to see 
> how this behaves in plain Lucene scenario.
> [1] : [http://markmail.org/message/re3ifmq2664bqfjk]
> [2] : 
> [https://github.com/apache/jackrabbit-oak/blob/trunk/oak-lucene/src/main/java/org/apache/jackrabbit/oak/plugins/index/lucene/writer/CommitMitigatingTieredMergePolicy.java]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Comment Edited] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492348#comment-16492348
 ] 

Dawid Weiss edited comment on LUCENE-8334 at 5/28/18 7:22 AM:
--

Hi Simon. This was made private in this patch:
{code}
-  // TODO: why is this public?
-  public SegmentReader(SegmentCommitInfo si, int createdVersionMajor, 
IOContext context) throws IOException {
-this.si = si;
+  SegmentReader(SegmentCommitInfo si, int createdVersionMajor, IOContext 
context) throws IOException {
{code}

I agree it's expert-level API, but it is useful (has no alternative) for 
manipulating parallel indexes (see TestDemoParallelLeafReader). I am actually 
using this stuff (outside of Lucene code) exactly like Mike's "demo" shows -- 
to maintain derived indexes from primary segments.

Can we make this a primary API citizen (or expose a public interface 
implemented by this class that would convey the information required for 
TestDemoParallelLeafReader to work, if it's moved from .index package)?


was (Author: dweiss):
Hi Simon. This was made private in this patch:
{code}
-  // TODO: why is this public?
-  public SegmentReader(SegmentCommitInfo si, int createdVersionMajor, 
IOContext context) throws IOException {
-this.si = si;
+  SegmentReader(SegmentCommitInfo si, int createdVersionMajor, IOContext 
context) throws IOException {
{code}

I agree it's expert-level API, but it is useful (has no alternative) for 
manipulating parallel indexes (see TestDemoParallelLeafReader). I am actually 
using this stuff (outside of Lucene code) exactly like Mike's "demo" shows -- 
to maintain derived indexes from primary segments.

Can we make a primary API citizen (or expose a public interface implemented by 
this class that would convey the information required for 
TestDemoParallelLeafReader to work, if it's moved from .index package)?

> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (LUCENE-8334) Ensure SR#getSementInfo() returns snapshot

[Dawid Weiss (JIRA)]

[ 
https://issues.apache.org/jira/browse/LUCENE-8334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492348#comment-16492348
 ] 

Dawid Weiss commented on LUCENE-8334:
-

Hi Simon. This was made private in this patch:
{code}
-  // TODO: why is this public?
-  public SegmentReader(SegmentCommitInfo si, int createdVersionMajor, 
IOContext context) throws IOException {
-this.si = si;
+  SegmentReader(SegmentCommitInfo si, int createdVersionMajor, IOContext 
context) throws IOException {
{code}

I agree it's expert-level API, but it is useful (has no alternative) for 
manipulating parallel indexes (see TestDemoParallelLeafReader). I am actually 
using this stuff (outside of Lucene code) exactly like Mike's "demo" shows -- 
to maintain derived indexes from primary segments.

Can we make a primary API citizen (or expose a public interface implemented by 
this class that would convey the information required for 
TestDemoParallelLeafReader to work, if it's moved from .index package)?

> Ensure SR#getSementInfo() returns snapshot
> --
>
> Key: LUCENE-8334
> URL: https://issues.apache.org/jira/browse/LUCENE-8334
> Project: Lucene - Core
>  Issue Type: Bug
>Reporter: Simon Willnauer
>Priority: Major
> Attachments: LUCENE-8334.patch
>
>
>  The SegmentCommitInfo passed to the segment reader is mutated concurrently.
> An instance obtained from SR#getSegmentInfo() might return wrong delete 
> counts
> or generation ids. This ensures that the SR will use a clone internally 
> while stil
> maintaining the original SI since it's needed inside IW for maintainance 
> like
> accessing pooled readers etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[jira] [Commented] (SOLR-10428) CloudSolrClient: Qerying multiple collection aliases leads to SolrException: Collection not found

[Philip Pock (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-10428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492337#comment-16492337
 ] 

Philip Pock commented on SOLR-10428:


Sorry for the late reply. The only mention I could find about providing 
multiple collections in the parameter is from 
[https://wiki.apache.org/solr/SolrCloud#Distributed_Requests]
{quote}Query all shards of multiple compatible collections, explicitly 
specified:

http://localhost:8983/solr/collection1/select?collection=collection1_NY,collection1_NJ,collection1_CT
{quote}
I also think that "collection" should be changed to "collectionName".

> CloudSolrClient: Qerying multiple collection aliases leads to SolrException: 
> Collection not found
> -
>
> Key: SOLR-10428
> URL: https://issues.apache.org/jira/browse/SOLR-10428
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: SolrJ
>Affects Versions: 6.4, 6.4.1, 6.4.2, 6.5, 7.0
>Reporter: Philip Pock
>Priority: Minor
>
> We have multiple collections and an alias is created for each of them. e.g.:
> alias-a -> collection-a, alias-b -> collection-b
> We search in multiple collections by passing the aliases of the collections 
> in the collections parameter.
> {code}solrClient.query("alias-a,alias-b", params, 
> SolrRequest.METHOD.POST){code}
> The client can't find the collection and throws an Exception. Relevant parts 
> of the stacktrace using v6.5.0:
> {noformat}
> org.apache.solr.common.SolrException: Collection not found: collection-a
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.getCollectionNames(CloudSolrClient.java:1394)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1087)
>   at 
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1057)
>   at 
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
>   at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
> {noformat}
> Everything works fine with a single alias.
> I think this issue was introduced with SOLR-9784. Please see my comment below.
> {code:title=org.apache.solr.client.solrj.impl.CloudSolrClient }
> Set getCollectionNames(String collection) {
> List rawCollectionsList = StrUtils.splitSmart(collection, ",", 
> true);
> Set collectionNames = new HashSet<>();
> for (String collectionName : rawCollectionsList) {
>   if (stateProvider.getState(collectionName) == null) {
> // I assume that collectionName should be passed to getAlias here
> String alias = stateProvider.getAlias(collection);
> if (alias != null) {
>   List aliasList = StrUtils.splitSmart(alias, ",", true);
>   collectionNames.addAll(aliasList);
>   continue;
> }
>   throw new SolrException(ErrorCode.BAD_REQUEST, "Collection not 
> found: " + collectionName);
> }
>   collectionNames.add(collectionName);
> }
> return collectionNames;
>   }
> {code}
> The suggested change is similar to the previous revision: 
> https://github.com/apache/lucene-solr/commit/5650939a8d41b7bad584947a2c9dcedf3774b8de#diff-c8d54eacd46180b332c86c7ae448abaeL1301



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_172) - Build # 22120 - Unstable!

[Policeman Jenkins Server]

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/22120/
Java: 64bit/jdk1.8.0_172 -XX:+UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.update.MaxSizeAutoCommitTest.deleteTest

Error Message:
Tlog size exceeds the max size bound. Tlog path: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.update.MaxSizeAutoCommitTest_DEA0D2D1D506D059-001/init-core-data-001/tlog/tlog.005,
 tlog size: 1265

Stack Trace:
java.lang.AssertionError: Tlog size exceeds the max size bound. Tlog path: 
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.update.MaxSizeAutoCommitTest_DEA0D2D1D506D059-001/init-core-data-001/tlog/tlog.005,
 tlog size: 1265
at 
__randomizedtesting.SeedInfo.seed([DEA0D2D1D506D059:CEEE372EAEA8E9A8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at 
org.apache.solr.update.MaxSizeAutoCommitTest.getTlogFileSizes(MaxSizeAutoCommitTest.java:379)
at 
org.apache.solr.update.MaxSizeAutoCommitTest.deleteTest(MaxSizeAutoCommitTest.java:200)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at 
com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at 
com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at 
org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at 
org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at 
org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at 
org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at 
org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at 

[jira] [Commented] (SOLR-12358) Autoscaling suggestions fail randomly and for certain policies

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492318#comment-16492318
 ] 

ASF subversion and git services commented on SOLR-12358:


Commit a875300a897521bc618d5072b20fcd60c8f13985 in lucene-solr's branch 
refs/heads/branch_7x from [~noble.paul]
[ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a875300 ]

SOLR-12358: Autoscaling suggestions fail randomly with sorting


> Autoscaling suggestions fail randomly and for certain policies
> --
>
> Key: SOLR-12358
> URL: https://issues.apache.org/jira/browse/SOLR-12358
> Project: Solr
>  Issue Type: Bug
>  Security Level: Public(Default Security Level. Issues are Public) 
>  Components: AutoScaling
>Affects Versions: 7.3.1
>Reporter: Jerry Bao
>Assignee: Noble Paul
>Priority: Critical
> Attachments: SOLR-12358.patch, SOLR-12358.patch, SOLR-12358.patch, 
> SOLR-12358.patch, diagnostics, nodes
>
>
> For the following policy
> {code:java}
> {"cores": "<4","node": "#ANY"}{code}
> the suggestions endpoint fails
> {code:java}
> "error": {"msg": "Comparison method violates its general contract!","trace": 
> "java.lang.IllegalArgumentException: Comparison method violates its general 
> contract!\n\tat java.util.TimSort.mergeHi(TimSort.java:899)\n\tat 
> java.util.TimSort.mergeAt(TimSort.java:516)\n\tat 
> java.util.TimSort.mergeCollapse(TimSort.java:441)\n\tat 
> java.util.TimSort.sort(TimSort.java:245)\n\tat 
> java.util.Arrays.sort(Arrays.java:1512)\n\tat 
> java.util.ArrayList.sort(ArrayList.java:1462)\n\tat 
> java.util.Collections.sort(Collections.java:175)\n\tat 
> org.apache.solr.client.solrj.cloud.autoscaling.Policy.setApproxValuesAndSortNodes(Policy.java:363)\n\tat
>  
> org.apache.solr.client.solrj.cloud.autoscaling.Policy$Session.applyRules(Policy.java:310)\n\tat
>  
> org.apache.solr.client.solrj.cloud.autoscaling.Policy$Session.(Policy.java:272)\n\tat
>  
> org.apache.solr.client.solrj.cloud.autoscaling.Policy.createSession(Policy.java:376)\n\tat
>  
> org.apache.solr.client.solrj.cloud.autoscaling.PolicyHelper.getSuggestions(PolicyHelper.java:214)\n\tat
>  
> org.apache.solr.cloud.autoscaling.AutoScalingHandler.handleSuggestions(AutoScalingHandler.java:158)\n\tat
>  
> org.apache.solr.cloud.autoscaling.AutoScalingHandler.handleRequestBody(AutoScalingHandler.java:133)\n\tat
>  
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)\n\tat
>  org.apache.solr.api.ApiBag$ReqHandlerToApi.call(ApiBag.java:242)\n\tat 
> org.apache.solr.api.V2HttpCall.handleAdmin(V2HttpCall.java:311)\n\tat 
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)\n\tat
>  org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat 
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)\n\tat
>  
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
>  
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)\n\tat
>  
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)\n\tat
>  
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
>  
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)\n\tat
>  
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
>  
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
>  
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>  
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
>  
> 

[jira] [Commented] (SOLR-9168) Add availability to specify own oom handing script

[Shawn Heisey (JIRA)]

[ 
https://issues.apache.org/jira/browse/SOLR-9168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16492310#comment-16492310
 ] 

Shawn Heisey commented on SOLR-9168:


I thought I had made this comment already, but it's not here, so I'll try again.

Watching the exit code of the following command will determine whether the flag 
is supported without any need to parse a version string (POSIX version shown)

{noformat}
$JAVA -Xmx1M -XX+ExitOnOutOfMemoryError -version > /dev/null 2> /dev/null
{noformat}

If the flag is supported, it can be combined with OnOutOfMemoryError calling an 
oom logging script.  If it's not supported, then we just use the existing oom 
killer script.

I'm not sure whether the -Xmx parameter is required with -version to keep 
memory usage down.


> Add availability to specify own oom handing script
> --
>
> Key: SOLR-9168
> URL: https://issues.apache.org/jira/browse/SOLR-9168
> Project: Solr
>  Issue Type: Improvement
>  Components: scripts and tools
>Affects Versions: 5.5.1
>Reporter: AngryDeveloper
>Priority: Major
>  Labels: oom
> Fix For: 5.5.1
>
> Attachments: 
> 0001-SOLR-9168-Allow-users-to-specify-their-own-OnOutOfMe.patch, 
> SOLR-9168-userdefined.patch, SOLR-9168.patch
>
>
> Right now the start script always uses $SOLR_TIP/bin/oom_solr.sh  to handle 
> OutOfMemoryException. This script only kills instance of solr.
> We need to do some additional things (e.g sent mail about this exception)
> What do you think about adding possibility to set up own script?
> Proposition:
> {code}
> if [ -z "$SOLR_OOM_SCRIPT" ]; then
>   SOLR_OOM_SCRIPT=$SOLR_TIP/bin/oom_solr.sh 
> fi
> [...]
> nohup "$JAVA" "${SOLR_START_OPTS[@]}" $SOLR_ADDL_ARGS \
>   "-XX:OnOutOfMemoryError=$SOLR_OOM_SCRIPT $SOLR_PORT $SOLR_LOGS_DIR" \
> -jar start.jar "${SOLR_JETTY_CONFIG[@]}" \
>   1>"$SOLR_LOGS_DIR/solr-$SOLR_PORT-console.log" 2>&1 & echo $! > 
> "$SOLR_PID_DIR/solr-$SOLR_PORT.pid"
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org