[JENKINS] Lucene-Solr-repro - Build # 3107 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-repro/3107/ [...truncated 29 lines...] [repro] Jenkins log URL: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/61/consoleText [repro] Revision: a248bc209ef77050dc23363eb00cb3da86387322 [repro] Ant options: -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt [repro] Repro line: ant test -Dtestcase=ChaosMonkeySafeLeaderWithPullReplicasTest -Dtests.seed=20A42861E500220B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=el-GR -Dtests.timezone=America/Rosario -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=SaslZkACLProviderTest -Dtests.method=testSaslZkACLProvider -Dtests.seed=20A42861E500220B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=de-DE -Dtests.timezone=America/Mazatlan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=SaslZkACLProviderTest -Dtests.seed=20A42861E500220B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=de-DE -Dtests.timezone=America/Mazatlan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=ShardSplitTest -Dtests.method=testSplitWithChaosMonkey -Dtests.seed=20A42861E500220B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-ME -Dtests.timezone=Asia/Taipei -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] Repro line: ant test -Dtestcase=ShardSplitTest -Dtests.seed=20A42861E500220B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-ME -Dtests.timezone=Asia/Taipei -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [repro] git rev-parse --abbrev-ref HEAD [repro] git rev-parse HEAD [repro] Initial local git branch/revision: 3fe4d0a30aab2fbe979c5d42014ac7d78260d0a4 [repro] git fetch [repro] git checkout a248bc209ef77050dc23363eb00cb3da86387322 [...truncated 2 lines...] [repro] git merge --ff-only [...truncated 1 lines...] [repro] ant clean [...truncated 6 lines...] [repro] Test suites by module: [repro]solr/core [repro] SaslZkACLProviderTest [repro] ChaosMonkeySafeLeaderWithPullReplicasTest [repro] ShardSplitTest [repro] ant compile-test [...truncated 3576 lines...] [repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=15 -Dtests.class="*.SaslZkACLProviderTest|*.ChaosMonkeySafeLeaderWithPullReplicasTest|*.ShardSplitTest" -Dtests.showOutput=onerror -Dtests.multiplier=2 -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.seed=20A42861E500220B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=de-DE -Dtests.timezone=America/Mazatlan -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [...truncated 233640 lines...] [repro] Setting last failure code to 256 [repro] Failures: [repro] 0/5 failed: org.apache.solr.cloud.api.collections.ShardSplitTest [repro] 3/5 failed: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest [repro] 3/5 failed: org.apache.solr.cloud.SaslZkACLProviderTest [repro] git checkout 3fe4d0a30aab2fbe979c5d42014ac7d78260d0a4 [...truncated 2 lines...] [repro] Exiting with code 256 [...truncated 6 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-master - Build # 3244 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/3244/ 3 tests failed. FAILED: org.apache.solr.cloud.HttpPartitionTest.test Error Message: Timeout occurred while waiting response from server at: http://127.0.0.1:40358 Stack Trace: org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:40358 at __randomizedtesting.SeedInfo.seed([347A62CA017B905C:BC2E5D10AF87FDA4]:0) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:660) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368) at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1055) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:830) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:763) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:224) at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:338) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1080) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at
[jira] [Resolved] (SOLR-13149) Implement a basic CategoryRoutedAlias class
[ https://issues.apache.org/jira/browse/SOLR-13149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gus Heck resolved SOLR-13149. - Resolution: Fixed > Implement a basic CategoryRoutedAlias class > --- > > Key: SOLR-13149 > URL: https://issues.apache.org/jira/browse/SOLR-13149 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) > Components: UpdateRequestProcessors >Reporter: Gus Heck >Priority: Major > > This ticket will add the core functionality for data driven routing to > collections in an alias based on the value of a particular field by fleshing > out the methods required by the RoutedAlias interface. This ticket will also > look for any synergies with the existing TimeRoutedAlias class and reuse code > if possible. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-13150) Enforce a max cardinality on category routed aliases
[ https://issues.apache.org/jira/browse/SOLR-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gus Heck resolved SOLR-13150. - Resolution: Fixed > Enforce a max cardinality on category routed aliases > > > Key: SOLR-13150 > URL: https://issues.apache.org/jira/browse/SOLR-13150 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) > Components: UpdateRequestProcessors >Affects Versions: master (9.0) >Reporter: Gus Heck >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > This ticket will add the check to enforce a (configurable) maximum number of > collections in a category routed alias. The purpose of this check is to > provide a safety valve to ensure that unexpected data values don't cause > run-away collection creation. This check should happen within the > validateRouteValue() method specified by RoutedAlias -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-13148) Move time based logic into TimeRoutedAlias class
[ https://issues.apache.org/jira/browse/SOLR-13148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gus Heck resolved SOLR-13148. - Resolution: Fixed > Move time based logic into TimeRoutedAlias class > > > Key: SOLR-13148 > URL: https://issues.apache.org/jira/browse/SOLR-13148 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) > Components: UpdateRequestProcessors >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Attachments: SOLR-13148.patch, SOLR-13148.patch, SOLR-13148.patch > > Time Spent: 50m > Remaining Estimate: 0h > > To pave the way for new types of routed aliases we need to get any and all > time related logic out of the URP and into TimeRoutedAlias. This ticket will > do that, Rename the URP and extract an initial proposed Generic RoutedAlias > interface implemented by both TimeRoutedAlias and a skeleton place holder for > CategoryRoutedAlias -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13370) Investigate Memory Footprint of CategorRoutedAliasUpdateProcessorTest
Gus Heck created SOLR-13370: --- Summary: Investigate Memory Footprint of CategorRoutedAliasUpdateProcessorTest Key: SOLR-13370 URL: https://issues.apache.org/jira/browse/SOLR-13370 Project: Solr Issue Type: Sub-task Security Level: Public (Default Security Level. Issues are Public) Components: Tests Affects Versions: master (9.0) Reporter: Gus Heck The test is failing too frequently, usually with OOM on the build servers. This sub task will track changes/investigation/discussion to improve that situation. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-13151) Provide regex based category validation for CategoryRoutedAliases
[ https://issues.apache.org/jira/browse/SOLR-13151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gus Heck resolved SOLR-13151. - Resolution: Fixed > Provide regex based category validation for CategoryRoutedAliases > - > > Key: SOLR-13151 > URL: https://issues.apache.org/jira/browse/SOLR-13151 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) > Components: UpdateRequestProcessors >Affects Versions: master (9.0) >Reporter: Gus Heck >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > This ticket will add the check to enforce a (configurable, optional) regular > expression to enreject requests that attempt to add an unexpected category, > or a malformed category to a category routed alias. The purpose of this check > is to provide a safety valve to ensure that unexpected data values don't > cause creation of collections inconsistent with the user's intended design. > This check should happen within the validateRouteValue() method specified by > RoutedAlias -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-13152) Update CREATEALIAS to support new parameters for category routed aliases
[ https://issues.apache.org/jira/browse/SOLR-13152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gus Heck resolved SOLR-13152. - Resolution: Fixed > Update CREATEALIAS to support new parameters for category routed aliases > > > Key: SOLR-13152 > URL: https://issues.apache.org/jira/browse/SOLR-13152 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Gus Heck >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Both V1 and V2 versions of CREATEALIAS api commands need to: > # Accept 'category' as a value for router.name > # Accept *router.maxCardinality* to place a limit on the total number of > collections that can be created (maybe required?) - must be an integer > # Accept *router.mustMatch* to provide pattern matching for valid data and > reject requests that would create an undesired collection (optional) - must > be a valid regular expression > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809400#comment-16809400 ] Gus Heck commented on SOLR-13131: - In opening 13 tar.gz files, I've found 8 of them have out of memory errors. I may have missed a prior OOM in the early files having seen later exceptions first when scrolling back up. At least 50% of the problem is tests running out of memory. So this might mean there's some need to work on skinnying up the test, but I'm not worried about the functionality from anything I've seen. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13220) Add connect Streaming Expression to create cached JDBC connections
[
https://issues.apache.org/jira/browse/SOLR-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809393#comment-16809393
]
Joel Bernstein edited comment on SOLR-13220 at 4/4/19 12:22 AM:
The security implications of the connect was too complex. Having the jdbc
stream lazily cache connections ensures that each jdbc call passes in the
credentials, rather then grabs an insecure handle to a connection.
The next release of the zeppelin-solr interpreter will allow you to configure a
default JDBC connection and will add the connection info to the outgoing jdbc
expression so users don't need to know or send a connection each time.
This gives us the best scenario I think:
1) Secure connections
2) Cached connections
3) A client can choose to add the connection info to the jdbc expression so
users don't have to know or deal with the connection info on each call.
was (Author: joel.bernstein):
The security implications of the connect was too complex. Having the jdbc
stream lazily cache connections ensures that each jdbc call passes in the
credentials, rather then grabs an insecure handle to a connection.
The next release of the zeppelin-solr interpreter will allow you to configure a
default JDBC connection and will add the connection info to the outgoing jdbc
expression so users don't need to know or send a connection each time.
This give us the best scenario I think:
1) Secure connections
2) Cached connections
3) A client can choose to add the connection info to the jdbc expression so
users don't have to know or deal with the connection info on each call.
> Add connect Streaming Expression to create cached JDBC connections
> --
>
> Key: SOLR-13220
> URL: https://issues.apache.org/jira/browse/SOLR-13220
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
>
> Currently the *jdbc* Streaming Expression requires that username, password
> and URL parameters are set to operate. This ticket will allow users to
> configure JDBC connections using the new *connect* Streaming Expression.
> The *jdbc* Streaming Expression will be changed to use the cached connections
> rather than accepting parameters directly.
> The *parallel* Streaming Expression can be used to setup connections on each
> node of a collection.
> The *Apache Hive* JDBC driver will also be included with Apache Solr as part
> of this ticket to allow seamless out-of-the-box connection to Apache Hive.
> Syntax:
> {code:java}
> connect(connectionName, url="", username="", password=""){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13220) Add connect Streaming Expression to create cached JDBC connections
[
https://issues.apache.org/jira/browse/SOLR-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809393#comment-16809393
]
Joel Bernstein commented on SOLR-13220:
---
The security implications of the connect was too complex. Having the jdbc
stream lazily cache connections ensures that each jdbc call passes in the
credentials, rather then grabs an insecure handle to a connection.
The next release of the zeppelin-solr interpreter will allow you to configure a
default JDBC connection and will add the connection info to the outgoing jdbc
expression so users don't need to know or send a connection each time.
This give us the best scenario I think:
1) Secure connections
2) Cached connections
3) A client can choose to add the connection info to the jdbc expression so
users don't have to know or deal with the connection info on each call.
> Add connect Streaming Expression to create cached JDBC connections
> --
>
> Key: SOLR-13220
> URL: https://issues.apache.org/jira/browse/SOLR-13220
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
>Reporter: Joel Bernstein
>Assignee: Joel Bernstein
>Priority: Major
>
> Currently the *jdbc* Streaming Expression requires that username, password
> and URL parameters are set to operate. This ticket will allow users to
> configure JDBC connections using the new *connect* Streaming Expression.
> The *jdbc* Streaming Expression will be changed to use the cached connections
> rather than accepting parameters directly.
> The *parallel* Streaming Expression can be used to setup connections on each
> node of a collection.
> The *Apache Hive* JDBC driver will also be included with Apache Solr as part
> of this ticket to allow seamless out-of-the-box connection to Apache Hive.
> Syntax:
> {code:java}
> connect(connectionName, url="", username="", password=""){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13369) TriLevelCompositeIdRoutingTest failure: same route prefix maped to multiple shards
Hoss Man created SOLR-13369:
---
Summary: TriLevelCompositeIdRoutingTest failure: same route prefix
maped to multiple shards
Key: SOLR-13369
URL: https://issues.apache.org/jira/browse/SOLR-13369
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Reporter: Hoss Man
thetaphi's 8x jenkins job just identified a reproducing seed that causes
TriLevelCompositeIdRoutingTest to fail after detecting 2 docs with matching
route prefixes on different shards...
{noformat}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TriLevelCompositeIdRoutingTest -Dtests.method=test
-Dtests.seed=A6B6F0104FE6018F -Dtests.multiplier=3 -Dtests.slow=true
-Dtests.locale=sr-Latn -Dtests.timezone=Pacific/Tongatapu -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 9.38s J0 | TriLevelCompositeIdRoutingTest.test <<<
[junit4]> Throwable #1: org.junit.ComparisonFailure: routePrefix
app9/2!user32 found in multiple shards expected: but was:
[junit4]>at
__randomizedtesting.SeedInfo.seed([A6B6F0104FE6018F:2EE2CFCAE11A6C77]:0)
[junit4]>at
org.apache.solr.cloud.TriLevelCompositeIdRoutingTest.test(TriLevelCompositeIdRoutingTest.java:122)
{noformat}
It's possible this is just a bug I introduced in SOLR-13210 due to a
missunderstanding in how routePrefixes that use a bit mask (ie: {{/2}} in the
assertion failure) are expected to work -- but I thought i had that squared
away based on shalin's feedback in SOLR-13210
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809361#comment-16809361 ] Gus Heck commented on SOLR-13131: - The first half a dozen I've looked out included 2 zk session expirations, 1 OOM Heap, one OOM GC limit exceeded and 2 Conection timeouts... so far nothing that's an actual logic failure still looking. Also, not sure why this would be rolled back instead of BadApple or AwaitsFix. > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-8.x-Windows (64bit/jdk-11) - Build # 186 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Windows/186/ Java: 64bit/jdk-11 -XX:+UseCompressedOops -XX:+UseG1GC 1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.impl.CloudHttp2SolrClientTest Error Message: ObjectTracker found 2 object(s) that were not released!!! [SolrCore, MockDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.(SolrCore.java:1063) at org.apache.solr.core.SolrCore.(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1193) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1103) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:99) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:779) at org.apache.solr.core.SolrCore.(SolrCore.java:976) at org.apache.solr.core.SolrCore.(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1193) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1103) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) expected null, but was:(SolrCore.java:1063) at org.apache.solr.core.SolrCore.(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1193) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1103) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:99) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:779) at org.apache.solr.core.SolrCore.(SolrCore.java:976) at org.apache.solr.core.SolrCore.(SolrCore.java:883) at
[JENKINS] Lucene-Solr-SmokeRelease-master - Build # 1298 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-SmokeRelease-master/1298/ No tests ran. Build Log: [...truncated 23437 lines...] [asciidoctor:convert] asciidoctor: ERROR: about-this-guide.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [asciidoctor:convert] asciidoctor: WARNING: aliases.adoc: line 25: list item index: expected 2, got 1 [asciidoctor:convert] asciidoctor: WARNING: aliases.adoc: line 26: list item index: expected 3, got 1 [asciidoctor:convert] asciidoctor: WARNING: aliases.adoc: line 224: list item index: expected 2, got 1 [asciidoctor:convert] asciidoctor: WARNING: aliases.adoc: line 225: list item index: expected 3, got 1 [asciidoctor:convert] asciidoctor: ERROR: solr-glossary.adoc: line 1: invalid part, must have at least one section (e.g., chapter, appendix, etc.) [asciidoctor:convert] asciidoctor: WARNING: aliases.adoc: line 25: list item index: expected 2, got 1 [asciidoctor:convert] asciidoctor: WARNING: aliases.adoc: line 26: list item index: expected 3, got 1 [asciidoctor:convert] asciidoctor: WARNING: aliases.adoc: line 224: list item index: expected 2, got 1 [asciidoctor:convert] asciidoctor: WARNING: aliases.adoc: line 225: list item index: expected 3, got 1 [java] Processed 2517 links (2062 relative) to 3336 anchors in 252 files [echo] Validated Links & Anchors via: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr-ref-guide/bare-bones-html/ -dist-changes: [copy] Copying 4 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/changes package: -unpack-solr-tgz: -ensure-solr-tgz-exists: [mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked [untar] Expanding: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/package/solr-9.0.0.tgz into /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/solr/build/solr.tgz.unpacked generate-maven-artifacts: resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-SmokeRelease-master/lucene/top-level-ivy-settings.xml resolve: ivy-availability-check: [loadresource] Do not set property disallowed.ivy.jars.list as its length is 0. -ivy-fail-disallowed-ivy-version: ivy-fail: ivy-configure: [ivy:configure] :: loading settings :: file =
[JENKINS] Lucene-Solr-8.x-Linux (64bit/jdk-11) - Build # 344 - Failure!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/344/ Java: 64bit/jdk-11 -XX:-UseCompressedOops -XX:+UseParallelGC 1 tests failed. FAILED: org.apache.solr.update.processor.CategoryRoutedAliasUpdateProcessorTest.testSliceRouting Error Message: Timeout occurred while waiting response from server at: http://127.0.0.1:35869/solr/myAlias__CRA__Heart_of_Gold_shard2_replica_n7 Stack Trace: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Timeout occurred while waiting response from server at: http://127.0.0.1:35869/solr/myAlias__CRA__Heart_of_Gold_shard2_replica_n7 at org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:125) at org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:46) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.directUpdate(BaseCloudSolrClient.java:485) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:964) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:830) at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:763) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207) at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:177) at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138) at org.apache.solr.update.processor.CategoryRoutedAliasUpdateProcessorTest.testSliceRouting(CategoryRoutedAliasUpdateProcessorTest.java:366) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[jira] [Commented] (SOLR-12809) Document recommended Java/Solr combinations (JDK 11?)
[ https://issues.apache.org/jira/browse/SOLR-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809300#comment-16809300 ] Jan Høydahl commented on SOLR-12809: Erick, you've done a great job researching and documenting all this. Kudos. I don't want to criticise your work or delay the 8.0 ref-guide. Really. But for the purpose of the "System Requirements" chapter I personally think four full pages of content is too overwhelming and makes the task of deciding a Java version for Solr look more complex and scary than it needs be. Can we try to stay within 2 PDF pages, perhaps using a simple table like in the HBase docs, and defer all details to the Java Champions blog plus a new web page for Lucene/Solr specific testing details etc, for those interested. We don't need to scrap any content, just prune what goes into RefGuide and link elsewhere for details. Some concrete feedback to the current text: * The link text for the Java Champions blog post should probably be "Java is still free by Java Champions", rather than currently "Java Champions". * In the list of OpenJDK sources, Oracle should come before Zulu, to be alphabetical/neutral, and perhaps write "in alphabetical order". * The link to Oracle's OpenJDK points to Oracle's paid JDK. Correct link is [https://jdk.java.net|https://jdk.java.net/] * The info box has a typo "While we reference the Jave Development (JDK) on this page, any *Jave* Runtime Environment..." * The text "Lucene/Solr 9 will almost certainly require Java 11" could skip "almost certainly" as this is already decided by vote * "There is considerable confusion about..." - choose another wording without negative vibes. E.g. "A frequently asked question on the user-list is..." * "There will be no vendor support or bug fixes for Java 8 however" - this is not correct according to the Java Champions blog, which states that several vendors will support Java 8, e.g. "Amazon offers long-term support for Corretto on AWS and includes performance enhancements and security updates for Corretto 8 until at least June 2023 at no cost.". * Header "Java 9 and 10" could perhaps be changed to "We recommend Java 11", since that is the conclusion? * "List of known issues" - this has been mentioned and linked to in the first chapter so why repeat? * "Lucene and Solr testing" - there is a problem with the link [[https://openjdk.java.net|OpenJDK]] > Document recommended Java/Solr combinations (JDK 11?) > - > > Key: SOLR-12809 > URL: https://issues.apache.org/jira/browse/SOLR-12809 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12809.patch, SolrSystemRequirements.pdf > > > JDK 8 will be EOL early next year (except for "premier support"). JDK 9, 10 > and 11 all have issues for Solr and Lucene IIUC. > Also IIUC Oracle will start requiring commercial licenses for 11. > This Jira is to discuss what we want to do going forward. Among the topics: > * Skip straight to 11, skipping 9 and 10? If so how to resolve current > issues? > * How much emphasis on OpenJDK .vs. Oracle's version > * What to do about dependencies that don't work (for whatever reason) with > the version of Java we go with? > * ??? > This may turn into an umbrella Jira with sub-tasks of course. Since JDK 11 > has had a GA release, I'd also like to have a record of where the current > issues are to refer people to. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12809) Document recommended Java/Solr combinations (JDK 11?)
[ https://issues.apache.org/jira/browse/SOLR-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809258#comment-16809258 ] Jan Høydahl commented on SOLR-12809: I built the guide PDF and attached the four pages to this issue: [^SolrSystemRequirements.pdf] for easier review. > Document recommended Java/Solr combinations (JDK 11?) > - > > Key: SOLR-12809 > URL: https://issues.apache.org/jira/browse/SOLR-12809 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12809.patch, SolrSystemRequirements.pdf > > > JDK 8 will be EOL early next year (except for "premier support"). JDK 9, 10 > and 11 all have issues for Solr and Lucene IIUC. > Also IIUC Oracle will start requiring commercial licenses for 11. > This Jira is to discuss what we want to do going forward. Among the topics: > * Skip straight to 11, skipping 9 and 10? If so how to resolve current > issues? > * How much emphasis on OpenJDK .vs. Oracle's version > * What to do about dependencies that don't work (for whatever reason) with > the version of Java we go with? > * ??? > This may turn into an umbrella Jira with sub-tasks of course. Since JDK 11 > has had a GA release, I'd also like to have a record of where the current > issues are to refer people to. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[
https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809251#comment-16809251
]
Bruno Roustant commented on LUCENE-8753:
{quote}I think this is similar to […] BlockTermsReader/Writer
{quote}
Indeed similar; it mainly differs from VariableGapTermsIndexWriter in the way
it selects the best term to start a block. It is based on the minimal
distinguishing prefix. The idea is to make the terms index FST more compact.
That way, given a target max heap memory, we can have potentially more blocks,
so smaller ones that are scanned faster. This requirement to consume less heap
was strong with lucene 7.1, now maybe less with the recent off-heap FST.
{quote}Are you also doing something different to encode/decode postings?
{quote}
No, the postings are written with the regular PostingsWriterBase.
{quote}Can you post results on the full wikimediumall?
{quote}
Good point. Will do tomorrow.
> New PostingFormat - UniformSplit
>
>
> Key: LUCENE-8753
> URL: https://issues.apache.org/jira/browse/LUCENE-8753
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
>Affects Versions: 8.0
>Reporter: Bruno Roustant
>Priority: Major
> Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a proposal to add a new PostingsFormat called "UniformSplit" with 4
> objectives:
> - Clear design and simple code.
> - Easily extensible, for both the logic and the index format.
> - Light memory usage with a very compact FST.
> - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance.
> (the pdf attached explains visually the technique in more details)
> The principle is to split the list of terms into blocks and use a FST to
> access the block, but not as a prefix trie, rather with a seek-floor pattern.
> For the selection of the blocks, there is a target average block size (number
> of terms), with an allowed delta variation (10%) to compare the terms and
> select the one with the minimal distinguishing prefix.
> There are also several optimizations inside the block to make it more
> compact and speed up the loading/scanning.
> The performance obtained is interesting with the luceneutil benchmark,
> comparing UniformSplit with BlockTree. Find it in the first comment and also
> attached for better formatting.
> Although the precise percentages vary between runs, three main points:
> - TermQuery and PhraseQuery are improved.
> - PrefixQuery and WildcardQuery are ok.
> - Fuzzy queries are clearly less performant, because BlockTree is so
> optimized for them.
> Compared to BlockTree, FST size is reduced by 15%, and segment writing time
> is reduced by 20%. So this PostingsFormat scales to lots of docs, as
> BlockTree.
> This initial version passes all Lucene tests. Use “ant test
> -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat.
> Subjectively, we think we have fulfilled our goal of code simplicity. And we
> have already exercised this PostingsFormat extensibility to create a
> different flavor for our own use-case.
> Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12809) Document recommended Java/Solr combinations (JDK 11?)
[ https://issues.apache.org/jira/browse/SOLR-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jan Høydahl updated SOLR-12809: --- Attachment: SolrSystemRequirements.pdf > Document recommended Java/Solr combinations (JDK 11?) > - > > Key: SOLR-12809 > URL: https://issues.apache.org/jira/browse/SOLR-12809 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12809.patch, SolrSystemRequirements.pdf > > > JDK 8 will be EOL early next year (except for "premier support"). JDK 9, 10 > and 11 all have issues for Solr and Lucene IIUC. > Also IIUC Oracle will start requiring commercial licenses for 11. > This Jira is to discuss what we want to do going forward. Among the topics: > * Skip straight to 11, skipping 9 and 10? If so how to resolve current > issues? > * How much emphasis on OpenJDK .vs. Oracle's version > * What to do about dependencies that don't work (for whatever reason) with > the version of Java we go with? > * ??? > This may turn into an umbrella Jira with sub-tasks of course. Since JDK 11 > has had a GA release, I'd also like to have a record of where the current > issues are to refer people to. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13364) Make Admin UI aware of logged-in users permissions
[
https://issues.apache.org/jira/browse/SOLR-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Høydahl updated SOLR-13364:
---
Description:
We should aim to add fine-grained permission checks to the UI. One way to do
this is to add a new REST-endpoint {{/admin/login/whoami}} that is always open
for all, and that responds with a JSON with current user's permissions. If no
user is logged in it will respond with empty list and "No user logged in". Else
it will respond with e.g.
{code:java}
{
"user": "john",
"roles": ["superuser", "searcher"],
"permissions": [
{"name": "security-edit"},
{"name": "read"},
{"name": "customFoo", "path": "/custom/path", "params": {"key": ["val1",
"val2"]} }
]
}{code}
The Admin UI can then request this endpoint and cache the info, so that it may
make decisions to hide/grey out certain menu options throughout the UI. E.g.
the create collection button would be disabled if the user lacks the predefined
permission "collection-admin-edit".
In theory the UI must also check if the user has a custom permission with path
{{/admin/collections}} and params {{action=CREATE}}, but it is not likely that
anyone would create a custom permission for something that is predefined.
was:
We should aim to add fine-grained permission checks to the UI. One way to do
this is to add a new REST-endpoint {{/admin/login/whoami}} that is always open
for all, and that responds with a JSON with current user's permissions. If no
user is logged in it will respond with empty list and "No user logged in". Else
it will respond with e.g.
{code:java}
{ "user": "john", "roles": ["superuser", "searcher"], "permissions":
["security-edit", "collectionadmin"...] }{code}
The Admin UI can then request this endpoint and cache the info, so that it may
make decisions to hide/grey out certain menu options throughout the UI.
> Make Admin UI aware of logged-in users permissions
> --
>
> Key: SOLR-13364
> URL: https://issues.apache.org/jira/browse/SOLR-13364
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI, Authentication, Authorization, security
>Reporter: Jan Høydahl
>Priority: Major
>
> We should aim to add fine-grained permission checks to the UI. One way to do
> this is to add a new REST-endpoint {{/admin/login/whoami}} that is always
> open for all, and that responds with a JSON with current user's permissions.
> If no user is logged in it will respond with empty list and "No user logged
> in". Else it will respond with e.g.
> {code:java}
> {
> "user": "john",
> "roles": ["superuser", "searcher"],
> "permissions": [
> {"name": "security-edit"},
> {"name": "read"},
> {"name": "customFoo", "path": "/custom/path", "params": {"key": ["val1",
> "val2"]} }
> ]
> }{code}
> The Admin UI can then request this endpoint and cache the info, so that it
> may make decisions to hide/grey out certain menu options throughout the UI.
> E.g. the create collection button would be disabled if the user lacks the
> predefined permission "collection-admin-edit".
> In theory the UI must also check if the user has a custom permission with
> path {{/admin/collections}} and params {{action=CREATE}}, but it is not
> likely that anyone would create a custom permission for something that is
> predefined.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[
https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809218#comment-16809218
]
Michael McCandless commented on LUCENE-8753:
I think this is similar to the terms dictionary format Lucene used to have
before {{BlockTree}}, still in Lucene's sources as {{BlockTermsReader/Writer}}.
Terms are assigned to fixed sized blocks and only the minimum unique prefix
needs to be enrolled in the terms index FST. But being able to do binary
search within a block is unique! That's very cool.
It's curious you see gains e.g. for {{AndHighLow}} – are you also doing
something different to encode/decode postings (not just terms dictionary)?
The 500K docs is a little small – can you post results on the full
{{wikimediumall}} set of documents?
> New PostingFormat - UniformSplit
>
>
> Key: LUCENE-8753
> URL: https://issues.apache.org/jira/browse/LUCENE-8753
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
>Affects Versions: 8.0
>Reporter: Bruno Roustant
>Priority: Major
> Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a proposal to add a new PostingsFormat called "UniformSplit" with 4
> objectives:
> - Clear design and simple code.
> - Easily extensible, for both the logic and the index format.
> - Light memory usage with a very compact FST.
> - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance.
> (the pdf attached explains visually the technique in more details)
> The principle is to split the list of terms into blocks and use a FST to
> access the block, but not as a prefix trie, rather with a seek-floor pattern.
> For the selection of the blocks, there is a target average block size (number
> of terms), with an allowed delta variation (10%) to compare the terms and
> select the one with the minimal distinguishing prefix.
> There are also several optimizations inside the block to make it more
> compact and speed up the loading/scanning.
> The performance obtained is interesting with the luceneutil benchmark,
> comparing UniformSplit with BlockTree. Find it in the first comment and also
> attached for better formatting.
> Although the precise percentages vary between runs, three main points:
> - TermQuery and PhraseQuery are improved.
> - PrefixQuery and WildcardQuery are ok.
> - Fuzzy queries are clearly less performant, because BlockTree is so
> optimized for them.
> Compared to BlockTree, FST size is reduced by 15%, and segment writing time
> is reduced by 20%. So this PostingsFormat scales to lots of docs, as
> BlockTree.
> This initial version passes all Lucene tests. Use “ant test
> -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat.
> Subjectively, we think we have fulfilled our goal of code simplicity. And we
> have already exercised this PostingsFormat extensibility to create a
> different flavor for our own use-case.
> Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-12) - Build # 23863 - Failure!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23863/ Java: 64bit/jdk-12 -XX:-UseCompressedOops -XX:+UseParallelGC All tests passed Build Log: [...truncated 15592 lines...] [junit4] JVM J0: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J0-20190403_192139_2312352148472700126707.sysout [junit4] >>> JVM J0 emitted unexpected output (verbatim) [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded [junit4] Dumping heap to /home/jenkins/workspace/Lucene-Solr-master-Linux/heapdumps/java_pid7798.hprof ... [junit4] Heap dump file created [486081724 bytes in 4.088 secs] [junit4] <<< JVM J0: EOF [...truncated 8964 lines...] BUILD FAILED /home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:633: The following error occurred while executing this line: /home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created: * java_pid7798.hprof Total time: 107 minutes 14 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2 [WARNINGS] Skipping publisher since build result is FAILURE Recording test results Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2 Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2 Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2 Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2 Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2 Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2 Setting ANT_1_8_2_HOME=/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2 - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[
https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809188#comment-16809188
]
Michael McCandless commented on LUCENE-8753:
{quote}I think PKLookup should be disregarded until it's fixed:
[https://github.com/mikemccand/luceneutil/issues/35] (feel free to comment
there if people have opinions)
{quote}
Note that the title on that issue was misleading (backwards from the reality) –
I just corrected it.
I don't think we should disregard {{PKLookup}} results: it's reporting the
performance when looking up actual IDs that do exist in the index. That is an
interesting result, but it is odd you are seeing varying/inconsistent results.
Note that if you add {{-jira}} into the luceneutil benchmark command-line it
will print results using the markup that Jira displays as a table, making it
easier for everyone to read.
> New PostingFormat - UniformSplit
>
>
> Key: LUCENE-8753
> URL: https://issues.apache.org/jira/browse/LUCENE-8753
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
>Affects Versions: 8.0
>Reporter: Bruno Roustant
>Priority: Major
> Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a proposal to add a new PostingsFormat called "UniformSplit" with 4
> objectives:
> - Clear design and simple code.
> - Easily extensible, for both the logic and the index format.
> - Light memory usage with a very compact FST.
> - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance.
> (the pdf attached explains visually the technique in more details)
> The principle is to split the list of terms into blocks and use a FST to
> access the block, but not as a prefix trie, rather with a seek-floor pattern.
> For the selection of the blocks, there is a target average block size (number
> of terms), with an allowed delta variation (10%) to compare the terms and
> select the one with the minimal distinguishing prefix.
> There are also several optimizations inside the block to make it more
> compact and speed up the loading/scanning.
> The performance obtained is interesting with the luceneutil benchmark,
> comparing UniformSplit with BlockTree. Find it in the first comment and also
> attached for better formatting.
> Although the precise percentages vary between runs, three main points:
> - TermQuery and PhraseQuery are improved.
> - PrefixQuery and WildcardQuery are ok.
> - Fuzzy queries are clearly less performant, because BlockTree is so
> optimized for them.
> Compared to BlockTree, FST size is reduced by 15%, and segment writing time
> is reduced by 20%. So this PostingsFormat scales to lots of docs, as
> BlockTree.
> This initial version passes all Lucene tests. Use “ant test
> -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat.
> Subjectively, we think we have fulfilled our goal of code simplicity. And we
> have already exercised this PostingsFormat extensibility to create a
> different flavor for our own use-case.
> Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13331) Atomic Update Multivalue remove does not work
[
https://issues.apache.org/jira/browse/SOLR-13331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809155#comment-16809155
]
Lucene/Solr QA commented on SOLR-13331:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Release audit (RAT) {color} |
{color:green} 1m 27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Check forbidden APIs {color} |
{color:green} 1m 27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} Validate source patterns {color} |
{color:green} 1m 27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 43m 33s{color}
| {color:red} core in the patch failed. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 49m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | solr.update.processor.AtomicUpdateRemovalJavabinTest |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | SOLR-13331 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12964692/SOLR-13331.patch |
| Optional Tests | compile javac unit ratsources checkforbiddenapis
validatesourcepatterns |
| uname | Linux lucene1-us-west 4.4.0-137-generic #163~14.04.1-Ubuntu SMP Mon
Sep 24 17:14:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | ant |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-SOLR-Build/sourcedir/dev-tools/test-patch/lucene-solr-yetus-personality.sh
|
| git revision | master / 6596ed1 |
| ant | version: Apache Ant(TM) version 1.9.3 compiled on July 24 2018 |
| Default Java | 1.8.0_191 |
| unit |
https://builds.apache.org/job/PreCommit-SOLR-Build/365/artifact/out/patch-unit-solr_core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-SOLR-Build/365/testReport/ |
| modules | C: solr solr/core U: solr |
| Console output |
https://builds.apache.org/job/PreCommit-SOLR-Build/365/console |
| Powered by | Apache Yetus 0.7.0 http://yetus.apache.org |
This message was automatically generated.
> Atomic Update Multivalue remove does not work
> -
>
> Key: SOLR-13331
> URL: https://issues.apache.org/jira/browse/SOLR-13331
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: UpdateRequestProcessors
>Affects Versions: 7.7, 7.7.1, 8.0
> Environment: Standalone Solr Server
>Reporter: Thomas Wöckinger
>Assignee: Jason Gerlowski
>Priority: Critical
> Labels: patch
> Fix For: 8.0
>
> Attachments: Fix-SOLR13331-Add-toNativeType-implementations.patch,
> SOLR-13331.patch
>
>
> When using JavaBinCodec the values of collections are of type
> ByteArrayUtf8CharSequence, existing field values are Strings so the remove
> Operation does not have any effect.
> The relevant code is located in class AtomicUpdateDocumentMerger method
> doRemove.
> The method parameter fieldVal contains the collection values of type
> ByteArrayUtf8CharSequence, the variable original contains the collection of
> Strings
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809141#comment-16809141 ] Gus Heck commented on SOLR-13131: - Hmm, I didn't see this on the bad apples reports? I'll take a look. I'd been looking at the bad-apple reports and assuming that if it didn't show there, then there wasn't a problem > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12809) Document recommended Java/Solr combinations (JDK 11?)
[ https://issues.apache.org/jira/browse/SOLR-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809126#comment-16809126 ] Erick Erickson commented on SOLR-12809: --- OK, here's a patch with the changes. It's just one page in the ref guide "Solr System Requirements". BTW, I love the page title “Solr System Requirements”. Start with a large interstellar gas cloud seeded with the remnants of supernovas, stir and bake for a really long time…. If there's no objection, I'll commit this this week. I'll then take the Wiki page and just reference this page so when the Wiki goes away we won't care. > Document recommended Java/Solr combinations (JDK 11?) > - > > Key: SOLR-12809 > URL: https://issues.apache.org/jira/browse/SOLR-12809 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12809.patch > > > JDK 8 will be EOL early next year (except for "premier support"). JDK 9, 10 > and 11 all have issues for Solr and Lucene IIUC. > Also IIUC Oracle will start requiring commercial licenses for 11. > This Jira is to discuss what we want to do going forward. Among the topics: > * Skip straight to 11, skipping 9 and 10? If so how to resolve current > issues? > * How much emphasis on OpenJDK .vs. Oracle's version > * What to do about dependencies that don't work (for whatever reason) with > the version of Java we go with? > * ??? > This may turn into an umbrella Jira with sub-tasks of course. Since JDK 11 > has had a GA release, I'd also like to have a record of where the current > issues are to refer people to. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809124#comment-16809124 ] Mike Sokolov commented on LUCENE-8753: -- The behavior I'm referring to isn't a problem with the benchmark _per se_, it's an intrinsic feature of trying to optimize blocks of things that work better when you have a bunch of similar things right next to each other sharing a common prefix. The PKLookup test is very sensitive to such optimizations (or failures to optimize) because it indexes a consecutive block of terms: a00, a01, a02, etc. and at the same time, this nice consecutive property can be disturbed by index fragmentation, especially when only each term is indexed for only a single document. > New PostingFormat - UniformSplit > > > Key: LUCENE-8753 > URL: https://issues.apache.org/jira/browse/LUCENE-8753 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: 8.0 >Reporter: Bruno Roustant >Priority: Major > Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt > > Time Spent: 10m > Remaining Estimate: 0h > > This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 > objectives: > - Clear design and simple code. > - Easily extensible, for both the logic and the index format. > - Light memory usage with a very compact FST. > - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. > (the pdf attached explains visually the technique in more details) > The principle is to split the list of terms into blocks and use a FST to > access the block, but not as a prefix trie, rather with a seek-floor pattern. > For the selection of the blocks, there is a target average block size (number > of terms), with an allowed delta variation (10%) to compare the terms and > select the one with the minimal distinguishing prefix. > There are also several optimizations inside the block to make it more > compact and speed up the loading/scanning. > The performance obtained is interesting with the luceneutil benchmark, > comparing UniformSplit with BlockTree. Find it in the first comment and also > attached for better formatting. > Although the precise percentages vary between runs, three main points: > - TermQuery and PhraseQuery are improved. > - PrefixQuery and WildcardQuery are ok. > - Fuzzy queries are clearly less performant, because BlockTree is so > optimized for them. > Compared to BlockTree, FST size is reduced by 15%, and segment writing time > is reduced by 20%. So this PostingsFormat scales to lots of docs, as > BlockTree. > This initial version passes all Lucene tests. Use “ant test > -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. > Subjectively, we think we have fulfilled our goal of code simplicity. And we > have already exercised this PostingsFormat extensibility to create a > different flavor for our own use-case. > Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12809) Document recommended Java/Solr combinations (JDK 11?)
[ https://issues.apache.org/jira/browse/SOLR-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erick Erickson updated SOLR-12809: -- Attachment: SOLR-12809.patch > Document recommended Java/Solr combinations (JDK 11?) > - > > Key: SOLR-12809 > URL: https://issues.apache.org/jira/browse/SOLR-12809 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Erick Erickson >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-12809.patch > > > JDK 8 will be EOL early next year (except for "premier support"). JDK 9, 10 > and 11 all have issues for Solr and Lucene IIUC. > Also IIUC Oracle will start requiring commercial licenses for 11. > This Jira is to discuss what we want to do going forward. Among the topics: > * Skip straight to 11, skipping 9 and 10? If so how to resolve current > issues? > * How much emphasis on OpenJDK .vs. Oracle's version > * What to do about dependencies that don't work (for whatever reason) with > the version of Java we go with? > * ??? > This may turn into an umbrella Jira with sub-tasks of course. Since JDK 11 > has had a GA release, I'd also like to have a record of where the current > issues are to refer people to. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] dsmiley commented on issue #455: SOLR-12638
dsmiley commented on issue #455: SOLR-12638 URL: https://github.com/apache/lucene-solr/pull/455#issuecomment-479617469 For detecting which document got where, consider the `[shard]` doc transformer https://lucene.apache.org/solr/guide/7_7/transforming-result-documents.html > SOLR-12638: Atomic Updates for nested documents. This enables atomic updates for nested documents, without the need to supply the whole nested hierarchy (which would be overwritten if absent). This is done by fetching the whole document hierarchy, updating the specific doc in the path that is to be updated, removing the old document hierarchy and indexing the new one with the atomic update merged into it. Thanks for taking a stab at this. Sometimes I think the "atomic"-ness gets more visibility in features/naming than the "partial-update"-ness that goes along with it, which is IMO more in the minds of those that use these features. I could be wrong. Maybe instead of "Atomic Updates" we say "Atomic/Partial Updates"? You could also mention that `[child]` now works in RTG. OH; we're missing updates to the Ref Guide! See `updating-parts-of-documents.adoc`. You could also add a link & mention in the indexing side of the page on nested docs you did. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12833) Use timed-out lock in DistributedUpdateProcessor
[ https://issues.apache.org/jira/browse/SOLR-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16809012#comment-16809012 ] jefferyyuan commented on SOLR-12833: thanks for the findings, [~ab], [~markrmil...@gmail.com] as you said, we don't need versionBucketLockTimeoutMs in every VersionBucket, I can create one pr to remove it from VersionBucket. One approach to fix this problem: * if there is not a lot of competition on same version bucket and the update usually finishes fast, customers don't specify versionBucketLockTimeoutMs value, then we use the old VersionBucket which has no lock and Condition objects, and its lock signalAll, awaitNanos methods will keep the old ways, use the intrinsic. * if there is a lot of competition on same version bucket, and update(like geo related updates) takes time, customer can specify versionBucketLockTimeoutMs explicitly, then we can create and use another class TimedVersionBucket that extends VersionBucket, and uses lock and Condition, so only one update on same bucket will actually go forward and get processed, other updates will fail fast. > Use timed-out lock in DistributedUpdateProcessor > > > Key: SOLR-12833 > URL: https://issues.apache.org/jira/browse/SOLR-12833 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: update, UpdateRequestProcessors >Affects Versions: 7.5, 8.0 >Reporter: jefferyyuan >Assignee: Mark Miller >Priority: Minor > Fix For: 7.7, 8.0 > > Attachments: SOLR-12833-noint.patch, SOLR-12833.patch, > SOLR-12833.patch > > Time Spent: 20m > Remaining Estimate: 0h > > There is a synchronize block that blocks other update requests whose IDs fall > in the same hash bucket. The update waits forever until it gets the lock at > the synchronize block, this can be a problem in some cases. > > Some add/update requests (for example updates with spatial/shape analysis) > like may take time (30+ seconds or even more), this would the request time > out and fail. > Client may retry the same requests multiple times or several minutes, this > would make things worse. > The server side receives all the update requests but all except one can do > nothing, have to wait there. This wastes precious memory and cpu resource. > We have seen the case 2000+ threads are blocking at the synchronize lock, and > only a few updates are making progress. Each thread takes 3+ mb memory which > causes OOM. > Also if the update can't get the lock in expected time range, its better to > fail fast. > > We can have one configuration in solrconfig.xml: > updateHandler/versionLock/timeInMill, so users can specify how long they want > to wait the version bucket lock. > The default value can be -1, so it behaves same - wait forever until it gets > the lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-13131) Category Routed Aliases
[ https://issues.apache.org/jira/browse/SOLR-13131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man reopened SOLR-13131: - Gus: The new CategoryRoutedAliasUpdateProcessorTest added in this jira is currently one of the most commonly failing test in our jenkins builds... http://fucit.org/solr-jenkins-reports/failure-report.html ||Suite?||Class||Method||Rate||Runs||Fails|| |false|CategoryRoutedAliasUpdateProcessorTest|testSliceRouting|4.26439232409382|469|20| |true|CategoryRoutedAliasUpdateProcessorTest|-|3.34928229665072|418|14| |false|CategoryRoutedAliasUpdateProcessorTest|test|0.639658848614072|469|3| |false|CategoryRoutedAliasUpdateProcessorTest|testMaxCardinality|0.427350427350427|468|2| |false|CategoryRoutedAliasUpdateProcessorTest|testNonEnglish|0.427350427350427|468|2| |false|CategoryRoutedAliasUpdateProcessorTest|testMustMatch|0.213675213675214|468|1| |false|CategoryRoutedAliasUpdateProcessorTest|testInvalidMustMatch|0.213675213675214|468|1| (some other night tests that run less often have a higher failure%, but in terms of raw numbers, this test is the biggest cause of when/why a jenkins buld fails) Skimming the logs from some of these failures, a lot of the failures seem to ultimately relate to request timeouts wating for response from Solr - but skimming the logs suggests a host of assorted problems under the covers, from not being able to initialize cores, to assertion errors, to NPEs. Are you looking into these failures? Do we have any sense of wether they are "test bugs" or "code bugs"? should we role back this issue (at least on 8x) until we can get them under control? > Category Routed Aliases > --- > > Key: SOLR-13131 > URL: https://issues.apache.org/jira/browse/SOLR-13131 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: SolrCloud >Affects Versions: master (9.0) >Reporter: Gus Heck >Assignee: Gus Heck >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: indexingWithCRA.png, indexingwithoutCRA.png, > indexintWithoutCRA2.png > > > This ticket is to add a second type of routed alias in addition to the > current time routed aliases. The new type of alias will allow data driven > creation of collections based on the values of a field and automated > organization of these collections under an alias that allows the collections > to also be searched as a whole. > The use case in mind at present is an IOT device type segregation, but I > could also see this leading to the ability to direct updates to tenant > specific hardware (in cooperation with autoscaling). > This ticket also looks forward to (but does not include) the creation of a > Dimensionally Routed Alias which would allow organizing time routed data also > segregated by device > Further design details to be added in comments. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13368) SchemaManager failures when processing schema update requests
Andrzej Bialecki created SOLR-13368:
Summary: SchemaManager failures when processing schema update
requests
Key: SOLR-13368
URL: https://issues.apache.org/jira/browse/SOLR-13368
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Affects Versions: 8.0, 8.1, master (9.0)
Reporter: Andrzej Bialecki
Assignee: Andrzej Bialecki
When sending a schema update requests occasionally {{SchemaManager}} produces
this error:
{code:java}
[junit4] 2> 508295 ERROR (qtp1376110895-5901) [n:127.0.0.1:48080_solr c:.system
s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.h.RequestHandlerBase
org.apache.solr.common.SolrException: Error reading input String Can't find
resource 'schema.xml' in classpath or '/configs/.system',
cwd=/var/lib/jenkins/jobs/Lucene-Solr-tests-master/workspace/solr/build/solr-core/test/J5
[junit4] 2> at
org.apache.solr.handler.SchemaHandler.handleRequestBody(SchemaHandler.java:94)
[junit4] 2> at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
[junit4] 2> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2566)
[junit4] 2> at
org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)
[junit4] 2> at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:394)
[junit4] 2> at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:340)
[junit4] 2> at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
[junit4] 2> at
org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
[junit4] 2> at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
[junit4] 2> at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
[junit4] 2> at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
[junit4] 2> at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
[junit4] 2> at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
[junit4] 2> at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
[junit4] 2> at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
[junit4] 2> at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
[junit4] 2> at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
[junit4] 2> at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
[junit4] 2> at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
[junit4] 2> at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
[junit4] 2> at
org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
[junit4] 2> at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:502)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
[junit4] 2> at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
[junit4] 2> at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
[junit4] 2> at
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
[junit4] 2> at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
[junit4] 2> at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.core.SolrResourceNotFoundException:
Can't find resource 'schema.xml' in classpath or '/configs/.system',
cwd=/var/lib/jenkins/jobs/Lucene-Solr-tests-master/workspace/solr/build/solr-core/test/J5
[junit4] 2> at
org.apache.solr.cloud.ZkSolrResourceLoader.openResource(ZkSolrResourceLoader.java:130)
[junit4] 2> at
org.apache.solr.schema.SchemaManager.getFreshManagedSchema(SchemaManager.java:422)
[junit4] 2> at
org.apache.solr.schema.SchemaManager.doOperations(SchemaManager.java:105)
[junit4] 2> at
org.apache.solr.schema.SchemaManager.performOperations(SchemaManager.java:83)
[junit4] 2> at
org.apache.solr.handler.SchemaHandler.handleRequestBody(SchemaHandler.java:90)
[junit4] 2> ... 31 more
{code}
This happens for regular SolrCloud collections bootstrapped from file-based
configs. If the configuration allows schema editing then {{solr.xml}} should be
initially present in ZK, then it's replaced by {{managed-schema}} but a backup
{{solr.xml.bak}} file still exists
[jira] [Commented] (SOLR-13262) Implement collection RENAME command
[ https://issues.apache.org/jira/browse/SOLR-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808976#comment-16808976 ] Andrzej Bialecki commented on SOLR-13262: -- Updated patch that prevents renaming of routed aliases. > Implement collection RENAME command > --- > > Key: SOLR-13262 > URL: https://issues.apache.org/jira/browse/SOLR-13262 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 8.0, master (9.0) >Reporter: Andrzej Bialecki >Assignee: Andrzej Bialecki >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: SOLR-13262.patch, SOLR-13262.patch > > > There's no RENAME collection command, which makes it unnecessarily difficult > to manage long-term collection life-cycles. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13262) Implement collection RENAME command
[ https://issues.apache.org/jira/browse/SOLR-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Bialecki updated SOLR-13262: - Attachment: SOLR-13262.patch > Implement collection RENAME command > --- > > Key: SOLR-13262 > URL: https://issues.apache.org/jira/browse/SOLR-13262 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 8.0, master (9.0) >Reporter: Andrzej Bialecki >Assignee: Andrzej Bialecki >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: SOLR-13262.patch, SOLR-13262.patch > > > There's no RENAME collection command, which makes it unnecessarily difficult > to manage long-term collection life-cycles. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13367) Highlighting fails for Range queries on Multi-valued String fields
Karl Wolf created SOLR-13367:
Summary: Highlighting fails for Range queries on Multi-valued
String fields
Key: SOLR-13367
URL: https://issues.apache.org/jira/browse/SOLR-13367
Project: Solr
Issue Type: Bug
Security Level: Public (Default Security Level. Issues are Public)
Components: highlighter
Affects Versions: 7.7.1, 7.5
Environment: RedHat Linux v7
Java 1.8.0_201
Reporter: Karl Wolf
Fix For: 5.1
Range queries against multi-valued string fields produces useless highlighting,
even though "hl.highlightMultiTerm":"true"
I have uncovered what I believe is a bug. At the very lease it is a difference
in behavior between Solr v5.1.0 and v7.5.0 (and v7.7.1).
I have a multi-valued string Field defined in my schema as:
I am using a query containing a Range clause and I am using highlighting to get
the list of values that actually matched the range query.
All examples below were using the appropriate Solr Admin Server SolrCore Query
page.
***
First, a correctly working example of a range query using Solr v5.1.0 which
produces useful results:
{
"responseHeader": {
"status": 0,
"QTime": 366,
"params": {
"q": "MyStringField:[A TO B}",
"hl": "true",
"indent": "true",
"hl.preserveMulti": "true",
"fl": "MyStringField,MyUniqueID",
"hl.requireFieldMatch": "true",
"hl.usePhraseHighlighter": "true",
"hl.fl": "MyStringField",
"wt": "json",
"hl.highlightMultiTerm": "true",
"_": "1553275722025"
}
},
"response": {
"numFound": 999,
"start": 0,
"docs": [
{
"MyStringField": [
"Stanley, Wendell M.",
"Avery, Roy"
],
"MyUniqueID": "UniqueID1"
},
{
"MyStringField": [
"Avery, Roy"
],
"MyUniqueID": "UniqueID2"
},
*** lots more docs correctly found
]
},
*** we get to the highlighting portion of the response
*** this indicates which values of each MyStringField
*** that actually matched the query
"highlighting": {
"UniqueID1": {
"MyStringField": [
"Avery, Roy"
]
},
"UniqueID2": {
"MyStringField": [
"Avery, Roy"
]
},
"UniqueID3": {
"MyStringField": [
"American Institute of Biological Sciences",
"Albritton, Errett C."
]
},
... etc.
*** lots more useful highlight values. Note the two matching values
*** for document UniqueID3.
}
***
* THE PROBLEM
* Now using newer versions of Solr
***
Using the exact same parameters with Solr v7.5.0 or v7.7.1, the top portion of
the
response is basically the same including the number of documents found
{
"responseHeader":{
"status":0,
"QTime":245,
"params":{
"q":"MyStringField:[A TO B}",
"hl":"on",
"hl.preserveMulti":"true",
"fl":"MyUniqueID, MyStringField",
"hl.requireFieldMatch":"true",
"hl.fl":"MyStringField",
"hightlightMultiTerm":"true",
"wt":"json",
"_":"1553105129887",
"usePhraseHighLighter":"true"}},
"response":{"numFound":999,"start":0,"docs":[
*** The problem is with the lighlighting portion of the results, which is
effectively empty.
*** There is no way to know what values in each document that actually matched
the query:
"highlighting":{
"UniqueID1":{},
"UniqueID2":{},
"UniqueID3":{},
... etc.
*** NOTE: The source data is the same for all of the tested Solr versions and
the Solr indexes
*** were properly rebuilt for each Solr version.
***
Changing the request to using the "unified" highlighter: "hl.method=unified",
the highlighting looks like:
"highlighting":{
"UniqueID1":{
"MyStringField":[]},
"UniqueID2":{
"MyStringField":[]},
"UniqueID3":{
"MyStringField":[]},
... etc.
*** The highlighting now properly lists the matching field but still no useful
values are listed.
***
NOTE: if I change the query from using a Range clause to using a Wildcard
query: q="MyStringField:A*"
the highlighting is correct in both Solr v7.5.0 and v7.7.1: These are GOOD
results!
"highlighting":{
"UniqueID1": {
"MyStringField": ["Avery, Roy"]},
"UniqueID2": {
"MyStringField": ["Avery, Roy"]},
"UniqueID3": {
"MyStringField": [
"American Institute of Biological Sciences",
"Albritton, Errett C."
]
},
... etc.
*** This makes me think there is some problem with the way a Range query
***
[jira] [Commented] (SOLR-13285) ByteArrayUtf8CharSequence cannot be cast to java.lang.String exception during replication
[
https://issues.apache.org/jira/browse/SOLR-13285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808923#comment-16808923
]
Gerald Bonfiglio commented on SOLR-13285:
-
I'm also seeing similar error doing Atomic Updates using removeregex
{code:java}
2019-04-03 16:51:07.565 ERROR (qtp343856911-37962) [c:mycollection s:shard2
r:core_node4 x:mycollection_shard2_replica_n2] o.a.s.s.HttpSolrCall
null:java.lang.ClassCastException:
org.apache.solr.common.util.ByteArrayUtf8CharSequence cannot be cast to
java.lang.String
at
org.apache.solr.update.processor.AtomicUpdateDocumentMerger.preparePatterns(AtomicUpdateDocumentMerger.java:437)
at
org.apache.solr.update.processor.AtomicUpdateDocumentMerger.doRemoveRegex(AtomicUpdateDocumentMerger.java:419)
at
org.apache.solr.update.processor.AtomicUpdateDocumentMerger.merge(AtomicUpdateDocumentMerger.java:116)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.getUpdatedDocument(DistributedUpdateProcessor.java:1422)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1106)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:693)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
at
org.apache.solr.update.processor.DocExpirationUpdateProcessorFactory$TTLUpdateProcessor.processAdd(DocExpirationUpdateProcessorFactory.java:346)
at
org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
at
org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:92)
at
org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:110)
at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:327)
at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readIterator(JavaBinUpdateRequestCodec.java:280)
at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:333)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:278)
at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readNamedList(JavaBinUpdateRequestCodec.java:235)
at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:298)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:278)
at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:191)
at
org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:126)
at
org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:123)
at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:70)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2551)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:710)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808911#comment-16808911 ] David Smiley commented on LUCENE-8753: -- I think PKLookup should be disregarded until it's fixed: [https://github.com/mikemccand/luceneutil/issues/35] (feel free to comment there if people have opinions) > New PostingFormat - UniformSplit > > > Key: LUCENE-8753 > URL: https://issues.apache.org/jira/browse/LUCENE-8753 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: 8.0 >Reporter: Bruno Roustant >Priority: Major > Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt > > Time Spent: 10m > Remaining Estimate: 0h > > This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 > objectives: > - Clear design and simple code. > - Easily extensible, for both the logic and the index format. > - Light memory usage with a very compact FST. > - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. > (the pdf attached explains visually the technique in more details) > The principle is to split the list of terms into blocks and use a FST to > access the block, but not as a prefix trie, rather with a seek-floor pattern. > For the selection of the blocks, there is a target average block size (number > of terms), with an allowed delta variation (10%) to compare the terms and > select the one with the minimal distinguishing prefix. > There are also several optimizations inside the block to make it more > compact and speed up the loading/scanning. > The performance obtained is interesting with the luceneutil benchmark, > comparing UniformSplit with BlockTree. Find it in the first comment and also > attached for better formatting. > Although the precise percentages vary between runs, three main points: > - TermQuery and PhraseQuery are improved. > - PrefixQuery and WildcardQuery are ok. > - Fuzzy queries are clearly less performant, because BlockTree is so > optimized for them. > Compared to BlockTree, FST size is reduced by 15%, and segment writing time > is reduced by 20%. So this PostingsFormat scales to lots of docs, as > BlockTree. > This initial version passes all Lucene tests. Use “ant test > -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. > Subjectively, we think we have fulfilled our goal of code simplicity. And we > have already exercised this PostingsFormat extensibility to create a > different flavor for our own use-case. > Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13366) AutoScalingConfig 'Invalid stage name' warnings after upgrade
[
https://issues.apache.org/jira/browse/SOLR-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808897#comment-16808897
]
Christine Poerschke commented on SOLR-13366:
Attached patch w.r.t. the code change mentioned above, also the Solr Reference
Guide for the
[SystemLogListener|https://lucene.apache.org/solr/guide/7_7/solrcloud-autoscaling-listeners.html#systemloglistener]
still mentioned {{WAITING}} stage.
> AutoScalingConfig 'Invalid stage name' warnings after upgrade
> -
>
> Key: SOLR-13366
> URL: https://issues.apache.org/jira/browse/SOLR-13366
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Reporter: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-13366.patch
>
>
> I noticed WARNings like this in some of our logs:
> {code:java}
> ... OverseerAutoScalingTriggerThread ... o.a.s.c.s.c.a.AutoScalingConfig
> Invalid stage name '.auto_add_replicas.system' in listener config, skipping:
> {beforeAction=[], afterAction=[], trigger=.auto_add_replicas, stage=[WAITING,
> STARTED, ABORTED, SUCCEEDED, FAILED, BEFORE_ACTION, AFTER_ACTION],
> class=org.apache.solr.cloud.autoscaling.SystemLogListener}
> {code}
> After some detective work I think I've tracked this down to 7.1.0
> [TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.1.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
> having a {{WAITING}} stage and that stage having been removed in 7.2.0
> [TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.2.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
> via the SOLR-11320 changes. Haven't tried to reproduce it but my theory is
> that the listener got auto-created (with the {{WAITING}} stage) when the
> cloud was running pre-7.2.0 code and then after upgrading the warnings start
> to appear.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-NightlyTests-8.x - Build # 61 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/61/ 6 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest Error Message: ObjectTracker found 6 object(s) that were not released!!! [MMapDirectory, SolrCore, InternalHttpClient, MMapDirectory, MMapDirectory, MMapDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:368) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:747) at org.apache.solr.core.SolrCore.(SolrCore.java:976) at org.apache.solr.core.SolrCore.(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1193) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1103) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:744) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:394) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:340) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.(SolrCore.java:1063) at org.apache.solr.core.SolrCore.(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1193) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1103) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at
[jira] [Updated] (SOLR-13366) AutoScalingConfig 'Invalid stage name' warnings after upgrade
[
https://issues.apache.org/jira/browse/SOLR-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke updated SOLR-13366:
---
Attachment: SOLR-13366.patch
> AutoScalingConfig 'Invalid stage name' warnings after upgrade
> -
>
> Key: SOLR-13366
> URL: https://issues.apache.org/jira/browse/SOLR-13366
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Reporter: Christine Poerschke
>Priority: Minor
> Attachments: SOLR-13366.patch
>
>
> I noticed WARNings like this in some of our logs:
> {code:java}
> ... OverseerAutoScalingTriggerThread ... o.a.s.c.s.c.a.AutoScalingConfig
> Invalid stage name '.auto_add_replicas.system' in listener config, skipping:
> {beforeAction=[], afterAction=[], trigger=.auto_add_replicas, stage=[WAITING,
> STARTED, ABORTED, SUCCEEDED, FAILED, BEFORE_ACTION, AFTER_ACTION],
> class=org.apache.solr.cloud.autoscaling.SystemLogListener}
> {code}
> After some detective work I think I've tracked this down to 7.1.0
> [TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.1.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
> having a {{WAITING}} stage and that stage having been removed in 7.2.0
> [TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.2.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
> via the SOLR-11320 changes. Haven't tried to reproduce it but my theory is
> that the listener got auto-created (with the {{WAITING}} stage) when the
> cloud was running pre-7.2.0 code and then after upgrading the warnings start
> to appear.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8742) SpanNearBuilder: add 'missing slop attribute' test coverage
[ https://issues.apache.org/jira/browse/LUCENE-8742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christine Poerschke resolved LUCENE-8742. - Resolution: Fixed Fix Version/s: master (9.0) 8.1 > SpanNearBuilder: add 'missing slop attribute' test coverage > --- > > Key: LUCENE-8742 > URL: https://issues.apache.org/jira/browse/LUCENE-8742 > Project: Lucene - Core > Issue Type: Test >Reporter: Christine Poerschke >Assignee: Christine Poerschke >Priority: Minor > Fix For: 8.1, master (9.0) > > Attachments: LUCENE-8742.patch > > > Add test coverage for the current behaviour. Subsequently LUCENE-8743 might > potentially change the exact exception thrown in this scenario. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_172) - Build # 7870 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7870/ Java: 64bit/jdk1.8.0_172 -XX:+UseCompressedOops -XX:+UseG1GC 10 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest Error Message: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 1) Thread[id=71, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[B2A1C6A593853DF]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 2) Thread[id=66, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[B2A1C6A593853DF]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 3) Thread[id=74, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[B2A1C6A593853DF]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 1) Thread[id=71, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[B2A1C6A593853DF]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 2) Thread[id=66, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[B2A1C6A593853DF]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 3) Thread[id=74, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[B2A1C6A593853DF]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) at __randomizedtesting.SeedInfo.seed([B2A1C6A593853DF]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest Error Message: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 1) Thread[id=94, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[B2A1C6A593853DF]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 2) Thread[id=88, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[B2A1C6A593853DF]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at
[jira] [Updated] (SOLR-13366) AutoScalingConfig 'Invalid stage name' warnings after upgrade
[
https://issues.apache.org/jira/browse/SOLR-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Christine Poerschke updated SOLR-13366:
---
Summary: AutoScalingConfig 'Invalid stage name' warnings after upgrade
(was: 'Invalid stage name' warnings after upgrade)
> AutoScalingConfig 'Invalid stage name' warnings after upgrade
> -
>
> Key: SOLR-13366
> URL: https://issues.apache.org/jira/browse/SOLR-13366
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Reporter: Christine Poerschke
>Priority: Minor
>
> I noticed WARNings like this in some of our logs:
> {code:java}
> ... OverseerAutoScalingTriggerThread ... o.a.s.c.s.c.a.AutoScalingConfig
> Invalid stage name '.auto_add_replicas.system' in listener config, skipping:
> {beforeAction=[], afterAction=[], trigger=.auto_add_replicas, stage=[WAITING,
> STARTED, ABORTED, SUCCEEDED, FAILED, BEFORE_ACTION, AFTER_ACTION],
> class=org.apache.solr.cloud.autoscaling.SystemLogListener}
> {code}
> After some detective work I think I've tracked this down to 7.1.0
> [TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.1.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
> having a {{WAITING}} stage and that stage having been removed in 7.2.0
> [TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.2.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
> via the SOLR-11320 changes. Haven't tried to reproduce it but my theory is
> that the listener got auto-created (with the {{WAITING}} stage) when the
> cloud was running pre-7.2.0 code and then after upgrading the warnings start
> to appear.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13366) 'Invalid stage name' warnings after upgrade
[
https://issues.apache.org/jira/browse/SOLR-13366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808865#comment-16808865
]
Christine Poerschke commented on SOLR-13366:
At a quick glance a small change in AutoScalingConfig.java e.g.
{code:java}
- LOG.warn("Invalid stage name '" + name + "' in listener config, skipping: " +
properties);
+ LOG.warn("Invalid stage name '" + stageName + "' in listener config,
skipping: " + properties);
{code}
could make it more apparent that {{WAITING}} is the invalid stage.
And upon further consideration the {{skipping: " + properties}} could be
rephrased perhaps since the code appears to skip only the invalid stage but not
the other stages i.e. not all of the properties are being skipped.
> 'Invalid stage name' warnings after upgrade
> ---
>
> Key: SOLR-13366
> URL: https://issues.apache.org/jira/browse/SOLR-13366
> Project: Solr
> Issue Type: Bug
> Components: AutoScaling
>Reporter: Christine Poerschke
>Priority: Minor
>
> I noticed WARNings like this in some of our logs:
> {code:java}
> ... OverseerAutoScalingTriggerThread ... o.a.s.c.s.c.a.AutoScalingConfig
> Invalid stage name '.auto_add_replicas.system' in listener config, skipping:
> {beforeAction=[], afterAction=[], trigger=.auto_add_replicas, stage=[WAITING,
> STARTED, ABORTED, SUCCEEDED, FAILED, BEFORE_ACTION, AFTER_ACTION],
> class=org.apache.solr.cloud.autoscaling.SystemLogListener}
> {code}
> After some detective work I think I've tracked this down to 7.1.0
> [TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.1.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
> having a {{WAITING}} stage and that stage having been removed in 7.2.0
> [TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.2.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
> via the SOLR-11320 changes. Haven't tried to reproduce it but my theory is
> that the listener got auto-created (with the {{WAITING}} stage) when the
> cloud was running pre-7.2.0 code and then after upgrading the warnings start
> to appear.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-13366) 'Invalid stage name' warnings after upgrade
Christine Poerschke created SOLR-13366:
--
Summary: 'Invalid stage name' warnings after upgrade
Key: SOLR-13366
URL: https://issues.apache.org/jira/browse/SOLR-13366
Project: Solr
Issue Type: Bug
Components: AutoScaling
Reporter: Christine Poerschke
I noticed WARNings like this in some of our logs:
{code:java}
... OverseerAutoScalingTriggerThread ... o.a.s.c.s.c.a.AutoScalingConfig
Invalid stage name '.auto_add_replicas.system' in listener config, skipping:
{beforeAction=[], afterAction=[], trigger=.auto_add_replicas, stage=[WAITING,
STARTED, ABORTED, SUCCEEDED, FAILED, BEFORE_ACTION, AFTER_ACTION],
class=org.apache.solr.cloud.autoscaling.SystemLogListener}
{code}
After some detective work I think I've tracked this down to 7.1.0
[TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.1.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
having a {{WAITING}} stage and that stage having been removed in 7.2.0
[TriggerEventProcessorStage|https://github.com/apache/lucene-solr/blob/releases/lucene-solr/7.2.0/solr/solrj/src/java/org/apache/solr/client/solrj/cloud/autoscaling/TriggerEventProcessorStage.java]
via the SOLR-11320 changes. Haven't tried to reproduce it but my theory is
that the listener got auto-created (with the {{WAITING}} stage) when the cloud
was running pre-7.2.0 code and then after upgrading the warnings start to
appear.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808860#comment-16808860 ] Mike Sokolov commented on LUCENE-8753: -- I've been working on some other FST-related changes, and running these benchmarks a bunch. PKLookup performance varies wildly (for me) depending on just how the terms are distributed among the segments. The PK's are a sequence (with no gaps) of base36 identifiers, and I see different performance characteristics in small index vs larger (wikimediumall). In general force merge gets me consistent results, but then may not be predictive of reality... > New PostingFormat - UniformSplit > > > Key: LUCENE-8753 > URL: https://issues.apache.org/jira/browse/LUCENE-8753 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: 8.0 >Reporter: Bruno Roustant >Priority: Major > Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt > > Time Spent: 10m > Remaining Estimate: 0h > > This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 > objectives: > - Clear design and simple code. > - Easily extensible, for both the logic and the index format. > - Light memory usage with a very compact FST. > - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. > (the pdf attached explains visually the technique in more details) > The principle is to split the list of terms into blocks and use a FST to > access the block, but not as a prefix trie, rather with a seek-floor pattern. > For the selection of the blocks, there is a target average block size (number > of terms), with an allowed delta variation (10%) to compare the terms and > select the one with the minimal distinguishing prefix. > There are also several optimizations inside the block to make it more > compact and speed up the loading/scanning. > The performance obtained is interesting with the luceneutil benchmark, > comparing UniformSplit with BlockTree. Find it in the first comment and also > attached for better formatting. > Although the precise percentages vary between runs, three main points: > - TermQuery and PhraseQuery are improved. > - PrefixQuery and WildcardQuery are ok. > - Fuzzy queries are clearly less performant, because BlockTree is so > optimized for them. > Compared to BlockTree, FST size is reduced by 15%, and segment writing time > is reduced by 20%. So this PostingsFormat scales to lots of docs, as > BlockTree. > This initial version passes all Lucene tests. Use “ant test > -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. > Subjectively, we think we have fulfilled our goal of code simplicity. And we > have already exercised this PostingsFormat extensibility to create a > different flavor for our own use-case. > Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[
https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808852#comment-16808852
]
Bruno Roustant commented on LUCENE-8753:
{quote}Is it due to the fact that it doesn't have the ability to fail lookups
early like BlockTree?
{quote}
This is one cause. While BlockTree builds a kind of prefix-trie and may stop if
the prefix is not matched, UniformSplit doesn't, so it loads a block.
That said I remarked that PKLookup performance varies a lot. It is sometimes in
favor of UniformSplit. Actually I don't know how the benchmark generates the
test set. It clearly has an influence on the metric.
> New PostingFormat - UniformSplit
>
>
> Key: LUCENE-8753
> URL: https://issues.apache.org/jira/browse/LUCENE-8753
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
>Affects Versions: 8.0
>Reporter: Bruno Roustant
>Priority: Major
> Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is a proposal to add a new PostingsFormat called "UniformSplit" with 4
> objectives:
> - Clear design and simple code.
> - Easily extensible, for both the logic and the index format.
> - Light memory usage with a very compact FST.
> - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance.
> (the pdf attached explains visually the technique in more details)
> The principle is to split the list of terms into blocks and use a FST to
> access the block, but not as a prefix trie, rather with a seek-floor pattern.
> For the selection of the blocks, there is a target average block size (number
> of terms), with an allowed delta variation (10%) to compare the terms and
> select the one with the minimal distinguishing prefix.
> There are also several optimizations inside the block to make it more
> compact and speed up the loading/scanning.
> The performance obtained is interesting with the luceneutil benchmark,
> comparing UniformSplit with BlockTree. Find it in the first comment and also
> attached for better formatting.
> Although the precise percentages vary between runs, three main points:
> - TermQuery and PhraseQuery are improved.
> - PrefixQuery and WildcardQuery are ok.
> - Fuzzy queries are clearly less performant, because BlockTree is so
> optimized for them.
> Compared to BlockTree, FST size is reduced by 15%, and segment writing time
> is reduced by 20%. So this PostingsFormat scales to lots of docs, as
> BlockTree.
> This initial version passes all Lucene tests. Use “ant test
> -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat.
> Subjectively, we think we have fulfilled our goal of code simplicity. And we
> have already exercised this PostingsFormat extensibility to create a
> different flavor for our own use-case.
> Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk1.8.0_172) - Build # 23862 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23862/ Java: 64bit/jdk1.8.0_172 -XX:-UseCompressedOops -XX:+UseSerialGC 13 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest Error Message: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 1) Thread[id=1874, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[47835EC7D25C7B95]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 2) Thread[id=1868, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[47835EC7D25C7B95]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 3) Thread[id=1871, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[47835EC7D25C7B95]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) Stack Trace: com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 1) Thread[id=1874, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[47835EC7D25C7B95]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 2) Thread[id=1868, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[47835EC7D25C7B95]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 3) Thread[id=1871, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[47835EC7D25C7B95]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) at __randomizedtesting.SeedInfo.seed([47835EC7D25C7B95]:0) FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest Error Message: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 1) Thread[id=1892, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[47835EC7D25C7B95]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 2) Thread[id=1889, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[47835EC7D25C7B95]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at
[jira] [Commented] (SOLR-13364) Make Admin UI aware of logged-in users permissions
[
https://issues.apache.org/jira/browse/SOLR-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808850#comment-16808850
]
Jan Høydahl commented on SOLR-13364:
Feel free to whip up a patch. I have no immediate plan to start on this one.
> Make Admin UI aware of logged-in users permissions
> --
>
> Key: SOLR-13364
> URL: https://issues.apache.org/jira/browse/SOLR-13364
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI, Authentication, Authorization, security
>Reporter: Jan Høydahl
>Priority: Major
>
> We should aim to add fine-grained permission checks to the UI. One way to do
> this is to add a new REST-endpoint {{/admin/login/whoami}} that is always
> open for all, and that responds with a JSON with current user's permissions.
> If no user is logged in it will respond with empty list and "No user logged
> in". Else it will respond with e.g.
> {code:java}
> { "user": "john", "roles": ["superuser", "searcher"], "permissions":
> ["security-edit", "collectionadmin"...] }{code}
> The Admin UI can then request this endpoint and cache the info, so that it
> may make decisions to hide/grey out certain menu options throughout the UI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808842#comment-16808842
]
Uwe Schindler commented on LUCENE-2562:
---
I screenshotted it:
!screenshot-1.png!
Basically, I'd remove the whole ConcurrentCache and all that stuff, as checking
if a class is a subtype of another one is very easy with
{{Class#isAssignableFrom(Class)}}.
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Uwe Schindler
>Priority: Major
> Labels: gsoc2014
> Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch,
> LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch,
> LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png, Luke-ALE-3.png,
> Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png, luke-javafx2.png,
> luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg, lukeALE-documents.png,
> screenshot-1.png, スクリーンショット 2018-11-05 9.19.47.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> see
> "RE: Luke - in need of maintainer":
> http://markmail.org/message/m4gsto7giltvrpuf
> "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q
> I think it would be great if there was a version of Luke that always worked
> with trunk - and it would also be great if it was easier to match Luke jars
> with Lucene versions.
> While I'd like to get GWT Luke into the mix as well, I think the easiest
> starting point is to straight port Luke to another UI toolkit before
> abstracting out DTO objects that both GWT Luke and Pivot Luke could share.
> I've started slowly converting Luke's use of thinlet to Apache Pivot. I
> haven't/don't have a lot of time for this at the moment, but I've plugged
> away here and there over the past work or two. There is still a *lot* to do.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-2562) Make Luke a Lucene/Solr Module
[ https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated LUCENE-2562: -- Attachment: screenshot-1.png > Make Luke a Lucene/Solr Module > -- > > Key: LUCENE-2562 > URL: https://issues.apache.org/jira/browse/LUCENE-2562 > Project: Lucene - Core > Issue Type: Task >Reporter: Mark Miller >Assignee: Uwe Schindler >Priority: Major > Labels: gsoc2014 > Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch, > LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch, > LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png, Luke-ALE-3.png, > Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png, luke-javafx2.png, > luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg, lukeALE-documents.png, > screenshot-1.png, スクリーンショット 2018-11-05 9.19.47.png > > Time Spent: 50m > Remaining Estimate: 0h > > see > "RE: Luke - in need of maintainer": > http://markmail.org/message/m4gsto7giltvrpuf > "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q > I think it would be great if there was a version of Luke that always worked > with trunk - and it would also be great if it was easier to match Luke jars > with Lucene versions. > While I'd like to get GWT Luke into the mix as well, I think the easiest > starting point is to straight port Luke to another UI toolkit before > abstracting out DTO objects that both GWT Luke and Pivot Luke could share. > I've started slowly converting Luke's use of thinlet to Apache Pivot. I > haven't/don't have a lot of time for this at the moment, but I've plugged > away here and there over the past work or two. There is still a *lot* to do. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13262) Implement collection RENAME command
[ https://issues.apache.org/jira/browse/SOLR-13262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808838#comment-16808838 ] Andrzej Bialecki commented on SOLR-13262: -- [~gus_heck] - these are very good points, thanks. I'm inclined to initially forbid renaming router aliases, at least for now. I believe eventually it can be supported but it needs more changes in the way routed aliases work (eg. storing the initial alias as a "prefixName" property in the alias properties, so that the current alias name wouldn't be used for constructing / parsing of collection names). I'm also inclined to forbid sending updates to a non-routed alias that consists of more than 1 collection - the current behavior is documented but other than that it's unintuitive and misleading - a scenario where users would want an update to succeed if they accidentally send it to a multi-collection alias seems very unlikely? Of course, if it's there I'm sure someone uses and relies on it, so there would be a back-compat aspect of this change... Collection admin commands rarely make sense, too, if the alias points to more than 1 collection. We could either refuse supporting any aliases (which was the situation before this patch ;) ) or accept only requests that refer to 1:1 aliases. Hence the need for recognizing this specific sub-class of alias. Re: documentation - yes, this needs clarification, which I'm happy to provide once we're ready with the functionality. > Implement collection RENAME command > --- > > Key: SOLR-13262 > URL: https://issues.apache.org/jira/browse/SOLR-13262 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Affects Versions: 8.0, master (9.0) >Reporter: Andrzej Bialecki >Assignee: Andrzej Bialecki >Priority: Major > Fix For: 8.1, master (9.0) > > Attachments: SOLR-13262.patch > > > There's no RENAME collection command, which makes it unnecessarily difficult > to manage long-term collection life-cycles. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808833#comment-16808833
]
Tomoko Uchida commented on LUCENE-2562:
---
{quote}Would it be possible to remove the isSubclass() code and replace by the
JDK one?
{quote}
I think you mean {{Class.isAssignableFrom()}}, I have not noticed this method.
I will remove isSubclass() by replacing that.
{quote}I made a comment on one of the pull requests
([https://github.com/apache/lucene-solr/pull/618]), but not sure if this is
reflected in the patch.
{quote}
I cannot find your comments in the past PRs, 'cause there are lots of files and
GitHub search {{commenter:uschindler}} does not work for finding them... :'(
Can you please tell me the points again?
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Uwe Schindler
>Priority: Major
> Labels: gsoc2014
> Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch,
> LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch,
> LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png, Luke-ALE-3.png,
> Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png, luke-javafx2.png,
> luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg, lukeALE-documents.png,
> スクリーンショット 2018-11-05 9.19.47.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> see
> "RE: Luke - in need of maintainer":
> http://markmail.org/message/m4gsto7giltvrpuf
> "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q
> I think it would be great if there was a version of Luke that always worked
> with trunk - and it would also be great if it was easier to match Luke jars
> with Lucene versions.
> While I'd like to get GWT Luke into the mix as well, I think the easiest
> starting point is to straight port Luke to another UI toolkit before
> abstracting out DTO objects that both GWT Luke and Pivot Luke could share.
> I've started slowly converting Luke's use of thinlet to Apache Pivot. I
> haven't/don't have a lot of time for this at the moment, but I've plugged
> away here and there over the past work or two. There is still a *lot* to do.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] mikemccand commented on a change in pull request #613: LUCENE-8671: Expose FST off/on-heap options on Lucene50PostingsFormat
mikemccand commented on a change in pull request #613: LUCENE-8671: Expose FST
off/on-heap options on Lucene50PostingsFormat
URL: https://github.com/apache/lucene-solr/pull/613#discussion_r271788046
##
File path:
lucene/core/src/java/org/apache/lucene/codecs/lucene50/Lucene50PostingsFormat.java
##
@@ -389,6 +390,33 @@
private final int minTermBlockSize;
private final int maxTermBlockSize;
+ private final FSTLoadMode fstLoadMode;
+
+ /**
+ * An enum that allows to control if term index FSTs are loaded into memory
or read off-heap
+ */
+ public enum FSTLoadMode {
+/**
+ * Always read FSTs from disk.
+ */
+OFF_HEAP,
+/**
+ * Never read FSTs from disk ie. all fields FSTs are loaded into memory
+ */
+ON_HEAP,
+/**
+ * Always read FSTs from disk.
+ * An exception is made for ID fields in an IndexWriter context which are
always loaded into memory.
+ * This is useful to guarantee best update performance even if a non
MMapDirectory is used which required for
+ * the {@link FSTLoadMode#AUTO} mode is used.
Review comment:
Two `is used` in the last sentence here?
Maybe include a warning here that this does not look at `Directory` type and
may put FST off-heap even for buffered directory implementations?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13075) Harden SaslZkACLProviderTest.
[ https://issues.apache.org/jira/browse/SOLR-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808815#comment-16808815 ] ASF subversion and git services commented on SOLR-13075: Commit 9f2e04b3f2367d0393a5515df9675faac1deafd0 in lucene-solr's branch refs/heads/branch_8x from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=9f2e04b ] SOLR-13075: Harden SaslZkACLProviderTest (Kevin Risden, Hrishikesh Gadre, Peter Cseh) Signed-off-by: Kevin Risden > Harden SaslZkACLProviderTest. > - > > Key: SOLR-13075 > URL: https://issues.apache.org/jira/browse/SOLR-13075 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13075) Harden SaslZkACLProviderTest.
[ https://issues.apache.org/jira/browse/SOLR-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808814#comment-16808814 ] ASF subversion and git services commented on SOLR-13075: Commit 6596ed1c16de72133fc9f86dafce46daf3bdf724 in lucene-solr's branch refs/heads/master from Kevin Risden [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6596ed1 ] SOLR-13075: Harden SaslZkACLProviderTest (Kevin Risden, Hrishikesh Gadre, Peter Cseh) Signed-off-by: Kevin Risden > Harden SaslZkACLProviderTest. > - > > Key: SOLR-13075 > URL: https://issues.apache.org/jira/browse/SOLR-13075 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808813#comment-16808813 ] Adrien Grand commented on LUCENE-8753: -- I haven't read the pdf or the patch yet. The LowTerm speedup makes me think that term lookups are faster, but then I'm surprised PKLookup is slower. Is it due to the fact that it doesn't have the ability to fail lookups early like BlockTree? > New PostingFormat - UniformSplit > > > Key: LUCENE-8753 > URL: https://issues.apache.org/jira/browse/LUCENE-8753 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: 8.0 >Reporter: Bruno Roustant >Priority: Major > Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt > > Time Spent: 10m > Remaining Estimate: 0h > > This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 > objectives: > - Clear design and simple code. > - Easily extensible, for both the logic and the index format. > - Light memory usage with a very compact FST. > - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. > (the pdf attached explains visually the technique in more details) > The principle is to split the list of terms into blocks and use a FST to > access the block, but not as a prefix trie, rather with a seek-floor pattern. > For the selection of the blocks, there is a target average block size (number > of terms), with an allowed delta variation (10%) to compare the terms and > select the one with the minimal distinguishing prefix. > There are also several optimizations inside the block to make it more > compact and speed up the loading/scanning. > The performance obtained is interesting with the luceneutil benchmark, > comparing UniformSplit with BlockTree. Find it in the first comment and also > attached for better formatting. > Although the precise percentages vary between runs, three main points: > - TermQuery and PhraseQuery are improved. > - PrefixQuery and WildcardQuery are ok. > - Fuzzy queries are clearly less performant, because BlockTree is so > optimized for them. > Compared to BlockTree, FST size is reduced by 15%, and segment writing time > is reduced by 20%. So this PostingsFormat scales to lots of docs, as > BlockTree. > This initial version passes all Lucene tests. Use “ant test > -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. > Subjectively, we think we have fulfilled our goal of code simplicity. And we > have already exercised this PostingsFormat extensibility to create a > different flavor for our own use-case. > Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] bruno-roustant opened a new pull request #633: LUCENE-8753 UniformSplit PostingsFormat
bruno-roustant opened a new pull request #633: LUCENE-8753 UniformSplit PostingsFormat URL: https://github.com/apache/lucene-solr/pull/633 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8753) New PostingFormat - UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Roustant updated LUCENE-8753: --- Attachment: luceneutil.benchmark.txt > New PostingFormat - UniformSplit > > > Key: LUCENE-8753 > URL: https://issues.apache.org/jira/browse/LUCENE-8753 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: 8.0 >Reporter: Bruno Roustant >Priority: Major > Attachments: Uniform Split Technique.pdf, luceneutil.benchmark.txt > > > This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 > objectives: > - Clear design and simple code. > - Easily extensible, for both the logic and the index format. > - Light memory usage with a very compact FST. > - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. > (the pdf attached explains visually the technique in more details) > The principle is to split the list of terms into blocks and use a FST to > access the block, but not as a prefix trie, rather with a seek-floor pattern. > For the selection of the blocks, there is a target average block size (number > of terms), with an allowed delta variation (10%) to compare the terms and > select the one with the minimal distinguishing prefix. > There are also several optimizations inside the block to make it more > compact and speed up the loading/scanning. > The performance obtained is interesting with the luceneutil benchmark, > comparing UniformSplit with BlockTree. Find it in the first comment and also > attached for better formatting. > Although the precise percentages vary between runs, three main points: > - TermQuery and PhraseQuery are improved. > - PrefixQuery and WildcardQuery are ok. > - Fuzzy queries are clearly less performant, because BlockTree is so > optimized for them. > Compared to BlockTree, FST size is reduced by 15%, and segment writing time > is reduced by 20%. So this PostingsFormat scales to lots of docs, as > BlockTree. > This initial version passes all Lucene tests. Use “ant test > -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. > Subjectively, we think we have fulfilled our goal of code simplicity. And we > have already exercised this PostingsFormat extensibility to create a > different flavor for our own use-case. > Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-8753) New PostingFormat - UniformSplit
[ https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruno Roustant updated LUCENE-8753: --- Description: This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 objectives: - Clear design and simple code. - Easily extensible, for both the logic and the index format. - Light memory usage with a very compact FST. - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. (the pdf attached explains visually the technique in more details) The principle is to split the list of terms into blocks and use a FST to access the block, but not as a prefix trie, rather with a seek-floor pattern. For the selection of the blocks, there is a target average block size (number of terms), with an allowed delta variation (10%) to compare the terms and select the one with the minimal distinguishing prefix. There are also several optimizations inside the block to make it more compact and speed up the loading/scanning. The performance obtained is interesting with the luceneutil benchmark, comparing UniformSplit with BlockTree. Find it in the first comment and also attached for better formatting. Although the precise percentages vary between runs, three main points: - TermQuery and PhraseQuery are improved. - PrefixQuery and WildcardQuery are ok. - Fuzzy queries are clearly less performant, because BlockTree is so optimized for them. Compared to BlockTree, FST size is reduced by 15%, and segment writing time is reduced by 20%. So this PostingsFormat scales to lots of docs, as BlockTree. This initial version passes all Lucene tests. Use “ant test -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. Subjectively, we think we have fulfilled our goal of code simplicity. And we have already exercised this PostingsFormat extensibility to create a different flavor for our own use-case. Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley was: This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 objectives: - Clear design and simple code. - Easily extensible, for both the logic and the index format. - Light memory usage with a very compact FST. - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. (the pdf attached explains visually the technique in more details) The principle is to split the list of terms into blocks and use a FST to access the block, but not as a prefix trie, rather with a seek-floor pattern. For the selection of the blocks, there is a target average block size (number of terms), with an allowed delta variation (10%) to compare the terms and select the one with the minimal distinguishing prefix. There are also several optimizations inside the block to make it more compact and speed up the loading/scanning. The performance obtained is interesting with the luceneutil benchmark, comparing UniformSplit with BlockTree. Find it in the first comment. Although the precise percentages vary between runs, three main points: - TermQuery and PhraseQuery are improved. - PrefixQuery and WildcardQuery are ok. - Fuzzy queries are clearly less performant, because BlockTree is so optimized for them. Compared to BlockTree, FST size is reduced by 15%, and segment writing time is reduced by 20%. So this PostingsFormat scales to lots of docs, as BlockTree. This initial version passes all Lucene tests. Use “ant test -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. Subjectively, we think we have fulfilled our goal of code simplicity. And we have already exercised this PostingsFormat extensibility to create a different flavor for our own use-case. Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley > New PostingFormat - UniformSplit > > > Key: LUCENE-8753 > URL: https://issues.apache.org/jira/browse/LUCENE-8753 > Project: Lucene - Core > Issue Type: Improvement > Components: core/codecs >Affects Versions: 8.0 >Reporter: Bruno Roustant >Priority: Major > Attachments: Uniform Split Technique.pdf > > > This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 > objectives: > - Clear design and simple code. > - Easily extensible, for both the logic and the index format. > - Light memory usage with a very compact FST. > - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. > (the pdf attached explains visually the technique in more details) > The principle is to split the list of terms into blocks and use a FST to > access the block, but not as a prefix trie, rather with a seek-floor pattern. > For the selection of the blocks, there is a target average block size (number > of terms), with an allowed delta variation (10%) to compare the terms and > select
[jira] [Commented] (LUCENE-8753) New PostingFormat - UniformSplit
[
https://issues.apache.org/jira/browse/LUCENE-8753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808799#comment-16808799
]
Bruno Roustant commented on LUCENE-8753:
Here's the Luceneutil benchmark with the wikimedium500k data set using Java 8.
This is a bit dated using Lucene 7.1; it'd be nice to update to master.
Report after iter 19:
TaskQPS blocktree StdDevQPS uniformsplit StdDev Pct diff
Fuzzy1 508.47 (3.8%) 221.37 (0.9%) {color:#59afe1}-56.5%{color} ( -58% - -53%)
Fuzzy2 171.73 (6.4%) 80.62 (1.4%) {color:#59afe1}-53.1%{color} ( -57% - -48%)
PKLookup 182.47 (2.4%) 149.62 (2.5%) {color:#59afe1}-18.0%{color} ( -22% -
-13%)
Wildcard 1788.74 (5.9%) 1729.37 (4.5%) {color:#59afe1}-3.3%{color} ( -12% - 7%)
IntNRQ 1561.48 (2.1%) 1564.33 (1.9%) {color:#59afe1}0.2%{color} ( -3% - 4%)
Prefix3 1759.69 (5.0%) 1829.74 (4.8%) {color:#59afe1}4.0%{color} ( -5% - 14%)
HighTermDayOfYearSort 586.06 (5.4%) 622.34 (8.2%) {color:#59afe1}6.2%{color} (
-6% - 20%)
MedPhrase 1204.85 (5.5%) 1282.89 (7.7%) {color:#59afe1}6.5%{color} ( -6% - 20%)
HighSpanNear 590.88 (4.1%) 629.64 (6.1%) {color:#59afe1}6.6%{color} ( -3% -
17%)
OrHighMed 1101.48 (4.5%) 1220.75 (6.2%) {color:#59afe1}10.8%{color} ( 0% - 22%)
HighTermMonthSort 2617.10 (2.6%) 2916.34 (4.6%) {color:#59afe1}11.4%{color} (
4% - 19%)
HighPhrase 961.04 (5.5%) 1073.62 (6.0%) {color:#59afe1}11.7%{color} ( 0% - 24%)
MedSloppyPhrase 604.56 (13.3%) 680.31 (13.7%) {color:#59afe1}12.5%{color} (
-12% - 45%)
LowSloppyPhrase 954.87 (8.1%) 1075.67 (5.4%) {color:#59afe1}12.7%{color} ( 0%
- 28%)
MedSpanNear 737.14 (5.8%) 830.68 (8.3%) {color:#59afe1}12.7%{color} ( -1% -
28%)
OrHighHigh 811.57 (5.7%) 915.01 (6.2%) {color:#59afe1}12.7%{color} ( 0% - 26%)
AndHighMed 1157.45 (5.3%) 1317.78 (5.1%) {color:#59afe1}13.9%{color} ( 3% -
25%)
AndHighHigh 1095.29 (5.7%) 1254.16 (4.9%) {color:#59afe1}14.5%{color} ( 3% -
26%)
HighSloppyPhrase 880.42 (8.2%) 1009.72 (7.0%) {color:#59afe1}14.7%{color} ( 0%
- 32%)
LowPhrase 1245.33 (6.0%) 1473.57 (4.4%) {color:#59afe1}18.3%{color} ( 7% - 30%)
Respell 81.10 (12.7%) 99.43 (10.3%) {color:#59afe1}22.6%{color} ( 0% - 52%)
HighTerm 3733.81 (6.1%) 4599.96 (6.8%) {color:#59afe1}23.2%{color} ( 9% - 38%)
OrHighLow 1960.13 (6.2%) 2415.81 (6.0%) {color:#59afe1}23.2%{color} ( 10% -
37%)
MedTerm 4411.60 (4.9%) 5450.56 (5.8%) {color:#59afe1}23.6%{color} ( 12% - 35%)
LowSpanNear 1944.27 (5.3%) 2416.29 (4.5%) {color:#59afe1}24.3%{color} ( 13% -
36%)
AndHighLow 1978.10 (7.6%) 2500.74 (5.8%) {color:#59afe1}26.4%{color} ( 12% -
43%)
LowTerm 4949.24 (4.8%) 6589.86 (5.3%) {color:#59afe1}33.1%{color} ( 22% - 45%)
> New PostingFormat - UniformSplit
>
>
> Key: LUCENE-8753
> URL: https://issues.apache.org/jira/browse/LUCENE-8753
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/codecs
>Affects Versions: 8.0
>Reporter: Bruno Roustant
>Priority: Major
> Attachments: Uniform Split Technique.pdf
>
>
> This is a proposal to add a new PostingsFormat called "UniformSplit" with 4
> objectives:
> - Clear design and simple code.
> - Easily extensible, for both the logic and the index format.
> - Light memory usage with a very compact FST.
> - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance.
> (the pdf attached explains visually the technique in more details)
> The principle is to split the list of terms into blocks and use a FST to
> access the block, but not as a prefix trie, rather with a seek-floor pattern.
> For the selection of the blocks, there is a target average block size (number
> of terms), with an allowed delta variation (10%) to compare the terms and
> select the one with the minimal distinguishing prefix.
> There are also several optimizations inside the block to make it more compact
> and speed up the loading/scanning.
> The performance obtained is interesting with the luceneutil benchmark,
> comparing UniformSplit with BlockTree. Find it in the first comment.
>
> Although the precise percentages vary between runs, three main points:
> - TermQuery and PhraseQuery are improved.
> - PrefixQuery and WildcardQuery are ok.
> - Fuzzy queries are clearly less performant, because BlockTree is so
> optimized for them.
> Compared to BlockTree, FST size is reduced by 15%, and segment writing time
> is reduced by 20%. So this PostingsFormat scales to lots of docs, as
> BlockTree.
>
> This initial version passes all Lucene tests. Use “ant test
> -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat.
> Subjectively, we think we have fulfilled our goal of code simplicity. And we
> have already exercised this PostingsFormat extensibility to create a
> different flavor for our own use-case.
>
> Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David
[jira] [Created] (LUCENE-8753) New PostingFormat - UniformSplit
Bruno Roustant created LUCENE-8753: -- Summary: New PostingFormat - UniformSplit Key: LUCENE-8753 URL: https://issues.apache.org/jira/browse/LUCENE-8753 Project: Lucene - Core Issue Type: Improvement Components: core/codecs Affects Versions: 8.0 Reporter: Bruno Roustant Attachments: Uniform Split Technique.pdf This is a proposal to add a new PostingsFormat called "UniformSplit" with 4 objectives: - Clear design and simple code. - Easily extensible, for both the logic and the index format. - Light memory usage with a very compact FST. - Focus on efficient TermQuery, PhraseQuery and PrefixQuery performance. (the pdf attached explains visually the technique in more details) The principle is to split the list of terms into blocks and use a FST to access the block, but not as a prefix trie, rather with a seek-floor pattern. For the selection of the blocks, there is a target average block size (number of terms), with an allowed delta variation (10%) to compare the terms and select the one with the minimal distinguishing prefix. There are also several optimizations inside the block to make it more compact and speed up the loading/scanning. The performance obtained is interesting with the luceneutil benchmark, comparing UniformSplit with BlockTree. Find it in the first comment. Although the precise percentages vary between runs, three main points: - TermQuery and PhraseQuery are improved. - PrefixQuery and WildcardQuery are ok. - Fuzzy queries are clearly less performant, because BlockTree is so optimized for them. Compared to BlockTree, FST size is reduced by 15%, and segment writing time is reduced by 20%. So this PostingsFormat scales to lots of docs, as BlockTree. This initial version passes all Lucene tests. Use “ant test -Dtests.codec=UniformSplitTesting” to test with this PostingsFormat. Subjectively, we think we have fulfilled our goal of code simplicity. And we have already exercised this PostingsFormat extensibility to create a different flavor for our own use-case. Contributors: Juan Camilo Rodriguez Duran, Bruno Roustant, David Smiley -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-13075) Harden SaslZkACLProviderTest.
[ https://issues.apache.org/jira/browse/SOLR-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808789#comment-16808789 ] Erick Erickson edited comment on SOLR-13075 at 4/3/19 2:41 PM: --- Thanks! If I saw a failure over the next few days I was going to start looking at the patch, one thing at a time. Now I don't have to hunt for it. was (Author: erickerickson): Thanks! If I saw a failure over the next few days I was going to start looking at the patch, one thing at a time. Now I don't have to hung for it. > Harden SaslZkACLProviderTest. > - > > Key: SOLR-13075 > URL: https://issues.apache.org/jira/browse/SOLR-13075 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13075) Harden SaslZkACLProviderTest.
[ https://issues.apache.org/jira/browse/SOLR-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808789#comment-16808789 ] Erick Erickson commented on SOLR-13075: --- Thanks! If I saw a failure over the next few days I was going to start looking at the patch, one thing at a time. Now I don't have to hung for it. > Harden SaslZkACLProviderTest. > - > > Key: SOLR-13075 > URL: https://issues.apache.org/jira/browse/SOLR-13075 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] moshebla commented on issue #455: SOLR-12638
moshebla commented on issue #455: SOLR-12638 URL: https://github.com/apache/lucene-solr/pull/455#issuecomment-479516796 > Can you please propose CHANGES.txt wording? SOLR-12638: Atomic Updates for nested documents. This enables atomic updates for nested documents, without the need to supply the whole nested hierarchy (which would be overwritten if absent). This is done by fetching the whole document hierarchy, updating the specific doc in the path that is to be updated, removing the old document hierarchy and indexing the new one with the atomic update merged into it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13075) Harden SaslZkACLProviderTest.
[ https://issues.apache.org/jira/browse/SOLR-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808765#comment-16808765 ] Peter Cseh commented on SOLR-13075: --- Thanks! I wasn't sure which ticket to use. :) > Harden SaslZkACLProviderTest. > - > > Key: SOLR-13075 > URL: https://issues.apache.org/jira/browse/SOLR-13075 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13075) Harden SaslZkACLProviderTest.
[ https://issues.apache.org/jira/browse/SOLR-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808761#comment-16808761 ] Kevin Risden commented on SOLR-13075: - [~gezapeti] attached slightly modified patch over here. > Harden SaslZkACLProviderTest. > - > > Key: SOLR-13075 > URL: https://issues.apache.org/jira/browse/SOLR-13075 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-13075) Harden SaslZkACLProviderTest.
[ https://issues.apache.org/jira/browse/SOLR-13075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated SOLR-13075: Attachment: SOLR-13075.patch > Harden SaslZkACLProviderTest. > - > > Key: SOLR-13075 > URL: https://issues.apache.org/jira/browse/SOLR-13075 > Project: Solr > Issue Type: Sub-task > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Mark Miller >Assignee: Erick Erickson >Priority: Major > Attachments: SOLR-13075.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13364) Make Admin UI aware of logged-in users permissions
[
https://issues.apache.org/jira/browse/SOLR-13364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808752#comment-16808752
]
Gus Heck commented on SOLR-13364:
-
I'd hope that this would populate a JS object that acts as a service and that
caches the information to avoid chatty requests for perms from N elements on
each portion of the ui.
In other (non angularjs) systems I've had this sort of info get populated by a
scriptlet or tag in the base file that sets a global JS value to avoid
providing a "have I elevated my perms" service (granted if they dig through the
HTML/JS on the page they can find it, but that's a little more work than
noticing a rest call-out that spells it out nice and clear. If one wants, one
can make the thing interpreted by the encoded so it's not instantly
recognizable via find command in a browser page source window).
It's just for rendering so caching it should be fine. If the perms change on
the back end the user might need to reload the page, but that doesn't seem like
a problem to me since the perm changes will start failing requests no longer
authorized. (one hopes).
> Make Admin UI aware of logged-in users permissions
> --
>
> Key: SOLR-13364
> URL: https://issues.apache.org/jira/browse/SOLR-13364
> Project: Solr
> Issue Type: New Feature
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Admin UI, Authentication, Authorization, security
>Reporter: Jan Høydahl
>Priority: Major
>
> We should aim to add fine-grained permission checks to the UI. One way to do
> this is to add a new REST-endpoint {{/admin/login/whoami}} that is always
> open for all, and that responds with a JSON with current user's permissions.
> If no user is logged in it will respond with empty list and "No user logged
> in". Else it will respond with e.g.
> {code:java}
> { "user": "john", "roles": ["superuser", "searcher"], "permissions":
> ["security-edit", "collectionadmin"...] }{code}
> The Admin UI can then request this endpoint and cache the info, so that it
> may make decisions to hide/grey out certain menu options throughout the UI.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-8.x - Build # 98 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-8.x/98/ 1 tests failed. FAILED: org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.testDelegationTokenRenew Error Message: expected:<200> but was:<403> Stack Trace: java.lang.AssertionError: expected:<200> but was:<403> at __randomizedtesting.SeedInfo.seed([8D849E0AC37F3A0A:BA1F6A14FBB3E7AE]:0) at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:834) at org.junit.Assert.assertEquals(Assert.java:645) at org.junit.Assert.assertEquals(Assert.java:631) at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.renewDelegationToken(TestDelegationWithHadoopAuth.java:120) at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.verifyDelegationTokenRenew(TestDelegationWithHadoopAuth.java:303) at org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth.testDelegationTokenRenew(TestDelegationWithHadoopAuth.java:321) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[jira] [Created] (SOLR-13365) Add autoscaling policy support for balancing cores across all availability zones
Shalin Shekhar Mangar created SOLR-13365:
Summary: Add autoscaling policy support for balancing cores across
all availability zones
Key: SOLR-13365
URL: https://issues.apache.org/jira/browse/SOLR-13365
Project: Solr
Issue Type: Improvement
Security Level: Public (Default Security Level. Issues are Public)
Components: AutoScaling
Reporter: Shalin Shekhar Mangar
Assignee: Shalin Shekhar Mangar
Fix For: 8.1, master (9.0)
The autoscaling policy supports spreading replicas of a shard across zones (via
sysprops) but that by itself does not guarantee that all availability zones are
actually used and that too in a balanced way.
For example:
{code}
{replica:#EQUAL, shard:#EACH, sysprop.az:#EACH}
{code}
The above policy might end up using only 2 out of 3 availability zones to
spread the replicas (assuming there are enough nodes per zone to not violate
any other policy rules) and not generate any violations. So although we still
have resilience against the loss of one AZ, we do not keep enough capacity as
we could have if we had used all 3 zones equally. In summary, this is a cluster
balancing problem.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8752) Apply a patch to kuromoji dictionary to properly handle Japanese new era '令和' (REIWA)
[
https://issues.apache.org/jira/browse/LUCENE-8752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808716#comment-16808716
]
Robert Muir commented on LUCENE-8752:
-
+1
> Apply a patch to kuromoji dictionary to properly handle Japanese new era '令和'
> (REIWA)
> -
>
> Key: LUCENE-8752
> URL: https://issues.apache.org/jira/browse/LUCENE-8752
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/analysis
>Reporter: Tomoko Uchida
>Priority: Minor
>
> As of May 1st, 2019, Japanese era '元号' (Gengo) will be set to '令和' (Reiwa).
> See this article for more details:
> [https://www.bbc.com/news/world-asia-47769566]
> Currently '令和' is splitted up to '令' and '和' by {{JapaneseTokenizer}}. It
> should be tokenized as one word so that Japanese texts including era names
> are searched as users expect. Because the default Kuromoji dictionary
> (mecab-ipadic) has not been maintained since 2007, a one-line patch to the
> source CSV file is needed for this era change.
> Era name is used in many official or formal documents in Japan, so it would
> be desirable the search systems properly handle this without adding a user
> dictionary or using phrase query. :)
> FYI, JDK DateTime API will support the new era (in the next updates.)
> [https://blogs.oracle.com/java-platform-group/a-new-japanese-era-for-java]
> The patch is available here:
> [https://github.com/apache/lucene-solr/pull/632]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-8.x-Linux (64bit/jdk-11) - Build # 342 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-8.x-Linux/342/ Java: 64bit/jdk-11 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC 4 tests failed. FAILED: org.apache.solr.cloud.TriLevelCompositeIdRoutingTest.test Error Message: routePrefix app9/2!user32 found in multiple shards expected: but was: Stack Trace: org.junit.ComparisonFailure: routePrefix app9/2!user32 found in multiple shards expected: but was: at __randomizedtesting.SeedInfo.seed([A6B6F0104FE6018F:2EE2CFCAE11A6C77]:0) at org.junit.Assert.assertEquals(Assert.java:115) at org.apache.solr.cloud.TriLevelCompositeIdRoutingTest.test(TriLevelCompositeIdRoutingTest.java:122) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[ https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808692#comment-16808692 ] Uwe Schindler commented on LUCENE-2562: --- The confusion came from the fact that you said "I updated the patch..." and I was looking for a patch. You did not add a link to the PR, I had to manually find it in my mails. We should only upload a patch at the end, for now you just have to know that the attached patches are all no longer valid. Would it be possible to remove the isSubclass() code and replace by the JDK one? The cache is also obsolete. > Make Luke a Lucene/Solr Module > -- > > Key: LUCENE-2562 > URL: https://issues.apache.org/jira/browse/LUCENE-2562 > Project: Lucene - Core > Issue Type: Task >Reporter: Mark Miller >Assignee: Uwe Schindler >Priority: Major > Labels: gsoc2014 > Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch, > LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch, > LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png, Luke-ALE-3.png, > Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png, luke-javafx2.png, > luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg, lukeALE-documents.png, > スクリーンショット 2018-11-05 9.19.47.png > > Time Spent: 50m > Remaining Estimate: 0h > > see > "RE: Luke - in need of maintainer": > http://markmail.org/message/m4gsto7giltvrpuf > "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q > I think it would be great if there was a version of Luke that always worked > with trunk - and it would also be great if it was easier to match Luke jars > with Lucene versions. > While I'd like to get GWT Luke into the mix as well, I think the easiest > starting point is to straight port Luke to another UI toolkit before > abstracting out DTO objects that both GWT Luke and Pivot Luke could share. > I've started slowly converting Luke's use of thinlet to Apache Pivot. I > haven't/don't have a lot of time for this at the moment, but I've plugged > away here and there over the past work or two. There is still a *lot* to do. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8752) Apply a patch to kuromoji dictionary to properly handle Japanese new era '令和' (REIWA)
[
https://issues.apache.org/jira/browse/LUCENE-8752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808691#comment-16808691
]
Tomoko Uchida commented on LUCENE-8752:
---
Could you please review this pull request? [~cm] (or someone else?)
[https://github.com/apache/lucene-solr/pull/632]
> Apply a patch to kuromoji dictionary to properly handle Japanese new era '令和'
> (REIWA)
> -
>
> Key: LUCENE-8752
> URL: https://issues.apache.org/jira/browse/LUCENE-8752
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/analysis
>Reporter: Tomoko Uchida
>Priority: Minor
>
> As of May 1st, 2019, Japanese era '元号' (Gengo) will be set to '令和' (Reiwa).
> See this article for more details:
> [https://www.bbc.com/news/world-asia-47769566]
> Currently '令和' is splitted up to '令' and '和' by {{JapaneseTokenizer}}. It
> should be tokenized as one word so that Japanese texts including era names
> are searched as users expect. Because the default Kuromoji dictionary
> (mecab-ipadic) has not been maintained since 2007, a one-line patch to the
> source CSV file is needed for this era change.
> Era name is used in many official or formal documents in Japan, so it would
> be desirable the search systems properly handle this without adding a user
> dictionary or using phrase query. :)
> FYI, JDK DateTime API will support the new era (in the next updates.)
> [https://blogs.oracle.com/java-platform-group/a-new-japanese-era-for-java]
> The patch is available here:
> [https://github.com/apache/lucene-solr/pull/632]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-12833) Use timed-out lock in DistributedUpdateProcessor
[ https://issues.apache.org/jira/browse/SOLR-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808686#comment-16808686 ] Andrzej Bialecki commented on SOLR-12833: -- It's not much, but we can get rid of the int (4 bytes.. meh) .. though this doesn't show up in the profiler - maybe due to alignment? > Use timed-out lock in DistributedUpdateProcessor > > > Key: SOLR-12833 > URL: https://issues.apache.org/jira/browse/SOLR-12833 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: update, UpdateRequestProcessors >Affects Versions: 7.5, 8.0 >Reporter: jefferyyuan >Assignee: Mark Miller >Priority: Minor > Fix For: 7.7, 8.0 > > Attachments: SOLR-12833-noint.patch, SOLR-12833.patch, > SOLR-12833.patch > > Time Spent: 20m > Remaining Estimate: 0h > > There is a synchronize block that blocks other update requests whose IDs fall > in the same hash bucket. The update waits forever until it gets the lock at > the synchronize block, this can be a problem in some cases. > > Some add/update requests (for example updates with spatial/shape analysis) > like may take time (30+ seconds or even more), this would the request time > out and fail. > Client may retry the same requests multiple times or several minutes, this > would make things worse. > The server side receives all the update requests but all except one can do > nothing, have to wait there. This wastes precious memory and cpu resource. > We have seen the case 2000+ threads are blocking at the synchronize lock, and > only a few updates are making progress. Each thread takes 3+ mb memory which > causes OOM. > Also if the update can't get the lock in expected time range, its better to > fail fast. > > We can have one configuration in solrconfig.xml: > updateHandler/versionLock/timeInMill, so users can specify how long they want > to wait the version bucket lock. > The default value can be -1, so it behaves same - wait forever until it gets > the lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[
https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808681#comment-16808681
]
Tomoko Uchida commented on LUCENE-2562:
---
Thanks [~thetaphi],
{quote}Nevertheless, we should clean up the attached files, there is no patch!
I only see the pull request: [https://github.com/apache/lucene-solr/pull/618]
{quote}
Yes, this is the latest pull request, sorry for the confusion. I am not sure
how to clean-up the old attachments. Should I create a patch including all the
changes and attach to the issue?
> Make Luke a Lucene/Solr Module
> --
>
> Key: LUCENE-2562
> URL: https://issues.apache.org/jira/browse/LUCENE-2562
> Project: Lucene - Core
> Issue Type: Task
>Reporter: Mark Miller
>Assignee: Uwe Schindler
>Priority: Major
> Labels: gsoc2014
> Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch,
> LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch,
> LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png, Luke-ALE-3.png,
> Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png, luke-javafx2.png,
> luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg, lukeALE-documents.png,
> スクリーンショット 2018-11-05 9.19.47.png
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> see
> "RE: Luke - in need of maintainer":
> http://markmail.org/message/m4gsto7giltvrpuf
> "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q
> I think it would be great if there was a version of Luke that always worked
> with trunk - and it would also be great if it was easier to match Luke jars
> with Lucene versions.
> While I'd like to get GWT Luke into the mix as well, I think the easiest
> starting point is to straight port Luke to another UI toolkit before
> abstracting out DTO objects that both GWT Luke and Pivot Luke could share.
> I've started slowly converting Luke's use of thinlet to Apache Pivot. I
> haven't/don't have a lot of time for this at the moment, but I've plugged
> away here and there over the past work or two. There is still a *lot* to do.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[GitHub] [lucene-solr] s1monw commented on a change in pull request #613: LUCENE-8671: Expose FST off/on-heap options on Lucene50PostingsFormat
s1monw commented on a change in pull request #613: LUCENE-8671: Expose FST
off/on-heap options on Lucene50PostingsFormat
URL: https://github.com/apache/lucene-solr/pull/613#discussion_r271709426
##
File path:
lucene/core/src/java/org/apache/lucene/codecs/blocktree/FieldReader.java
##
@@ -84,21 +85,18 @@
// if (DEBUG) {
// System.out.println("BTTR: seg=" + segment + " field=" +
fieldInfo.name + " rootBlockCode=" + rootCode + " divisor=" + indexDivisor);
// }
-
rootBlockFP = (new ByteArrayDataInput(rootCode.bytes, rootCode.offset,
rootCode.length)).readVLong() >>> BlockTreeTermsReader.OUTPUT_FLAGS_NUM_BITS;
-
+// Initialize FST offheap if index is MMapDirectory and
+// docCount != sumDocFreq implying field is not primary key
+isFSTOffHeap = ((this.docCount != this.sumDocFreq) || openedFromWriter ==
false) && loadFSTOffHeap;
Review comment:
sure we can do that
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13355) RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
[
https://issues.apache.org/jira/browse/SOLR-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808637#comment-16808637
]
ASF subversion and git services commented on SOLR-13355:
Commit ec1d13a372cba7ccfb28c2b2e584fd1735798068 in lucene-solr's branch
refs/heads/master from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ec1d13a ]
SOLR-13355: Add missing CHANGES.txt entry
> RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
> ---
>
> Key: SOLR-13355
> URL: https://issues.apache.org/jira/browse/SOLR-13355
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Affects Versions: 7.5, 8.0, master (9.0)
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-13355.patch
>
>
> RuleBasedAuthorizationPlugin defines a set of predefined permission rules
> that users can use ootb to lock down sets of APIs to different roles (and
> ultimately, users). The widest of these, the "all" permission is intended to
> be a catch-all that covers all requests not handled by an earlier rule.
> But in practice, "all" doesn't seem to have any effect on most endpoints.
> For example, the security.json below will still allow the readonly user to
> hit almost all endpoints!
> {code}
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "readonly": "",
> "admin": ""}},
> "authorization": {
> "class": "solr.RuleBasedAuthorizationPlugin",
> "permissions": [
> {"name":"read","role": "*"},
> {"name":"schema-read", "role":"*"},
> {"name":"config-read", "role":"*"},
> {"name":"collection-admin-read", "role":"*"},
> {"name":"metrics-read", "role":"*"},
> {"name":"core-admin-read","role":"*"},
> {"name": "all", "role": "admin_role"}
> ],
> "user-role": {
> "readonly": "readonly_role",
> "admin": "admin_role"
> }}}
> {code}
> It looks like this happens because we neglect to check for the "all" special
> case in the branch of code that gets triggered for Handlers that implement
> PermissionNameProvider. See
> [here|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L122].
> e.g. With the security.json above if the "readonly" user makes a request to
> {{/admin/authorization}}, the PermissionNameProvider will return
> {{SECURITY_EDIT}}. When deciding whether the "all" permission applies to
> that endpoint, the code compares SECURITY_EDIT to ALL, sees they don't match,
> and decides that "all" doesn't apply.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13355) RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
[
https://issues.apache.org/jira/browse/SOLR-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808636#comment-16808636
]
ASF subversion and git services commented on SOLR-13355:
Commit fdeedce21f17d8a14b35e5d123bfde873770271f in lucene-solr's branch
refs/heads/branch_8x from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=fdeedce ]
SOLR-13355: Add missing CHANGES.txt entry
> RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
> ---
>
> Key: SOLR-13355
> URL: https://issues.apache.org/jira/browse/SOLR-13355
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Affects Versions: 7.5, 8.0, master (9.0)
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-13355.patch
>
>
> RuleBasedAuthorizationPlugin defines a set of predefined permission rules
> that users can use ootb to lock down sets of APIs to different roles (and
> ultimately, users). The widest of these, the "all" permission is intended to
> be a catch-all that covers all requests not handled by an earlier rule.
> But in practice, "all" doesn't seem to have any effect on most endpoints.
> For example, the security.json below will still allow the readonly user to
> hit almost all endpoints!
> {code}
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "readonly": "",
> "admin": ""}},
> "authorization": {
> "class": "solr.RuleBasedAuthorizationPlugin",
> "permissions": [
> {"name":"read","role": "*"},
> {"name":"schema-read", "role":"*"},
> {"name":"config-read", "role":"*"},
> {"name":"collection-admin-read", "role":"*"},
> {"name":"metrics-read", "role":"*"},
> {"name":"core-admin-read","role":"*"},
> {"name": "all", "role": "admin_role"}
> ],
> "user-role": {
> "readonly": "readonly_role",
> "admin": "admin_role"
> }}}
> {code}
> It looks like this happens because we neglect to check for the "all" special
> case in the branch of code that gets triggered for Handlers that implement
> PermissionNameProvider. See
> [here|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L122].
> e.g. With the security.json above if the "readonly" user makes a request to
> {{/admin/authorization}}, the PermissionNameProvider will return
> {{SECURITY_EDIT}}. When deciding whether the "all" permission applies to
> that endpoint, the code compares SECURITY_EDIT to ALL, sees they don't match,
> and decides that "all" doesn't apply.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13355) RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
[
https://issues.apache.org/jira/browse/SOLR-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808632#comment-16808632
]
ASF subversion and git services commented on SOLR-13355:
Commit b135c49d386c14ac2a6476fb9e57ab8a5c74ecf7 in lucene-solr's branch
refs/heads/branch_7_7 from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=b135c49 ]
SOLR-13355: Obey 'ALL' for handlers with other predefined perms
Prior to this commit, RuleBasedAuthorizationPlugin would check for the
predefined 'ALL' permission only when the endpoint being hit wasn't
associated with another predefined-permission.
This resulted in some very unintuitive behavior. For example, the
permission {name:all, role:admin} would correctly prevent a
role:foo user from accessing /admin/info/properties, but would allow
write access to /admin/authorization because of the SECURITY_EDIT
predefined perm associated with that endpoint.
This commit fixes this bug so that the 'all' permission is always
consulted whether or not the endpoint is associated with other predefined
permissions.
> RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
> ---
>
> Key: SOLR-13355
> URL: https://issues.apache.org/jira/browse/SOLR-13355
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Affects Versions: 7.5, 8.0, master (9.0)
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-13355.patch
>
>
> RuleBasedAuthorizationPlugin defines a set of predefined permission rules
> that users can use ootb to lock down sets of APIs to different roles (and
> ultimately, users). The widest of these, the "all" permission is intended to
> be a catch-all that covers all requests not handled by an earlier rule.
> But in practice, "all" doesn't seem to have any effect on most endpoints.
> For example, the security.json below will still allow the readonly user to
> hit almost all endpoints!
> {code}
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "readonly": "",
> "admin": ""}},
> "authorization": {
> "class": "solr.RuleBasedAuthorizationPlugin",
> "permissions": [
> {"name":"read","role": "*"},
> {"name":"schema-read", "role":"*"},
> {"name":"config-read", "role":"*"},
> {"name":"collection-admin-read", "role":"*"},
> {"name":"metrics-read", "role":"*"},
> {"name":"core-admin-read","role":"*"},
> {"name": "all", "role": "admin_role"}
> ],
> "user-role": {
> "readonly": "readonly_role",
> "admin": "admin_role"
> }}}
> {code}
> It looks like this happens because we neglect to check for the "all" special
> case in the branch of code that gets triggered for Handlers that implement
> PermissionNameProvider. See
> [here|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L122].
> e.g. With the security.json above if the "readonly" user makes a request to
> {{/admin/authorization}}, the PermissionNameProvider will return
> {{SECURITY_EDIT}}. When deciding whether the "all" permission applies to
> that endpoint, the code compares SECURITY_EDIT to ALL, sees they don't match,
> and decides that "all" doesn't apply.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13355) RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
[
https://issues.apache.org/jira/browse/SOLR-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808631#comment-16808631
]
ASF subversion and git services commented on SOLR-13355:
Commit 4ad797a106483819603ee971638e36ee21c57645 in lucene-solr's branch
refs/heads/branch_7_7 from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=4ad797a ]
SOLR-13355: Small refactors to RuleBasedAuthorizationPlugin
> RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
> ---
>
> Key: SOLR-13355
> URL: https://issues.apache.org/jira/browse/SOLR-13355
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Affects Versions: 7.5, 8.0, master (9.0)
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-13355.patch
>
>
> RuleBasedAuthorizationPlugin defines a set of predefined permission rules
> that users can use ootb to lock down sets of APIs to different roles (and
> ultimately, users). The widest of these, the "all" permission is intended to
> be a catch-all that covers all requests not handled by an earlier rule.
> But in practice, "all" doesn't seem to have any effect on most endpoints.
> For example, the security.json below will still allow the readonly user to
> hit almost all endpoints!
> {code}
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "readonly": "",
> "admin": ""}},
> "authorization": {
> "class": "solr.RuleBasedAuthorizationPlugin",
> "permissions": [
> {"name":"read","role": "*"},
> {"name":"schema-read", "role":"*"},
> {"name":"config-read", "role":"*"},
> {"name":"collection-admin-read", "role":"*"},
> {"name":"metrics-read", "role":"*"},
> {"name":"core-admin-read","role":"*"},
> {"name": "all", "role": "admin_role"}
> ],
> "user-role": {
> "readonly": "readonly_role",
> "admin": "admin_role"
> }}}
> {code}
> It looks like this happens because we neglect to check for the "all" special
> case in the branch of code that gets triggered for Handlers that implement
> PermissionNameProvider. See
> [here|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L122].
> e.g. With the security.json above if the "readonly" user makes a request to
> {{/admin/authorization}}, the PermissionNameProvider will return
> {{SECURITY_EDIT}}. When deciding whether the "all" permission applies to
> that endpoint, the code compares SECURITY_EDIT to ALL, sees they don't match,
> and decides that "all" doesn't apply.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13355) RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
[
https://issues.apache.org/jira/browse/SOLR-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808626#comment-16808626
]
ASF subversion and git services commented on SOLR-13355:
Commit 68918206f56fc7a65ce9b84a9cf6a30edf8ce7c2 in lucene-solr's branch
refs/heads/branch_8x from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6891820 ]
SOLR-13355: Small refactors to RuleBasedAuthorizationPlugin
> RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
> ---
>
> Key: SOLR-13355
> URL: https://issues.apache.org/jira/browse/SOLR-13355
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Affects Versions: 7.5, 8.0, master (9.0)
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-13355.patch
>
>
> RuleBasedAuthorizationPlugin defines a set of predefined permission rules
> that users can use ootb to lock down sets of APIs to different roles (and
> ultimately, users). The widest of these, the "all" permission is intended to
> be a catch-all that covers all requests not handled by an earlier rule.
> But in practice, "all" doesn't seem to have any effect on most endpoints.
> For example, the security.json below will still allow the readonly user to
> hit almost all endpoints!
> {code}
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "readonly": "",
> "admin": ""}},
> "authorization": {
> "class": "solr.RuleBasedAuthorizationPlugin",
> "permissions": [
> {"name":"read","role": "*"},
> {"name":"schema-read", "role":"*"},
> {"name":"config-read", "role":"*"},
> {"name":"collection-admin-read", "role":"*"},
> {"name":"metrics-read", "role":"*"},
> {"name":"core-admin-read","role":"*"},
> {"name": "all", "role": "admin_role"}
> ],
> "user-role": {
> "readonly": "readonly_role",
> "admin": "admin_role"
> }}}
> {code}
> It looks like this happens because we neglect to check for the "all" special
> case in the branch of code that gets triggered for Handlers that implement
> PermissionNameProvider. See
> [here|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L122].
> e.g. With the security.json above if the "readonly" user makes a request to
> {{/admin/authorization}}, the PermissionNameProvider will return
> {{SECURITY_EDIT}}. When deciding whether the "all" permission applies to
> that endpoint, the code compares SECURITY_EDIT to ALL, sees they don't match,
> and decides that "all" doesn't apply.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-13355) RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
[
https://issues.apache.org/jira/browse/SOLR-13355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808627#comment-16808627
]
ASF subversion and git services commented on SOLR-13355:
Commit d5b9fbee374e83045613fba5cb87bd550afcfa2b in lucene-solr's branch
refs/heads/branch_8x from Jason Gerlowski
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=d5b9fbe ]
SOLR-13355: Obey 'ALL' for handlers with other predefined perms
Prior to this commit, RuleBasedAuthorizationPlugin would check for the
predefined 'ALL' permission only when the endpoint being hit wasn't
associated with another predefined-permission.
This resulted in some very unintuitive behavior. For example, the
permission {name:all, role:admin} would correctly prevent a
role:foo user from accessing /admin/info/properties, but would allow
write access to /admin/authorization because of the SECURITY_EDIT
predefined perm associated with that endpoint.
This commit fixes this bug so that the 'all' permission is always
consulted whether or not the endpoint is associated with other predefined
permissions.
> RuleBasedAuthorizationPlugin ignores "all" permission for most handlers
> ---
>
> Key: SOLR-13355
> URL: https://issues.apache.org/jira/browse/SOLR-13355
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: security
>Affects Versions: 7.5, 8.0, master (9.0)
>Reporter: Jason Gerlowski
>Assignee: Jason Gerlowski
>Priority: Major
> Attachments: SOLR-13355.patch
>
>
> RuleBasedAuthorizationPlugin defines a set of predefined permission rules
> that users can use ootb to lock down sets of APIs to different roles (and
> ultimately, users). The widest of these, the "all" permission is intended to
> be a catch-all that covers all requests not handled by an earlier rule.
> But in practice, "all" doesn't seem to have any effect on most endpoints.
> For example, the security.json below will still allow the readonly user to
> hit almost all endpoints!
> {code}
> {
> "authentication": {
> "blockUnknown": true,
> "class": "solr.BasicAuthPlugin",
> "credentials": {
> "readonly": "",
> "admin": ""}},
> "authorization": {
> "class": "solr.RuleBasedAuthorizationPlugin",
> "permissions": [
> {"name":"read","role": "*"},
> {"name":"schema-read", "role":"*"},
> {"name":"config-read", "role":"*"},
> {"name":"collection-admin-read", "role":"*"},
> {"name":"metrics-read", "role":"*"},
> {"name":"core-admin-read","role":"*"},
> {"name": "all", "role": "admin_role"}
> ],
> "user-role": {
> "readonly": "readonly_role",
> "admin": "admin_role"
> }}}
> {code}
> It looks like this happens because we neglect to check for the "all" special
> case in the branch of code that gets triggered for Handlers that implement
> PermissionNameProvider. See
> [here|https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/security/RuleBasedAuthorizationPlugin.java#L122].
> e.g. With the security.json above if the "readonly" user makes a request to
> {{/admin/authorization}}, the PermissionNameProvider will return
> {{SECURITY_EDIT}}. When deciding whether the "all" permission applies to
> that endpoint, the code compares SECURITY_EDIT to ALL, sees they don't match,
> and decides that "all" doesn't apply.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-12833) Use timed-out lock in DistributedUpdateProcessor
[ https://issues.apache.org/jira/browse/SOLR-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrzej Bialecki updated SOLR-12833: - Attachment: SOLR-12833-noint.patch > Use timed-out lock in DistributedUpdateProcessor > > > Key: SOLR-12833 > URL: https://issues.apache.org/jira/browse/SOLR-12833 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) > Components: update, UpdateRequestProcessors >Affects Versions: 7.5, 8.0 >Reporter: jefferyyuan >Assignee: Mark Miller >Priority: Minor > Fix For: 7.7, 8.0 > > Attachments: SOLR-12833-noint.patch, SOLR-12833.patch, > SOLR-12833.patch > > Time Spent: 20m > Remaining Estimate: 0h > > There is a synchronize block that blocks other update requests whose IDs fall > in the same hash bucket. The update waits forever until it gets the lock at > the synchronize block, this can be a problem in some cases. > > Some add/update requests (for example updates with spatial/shape analysis) > like may take time (30+ seconds or even more), this would the request time > out and fail. > Client may retry the same requests multiple times or several minutes, this > would make things worse. > The server side receives all the update requests but all except one can do > nothing, have to wait there. This wastes precious memory and cpu resource. > We have seen the case 2000+ threads are blocking at the synchronize lock, and > only a few updates are making progress. Each thread takes 3+ mb memory which > causes OOM. > Also if the update can't get the lock in expected time range, its better to > fail fast. > > We can have one configuration in solrconfig.xml: > updateHandler/versionLock/timeInMill, so users can specify how long they want > to wait the version bucket lock. > The default value can be -1, so it behaves same - wait forever until it gets > the lock. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11208) Usage SynchronousQueue in Executors prevent large scale operations
[
https://issues.apache.org/jira/browse/SOLR-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808615#comment-16808615
]
Roger Lehmann commented on SOLR-11208:
--
A quick and helpful workaround of this design flaw would be to make the maximum
amount of threads available to the MDCAwareThreadPool configurable in the Solr
config.
It is currently preventing me from using SolrCloud 7.7.1 in an AWS auto scaling
group where instances are added and deleted dynamically.
I'm unfortunately not fit to do these changes myself, anyone up to that? Would
be great, thanks!
> Usage SynchronousQueue in Executors prevent large scale operations
> --
>
> Key: SOLR-11208
> URL: https://issues.apache.org/jira/browse/SOLR-11208
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
>Affects Versions: 6.6
>Reporter: Björn Häuser
>Priority: Major
>
> I am not sure where to start with this one.
> I tried to post this already on the mailing list:
> https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201708.mbox/%3c48c49426-33a2-4d79-ae26-a4515b8f8...@gmail.com%3e
> In short: the usage of a SynchronousQueue as the workQeue prevents more tasks
> than max threads.
> For example, taken from OverseerCollectionMessageHandler:
> {code:java}
> ExecutorService tpe = new ExecutorUtil.MDCAwareThreadPoolExecutor(5, 10,
> 0L, TimeUnit.MILLISECONDS,
> new SynchronousQueue<>(),
> new
> DefaultSolrThreadFactory("OverseerCollectionMessageHandlerThreadFactory"));
> {code}
> This Executor is used when doing a REPLACENODE (= ADDREPLICA) command. When
> the node has more than 10 collections this will fail with the mentioned
> java.util.concurrent.RejectedExecutionException.
> I am also not sure how to fix this. Just replacing the queue with a different
> implementation feels wrong to me or could cause unwanted side behaviour.
> Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8738) Bump minimum Java version requirement to 11
[ https://issues.apache.org/jira/browse/LUCENE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808610#comment-16808610 ] Adrien Grand commented on LUCENE-8738: -- +1 The other part that needs reviews is the replacement of Observable/Observer with utility classes from java.beans. > Bump minimum Java version requirement to 11 > --- > > Key: LUCENE-8738 > URL: https://issues.apache.org/jira/browse/LUCENE-8738 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build >Reporter: Adrien Grand >Priority: Minor > Labels: Java11 > Fix For: master (9.0) > > > See vote thread for reference: https://markmail.org/message/q6ubdycqscpl43aq. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8752) Apply a patch to kuromoji dictionary to properly handle Japanese new era '令和' (REIWA)
[
https://issues.apache.org/jira/browse/LUCENE-8752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808599#comment-16808599
]
Tomoko Uchida commented on LUCENE-8752:
---
The POS tag and costs for "令和" is set after the current era "平成" to not to
affect on current analysis behaviour.
{{mecab-ipadic-2.7.0-20070801/Noun.proper.csv}}
{code:java}
平成,1288,1288,5904,名詞,固有名詞,一般,*,*,*,平成,ヘイセイ,ヘイセイ
...
令和,1288,1288,5904,名詞,固有名詞,一般,*,*,*,令和,レイワ,レイワ // added in the patch
{code}
> Apply a patch to kuromoji dictionary to properly handle Japanese new era '令和'
> (REIWA)
> -
>
> Key: LUCENE-8752
> URL: https://issues.apache.org/jira/browse/LUCENE-8752
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/analysis
>Reporter: Tomoko Uchida
>Priority: Minor
>
> As of May 1st, 2019, Japanese era '元号' (Gengo) will be set to '令和' (Reiwa).
> See this article for more details:
> [https://www.bbc.com/news/world-asia-47769566]
> Currently '令和' is splitted up to '令' and '和' by {{JapaneseTokenizer}}. It
> should be tokenized as one word so that Japanese texts including era names
> are searched as users expect. Because the default Kuromoji dictionary
> (mecab-ipadic) has not been maintained since 2007, a one-line patch to the
> source CSV file is needed for this era change.
> Era name is used in many official or formal documents in Japan, so it would
> be desirable the search systems properly handle this without adding a user
> dictionary or using phrase query. :)
> FYI, JDK DateTime API will support the new era (in the next updates.)
> [https://blogs.oracle.com/java-platform-group/a-new-japanese-era-for-java]
> The patch is available here:
> [https://github.com/apache/lucene-solr/pull/632]
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[ https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808579#comment-16808579 ] Uwe Schindler commented on LUCENE-2562: --- Nevertheless, we should clean up the attached files, there is no patch! I only see the pull request: https://github.com/apache/lucene-solr/pull/618 > Make Luke a Lucene/Solr Module > -- > > Key: LUCENE-2562 > URL: https://issues.apache.org/jira/browse/LUCENE-2562 > Project: Lucene - Core > Issue Type: Task >Reporter: Mark Miller >Assignee: Uwe Schindler >Priority: Major > Labels: gsoc2014 > Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch, > LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch, > LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png, Luke-ALE-3.png, > Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png, luke-javafx2.png, > luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg, lukeALE-documents.png, > スクリーンショット 2018-11-05 9.19.47.png > > Time Spent: 50m > Remaining Estimate: 0h > > see > "RE: Luke - in need of maintainer": > http://markmail.org/message/m4gsto7giltvrpuf > "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q > I think it would be great if there was a version of Luke that always worked > with trunk - and it would also be great if it was easier to match Luke jars > with Lucene versions. > While I'd like to get GWT Luke into the mix as well, I think the easiest > starting point is to straight port Luke to another UI toolkit before > abstracting out DTO objects that both GWT Luke and Pivot Luke could share. > I've started slowly converting Luke's use of thinlet to Apache Pivot. I > haven't/don't have a lot of time for this at the moment, but I've plugged > away here and there over the past work or two. There is still a *lot* to do. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-8752) Apply a patch to kuromoji dictionary to properly handle Japanese new era '令和' (REIWA)
Tomoko Uchida created LUCENE-8752:
-
Summary: Apply a patch to kuromoji dictionary to properly handle
Japanese new era '令和' (REIWA)
Key: LUCENE-8752
URL: https://issues.apache.org/jira/browse/LUCENE-8752
Project: Lucene - Core
Issue Type: Improvement
Components: modules/analysis
Reporter: Tomoko Uchida
As of May 1st, 2019, Japanese era '元号' (Gengo) will be set to '令和' (Reiwa). See
this article for more details:
[https://www.bbc.com/news/world-asia-47769566]
Currently '令和' is splitted up to '令' and '和' by {{JapaneseTokenizer}}. It
should be tokenized as one word so that Japanese texts including era names are
searched as users expect. Because the default Kuromoji dictionary
(mecab-ipadic) has not been maintained since 2007, a one-line patch to the
source CSV file is needed for this era change.
Era name is used in many official or formal documents in Japan, so it would be
desirable the search systems properly handle this without adding a user
dictionary or using phrase query. :)
FYI, JDK DateTime API will support the new era (in the next updates.)
[https://blogs.oracle.com/java-platform-group/a-new-japanese-era-for-java]
The patch is available here:
[https://github.com/apache/lucene-solr/pull/632]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2562) Make Luke a Lucene/Solr Module
[ https://issues.apache.org/jira/browse/LUCENE-2562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808574#comment-16808574 ] Uwe Schindler commented on LUCENE-2562: --- Hi [~Tomoko Uchida], thanks for adding the patch. I am now a bit confused about the pull requests related to that patch. Which one is the uptodate one? I made a comment on one of the pull requests (https://github.com/apache/lucene-solr/pull/618), but not sure if this is reflected in the patch. I'd like to remove the complexits in isSubclassOf, as this can be handled easily with standard java code. > Make Luke a Lucene/Solr Module > -- > > Key: LUCENE-2562 > URL: https://issues.apache.org/jira/browse/LUCENE-2562 > Project: Lucene - Core > Issue Type: Task >Reporter: Mark Miller >Assignee: Uwe Schindler >Priority: Major > Labels: gsoc2014 > Attachments: LUCENE-2562-Ivy.patch, LUCENE-2562-Ivy.patch, > LUCENE-2562-Ivy.patch, LUCENE-2562-ivy.patch, LUCENE-2562.patch, > LUCENE-2562.patch, Luke-ALE-1.png, Luke-ALE-2.png, Luke-ALE-3.png, > Luke-ALE-4.png, Luke-ALE-5.png, luke-javafx1.png, luke-javafx2.png, > luke-javafx3.png, luke1.jpg, luke2.jpg, luke3.jpg, lukeALE-documents.png, > スクリーンショット 2018-11-05 9.19.47.png > > Time Spent: 50m > Remaining Estimate: 0h > > see > "RE: Luke - in need of maintainer": > http://markmail.org/message/m4gsto7giltvrpuf > "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q > I think it would be great if there was a version of Luke that always worked > with trunk - and it would also be great if it was easier to match Luke jars > with Lucene versions. > While I'd like to get GWT Luke into the mix as well, I think the easiest > starting point is to straight port Luke to another UI toolkit before > abstracting out DTO objects that both GWT Luke and Pivot Luke could share. > I've started slowly converting Luke's use of thinlet to Apache Pivot. I > haven't/don't have a lot of time for this at the moment, but I've plugged > away here and there over the past work or two. There is still a *lot* to do. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS-EA] Lucene-Solr-master-Windows (64bit/jdk-13-ea+12) - Build # 7869 - Failure!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7869/ Java: 64bit/jdk-13-ea+12 -XX:-UseCompressedOops -XX:+UseParallelGC 2 tests failed. FAILED: org.apache.solr.cloud.BasicDistributedZkTest.test Error Message: KeeperErrorCode = ConnectionLoss for /solr/collections/oneInstanceCollection2/state.json Stack Trace: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /solr/collections/oneInstanceCollection2/state.json at __randomizedtesting.SeedInfo.seed([9BBB16D6D661EA9D:13EF290C789D8765]:0) at org.apache.zookeeper.KeeperException.create(KeeperException.java:102) at org.apache.zookeeper.KeeperException.create(KeeperException.java:54) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1221) at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:358) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:358) at org.apache.solr.common.cloud.SolrZkClient.printLayout(SolrZkClient.java:615) at org.apache.solr.common.cloud.SolrZkClient.printLayout(SolrZkClient.java:640) at org.apache.solr.common.cloud.SolrZkClient.printLayout(SolrZkClient.java:640) at org.apache.solr.common.cloud.SolrZkClient.printLayout(SolrZkClient.java:640) at org.apache.solr.common.cloud.SolrZkClient.printLayout(SolrZkClient.java:640) at org.apache.solr.common.cloud.SolrZkClient.printLayoutToStream(SolrZkClient.java:653) at org.apache.solr.cloud.AbstractDistribZkTestBase.printLayout(AbstractDistribZkTestBase.java:315) at org.apache.solr.cloud.BasicDistributedZkTest.testStopAndStartCoresInOneInstance(BasicDistributedZkTest.java:632) at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:427) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:567) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938) at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974) at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082) at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817) at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at
[GitHub] [lucene-solr] mocobeta opened a new pull request #632: Add a Kuromoji dictionary entry for Japanese new era 令和(Reiwa)
mocobeta opened a new pull request #632: Add a Kuromoji dictionary entry for Japanese new era 令和(Reiwa) URL: https://github.com/apache/lucene-solr/pull/632 As of May 1st, 2019, Japanese era '元号' (Gengo) will be set to '令和' (Reiwa). Currently '令和' is splitted up to '令' and '和' by `JapaneseTokenizer`. It should be tokenized as one word so that Japanese texts including era names are searched as users expect. Because the default Kuromoji dictionary (mecab-ipadic) is not maintained since 2007, this applies a patch to the source CSV file before building the dictionary. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8738) Bump minimum Java version requirement to 11
[
https://issues.apache.org/jira/browse/LUCENE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808555#comment-16808555
]
Uwe Schindler commented on LUCENE-8738:
---
I have some comment about the deprecation removal:
The {{Class#newInstance()}} stuff was replaced by
{{clazz.getDeclaredConstructor().newInstance()}}. This is perfectly fine and
also suggested in the deprecation notice. But I'd like to use
{{getConstructor()}} instead, as we are only interested in "public"
constructors.
We should also think about the fact that the code may now throw different
exceptions. About the changes [~jpountz] did: Most code ways already wrapped
with a general catch-all code, but maybe we can improve and unwrap runtime
excetions from the InvocationTargetException. I will have a closer look again.
> Bump minimum Java version requirement to 11
> ---
>
> Key: LUCENE-8738
> URL: https://issues.apache.org/jira/browse/LUCENE-8738
> Project: Lucene - Core
> Issue Type: Improvement
> Components: general/build
>Reporter: Adrien Grand
>Priority: Minor
> Labels: Java11
> Fix For: master (9.0)
>
>
> See vote thread for reference: https://markmail.org/message/q6ubdycqscpl43aq.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8738) Bump minimum Java version requirement to 11
[ https://issues.apache.org/jira/browse/LUCENE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808542#comment-16808542 ] Uwe Schindler commented on LUCENE-8738: --- I removed the MR-JAR parts. To easily restore it in case we would like to use it for Java 12+, this commit should survive in GIT, so we should not squash the branch of this issue. I kept the documentation unmodified in the README.maven file, as the artifacts built by Maven should never be used in production, which is unrelated to MR-JAR. > Bump minimum Java version requirement to 11 > --- > > Key: LUCENE-8738 > URL: https://issues.apache.org/jira/browse/LUCENE-8738 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build >Reporter: Adrien Grand >Priority: Minor > Labels: Java11 > Fix For: master (9.0) > > > See vote thread for reference: https://markmail.org/message/q6ubdycqscpl43aq. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8738) Bump minimum Java version requirement to 11
[ https://issues.apache.org/jira/browse/LUCENE-8738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808537#comment-16808537 ] ASF subversion and git services commented on LUCENE-8738: - Commit ba01df2c22840b699a758df930e0212e1682457b in lucene-solr's branch refs/heads/jira/LUCENE-8738 from Uwe Schindler [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ba01df2 ] LUCENE-8738, LUCENE-7966: Remove MR-JAR classfile patching from Lucene 9 completely (minimum Java 11) > Bump minimum Java version requirement to 11 > --- > > Key: LUCENE-8738 > URL: https://issues.apache.org/jira/browse/LUCENE-8738 > Project: Lucene - Core > Issue Type: Improvement > Components: general/build >Reporter: Adrien Grand >Priority: Minor > Labels: Java11 > Fix For: master (9.0) > > > See vote thread for reference: https://markmail.org/message/q6ubdycqscpl43aq. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-7966) build mr-jar and use some java 9 methods if available
[
https://issues.apache.org/jira/browse/LUCENE-7966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808538#comment-16808538
]
ASF subversion and git services commented on LUCENE-7966:
-
Commit ba01df2c22840b699a758df930e0212e1682457b in lucene-solr's branch
refs/heads/jira/LUCENE-8738 from Uwe Schindler
[ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=ba01df2 ]
LUCENE-8738, LUCENE-7966: Remove MR-JAR classfile patching from Lucene 9
completely (minimum Java 11)
> build mr-jar and use some java 9 methods if available
> -
>
> Key: LUCENE-7966
> URL: https://issues.apache.org/jira/browse/LUCENE-7966
> Project: Lucene - Core
> Issue Type: Improvement
> Components: core/other, general/build
>Reporter: Robert Muir
>Assignee: Uwe Schindler
>Priority: Major
> Labels: Java9
> Fix For: 7.3, 8.0
>
> Attachments: LUCENE-7966-7x-backwards.patch,
> LUCENE-7966-7x-backwards.patch, LUCENE-7966-v2.patch, LUCENE-7966.patch,
> LUCENE-7966.patch, LUCENE-7966.patch, LUCENE-7966.patch, LUCENE-7966.patch
>
>
> See background: http://openjdk.java.net/jeps/238
> It would be nice to use some of the newer array methods and range checking
> methods in java 9 for example, without waiting for lucene 10 or something. If
> we build an MR-jar, we can start migrating our code to use java 9 methods
> right now, it will use optimized methods from java 9 when thats available,
> otherwise fall back to java 8 code.
> This patch adds:
> {code}
> Objects.checkIndex(int,int)
> Objects.checkFromToIndex(int,int,int)
> Objects.checkFromIndexSize(int,int,int)
> Arrays.mismatch(byte[],int,int,byte[],int,int)
> Arrays.compareUnsigned(byte[],int,int,byte[],int,int)
> Arrays.equal(byte[],int,int,byte[],int,int)
> // did not add char/int/long/short/etc but of course its possible if needed
> {code}
> It sets these up in {{org.apache.lucene.future}} as 1-1 mappings to java
> methods. This way, we can simply directly replace call sites with java 9
> methods when java 9 is a minimum. Simple 1-1 mappings mean also that we only
> have to worry about testing that our java 8 fallback methods work.
> I found that many of the current byte array methods today are willy-nilly and
> very lenient for example, passing invalid offsets at times and relying on
> compare methods not throwing exceptions, etc. I fixed all the instances in
> core/codecs but have not looked at the problems with AnalyzingSuggester. Also
> SimpleText still uses a silly method in ArrayUtil in similar crazy way, have
> not removed that one yet.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-8750) Implement setMissingValue for numeric ValueSource sortFields
[ https://issues.apache.org/jira/browse/LUCENE-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Woodward resolved LUCENE-8750. --- Resolution: Fixed Assignee: Alan Woodward Fix Version/s: 8.1 Thanks [~msoko...@gmail.com]! > Implement setMissingValue for numeric ValueSource sortFields > > > Key: LUCENE-8750 > URL: https://issues.apache.org/jira/browse/LUCENE-8750 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Mike Sokolov >Assignee: Alan Woodward >Priority: Major > Fix For: 8.1 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > We currently have setMissingValue for SortFields based on concrete numeric > fields, but not for SortFields derived from LongValuesSource and > DoubleValuesSource. > This issue is for implementing > LongValuesSource.LongValuesSortField.setMissingValue and > DoubleValuesSource.DoubleValuesSortField.setMissingValue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-8750) Implement setMissingValue for numeric ValueSource sortFields
[ https://issues.apache.org/jira/browse/LUCENE-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16808521#comment-16808521 ] ASF subversion and git services commented on LUCENE-8750: - Commit 6f47062a9b0a4d76562f15c4f1a3aec6acf27295 in lucene-solr's branch refs/heads/branch_8x from Alan Woodward [ https://gitbox.apache.org/repos/asf?p=lucene-solr.git;h=6f47062 ] LUCENE-8750: Add setMissingValue to sorts from Double/LongValuesSource > Implement setMissingValue for numeric ValueSource sortFields > > > Key: LUCENE-8750 > URL: https://issues.apache.org/jira/browse/LUCENE-8750 > Project: Lucene - Core > Issue Type: Improvement >Reporter: Mike Sokolov >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > We currently have setMissingValue for SortFields based on concrete numeric > fields, but not for SortFields derived from LongValuesSource and > DoubleValuesSource. > This issue is for implementing > LongValuesSource.LongValuesSortField.setMissingValue and > DoubleValuesSource.DoubleValuesSortField.setMissingValue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
